Cardiff University  
 
 

 


Booking form Evaluation form Contact RCBN



      

SUMMARY OF 'MODELS OF RESEARCH IMPACT: A CROSS-SECTOR REVIEW OF LITERATURE AND PRACTICE' (LSDA)

 

Models of research impact: a cross-sector view of literature and practice
(Building effective research: 4th report in the series)
Learning and Skills Research Centre (2003)
Authors: Sandra Nutley, Janie Percy-Smith and William Solesbury
ISBN 1 85338 857 2

This major national study sets out practical steps to enhance the impact of research on practice and policy. It analyses evidence from both research literature and actual practice across education, social care, criminal justice and healthcare sectors, and offers recommendations to the learning and skills community.

The summary is divided into two parts: (click on either to navigate the summary)

To see the complete report in pdf format please click here.

The RCBN have also established an online discussion forum to discuss these models of research impact further. To join the discussion please click here.

SUMMARY OF FINDINGS (Return to top)

Aims | Literature Review | Case Studies | Practices for Research Impact | Effectiveness
Barriers and Enablers | Table 1 Practices for Enhancing Research Impact

Aims (Return to top)

• To review the literature on research impact;
• To assess practices in comparable organisations;
• To characterise models, strategies and actions that would be useful in the learning and skills sector; and
• To develop guidance for the designers and managers of LSRC research.

Literature Review (Return to top)

5,800 references initially identified
750 examined in full
341 selected for detailed analysis
Of these:

155 = conceptual papers
2 = methodological
59 = provided background
125 = empirical papers (60% from healthcare)

[More information about the literature review can be found in Appendix 1 of the report, pp.33-42; Empirical references listed on pp.43- 49]

Distinction between two kinds of impact:
1. ‘Conceptual’ value of research (or the ‘enlightenment’) – changes of knowledge, understanding and belief.
2. ‘Instrumental’ value of research (or ‘engineering’) – directly changing behaviour in policy and practice.

Most literature considers impact as ‘instrumental’ or to change policy/practice behaviour.

Case Studies (Return to top)

Characteristics in selecting case studies (at least one or more present):
• Research users as well as research producers and/or research funders;
• Both a national and a regional or local remit;
• Established reputations for effectively relating research to policy or practice; and/or
• A span of diverse sectors and disciplines.

Five organisations chosen:
1. Barnardo’s
2. Joseph Rowntree Foundation (JRF)
3. Local and Regional Government Research Unit (LRGRU)
4. Local Government Association (LGA)
5. MORI Social Research Institute (MORI)

[More information about the case studies can be found in the Appendix 2, pp.51-62]

Practices for Research Impact (Return to top)

Each source analysed to:
• Identify the range of practices revealed in the source;
• Characterise each practice in terms of its underpinning model of research use;
• Note what is known about their effectiveness and the contextual barriers to, and enablers of, their use.

27 distinct practices recorded (see Table 1). These can be summarised under 6 headings:
1. Tailored presentation of research findings;
2. Tailoring research to users’ needs;
3. Increasing communication between researchers and users;
4. Support for developing research-informed practice;
5. Rewarding and reinforcing research-informed practice; and
6. Staff development, education and training.

Effectiveness (Return to top)

• Tangible impact = changes in policy, organisation, resourcing or delivery
• Intangible impact = changes in understanding, attitudes and behaviour (individuals’ and organisations’)

Measuring impact = few cases – where reported these are diverse: objective measures to self-reporting and observation by participants)

‘Effective’ practices:
• Active dissemination
• Closer links between researchers and practitioners
• Facilitation of research use
• Recruiting expert and peer opinion leaders

General characteristics of ‘effective’ practices: Multiple practices with typically long-term impacts

Barriers and Enablers (Return to top)

Culture – inclination to engage with research and/or policy/practice.
Salience – research perceived as helpful where it is relevant and timely.
Competing evidence – research must always compete with other influences, therefore reliability and reputation are important; where research is contradictory then practitioners will need guidance on their interpretation.
Communication channels – effective communication through appropriate media; communication of a range of factors, e.g. exigencies of practice, research craft, and results.
Resources and skills – time and skills are necessary and therefore should be organisational priorities.

Table 1 Practices for Enhancing Research Impact (source: p.15-6) (Return to top)

  Practice Examples Effectiveness?
Tailored presentation of research findings
1
Practice guidelines – informed by research Growing medical practice, sometimes developed through consensus processes; LGA produces good practice examples for use by local authorities Single guideline has low impact in changing practice; needs reinforcement through reminders, incentives, peer endorsement
2
Use of mass media – through press notices and briefing Common practice in case studies; JRF is particularly successful in securing coverage Evidence from systematic reviews in healthcare that media coverage can promote behavioural change in practitioners and consumers
3
Print publications – reports, summaries, newsletters Practised in MORI, JRG, LRGRU and LGA for external audiences; common practice found in literature review Can increase awareness, but unlikely by itself to change behaviour
4
Workshop/seminar presentations MORI is proactive in presentations at others’ or own events Literature suggests oral presentations with opportunity for interaction impacts well on practitioners; source and timing are important
5
Lobbying – including through consultation responses JRF sends letter and research summary to 10 most influential people in relevant practice field Ad hoc evidence of effectiveness in raising awareness, but repetition becomes counterproductive
6
Tailored material – for target audiences JRF is responsive to such opportunities, eg policy reviews Can help to change attitudes; requires empathy and/or cooperation with practitioners to get it right
Tailoring research to users’ needs
7
Research planning – to improve relevance Both MORI and LRGRU negotiate research plans with clients Ad hoc evidence of effectiveness by encouraging early engagement by research units
8
Demonstration projects – to carry research into practice Beacon schools, pilot or pathfinder projects, taskforces, practical projects Ad hoc evidence of effectiveness; believed to add credibility to research and sharpen its practical application
9
Standards for research – to improve quality Barnardo’s has standards for research linked to organisational quality commitments Ad hoc evidence that it works we as part of a wider framework of change management
10
Research programmes – rather than small projects JRF and LRGRU have moved to larger research programmes Ad hoc evidence that cumulative evidence increases awareness
Increasing communication between researchers and users
11
Networking – personal contacts between researcher and practitioner Barnardo’s, LRGRU, LGA all actively seek such involvement Long-term effects in fostering awareness
12
Project partnerships – between researchers and practitioners Examples from literature of action research, of partnership projects in medicine and teaching, and of joint promotion of research; JRD project advisory groups contain both researchers and practitioners Some evidence that research is more practice-relevant, with consequent greater practitioner engagement
13
Ongoing partnerships – between research and practice organisations TTA-funded school-university consortia; JRF-funded social care network Some evidence of impact on attitudes and behaviour; time and commitment are necessary conditions
14
Co-location of researchers and practitioners Barnardo’s devolution of researchers to regions; LRGRU’s location within policy directorate Anecdotal evidence of promotion of mutual understanding and responsiveness
Support for developing research-informed practice
15
Information and inquiry services – on research by practitioners Literature reports examples in education and social care; most case study organisations have research websites Commonly judged useful in supporting evidence-informed practice; but effectiveness depends on promoter’s enthusiasm and credibility within an organisation
16
IT support systems Barnardo’s intranet connects dispersed researchers and practitioners IT systems are effective in supporting change in healthcare practice
17
Facilitation of research impact actions – through training, funding, support Examples from UK and US literature of such programmes in medicine, criminal justice and social care Programmes need good resources and leadership, but changes in conceptual and instrumental impact have been achieved
18
Office systems – other than IT Healthcare experience of specifying care pathways and using checklists Office tools and teamwork can improve practice
19
Research champions – recruiting opinion leaders to this role Role of opinion leaders in Promoting Action in Clinical Excellence (PACE) programme Some evidence of influence of colleagues on research awareness and practice change
20
Organisational initiatives – fostering evidence-informed practice Healthcare experience of using quality improvement initiatives to increase the flow of evidence within organisations Some evidence of success in changing attitudes and behaviours; key factors are leadership, clear goals, supportive infrastructure and integration with practice
Rewarding and reinforcing research-informed practice
21
Incentives – for researchers’ dissemination Financial or status rewards for GPs for adoption of particular practices Successful in increasing dissemination
22
Incentives – for practitioners’ uptake Financial rewards for GPs Mixed results
23
Reminders and prompts – oral, print or electronic Use of computerised prompts in primary healthcare practice Effective – reinforcement of messages influences behaviour
24
Audit and feedback – of practice Use of audit and feedback regimes to change prescribing behaviour of GPs Mixed results in changing practice; feedback most effective with peer comparison
25
Targeting service users – to provide leverage on practitioners Providing research evidence on good practice to healthcare patients Successful in changing clinical practice
Staff development, education and training  
26
Outreach – researchers visiting practitioners Found in healthcare field Only modest effects in changing practice as a single intervention
27
Training and staff development – of both researchers and practitioners US literature on training for research-based practice in medicine and social care; Barnardo’s trains practitioners in research appraisal Mixed success in achieving practice change; but ongoing support and favourable organisational culture seems crucial

 

STRATEGIES AND ACTIONS FOR THE LEARNING AND SKILLS SECTOR (Return to top)

Sector Barriers and Enablers | A Model of Research Impact | A Typology of Practices

Sector Barriers and Enablers (Return to top)

Particular to the learning and skills sector:

Culture - organisational diversity; affects of influential individuals in the organisation; predominantly instrumental (research valued where immediate practical impact); low level of professionalisation may inhibit research awareness.
Salience – political agenda = targets, performance and reform agenda; greater salience where research relates directly to this agenda.
Competing evidence – multiple sources of research material, therefore leads to reliance on ‘trusted’ and ‘authoritative’ sources or wait for greater support for change. Systematic reviews can help.
Communication – no single channel for communication; diverse and fragmented sector not well connected.
Resources and skills – skills and resources for research awareness and use is patchy; practitioners lack: time to keep up to date, access to research outputs, and skills to interpret and apply research findings.

A Model of Research Impact (Return to top)

Main models are:

Self-motivated learning – where the individual/organisation is an active consumer of new knowledge;
Incentives and rewards - where behaviour change is encouraged and conditioned by reinforcement;
Rational choice – where individuals/organisations balance the pros and cons of using particular research findings in their work;
Social influence – whereby peer opinion shapes attitudes and behaviours towards research and its use;
Facilitation – where senior management in an organisation enables research use by ensuring that organisational support systems are in place; and
Knowledge management – where information systems facilitate the flow of information and the accumulation of knowledge.

A Typology of Practices (Return to top)

Four categories of practice:

Category 1 (interactions)

Most commonly argued practices for impact = improve interactions (e.g. 11 of the 27 practices (listed in Table 1) fall into this category).
Potentially effective practices in learning and skills sector:
o use of mass media
o presentations
o lobbying
o tailored material
o networking
o project partnerships
o ongoing partnerships
o reminders and prompts

Less effective practices in learning and skills sector (due to diverse and hard-to-teach targets):
o print publications
o targeted service users
o outreach

Varieties of interaction:

 

One-way Two-way
Direct Lobbying
Tailored materials
Reminders and prompts
Workshops/seminars
Project partnerships
Outreach
Indirect Mass media
Print publications
Targeting service users
Networking
Ongoing partnerships

Category 2 (research production)

Potentially effective practices:
o Research planning
o Research programmes
o Training and staff development

Less effective practices:
o Standards for research
o Incentives – for dissemination

Category 3 (research use)

Potentially effective practices:
o Practice guidelines
o Demonstration projects
o Research champions
o Training and staff development

Less effective practices:
o Incentives – for research uptake
o Audit and feedback

Category 4 (organisational contexts)

Potentially effective practices:
o Information and inquiry services
o Facilitation of research impact action
o Organisational initiatives

Less effective practices:
o Co-location of researchers and practitioners
o IT support systems
o Office systems

(Return to top)

Chris Taylor
May 2003

       
 
Home > Activities > Impact and dissemination of teaching and learning research > Summary of Models of Research Impact
   
Return to top of page
This page was last updated 18th February 2004
Email: RCBN@Cardiff.ac.uk