IT-assisted research

SUMMARY OF 'MODELS OF RESEARCH IMPACT: A CROSS-SECTOR REVIEW OF LITERATURE AND PRACTICE' (LSDA)

Models of research impact: a cross-sector view of literature and practice
(Building effective research: 4th report in the series)
Learning and Skills Research Centre (2003)
Authors: Sandra Nutley, Janie Percy-Smith and William Solesbury
ISBN 1 85338 857 2

This major national study sets out practical steps to enhance the impact of research on practice and policy. It analyses evidence from both research literature and actual practice across education, social care, criminal justice and healthcare sectors, and offers recommendations to the learning and skills community.

The summary is divided into two parts: (click on either to navigate the summary)

Summary of Findings
Strategies and Actions for the Learning and Skills Sector

To see the complete report in pdf format please click here.

The RCBN have also established an online discussion forum to discuss these models of research impact further. To join the discussion please click here.

SUMMARY OF FINDINGS (Return to top)

Aims (Return to top)

• To review the literature on research impact;
• To assess practices in comparable organisations;
• To characterise models, strategies and actions that would be useful in the learning and skills sector; and
• To develop guidance for the designers and managers of LSRC research.

Literature Review (Return to top)

5,800 references initially identified
750 examined in full
341 selected for detailed analysis
Of these:

155 = conceptual papers
2 = methodological
59 = provided background
125 = empirical papers (60% from healthcare)

[More information about the literature review can be found in Appendix 1 of the report, pp.33-42; Empirical references listed on pp.43- 49]

Distinction between two kinds of impact:
1. ‘Conceptual’ value of research (or the ‘enlightenment’) – changes of knowledge, understanding and belief.
2. ‘Instrumental’ value of research (or ‘engineering’) – directly changing behaviour in policy and practice.

Most literature considers impact as ‘instrumental’ or to change policy/practice behaviour.

Case Studies (Return to top)

Characteristics in selecting case studies (at least one or more present):
• Research users as well as research producers and/or research funders;
• Both a national and a regional or local remit;
• Established reputations for effectively relating research to policy or practice; and/or
• A span of diverse sectors and disciplines.

Five organisations chosen:
1. Barnardo’s
2. Joseph Rowntree Foundation (JRF)
3. Local and Regional Government Research Unit (LRGRU)
4. Local Government Association (LGA)
5. MORI Social Research Institute (MORI)

[More information about the case studies can be found in the Appendix 2, pp.51-62]

Practices for Research Impact (Return to top)

Each source analysed to:
• Identify the range of practices revealed in the source;
• Characterise each practice in terms of its underpinning model of research use;
• Note what is known about their effectiveness and the contextual barriers to, and enablers of, their use.

27 distinct practices recorded (see Table 1). These can be summarised under 6 headings:
1. Tailored presentation of research findings;
2. Tailoring research to users’ needs;
3. Increasing communication between researchers and users;
4. Support for developing research-informed practice;
5. Rewarding and reinforcing research-informed practice; and
6. Staff development, education and training.

Effectiveness (Return to top)

• Tangible impact = changes in policy, organisation, resourcing or delivery
• Intangible impact = changes in understanding, attitudes and behaviour (individuals’ and organisations’)

Measuring impact = few cases – where reported these are diverse: objective measures to self-reporting and observation by participants)

‘Effective’ practices:
• Active dissemination
• Closer links between researchers and practitioners
• Facilitation of research use
• Recruiting expert and peer opinion leaders

General characteristics of ‘effective’ practices: Multiple practices with typically long-term impacts

Barriers and Enablers (Return to top)

• Culture – inclination to engage with research and/or policy/practice.
• Salience – research perceived as helpful where it is relevant and timely.
• Competing evidence – research must always compete with other influences, therefore reliability and reputation are important; where research is contradictory then practitioners will need guidance on their interpretation.
• Communication channels – effective communication through appropriate media; communication of a range of factors, e.g. exigencies of practice, research craft, and results.
• Resources and skills – time and skills are necessary and therefore should be organisational priorities.

Table 1 Practices for Enhancing Research Impact (source: p.15-6) (Return to top)

	Practice	Examples	Effectiveness?
*Tailored presentation of research findings*
1	Practice guidelines – informed by research	Growing medical practice, sometimes developed through consensus processes; LGA produces good practice examples for use by local authorities	Single guideline has low impact in changing practice; needs reinforcement through reminders, incentives, peer endorsement
2	Use of mass media – through press notices and briefing	Common practice in case studies; JRF is particularly successful in securing coverage	Evidence from systematic reviews in healthcare that media coverage can promote behavioural change in practitioners and consumers
3	Print publications – reports, summaries, newsletters	Practised in MORI, JRG, LRGRU and LGA for external audiences; common practice found in literature review	Can increase awareness, but unlikely by itself to change behaviour
4	Workshop/seminar presentations	MORI is proactive in presentations at others’ or own events	Literature suggests oral presentations with opportunity for interaction impacts well on practitioners; source and timing are important
5	Lobbying – including through consultation responses	JRF sends letter and research summary to 10 most influential people in relevant practice field	Ad hoc evidence of effectiveness in raising awareness, but repetition becomes counterproductive
6	Tailored material – for target audiences	JRF is responsive to such opportunities, eg policy reviews	Can help to change attitudes; requires empathy and/or cooperation with practitioners to get it right
*Tailoring research to users’ needs*
7	Research planning – to improve relevance	Both MORI and LRGRU negotiate research plans with clients	Ad hoc evidence of effectiveness by encouraging early engagement by research units
8	Demonstration projects – to carry research into practice	Beacon schools, pilot or pathfinder projects, taskforces, practical projects	Ad hoc evidence of effectiveness; believed to add credibility to research and sharpen its practical application
9	Standards for research – to improve quality	Barnardo’s has standards for research linked to organisational quality commitments	Ad hoc evidence that it works we as part of a wider framework of change management
10	Research programmes – rather than small projects	JRF and LRGRU have moved to larger research programmes	Ad hoc evidence that cumulative evidence increases awareness
*Increasing communication between researchers and users*
11	Networking – personal contacts between researcher and practitioner	Barnardo’s, LRGRU, LGA all actively seek such involvement	Long-term effects in fostering awareness
12	Project partnerships – between researchers and practitioners	Examples from literature of action research, of partnership projects in medicine and teaching, and of joint promotion of research; JRD project advisory groups contain both researchers and practitioners	Some evidence that research is more practice-relevant, with consequent greater practitioner engagement
13	Ongoing partnerships – between research and practice organisations	TTA-funded school-university consortia; JRF-funded social care network	Some evidence of impact on attitudes and behaviour; time and commitment are necessary conditions
14	Co-location of researchers and practitioners	Barnardo’s devolution of researchers to regions; LRGRU’s location within policy directorate	Anecdotal evidence of promotion of mutual understanding and responsiveness
*Support for developing research-informed practice*
15	Information and inquiry services – on research by practitioners	Literature reports examples in education and social care; most case study organisations have research websites	Commonly judged useful in supporting evidence-informed practice; but effectiveness depends on promoter’s enthusiasm and credibility within an organisation
16	IT support systems	Barnardo’s intranet connects dispersed researchers and practitioners	IT systems are effective in supporting change in healthcare practice
17	Facilitation of research impact actions – through training, funding, support	Examples from UK and US literature of such programmes in medicine, criminal justice and social care	Programmes need good resources and leadership, but changes in conceptual and instrumental impact have been achieved
18	Office systems – other than IT	Healthcare experience of specifying care pathways and using checklists	Office tools and teamwork can improve practice
19	Research champions – recruiting opinion leaders to this role	Role of opinion leaders in Promoting Action in Clinical Excellence (PACE) programme	Some evidence of influence of colleagues on research awareness and practice change
20	Organisational initiatives – fostering evidence-informed practice	Healthcare experience of using quality improvement initiatives to increase the flow of evidence within organisations	Some evidence of success in changing attitudes and behaviours; key factors are leadership, clear goals, supportive infrastructure and integration with practice
*Rewarding and reinforcing research-informed practice*
21	Incentives – for researchers’ dissemination	Financial or status rewards for GPs for adoption of particular practices	Successful in increasing dissemination
22	Incentives – for practitioners’ uptake	Financial rewards for GPs	Mixed results
23	Reminders and prompts – oral, print or electronic	Use of computerised prompts in primary healthcare practice	Effective – reinforcement of messages influences behaviour
24	Audit and feedback – of practice	Use of audit and feedback regimes to change prescribing behaviour of GPs	Mixed results in changing practice; feedback most effective with peer comparison
25	Targeting service users – to provide leverage on practitioners	Providing research evidence on good practice to healthcare patients	Successful in changing clinical practice
*Staff development, education and training*
26	Outreach – researchers visiting practitioners	Found in healthcare field	Only modest effects in changing practice as a single intervention
27	Training and staff development – of both researchers and practitioners	US literature on training for research-based practice in medicine and social care; Barnardo’s trains practitioners in research appraisal	Mixed success in achieving practice change; but ongoing support and favourable organisational culture seems crucial

STRATEGIES AND ACTIONS FOR THE LEARNING AND SKILLS SECTOR (Return to top)

Sector Barriers and Enablers | A Model of Research Impact | A Typology of Practices

Sector Barriers and Enablers (Return to top)

Particular to the learning and skills sector:

•	Culture	- organisational diversity; affects of influential individuals in the organisation; predominantly instrumental (research valued where immediate practical impact); low level of professionalisation may inhibit research awareness.
•	Salience	– political agenda = targets, performance and reform agenda; greater salience where research relates directly to this agenda.
•	Competing evidence	– multiple sources of research material, therefore leads to reliance on ‘trusted’ and ‘authoritative’ sources or wait for greater support for change. Systematic reviews can help.
•	Communication	– no single channel for communication; diverse and fragmented sector not well connected.
•	Resources and skills	– skills and resources for research awareness and use is patchy; practitioners lack: time to keep up to date, access to research outputs, and skills to interpret and apply research findings.

A Model of Research Impact (Return to top)

Main models are:

•
Self-motivated learning – where the individual/organisation is an active consumer of new knowledge;

•
Incentives and rewards - where behaviour change is encouraged and conditioned by reinforcement;

•
Rational choice – where individuals/organisations balance the pros and cons of using particular research findings in their work;

•
Social influence – whereby peer opinion shapes attitudes and behaviours towards research and its use;

•
Facilitation – where senior management in an organisation enables research use by ensuring that organisational support systems are in place; and

•
Knowledge management – where information systems facilitate the flow of information and the accumulation of knowledge.

A Typology of Practices (Return to top)

Four categories of practice:

Category 1 (interactions)

Most commonly argued practices for impact = improve interactions (e.g. 11 of the 27 practices (listed in Table 1) fall into this category).
Potentially effective practices in learning and skills sector:
o use of mass media
o presentations
o lobbying
o tailored material
o networking
o project partnerships
o ongoing partnerships
o reminders and prompts

Less effective practices in learning and skills sector (due to diverse and hard-to-teach targets):
o print publications
o targeted service users
o outreach

Varieties of interaction:

One-way Two-way

Direct Lobbying
Tailored materials
Reminders and prompts
Workshops/seminars
Project partnerships
Outreach

Indirect Mass media
Print publications
Targeting service users
Networking
Ongoing partnerships

Category 2 (research production)

Potentially effective practices:
o Research planning
o Research programmes
o Training and staff development

Less effective practices:
o Standards for research
o Incentives – for dissemination

Category 3 (research use)

Potentially effective practices:
o Practice guidelines
o Demonstration projects
o Research champions
o Training and staff development

Less effective practices:
o Incentives – for research uptake
o Audit and feedback

Category 4 (organisational contexts)

Potentially effective practices:
o Information and inquiry services
o Facilitation of research impact action
o Organisational initiatives

Less effective practices:
o Co-location of researchers and practitioners
o IT support systems
o Office systems

(Return to top)

Chris Taylor
May 2003


	Home > Activities > Impact and dissemination of teaching and learning research > Summary of Models of Research Impact
	Return to top of page This page was last updated 18th February 2004 Email: RCBN@Cardiff.ac.uk