Educational research commissioned by/for policy audiences
Senior Policy Adviser for Research, General Teaching Council for England and Visiting Professor, Institute of Education, London
The ideas and material for this resource are embedded in the very practical and specific context of my day-job at the General Teaching Council for England . The Council was established by the Teaching and Higher Education Act of 1998 and began work in September 2000. Amongst its several duties, the Council has a statutory responsibility to give advice to the Secretary of State for Education on matters concerned with teachers and teaching, and our commitment is that such advice, all our policy work indeed, should be firmly grounded in research and evidence.
My theme – the relationship between research and policy in education – is not new, although I hope I have brought one or two new insights to it from my seven years' work at the GTC and, prior to that, from my short-term secondment to the World Bank education headquarters and my thirteen years' research efforts, mainly on policy-related projects, at the National Foundation for Educational Research. I am more than ever convinced of the importance of strengthening the relationship between research and policy: though I do not underestimate the difficulty, despite unprecedented national interest and investment of resource, of doing so. I believe policy-makers and researchers alike cannot afford to be naïve, cynical or complacent about either the challenges or the possibilities…
In putting these materials together, I have of course drawn on the work of many colleagues, both in universities and in the GTC and other national policy bodies; I am most especially indebted to those brave souls who continue to sow the borderlands between research and policy.
Aims of this resource
to provide an overview of how research is, and could be, used to support policy in education by:
- opening up the process of policy-making to researchers
- assisting researchers to understand the issues, challenges, opportunities and significance of policy-related research
- giving insights how research proposals may be judged
Decision-making in policy: what’s involved?
policy-makers by and large need to:
- grasp a complex remit quickly, and make decisions/take actions within very short time-frames
- interpret and adapt available evidence and/or use own experience/judgement, especially if research evidence is unavailable or contradictory
- be intellectually agile in an unpredictable environment
- create consensus among a wide range of vested interests
- actively shape the intellectual and political environments
- manage coded discussions and difficult negotiations with ‘stakeholder’ groups
- be guided by factors other than evidence, for example, particular principles and values; the human and financial resources available; the democratic process
- know what will count as success and exert leadership to achieve it
Who makes/influences policy decisions?
policy decisions are made by a range of people and influenced by many more, for example:
- government ministers, secretaries of state, etc.
- back-bench and opposition MPs
- civil servants in government departments and their equivalents in non-departmental public bodies (NDPBs)
- elected members and officers in local authorities
- regulatory bodies
- professional bodies
- campaigning organisations
- media and the public, in creating/changing a general climate
Research and policy: the ‘rationalist ideal’
It takes an extraordinary concatenation of circumstances for research to influence policy directly…. [rather] research helps people reconsider issues, it helps them think differently, it helps them re-conceptualise what the problem is and how prevalent it is, it helps them discard some old assumptions, it punctures old myths.
(Weiss 1991 – my emphasis)
Why do policy-related research?
- to exert influence on policy environment and/or on particular initiatives (and by extension on practice)
- to make a contribution to current policy debates (academics as ‘public intellectuals’?)
- to generate income
- to produce publications and other visible outputs/outcomes
Some pressures for research in a policy environment
- the relentless focus on ‘deliverables’
- the need for rapid, responsive, just-in-time, ‘good enough’ knowledge
- the replacement of traditional knowledge creation and dissemination by global/local networks, with attendant risks of ‘info-nuggets’ and ‘evidence-lite’ (Galvin 2004)
- adoption/adaptation of business and corporate research techniques like scenario-building, focus groups, futurising
- academics sometimes being seen as part of problem in education policy making – though they are not alone in this: the relationship between policy and the social sciences more generally is often construed as problematic compared with the natural and engineering sciences (see Commission on the Social Sciences 2003)
- a lack of understanding on both sides about the complex nature of research ‘impact’, use and influence (the Centre for the Use of Research and Evidence in Education website may be a helpful resource: http://www.curee-paccts.com/index.jsp)
Some risks when policy-makers look for ‘evidence’
- they may rely on known and trusted researchers
- due at least in part to time constraints and the difficulty of accessing academic research, they may be unduly swayed by well-known or highly visible research
- they may use in-house or own-commissioned research in preference to externally-generated research
- their contracting procedures for commissioning research may be unduly constrained by procurement rules which can limit the salience of research (see Social Research Association 2002)
- they may draw largely on their own judgement and personal networks and make little use of research digests, e-mail alerts, etc.
- they may be unaware of existing research and its implications, particularly grant-funded work whose outcomes reside mainly in academic journals
- they may use research findings selectively
- they may want research to provide confirmation of policies rather than an investigation of them
- they may lack the requisite skills for research appraisal and interpretation (but see the civil service initiative ‘Professional Skills for Government’, at: http://www.civilservice.gov.uk/archive/delivery_and_reform/publications/doc/briefingpsg_23may05.doc)
- they may delegate the responsibility for research management, interpretation and use to specialist officers who tend not to occupy senior positions in the organisational hierarchy
- there may be little or no ‘organisational memory’: key staff move on and take their knowledge and understanding of research with them
Implications for researchers
Researchers in education may wish to consider the following:
- what is the nature of the educational researcher’s responsibility to engage with decision-makers in policy circles, and how can it be enacted?
- what is the nature of the educational researcher’s responsibility to engage with the public, and how can it be enacted?
- how can university-based education researchers continue to improve their competitive edge over commercial or independent research organisations in their understanding of policy dilemmas, their conceptualisation of key issues, their critical understanding of the existing evidence and their research designs?
- how can university-based education researchers draw on other disciplines, substantively and methodologically, to add weight to the exploration of policy issues?
Research for or about policy?
Researchers in education may wish to consider the possibilities for intervening in:
- policy formation (‘upstream’)
- policy evaluation (‘downstream’)
- policy critique (‘on the bank’)
- policy influence through ‘big ideas’ (‘the wider environment’)
and to think about:
- which of these is most useful for policy environments and under what conditions
- what role is played, and what affordances are offered, by (i) grant-funded research (ii) directly commissioned research, respectively
- how the influence of grant-funded research on policy can be optimised (for example, by alerting selected policy-makers to the research before it starts, as well as to its outcomes; by setting up an advisory group with invited policy-makers as members; by producing policy-relevant briefings as well as academic papers; by creating interest and engagement amongst school leaders as influencers of policy; etc.)
What do policy clients look for in proposals for commissioned research?
This is not a definitive list but researchers may wish to think particularly about:
- the researchers’ understanding of the current policy environment, specific and general, and what its ‘direction of travel’ is likely to be
- the researchers’ appreciation of the particular policy challenges under consideration
- the intellectual rigour of the proposal: a clear rationale that traces the argument from the project aims, through the research issues (grounded in a critical understanding of existing research) and the research methods, to the proposed outcomes
- a willingness to challenge the brief and suggest alternative conceptualisations
- a track record of delivery to time and budget, plus a clear and comprehensive account of potential risks and their management
- value for money (which is not the same as lowest price)
BALL, S.J. (1990). Politics and Policy Making in Education . London : Routledge.
BERA. (2003). Educational Policy and Research across the UK . Report of a BERA Colloquium held at the University of Edinburgh , 7–8 November 2002. Nottingham : British Educational Research Association.
BURTON , P. (2006). ‘Modernising the policy process: making policy research more significant?', Policy Studies , 27 , 3, 173–95.
CAMPBELL , S., BENITA, S., COATES, E., DAVIES, P. and PENN, G. (2007). Analysis for policy: evidence-based policy in practice . London : Government Social Research Unit http://www.gsr.gov.uk/downloads/resources/pu256_160407.pdf
COMMISSION ON THE SOCIAL SCIENCES. (2003). Great Expectations: the Social Sciences in Britain . Downloadable from: http://www.the-academy.org.uk/
DAVIES, P. (2004). ‘Is evidence-based government possible?' The 2004 Jerry Lee Lecture, Campbell Collaboration Colloquium, Washington DC , 19 February.
DAVIES, H.T.O, NUTLEY , S.M. and SMITH, P.C. (2000). (Eds) What Works? Evidence-based Policy and Practice in Public Services . Bristol : The Policy Press.
DEPARTMENT OF EDUCATION, TRAINING AND YOUTH AFFAIRS (DETYA) (2000). The Impact of Educational Research . Canberra : Commonwealth of Australia . Downloaded June 2002 from: http://www.detya.gov.au/highered/respubs/impact/overview.htm
DUNCAN, S. and HARROP, A. (2006). ‘A user perspective on research quality', International Journal of Social Research Methodology , 9 , 2, 159–74.
FLYVBJERG, B. (2001). Making Social Science Matter. Why Social Inquiry Fails And How It Can Succeed Again . Cambridge : Cambridge University Press.
FURLONG, J. and OANCEA, A. (2005). Assessing Quality in Applied and Practice-based Educational Research: a Framework for Discussion (ESRC Report RES-618-25-6001). Oxford : Oxford University Department of Educational Studies.
GALVIN, C. (2004). ‘The making of public policy in a digital age: some reflections on changing practice'. Paper presented at Market Research Society Conference ‘Social Policy Research in the Information Age', London , 17 February.
HANNEY, S. (2004). ‘Assessing the impact of research on policy: concepts, models and methods'. Paper presented at a seminar, Assessing Research Impact , St Andrews University , 15 January.
HUMES, W. and BRYCE, T. (2001). Scholarship, research and the evidential basis of policy development in education', British Journal of Educational Studies , 49 , 3, 329–52.
LEVIN, B. (2001). ‘Knowledge and action in education policy and politics', presentation at conference on empirical issues in Canadian education, Ottawa , November. Downloaded 24 July 2003 from: http://home.cc.umanitoba.ca/~levin/sweetmanconf_files/frame.htm
LEVIN, B. (2003a). ‘Increasing the impact and value of research in education', Educators' Notebook , 14 , 1.
LEVIN, B. (2003b). ‘Increasing the impact of research', presentation to University of South Australia , 3 March. Downloaded 24 July 2003 from: http://home.cc.umanitoba.ca/~levin/increasing_impact_ofresearch_files/frame.htm
LEVIN, B. (2003c). ‘Research, policy and the state in education', presentation to Deakin University , 28 March. Downloaded 24 July 2003 from: http://home.cc.umanitoba.ca/~levin/deakin%20mar03_files/frame.htm
LEVIN, B. (2004). ‘Making research matter more', Education Policy Analysis Archives , 12 , 56. Downloaded 12 December 2006 from: http://epaa.asu.edu/epaa/v12n56/
LEVITT, R. and SOLESBURY, W. (2005). ‘Evidence-informed policy: what difference do outsiders in Whitehall make?' ESRC UK Centre for Evidence Based Policy and Practice: Working Paper 23.
MUNN, P. (2005). ‘Researching policy and policy research', Scottish Educational Review , 37 , 1, 17–28.
ORGANISATION FOR ECONOMIC CO-OPERATION AND DEVELOPMENT, CENTRE FOR EDUCATIONAL RESEARCH AND INNOVATION. (2007). Draft outline for the forthcoming CERI publication ‘Research and Evidence in Educational Policy-making: New Challenges'. Downloadable from: http://www.oecd.org/document/31/0,2340,en_2649_35845581_37813919_1_1_1_1,00.html
NUTLEY , S. (2003). ‘Bridging the policy/research divide: reflections and lessons from the UK .' Keynote paper presented at ‘Facing the future: engaging stakeholders and citizens in developing public policy', National Institute of Governance Conference, Canberra , 23 April.
NUTLEY , S.M., PERCY-SMITH, J. and SOLESBURY, W. (2003). Models of Research Impact: a Cross-Sector Review of Literature and Practice . London : Learning and Skills Development Agency.
NUTLEY , S.M., WALTER, I. and DAVIES, H. (2007). Using evidence: how research can inform public services . Bristol : Policy Press
NUTLEY , S.M. and WEBB, J. (2000). ‘Evidence and the policy process' . In: DAVIES, H.T.O., NUTLEY , S.M. and SMITH, P.C. (Eds) What works? Evidence-based policy and practice in public services . Bristol : The Policy Press.
PAWSON, R . (2006). Evidence-Based Policy: A Realist Perspective . London : Sage.
RICKINSON, M. (2007). Research-based Knowledge Creation for Evidence-based Policy and Practice in Education and Training. Working Paper for the European Commission.
SAUNDERS, L. (2004). Grounding the Democratic Imagination: Developing the Relationship between Research and Policy in Education. Professorial Lecture. London : Institute of Education .
SOCIAL RESEARCH ASSOCIATION. (2002). Commissioning Social Research: A Good Practice Guide . London : SRA. Downloadable at http://www.the-sra.org.uk/commissioning.pdf
START, D. and HOVLAND, I. (2004). Tools for Policy Impact: A Handbook for Researchers . Overseas Development Institute. Downloadable from: http://www.odi.org.uk/RAPID/Lessons/Index.html
STILGOE, J., IRWIN, A. and JONES, K. (2006). The Received Wisdom: Opening Up Expert Advice . London : Demos. Downloadable from: http://www.demos.co.uk/files/receivedwisdom.pdf
SUNDQUIST, J.L. (1978). ‘Research brokerage: the weak link'. In: LYNN , L.E. (Ed) Knowledge and Policy: The Uncertain Connection . Washington DC : National Academy of Sciences.
TEMPLE , P. (2003). ‘Educational research and policymaking: findings from some transitional countries', London Review of Education , 1 , 3, 217–28.
WALTER, I. , NUTLEY , S. and DAVIES, H. (2003). ‘Developing a taxonomy of interventions used to increase the impact of research'. Unpublished paper.
WEISS, C.H. (1979). ‘The many meanings of research utilisation', Public Administration Review , 39 , 5, 426–31.
WEISS, C.H. (1991). ‘Policy research as advocacy', Knowledge and Policy: The International Journal of Knowledge TRansfer , 4 , 1 & 2, 37–55.
WEISS, C.H. (1991). ‘Policy research: data, ideas or arguments?' In: WAGNER, P., WEISS, C.H., WITTROCK, B. and WOLLMAN, H. (Eds) Social Sciences and Modern States : National Experiences and Theoretical Crossroads . Cambridge : Cambridge University Press.
WHITTY, G. (2002). Making Sense of Education Policy . London : Paul Chapman.
WILLINSKY, J. (2000). If Only We Knew: Increasing the Public Value of Social-Science Research . London : Routledge.
WILLINSKY, J. (2003). ‘Policymakers' online use of academic research', Education Policy Analysis Archives, 11, 2. Downloaded 20 February 2003 from: http://epaa.asu.edu/epaa/v11n2/
|How to reference this page:
||Saunders, L. (2007) Educational research commissioned by/for policy audiences. London: TLRP. Online at http://www.tlrp.org/capacity/rm/wt/saunders1.html (accessed