Resources from RCBN and Journal: research designs
Chris is a senior lecturer at the School of Social Sciences at the University of Cardff and co-director of the NCRM node Qualiti in Cardiff
This guide, draws heavily on resources developed by the Research Capacity Building Network (RCBN) and its journal, Building Research Capacity, between 2000 and 2005, supplemented with other material.
In this section a number of important aspects of research design are considered. Many of them are essential to the successful completion of a research study. This section also contains access to an extensive methodological database of references.
Methodological References Database
The TLRP RCBN compiled a database of over 200 methodological texts in EndNote. These references include journal articles and book chapters. The database was intended to be as comprehensive as possible and, as such, made no attempt to review the texts listed. Therefore inclusion of an item in the database is in no way indicative of recommendation by the RCBN.
Each entry has a full reference and is coded to 35 key terms under the Label heading. This means that the database can be searched by one or a combination of key terms (such as Ethics, Generalisation or Systematic Review). And since the database is an EndNote database this can be appended to other reference lists you may use. EndNote can also be linked with any Word document in order to Cite While You Write (CWYW), and produces a fast and simple way of producing a bibliography in any Word document. You can download a trial version of EndNote from the official website: http://www.endnote.com/endemo.asp
The formulation and development of research questions is an essential stage of the research process. This 10-page guide by Anne Edwards, produced for the TLRP RCBN, first discusses how our assumptions about education and social science research shape the research questions before she guides the reader through the process of determining the aims and objectives of a research study. Edwards then attempts to link different kinds of research questions to the different kinds of research designs that one may wish to pursue:
Moving On: Linking Research Questions to Research Design (Anne Edwards 2003)
The TLRP RCBN also collated a number of references that discuss the formulation, development and refinement of research questions:
Qualitative Methods in Education Research
Amanda Coffey considers the role of qualitative work within educational research and addresses the capacity of such work to respond to contemporary developments and critical debates. Coffey argues that many critiques of qualitative research ignore the distinctive contribution that such work has contributed to understanding teaching and learning. Utilising Lincoln and Denzin's (2000) historical framework of qualitative methods Coffey presents the progress that has been made in the use of qualitative methods in educational research.
Coffey, A. (2002) Reflections on the history of qualitative research in education, Building Research Capacity , 2, 5-8
Here are a number of recommended references from colleagues within the education research community on qualitative research methods: http://www.tlrp.org/rcbn/capacity/References/Qualitative.html
The Role of Numbers in Research
Social science is facing increasing demands for research involving 'quantitative' approaches. Among these are the need, expressed by policy-makers, for practical evidence about what works, and the demand, exemplified by the new ESRC guidelines for research training, that all researchers learn something about techniques of analysis involving numbers. This paper by Stephen Gorard, Ken Prandy and Karen Roberts (Cardiff University) shows how a re- consideration of the nature and function of probabilities (or uncertainties) in research suggests a new approach to research training that will be more appropriate than traditional courses on statistics for all learners, and that will help researchers explain their findings to policy-makers.
Gorard, S., Prandy, K. & Roberts K. (2002) An Introduction to the simple role of numbers in social science research, Cardiff University School of Social Sciences Occasional Paper 53
The following article by Stephen Gorard (University of Birmingham) attempts to show how a change in the way we think about, teach and represent probabilities will help us use percentages and numerical findings more appropriately.
Gorard, S, (2003) Anyone can calculate conditional probabilities, Building Research Capacity , 5, 9-11
Ken Prandy suggests in his short article that we should be more sceptical about our use of numbers in social science and education research. Drawing upon the principles of ‘measurement', Prandy argues that quantitative research must involve theory or qualitative observation in order to avoid what he calls ‘pseudo-quantification'.
Prandy, K. (2002) Measuring quantities: the qualitative foundation of quantity, Building Research Capacity , 2, 3-4
A further useful reference in considering the role of numbers in education research:
Gorard, S. (2001) Quantitative Methods in Educational Research: The role of numbers made easy, London : Continuum
Here are some further recommended references by colleagues from within the education research community on various quantitative approaches or methods:
One of the key dimensions in all social science research is time – in how things change and how time impacts upon social processes. Time has a significant role to play in all education research, both as a framework from which to study phenomena or educational processes and as a factor in helping us to understand these phenomena and processes. Longitudinal research does not exclusively employ quantitative methods (see the recent ESRC-funded research programme, Changing Lives and Times: A Qualitative Longitudinal Network (“Timescapes”) but much longitudinal data, through the wide availability of large-scale, national and repeated surveys, is very suitable for quantitative analysis. Paul Croll (University of Reading) provides an introductory presentation of longitudinal research in education, drawing upon his own research using two surveys of special educational needs in English primary schools (Croll and Moses, 2003) and secondary analysis of the British Household Panel Survey and, in particular, the Young Person's Survey (Croll, 2003, 2004). Croll's presentation can be viewed in Adobe Acrobat:
Longitudinal Research: An Introduction (Paul Croll 2004)
A further detailed presentation by Paul Lambert and Vernon Gayle (both University of Stirling ) introduces the main quantitative approaches to longitudinal data analysis , primarily by focussing upon large-scale secondary datasets. It also outlines numerous datasets that are readily available for longitudinal data analysis and provides examples of their use in education research.
Quantitative Approaches to Longitudinal Research (Paul Lambert and Vernon Gayle 2004)
In Ray Crozier's ( University of East Anglia ) short article the complementarity between longitudinal datasets (qualitative and quantitative) is considered. In this Crozier provides examples of complementary longitudinal data and discusses the advantages and challenges of attempting to combine various such datasets, including the combination of quantitative and qualitative data . He concludes with a consideration of these issues explicitly in the context of education research.
Crozier, R. (2003) Complementarity between longitudinal studies, Building Research Capacity , 6, 7-10
Use of Large-Scale Secondary Datasets
Here are several examples of educational research that have used large-scale secondary datasets. The first by Paul Croll (University of Reading) provides an introductory presentation of longitudinal research in education, drawing upon his own research using two surveys of special educational needs in English primary schools (Croll and Moses, 2003) and secondary analysis of the British Household Panel Survey and, in particular, the Young Person's Survey (Croll, 2003, 2004). Croll's presentation can be viewed in Adobe Acrobat:
Longitudinal Research: An Introduction (Paul Croll 2004)
A further detailed presentation by Paul Lambert and Vernon Gayle (both University of Stirling ) introduces the main quantitative approaches to longitudinal data analysis, primarily by focussing upon large-scale secondary datasets. It also outlines numerous datasets that are readily available for longitudinal data analysis and provides examples of their use in education research.
Quantitative Approaches to Longitudinal Research (Paul Lambert and Vernon Gayle 2004)
The book by Stephen Gorard and Chris Taylor argues the case for combining multiple research methods and provide practical guidance for researchers with want to use this mixed methods approach. Central to this book is the view that all research has an overarching logic such that the combined use of quantitative and qualitative research methods is not only possible but also very fruitful. The book outlines and evaluates methods across and within studies, including complex interventions, Bayesian approaches, new political arithmetic, triangulation, life histories and design studies. It also offers a radical, new and very simple way of working with numbers.
Gorard, S. and Taylor, C. (2004) Combining Methods in Educational and Social Research, Berkshire: Open University Press
Roy Nash presents a realist approach to social explanation and argues that a ‘numbers and narratives’ methodology offers a way of combining data.
Nash, R (2002) A realist scheme for social explanation: on 'numbers and narratives', Building Research Capacity, 4, 1-4 http://www.tlrp.org/rcbn/capacity/Journal/issue4.pdf
Ken Prandy suggests in his short article that we should be more sceptical about our use of numbers in social science and education research. Drawing upon the principles of ‘measurement’ Prandy argues that quantitative research must involve theory or qualitative observation in order to avoid what he calls ‘pseudo-quantification’.
Prandy, K. (2002) Measuring quantities: the qualitative foundation of quantity, Building Research Capacity, 2, 3-4 http://www.tlrp.org/rcbn/capacity/Journal/issue2.pdf
In Ray Crozier’s (University of East Anglia) short article the complementarity between longitudinal datasets (qualitative and quantitative) is considered as an example of how one can combine qualitative and quantitative data. In this Crozier provides examples of complementary longitudinal data and discusses the advantages and challenges of attempting to combine various such datasets, including the combination of quantitative and qualitative data. He concludes with a consideration of these issues explicitly in the context of education research.
Crozier, R. (2003) Complementarity between longitudinal studies, Building Research Capacity, 6, 7-10 http://www.tlrp.org/rcbn/capacity/Journal/issue6.pdf
A further use of complementary approaches to combining qualitative and quantitative research is offered using the notion of triangulation. This brief paper by Stephen Gorard (University of Birmingham) and Chris Taylor (Cardiff University) considers the metaphor/analogy of triangulation in education research. It offers a very simple way of considering the potential benefits of triangulation in social science research and suggests that a complementary notion of triangulation is very useful when trying to combine qualitative and quantitative data.
Gorard, S. and Taylor, C. (2004) What is ‘triangulation’? Building Research Capacity, 7, 7-9
In the following article Jane Elliott (Institute of Education, University of London) presents her work on combining quantitative and qualitative approaches using children’s essays from the 1958 British birth cohort study, the National Child Development Study (NCDS). Elliott usefully highlights the existence of qualitative data in secondary datasets that are often considered to be solely quantitative in nature. The article also briefly summarises the quantitative and qualitative analysis of the children’s essays from the NCDS.
Elliott, J. (2007) Combining qualitative and quantitative approaches to research: a case study examining the analysis of children’s essays, Building Research Capacity, 12, 6-8
Professor Stephen Gorard (University of Birmingham) also considers, in the following article, the role of secondary data in combining quantitative and qualitative approaches to research.
Gorard, S. (2002) The role of secondary data in combining methodological approaches, Educational Review, 54, 3, 231-237
Randomised Controlled Trials in Education Research
In this short article Stephen Gorard (University of Birmingham) usefully sets out a rationale for the increased use of randomised controlled trials and its variants in educational research. In doing so it attempts to respond to the main concerns or barriers to their further use. Gorard concludes by introducing a new ESRC-funded project at the University of York to support the training and development of researchers in the use of trials.
Gorard, S. (2006) The concept of a ‘fair test is relevant to all researchers: introducing the Trials in Public Policy project, Building Research Capacity, 11, 6-9
Laurence Moore (Cardiff University) in this short article responds to the key challenges in applying randomised controlled trials to the evaluation of complex interventions, such as educational interventions, and how these can be overcome.
Moore, L. (2002) Research design for the rigorous evaluation of complex educational interventions: lessons from health services research, Building Research Capacity, 1, 4-5
Design experiments (or ‘design studies’, ‘design experiments’ or ‘teaching experiments’) are relatively new but growing approach to conducting educational research. This approach is largely based on research in the field of engineering, and is loosely based on a hybrid cycle of prototyping, classroom field-testing, and laboratory study. One of the difficulties in developing these methods in educational research has been the rather limited literature or guidance on how to ‘do’ design experiments. A large study funded by the National Science Foundation in the US was established to investigate more systematically the parameters, capabilities and distinctiveness of design experiments in educational research. In this short article Eamonn Kelly and Richard Lesh outline the thinking behind this study and design experiments.
Kelly, E. and Lesh, R. (2002) Understanding and explicating the design experiment methodology, Building Research Capacity, 3, 1-3
Here are a couple of further useful references on the use of design experiments that have been recommended by colleagues within the education research community:
ICT in Education Research
There has been a dramatic growth in the use of ICT to undertake research of all kinds. One example of this has been the use of video technology in education research, and video ethnography in particular. This MS PowerPoint presentation by Sarah Pink (University of Loughborough), commissioned by the TLRP RCBN provides an overview of the role and use of video in ethnographic research. It introduces a historical context of film-making in ethnography, ethical considerations in the use of video and video as part of a wider interest in visual research methods. There are external website links provided, that lead to relevant examples of research using video and other visual methods.
Video in research: introducing the video ethnography process (Sarah Pink 2004)
Other examples in the use of ICT in research is computer mediated communication (CMC) – communication of any form which takes place via computers, such as email, internet chat and bulletin boards, newsgroups, blogs and mobile phone text messaging. In this short article Neil Selwyn outlines the main potential advantages and disadvantages of using such ICT-based methods in educational research. In particular, Selwyn highlights the uneven population access to ICT communication and suggests that such electronic methodologies should perhaps only be considered a valid alternative to traditional techniques when targeting specific or narrowly defined populations with easy and unproblematic access to technology.
Selwyn, N. (202) Using computer-mediated communication in educational research, Building Research Capacity, 2, 8-10. http://www.tlrp.org/rcbn/capacity/Journal/issue2.pdf
In the next example, Fay Smith and Frank Hardman (both University of Newcastle) discuss the use of ICT in classroom observations. Based on their own study, the authors discuss the advantages and disadvantages of a Classroom Interaction System (CIS) using Observer software on handheld computer devices. They conclude that the advantages of CIS are ideal as a professional development tool with teachers.
Smith, F. and Hardman, F. (2003) Investigating classroom interaction using computerised observation, Building Research Capacity, 6, 5-7
Another important development in the use of ICT has been in the analysis of qualitative data through Computer Aided Qualitative Data Analysis Software (CAQDAS). Such technological and software developments are ongoing, but this short article usefully reviews a number of recent software packages in order to help researchers choose the most appropriate tool. The useful article first distinguishes between three main methods of analysis before discussing three sets of software packages: Atlas.ti; NUDIST and NVivo; and Hypertext.
Williams, M., Mason, B. and Renold, E. (2004) Using computers in qualitative research: a review of software packages, Building Research Capacity, 7, 4-7
Writing ‘Natural Histories' of Research
Often it is useful to formalise one's experiences of the research process in order to develop your own research expertise and that of others. Professor Martyn Hammersley (The Open University) suggests that one way of doing this is to write ‘natural histories' of research. This may be useful to experienced researchers who wish to write about their approach(es) to research. And in doing this experienced researchers will be able to contribute to the development of new career researchers. Professor Hammersley produced a guide to writing ‘natural histories' for the TLRP RCBN. This includes a 5-page introduction and outline to writing natural histories followed by a fairly exhaustive reference list and collection of research biographies, including many from the field of education research.
Guide to Natural Histories of Research (Martyn Hammersley 2003)
|How to reference this page:
||Taylor, C. (2007) Resources from the Research Capacity Building Network and Journal. London: TLRP. Online at http://www.tlrp.org/capacity/rm/wt/taylor (accessed