Educational Research and policy: epistemological perspectives
By John Elliott and Dominik Lukeš, University of East Anglia
[Note: the full version of the paper which is summarised and referred to here will be published as a special issue of the Journal of Philosophy of Education and subsequently as a book edited by David Bridges, Paul Smeyers and Richard Smith and published by Blackwell in 2008]
Summary: This paper examines the ethnographic case study in education in the context of policy making with particular emphasis on the practice of research and policy making. The central claim of the paper is that it is impossible to establish a transcendental epistemology of the case study on instrumental rationality. Instead it argues for the notion of situated judgment that needs to be made by practitioners in context, practitioners being both researchers and policy makers. In other words, questions about the level of confidence or warrant that can be placed in different sorts of research evidence and findings cannot be answered independently of forming a view about the appropriateness of the policy culture that shapes political decision-making.
The paper draws a distinction between the general which is internal to the data as construed by a particular discipline and the universal which is the result of embedded human deliberation. This applies to all research findings and not only to case study, although since case study has had to long defend itself against accusations of the lack of generality, it can be a useful starting point for the discussion. This paper is not meant to be yet another defence of the case study research genre, although a summary of other defences is offered. Rather it focuses on how the case of the case study points to the limits of epistemology as rationality and offers the view of epistemology as ethics.
Outline of argument: We start out by delimiting the scope of case study. We see educational case study as a form of inquiry into a particular instance of a general class of things that can be given sufficiently detailed attention to illuminate its educationally significant features. Although we acknowledge that case study and ethnography in education are frequently coterminous we go on to point out additional perspectives. The ethnographic approach is built around the metaphor of the understanding of cultures one is not familiar with (represented by the work of scholars like Becker, Willis, Lacey, Wallace, Ball, and many others). In opposition to this, the tradition of responsive and democratic evaluation and portrayal aims to break down the barriers between the researcher, the researched and the audience. It recognizes the situated nature of all actors in the process and is particularly relevant to the concerns of the policy maker. This approach is represented by the work of represented by the work of scholars such as MacDonald, Simons, Stake, Parlett, and others. A third strand is represented by the work of Lawrence Stenhouse and his contemporary history approach. Stenhouse provides a perspective that allows the researcher to marry the responsibility to data with the responsibility to his or her environment in a way that escapes the stereotypes associated with the ‘pure forms' of both quantitative and qualitative research. A comprehensive historian-like approach to data collection, retention and dissemination that allows multiple interpretations by multiple actors accounts both for the complexity of the data and the situation in the context of which it is collected and interpreted.
Together, and through their creative differences, these individual scholars and schools of thought have set out a qualitative research agenda (in work ranging roughly but not exclusively from the late 1960s to the mid-1980s) that is singularly coherent in its insistence on the relevance of case-study-based approach to research results. They have asked and answered almost every conceivable question about the reliability, replicability, generalizability and applicability of qualitative research data, and their answers offer a clear vision of the case study researcher's position in the context of pure method, data, the researched and the clients (consumers or audience) of research. Their answers reveal an irreducible complexity of this interaction that cannot be represented by a situation-independent method. We briefly outline some of the arguments for and against the single case study and put forward the conclusion that this type of qualitative data can provide just a good a basis for the kinds of generalization a policy maker would make. However, we believe, this is not where the discussion should stop.
First, we need to question the very notion of generalization. Generalization is a natural cognitive act so there is no question that generalizations are made based on any kind of research inasmuch as that research is a part of cognitive experience. Quantitative generalization can offer particular guidelines for qualitative research based on statistical distributions. This was described by Kennedy as early as 1979 and others before her. However, as a result generalizations become properties of the data with respect to larger populations and the quality judgements that can be made about these generalizations remain firmly in the domain of a particular discipline or method. In order to be made relevant for policy making, we need to seek universal conclusions in the data. Following Nussbaum, we take the universal judgements to be such that discern the commonality of different cases without ignoring their particularity. Such judgements can only be made by individuals in the context of their practice.
The central problem, we argue, lies in the insistence on method. We show that it is dangerous to simply look at the surface features of a research genre such as case study and expect it to provide reliable information in its sphere of influence (such as thick description). Just because case study looks at the situated condition of the individual does not mean that its utility for the policy maker lies in the small scale and the personal. Conversely, neither does the broad reach of large scale population studies mean that they generate automatically valid generalizations. Having established the validity and value of the case study as a research instrument, we now need to question the very reliance of the practitioner (including policy makers) on any instrument. Furthermore we need to raise ask whether it is appropriate for a philosopher to assume the role of an external expert, an arbiter of method and the resultant data.
It is the practitioner's situated judgement rather than the philosopher's instrumental rationality that needs to be the final judge of the reliability of research method and data. This judgement is ultimately an ethical responsibility of each agent in a role where such a decision needs to be made and cannot be left to transcendental epistemology. Inasmuch as ethics is a continual struggle with the question of “how do I act in ways that are consonant with my values and goals?”, all epistemology in the context of action (such as policy making) needs to be concerned with ethics just as much as with cognition.