Value Added Analysis of Pupil and School performance

Sally Thomas
Sally is a Reader in Education at the University of Bristol.

Aims of this resource

Value added analysis provides one quantitative method of estimating pupil and school performance that can be used for a variety of purposes, for example, learner, practitioner and school self evaluation, programme evaluation and knowledge-base research.

The value added concept is based on the assumption that teachers and schools add ‘value' to the achievement of their students. It is based on the idea of measuring student progress, in academic outcomes such as reading or mathematics attainment over a given period of time. However, the concept can also be applied to other outcomes such as attitudes, well-being or vocational qualifications. To measure student progress baseline and outcome measures are required at the start and end of a particular time period (for example covering all or part of the primary or secondary phases of education or over the time period when a new educational programme or innovation is implemented). Of course, as students grow older one would expect progress or improvement to be made and average attainment levels to rise. What is of interest is the extra value that is added by schools and teaching to student attainment over and above the progress or improvement that might be expected on average. Value added measures thus seek to establish whether students in some schools or classes make relatively greater or less progress than those in other schools or classes over a specified period of time. The most effective of schools and classes are estimated to be those in which student progress exceeds expectations. For individual students, it may also be of interest to estimate whether their performance is above or below what would be expected, although the interpretation of a single performance measure must of course be approached with caution, and the consequences for any individual must be seriously considered.

Of course in the high stakes context of external school evaluation and accountability, value added measures provide only one source of comparative information about a school's effectiveness. It is important to remember that the value of schools' educational quality is broader than what can be measured by attainment in a few specific areas of student activity. A comprehensive value added evaluation framework might also encompass measures related to numerous other aspects of a school's mission, processes and outcomes. Moreover value added measures of school effectiveness have real limitations which need to be understood by any users and are explored in more detail in the references below.

Uses Of Value Added measures

Research - overall trends in pupil and school performance, differential school effects, regional/international comparisons, case studies of more/less effective schools, linking effectiveness measures to improvement. For example, see Thomas, SM, Peng, W-J, Gray, J. (2007) Value added trends in English secondary school performance over ten years, Oxford Review of Education , 33 (3):261-295 http://www.eric.ed.gov/ERICWebPortal/custom/portlets/recordDetails/detailmini.jsp?_nfpb=true
&_&ERICExtSearch_SearchValue_0=EJ765898&ERICExtSearch_SearchType_0=eric_accno&accno=EJ765898

Programme Evaluation - evaluating the impact of a new reading or maths scheme across schools and/or classes. For example, see Pituch, KA & Miller, JW (1999) Strengthening Multisite Educational Interventions: An Illustration with Multilevel Modeling. Educational Research and Evaluation. 5(1):62-75.

http://www.eric.ed.gov/ERICWebPortal/custom/portlets/recordDetails/detailmini.jsp?_nfpb=true
&_&ERICExtSearch_SearchValue_0=EJ590599&ERICExtSearch_SearchType_0=eric_accno&accno=EJ590599

School Self-Evaluation - providing teachers with information that will allow them to reflect on, evaluate and improve their educational practice. For examples see introductory workshop materials for educational policy makers and researchers at http://www.cmm.bristol.ac.uk/MLwiN/tech-support/workshops/materials/educ-policy-makers.ppt and at http://www.cmm.bristol.ac.uk/MLwiN/tech-support/workshops/materials/lancashire-eg.ppt

External Evaluation - providing school inspectors with information about relative school performance that can be used alongside their own judgements and other evidence of quality and standards. DCSF paper describing developments in Value Added measures can be found at

http://www.dfes.gov.uk/research/data/uploadfiles/RW85.pdf

Accountability - publication of value added school league tables and parental choice. DCSF resources describing value added measures in league tables can be found at

http://www.dcsf.gov.uk/performancetables/schools_05/sec4.shtml and at

http://www.standards.dfes.gov.uk/performance/1316367/CVAinPAT2005/

Calculating Value Added measures using Multilevel Modeling

The statistical method used to calculate value added measures at different (hierarchical) levels of the education system (eg region, school, class, student) is multilevel modeling, which is widely regarded as the most appropriate and flexible technique for analysing educational data. This is because students are almost always clustered in groups such as schools and classes, at least for their formal schooling, and therefore have common educational experiences with their peer group(s). In other words measures of student performance within a group, class or school cannot be assumed to be independent and this aspect is explicitly taken into account in multilevel modeling analysis. Several web resources provide guidance in creating and interpreting value added measures using multilevel modeling:

Worksheet and dataset that introduces a simple 2 level model using educational data to examine value added measures of school effectiveness can be found at http://www.cmm.bristol.ac.uk/research/Lemma/index.shtml

Video describing why multilevel modelling analysis is appropriate to measure school effects can be found at http://www.cmm.bristol.ac.uk/learning-training/videos/index.shtml

Description of School Effectiveness Research using Multilevel models can be found at

http://www.cmm.bristol.ac.uk/research/Lemma/sch-effec.shtml

Other Web Resources – Selected Value Added Research and References

Saunders, L (1999) Value added measurement of school effectiveness: a critical review. NFER: Slough .

http://www.nfer.ac.uk/publications/aries-data/value-add-measurement-of-school-effectiveness-a-critical-review.cfm

Scheerens, J, Glas, C & Thomas, S (2003) Educational Evaluation, Assessment and Monitoring: A Systemic Approach. Swets & Zeitlinger: Lisse, Abingdon, Exton (PA). http://www.informaworld.com/smpp/title~content=t735008102

Thomas, S (2001) Dimensions of Secondary School Effectiveness: Comparative Analyses Across Regions , School Effectiveness & School Improvement Journal . 12(3): 285-322. http://www.eric.ed.gov/ERICWebPortal/custom/portlets/recordDetails/detailmini.jsp?_nfpb=true
&_&ERICExtSearch_SearchValue_0=EJ634729&ERICExtSearch_SearchType_0=eric_accno&accno=EJ634729

Thomas, S & Mortimore, P (1996) Comparison of Value Added Models for Secondary School Effectiveness, Research Papers in Education , 11, (1): 5-33. http://www.eric.ed.gov/ERICWebPortal/custom/portlets/recordDetails/detailmini.jsp?_nfpb=true
&_&ERICExtSearch_SearchValue_0=EJ526798&ERICExtSearch_SearchType_0=eric_accno&accno=EJ526798

TLRP Resources for Research in Education by Teaching and Learning Research Programme is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 2.0 UK: England & Wales License