TLRP & ESRC  
  home  news  search  vre  contact  sitemap
 AIMS
 FINDINGS
 PROJECTS
 THEMES
 CAPACITY
 EVENTS
 PUBLICATIONS
 RESOURCES
USERS
 INTERNATIONAL
 MANAGEMENT
  
 Capacity building resources
   

Experimental Designs

Stephen Gorard

image

Stephen is ...


Contents  

1. Introduction
            1.1 We all need trials
            1.2 The basic idea
2. The value of trials
            2.1 Comparators
            2.2 Matching groups
            2.3 Causal models
3. Simple trial designs
            3.1 Sampling
            3.2 Design
4. Related issues
            4.1 Ethical considerations
            4.2 Warranting conclusions
            4.3 The full cycle of research
5. Resources for those conducting trials
            5.1 Reporting trials
            5.2 Tips for those in the field
6. Alternatives to trials
            6.1 Regression discontinuity
            6.2 Design experiments
7. Some examples of trials
            7.1 STAR
            7.2 High/Scope
            7.3 ICT
8. References

How to reference this page

7. Some examples of trials

By Carole Torgerson, University of York

7.1 Project STAR (Tennessee Class-Size Experiment)

Probably the most famous randomised controlled trial in education is Project STAR, the RCT designed as the first phase of the Tennessee Class-Size Experiment (see http://www.library.ca.gov/CRB/clssz/). This was a three-phase study designed to determine the relative effectiveness of small and regular sized classrooms on pupil academic performance in the early years of education (Mosteller 1995, Boyd-Zaharius 1999, Hedges 2000). The first phase, Project STAR (for Pupil-Teacher Achievement Ratio) was begun in 1985 comparing the performance of pupils aged between 5 and 8, taught in classes of 13-17 pupils with pupils of the same age taught in classes of 22-25 pupils (with or without a teaching assistant). The main thrust for the study was a recently completed systematic review reported by Glass and colleagues in 1982 which demonstrated (controversially, because of lack of heterogeneity of the included studies) that class sizes of 15 or fewer made a noticeable improvement in performance. In the first year of the Tennessee experiment about 6500 pupils in about 330 classes in 80 schools were randomly assigned to one of the three arms of the trial. After one year the effect sizes on performance in reading and maths in small classes compared with performance in regular classes were between 0.13 and 0.27.

7.2 High/Scope Perry Pre-School Study

The High/Scope Perry Pre-school Study began in 1962, and evaluated a high-cost, high-quality pre-school educational intervention (Schweinhart et al. 1993, Barnett 1996, Schweinhart et al. 2005). Participants were 123 African-American children of low socio-economic status at high-risk of educational failure. 58 children were randomly assigned to a pre-school intervention programme that emphasized participants’ intellectual and social development, and was based on a theoretical model: expectation-achievement-motivation for benefits. The evaluation demonstrated immediate achievement gains, fewer special education placements and higher high school graduation rates in the intervention group. At the 22-year follow-up with 95% of all children in the intervention and control groups interviewed to look at long-tem effects, there were significantly lower crime and delinquency rates, lower incidence of teenage pregnancy and welfare dependency and significantly higher rates of pro-social behaviour, academic achievement, employment, income and family stability in the programme group compared with the control group. A cost-benefit analysis calculated $7 benefit for each $1 cost.

7.3 Recent large field trials in the area of ICT and literacy

Two large trials in the field of ICT and literacy learning have recently been published. Rouse et al. (2004) evaluated a popular instructional computer programme Fast ForWord in elementary schools in the US, using an individually randomised trial with about 500 participants. The computer programme is designed to improve language and reading skills. The effectiveness of the programme was assessed using four different measures of language and reading skills. Although use of the computer programme appeared to improve some aspects of the children’s language skills, these gains did not translate into broad measures of language acquisition or actual reading scores. The computer package had already been widely implemented before it was evaluated, at an estimated cost of 90 million dollars.

A pragmatic randomised controlled trial was undertaken among pupils aged 11 and 12 in a school in the North of England (Brooks et al. 2006). It tested the hypothesis that a widely used computer programme to support literacy learning would lead to improvements in spelling and reading scores. However, at post-test this had very similar results to the US study – there was no evidence of a statistically significant benefit of the computer intervention.

 

 

How to reference this page: Gorard, S. (2007) Experimental Designs. London: TLRP. Online at http://www.tlrp.org/capacity/rm/wt/gorard (accessed )

Creative Commons License TLRP Resources for Research in Education by Teaching and Learning Research Programme is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 2.0 UK: England & Wales License

 


 
   

 

 
homepage ESRC