Conceptual review and meta-analysis of school effectiveness

Integration
Of the five effectiveness-oriented educational research types, which were reviewed, two focused on “material” school characteristics (such as teacher salaries, building facilities and teacher/pupil ratio). The results were rather disappointing in that no substantial positive correlations of these material investments and educational achievement could be established in a consistent way across individual studies. On the basis of more recent studies these rather pessimistic conclusions have been challenged, although methodological critique indicates that the earlier pessimistic conclusions are more realistic. In-depth process studies connected with large-scale evaluations of compensatory programs pointed out that programs which used direct, i.e. structured, teaching approaches were superior to more “open” approaches. The research movement known as research on exemplary effective schools (or briefly: effective schools research) focused more on the internal functioning of schools than the earlier tradition of input-output studies.
These studies produced evidence that factors like strong educational leadership, emphasis on basic skills, an orderly and secure climate, high expectations of pupil achievement and frequent assessment of pupil progress were indicative of unusually effective schools.
Research results in the field of instructional effectiveness are centered around three major factors: effective learning time, structured teaching and opportunity to learn in the sense of a close alignment between items taught and items tested.
Although all kinds of nuances and specificities should be taken into account when interpreting these general results they appear to be fairly robust - as far as educational setting and type of students is concerned. The overall message is that an emphasis on basic subjects, an achievement-oriented orientation, an orderly school environment and structured teaching, which includes frequent assessment of progress, is effective in the attainment of learning results in the basic school subjects.

Table 6 summarizes the main characteristics of the five research traditions.

Table 6: General characteristics of types of school effectiveness research

	independent variable type	dependent variable type	Discipline	main study type
(un)equal opportunities	socio-economic status and IQ of pupil, material school characteristics	attainment	Sociology	Survey
production functions	material school characteristics	achievement level	Economics	Survey
evaluation compensatory programs	specific curricula	achievement level	interdisciplinary pedagogy	quasi-experiment
effective schools	“process” characteristics of schools	achievement level	interdisciplinary pedagogy	case-study
effective instruction	characteristics of teachers, instruction, class organization	achievement level	educational psychology	Experiment observation

In recent school effectiveness studies these various approaches to educational effectiveness have become integrated. Integration was manifested in the conceptual modelling and the choice of variables. At the technical level multi-level analysis has contributed significantly to this development. In contributions to the conceptual modelling of school effectiveness, schools became depicted as a set of “nested layers” (Purkey and Smith, 1983), where the central assumption was that higher organizational levels facilitated effectiveness enhancing conditions at lower levels (Scheerens & Creemers, 1989). In this way a synthesis between production functions, instructional effectiveness and school effectiveness became possible, by including the key variables from each tradition, each at the appropriate “layer” or level of school functioning [the school environment, the level of school organization and management, the classroom level and the level of the individual student]. Conceptual models that were developed according to this integrative perspective are those by Scheerens (1990), Creemers (1994), and Stringfield and Slavin (1992). Since the Scheerens model was used as the starting point of the meta-analyses described in subsequent sections it is shown in Figure 2.

Figure 2: An integrated model of school effectiveness (from Scheerens, 1990)
The choice of variables in this model is supported by the “review of reviews” on school effectiveness research that will be presented in the next section.
Exemplary cases of integrative, multi-level school effectiveness studies are those by Mortimore et al. (1988), Brandsma (1993), Hill et al. (1995), Sammons et al. (1995) and Grisay (1996).