Conceptual review and meta-analysis of school effectiveness

Discussion

Educational effectiveness studies in OECD-countries are carried out in three main strands: studies in the education production function traditions, studies of effective schools and studies about instructional effectiveness. During the last decade these three approaches are being integrated in more comprehensive multi-level studies. Broadly summarized, the results indicate a relatively minor impact of resource input variables and school organizational conditions and a larger impact of instructional characteristics at classroom level.

In school effectiveness studies in developing countries there is a strong predominance of studies carried out in the education production function tradition. When the results of the small minority of studies that include school organizational and instructional conditions is taken into account the overall picture of results is that facilities and the quality of human resource inputs have a larger impact than variation in instruction and pedagogy.
The implication of this state of affairs for the future of effectiveness research in developing countries should not be that – given the relatively small impact demonstrated so far – school organizational and instructional conditions deserve less attention. On the contrary, as others have pointed out (Fuller & Clarke, 1994), as material and quality of human resources conditions are improving in developing countries it is to be expected that managerial and instructional conditions will start to make more of a difference. One could argue that in early stages of development primacy ought to be given to putting in place basic provisions in terms of buildings, equipment, textbooks and qualified teachers. After this would have been realized conditions of education would be more comparable to conditions in OECD-countries, with a greater likelihood of effectiveness studies beginning to show the impact of instructional factors.

The strength of the school effectiveness knowledge base does not just depend on the accumulation of evidence on the strengths of associations between antecedent conditions of schooling and outcomes. The existence of an integrated conceptual framework and theory-embedded principles that could explain “why” certain factors work, is just as important. In this sense the study of educational effectiveness in OECD-countries is still very weak. Although there is some conceptual multi-level modeling, where the interactions between school environmental, school managerial and instructional conditions are substantively interpreted and analytically modeled (Scheerens & Bosker, 1997) use of theories is largely eclected. At the level of classroom instruction there is a latent controversy between behavioristic principles that seem to be better able to account for the outcomes of empirical studies so far and currently popular constructivist approaches.

The influence of economics and micro-economic theory on effectiveness studies in developing countries provides a more unilateral framework for these studies as compared to the eclectic use of theory in OECD-countries, where micro-economic theory is just one of the theories that is referred to.

A perspective that is interesting and innovative – also for studies in OECD-countries – is the use of cultural contingencies in interpreting results of effectiveness studies (Fuller & Clarke, 1994).

Concerning methodology a blending of experiences and approaches from studies in the two “worlds” would be desirable too. As emphasized by Riddell (1997), multi-level modeling should be applied more frequently in effectiveness studies in developing countries. (A condition that would be enforced if conditions at school and classroom level would be taken into account in study-designs).

Econometric modeling of selection-bias, seen more frequently in studies in developing countries, has a lot of potential for studies elsewhere too, as do cost-benefit analyses, and simulations. Another valuable idea from econometrics is the rendering of effects in terms of elasticities.
As far as research “logistics” is concerned, a lot is to be said for exploiting two kinds of synergy:

• 
  integrating effectiveness studies with evaluations of development projects (the study by Van der Werf et al. (1999) that was referred to earlier, is an example);
  
• 
  integrating project evaluation with the development and implementation of monitoring systems at national level, possibly exploring the “spin-off” of such monitoring instruments for school self-evaluation purposes as well^*).
  

When such integrations can be realized it is more likely that core data on inputs, processes and outcomes of schooling will be available on a longitudinal basis, and this would be a condition that would greatly enhance the validity and applicability of effectiveness studies.

Finally the value and strength of the school effectiveness knowledge base for practical application should be looked at. Two types of application can be considered: planning on the one hand and monitoring and evaluation on the other. The first is a stronger and more pretentious type of application than the latter.

When the input and process conditions that have received empirical support in educational effectiveness research are merely used as a basis for identifying variables to be included in indicator, monitoring and school self-evaluation instruments there is only an indirect link with prescribing policy and practice. At the same time there would be ample room for local and within-culture interpretation.
Given the weaknesses and “blank spots” in the empirical knowledge base and the need to contextualize results, the evaluative use of educational effectiveness knowledge seems to be the most appropriate one.

Nevertheless there is robustness in the general principles that emerge from the research-based knowledge of more than three decades of educational effectiveness research in industrialized and developing countries. From these principles the following tentative suggestions for educational projects in developing countries could be derived:

• 
  describe the general conditions of education on the basis of a core set of indicators, including poverty conditions per region, participative rates and the availability of basic resources;
  
• 
  at early stages of development emphasize conditions that stimulate intended participation levels and basic resources and facilities (e.g. buildings, classroom);
  
• 
  invest in substantive educational programs containing four well-integrated parts: a national examination or assessment program, national curriculum priorities in core subjects, teacher training (focused at subject matter mastery and instructional principles) and a national monitoring system;
  
• 
  decentralize school management, create conditions of local participation and control over financial conditions and teachers’ conditions of labor;
  
• 
  use different media (distance education, training courses, model-curricula, school self-evaluation) to enhance classroom management, effective learning time and structured teaching (with diagnosis, feedback and immediate remediation at its core) to stimulate active learning; adapt these general instructional conditions to aspects of the local culture.
  

For various reasons (larger between school variances in outcomes, practical relevance, blending of methodological principles, synergy with program evaluation and monitoring) it is to be expected that the “cutting edge” of educational effectiveness research in the near future will be situated in developing countries, more than in the industrialized part of the world.