Conceptual review and meta-analysis of school effectiveness

Part II Evidence from developing countries

In this part the evidence about effectiveness enhancing conditions of schooling in developing countries will be reviewed. The review sets out by referring to earlier review articles, particularly those by Hanushek (1995) and by Fuller and Clarke (1994), which in itself incorporates results of reviews by Fuller (1987), Lockheed & Hanushek (1988), and Lockheed & Verspoor (1991). Next a schematic description of 13 studies conducted after 1993 is provided. Conclusions are drawn about the state of the art of educational effectiveness research in developing countries, in terms of predominance of the type of factors that are studied, outcome comparison with results from industrialized countries, relevant research innovations and implications for policy and practice applications.

Scope and limitations of the school effectiveness model, particularly when applied to developing countries

Although the integrated model of school effectiveness is comprehensive in that it encompasses input, process, output and context conditions and recognizes the multi-level structure of educational systems it has a number of limitations.

1. 
   The model has the level of the individual school as its focus, and leaves important issues of a proper functioning of national education systems untreated; I shall refer to this as the aggregation limitation. To the extent that subsidiarity^*) is applied and schools are autonomous this limitation is less severe, since, by definition, the school would have more formal responsibilities.
   
2. 
   The model has a strongly instrumental focus, treating educational goals and objectives as largely “given”. Extending the model according to the larger perspective of organizational effectiveness, as briefly referred to in part I, can partly compensate for this limitation, by taking into account the responsiveness of the school vis à vis changing environmental constraints. It is again dependent on the pattern of functional decentralization in an educational system, to what extent adoption mechanisms at school level are important as compared to the provision of such levels at the macro level. We shall refer to this limitation as the instrumentality limitation.
   
3. 
   Although the model is amenable to include questions of equity and efficiency, the actual research practice has not lived up to expectations in this area. Moreover, the way school effectiveness research is dealing with these issues is also determined by the other two limitations concerning level of aggregation and instrumentality. The argument is that, particularly in developing countries, these issues deserve to be dealt with from a broader perspective than the school effectiveness model. This limitation will be referred to as the relatively narrow quality orientation.
   

Before discussing these three limitations further and examining their seriousness for developing countries, some clarification will be provided with respect to the concepts of “functional decentralization” and “subsidiarity”. These concepts provide a basis to determine the relative importance of the school as a decision-making level in education systems, and moreover differentiate the answer to this question according to particular domains of decision-making.

In the history of education in the Netherlands the term subsidiarity was used to refer to a specific way in which denominational pressure groups in education linked to see the relationship between the state and corporations representing interest groups in the educational field. According to the subsidiarity principle the state should not interfere in matters that can be dealt with by organized units of professionals. In the original case these organized units were the denominationally based corporations or pressure groups of representatives in the education field, their umbrella organizations in particular. “Subsidiarity” was the term preferred by the Roman-Catholic denomination, while the Protestants spoke of “sovereignty in one’s own circle”. Leune (1987, 379-380) points at the corporatistic nature of this kind of concepts. According to the subsidiarity principle the state only acts subsidiary, that is, it only interferes as a replacement, when needed. A simple example of subsidiarity is a driving-instructor, who takes over the steering of a vehicle when the trainee makes a mistake, but in all other cases quietly watches without interference. Within the context of the European Commission the term subsidiarity is used to express the principle that what can be accomplished by the member states should not be done by the central organs of the Union.

Of course it is debatable to what extent subsidiarity should be applied to schooling, in other words which functions the schools could accomplish without interference from higher administrative levels. The concept of functional decentralization helps in nuancing this discussion by taking into account that a system can decentralize in some domains, but not in others.
Although various classifications are available in the literature (cf. Van Amelsvoort & Scheerens, 1997) the most commonly recognized educational domains are:

• 
  the curriculum (including goals and standards)
  
• 
  finance
  
• 
  the conditions of labor and personnel policy
  
• 
  school management
  
• 
  teaching methods
  
• 
  quality control
  

A well-known pattern of functional decentralization is a liberalization of finance (e.g. block grants), management (cf. “school-based management”), and teaching methods, accompanied by a centralized core curriculum. In actual practice it appears hard to relax central regulations concerning the conditions of labor of educational personnel, under conditions of collective bargaining by trade unions.
A further qualification with respect to the degree of decentralization is possible by recognizing that sometimes government units are merely dispersed (“deconcentration”), that decision-making authority is sometimes only partly shed (“delegation”) and in other cases is completely given to local bodies (“devolution”) (cf. Bray, 1994).

re 1) aggregation limitation

At first sight it might look trivial to point out that the school effectiveness approach does not address all that is relevant in developing and improving education and that there is a place for macro level educational policy measures. The statement is made in this context for the following reasons:

• 
  to explicitly avoid the impression that the school effectiveness approach could be seen as a paracee for all problems in education in developing countries;
  
• 
  to introduce the idea of optimal patterns of functional centralization and (functional) school autonomy.
  

When considering macro-level stimulants and constraints, next to the school as a lever for educational improvement radical decentralization and privatization seems to be left aside as a policy option a priori. The relevant literature (e.g. James, 1991) points out that private education is likely to be either regulated or funded (or both) by the state, so that mixtures of state regulation and school autonomy are the most common situation, even in private and semi-private government-dependent school systems. Indeed the very decision as to the degree of privatization and decentralization is an example of a relevant state-controlled measure. Other relevant issues that are likely to be settled at the state level, particularly in developing countries, are:

• 
  dealing with trade-offs between quantity and quality of schooling (enlarging enrolments or investing in quality) and between quality and efficiency, for example, with respect to discussions on school and class size;
  
• 
  providing teacher training, or at least setting standards for teacher training;
  
• 
  arranging for equitable distribution of scarce resources (e.g. by stimulating poor rural areas on the basis of a progressive income tax measure);
  
• 
  setting (core) educational objectives and standards to provide vertical coordination between levels of schooling, e.g. between primary and secondary schools;
  
• 
  stimulating accountability, by introducing evaluation and feedback mechanisms, which can serve information requirements of consumers of education and administrators.
  

Although the empirical evidence is scarce, there appears to be some support for the hypothesis that functional centralization on curriculum standards and assessment enhances educational performance (e.g.Conley, 1997). Setting achievement standards and assessing student achievement relate favorably to effectiveness enhancing conditions at the school level. Having clear, accessible objectives can add to the overall purposefulness and achievement orientation of the school. It can, likewise, be seen as a supportive condition for “instructional leadership”, and, if information is properly fed back to stakeholders, as a basis for organizational learning, accountability and improved “consumerism”.

A further hypothesis, regarding developing countries is that the lower the level of schooling of parents and the poorer the catchment area of the school the more effective these measures of functional centralization are likely to be.

re 2) instrumentality limitation

Another aspect of the school effectiveness model is the “goal immanent” orientation. A function of “goal detection” or adaptation of goals according to changing societal and contextual conditions is missing. When the school effectiveness model is broadened in scope, by taking into account additional criteria such as responsiveness, participant satisfaction and formal structure (cf. Faerman & Quinn, 1985) this situation is improved. In developing countries material support from the local community appears to be particularly important, and part of the schools’ effort would be needed to acquire this support.
Given its technical and instrumental orientation the school effectiveness model is not strongly oriented towards incentives, and trade-offs between task-related and person-related interest. This is one of the reasons to attempt to connect microeconomic theory and school effectiveness modeling (cf. Scheerens & Van Praag, 1998).
Again, in developing countries “adaptability” and provisions of conditions that create incentives for good performance also deserve to be dealt with at macro level.

re 3) relatively narrow quality orientation

The school effectiveness models is, at its core, an instrumental model of direct school outputs (as compared to more long term, societal outcomes of schooling), in other words quality is addressed as technical effectiveness. The origin of school effectiveness research lies in improving education in poorer “inner city” districts in US cities, and, among studies, there is definitively a bias towards less “privileged” educational contexts, and therefore the research findings have a certain relevance to creating more equal educational provisions. Equity is more directly addressed in studies on so-called “differential effectiveness”, where the effectiveness of a school is differentiated according to sub-groups; i.e. boys/girls and children with high and low SES backgrounds. These studies are scarce, and the results inconclusive, however. The same applies to studies that have addressed cost-effectiveness. This state of affairs underlines a previous conclusion that the school effectiveness model inadequately addresses equity and efficiency of educational provisions at large and that, particularly in developing countries, these issues should be addressed primarily at the level of macro level educational policies.