Democratic School Turnarounds: Pursuing Equity and Learning from Evidence

Excerpts reprinted with permission from Trujillo, T. & Renée, M. (2012).    Democratic School Turnarounds: Pursuing Equity and Learning from Evidence. Boulder, CO: National Education Policy Center.

The report Democratic School Turnarounds considers the democratic tensions inherent in the federal School Improvement Grant (SIG) policy’s market-based school reforms and critiques the research base that many of these reforms are based on. It concludes with a set of recommendations that re-center the purposes of public education for low-income students, students of color, and local communities and that are intended to guide federal, state, and local policymakers toward more equitable, democratic turnaround processes. Each recommendation stems from the provisional lessons that are emerging from current SIG-inspired turnarounds, from research on earlier efforts to improve school and district effectiveness, and from pockets of promising community-based practices that are developing at local and national levels.

In this article, we present a selection of key points from the report, along with the complete recommendations. For the complete literature review and analysis, please see the full report.

THE FEDERAL SCHOOL IMPROVEMENT GRANT PROGRAM

The School Improvement Grant program was established in 2002 as part of the federal No Child Left Behind Act to provide financial support for the development and implementation of NCLB’s corrective actions, but was not funded until 2007 (GAO 2011). In 2009, the SIG program was transformed in size and scope by the passage of President Obama’s American Recovery and Reinvestment Act (ARRA). The SIG budget increased from $125 million in 2007 to $3.5 billion in the 2010-2011 school year.

After this one-time ARRA infusion of funding, the SIG program was funded at $546 million for the 2011-2012 school year and $535 million for the 2012-2013 school year (USDOE 2011). The administration’s explanation for reinventing the SIG program was that dramatically turning around schools requires financial investment alongside significant structural changes.

Currently, each SIG school can receive up to $2 million per year for three years. For impoverished schools already struggling to meet students’ needs during local and state fiscal crises, the amount of money is significant.

However, under the SIG program grant recipients revert to their original funding levels after the three-year federal commitment expires. In this way, the one-time spending increase does not fundamentally alter basic federal spending structures – structures whose inequitable, inadequate distribution across lines of poverty and race have been well documented (Oakes 2002). Along with the infusion of money came a mandate to prioritize the bottom five percent of each state’s schools and to adopt one of four prescriptive federal models of school improvement (USDOE 2009).

Unlike the testing and accountability policies that came before, the 2009 reinvention of the SIG program includes more funding for implementation. Nevertheless, the SIG policy remains grounded squarely in market-based ideas. It assumes that strong external threats motivate teachers and principals to improve, that standardized test scores are reliable measures of student performance, that meaningful, sustainable changes can be spurred by competition, and that outcome-oriented accountability reforms can effectively interrupt historical patterns of low performance (Trujillo 2012).

THE SIG PROGRAM RESEARCH BASE

While the present-day concept of school turnaround rose to prominence seemingly overnight with the rollout of the SIG program, the roots of these dramatic reforms run deep in the literature on educational effectiveness and improvement.

Issues in Methodologies

School and district effectiveness studies of the late 1970s and 1980s did much to focus the attention of scholars, policymakers, and practitioners on the aspects of schools and districts that might be strengthened to improve the performance of children of color and children from low-income families. However, critics pointed to several methodological and conceptual limitations of these studies (Rutter 1983; Good & Brophy 1986; Creemers 1991; Scheerens 1992; Teddlie & Stringfield 1993; Sammons 1999; Bowers 2010).¹ The methodologies of both the school and district research traditions relied on small, skewed samples, usually based on unusually high student test scores.² The studies were also often conducted on samples of convenience or samples based on anecdotal reports rather than on systematically selected cases. This selection process meant that the results of the studies did not represent the range of experiences across the nation’s schools. Likewise, much of this research was based on short-term, snapshot evidence, not on data collected over the entire length of the reform. Such designs incorrectly assumed that the test score gains would be sustained (Bowers 2010). Further, while later studies expanded the sources of data used to explain effectiveness (Teddlie & Reynolds 2000), the bulk of this research drew conclusions about the factors that influenced student performance based largely on self-reports from administrators or small, unrepresentative samples of teacher interviews (Teddlie & Stringfield 1993; Trujillo, forthcoming). This severely limited how much the lessons from these studies could be applied to schools or districts with different characteristics. The limited data sources also led researchers to produce somewhat fragmented, incomplete interpretations of the classroom, school, and community dynamics that shaped – and were shaped by – the reforms.

Over-Reliance on Standardized Test Scores

Conceptually, one of the most frequent critiques of these studies was that they relied on a single measure of effectiveness – standardized test scores. While relying on standardized test scores was methodologically problematic because it falsely assumed that the assessments were valid and reliable, doing so as the sole measure of effectiveness also led to narrow conceptions of student success and the purposes of education – ignoring the social, civic, and broader academic aspects of schooling. This narrow, test-based definition of effectiveness is characteristic of market- based arguments that assume that education’s primary functions are economic. From this viewpoint, test scores are often employed as the only indicator that schools are preparing students for competition in the workplace (Rose 1995; Ball 1998). This perspective contrasts with arguments that focus on the democratic purposes of schooling, which frame schools as vehicles for fostering the values and skills necessary for collective, democratic participation and civic engagement (Orr & Rogers 2011). Student scores on standardized tests are far too narrow to be the sole indicators of school success in the democratic model of schooling.

The Impact of Socio-political Contexts

Finally, these research traditions were critiqued for their inadequate treatment of the socio-political and normative contexts of schooling (Welner 2001; Oakes & Lipton 2003; Thrupp & Willmott 2003; Oakes & Rogers 2006). The studies discounted the inherently political nature of schools, as seen in issues of who has access to power and resources, who can make decisions, and how resources are allocated. They also overlooked the ways in which norms and beliefs about what quality schooling looks like, and to whom it should be directed, shaped educators’ and communities’ support or rejection of certain reforms. Instead, studies of effectiveness were limited to questions about curriculum, time on task, monitoring, and the like – the technical dimensions of schooling. As a result, the research overestimated the relationship between schools’ technical changes and student learning. It also discounted the ideological opposition certain school reforms may provoke, the influence of resources like funding and stable staffing, and the vulnerability of even those schools deemed “effective” to the structural effects of poverty and racism (Coleman 1966).

Results : Limited Impact and Sustainability

In the educational literature, a sizeable body of rigorous, systematic research on early reconstitution reforms shows that firing and replacing school staffs has usually failed to achieve the intended effects. One meta-analysis showed that reconstituted schools in San Francisco continued to show up on lists of low-performing schools (Mathis 2009). In Chicago, longitudinal research on reconstitution revealed that staff replacements were no higher in quality than their predecessors and that teacher morale deteriorated under these reforms (Hess 2003). And a comprehensive, long-term study in Maryland demonstrated that reconstitution inadvertently reduced the social stability and climate of schools and was not associated with either organizational improvements or heightened student performance (Malen et al. 2002).

Also implicit in the claims about the efficacy of reconstitution is the assumption that the benefits accrued from replacing the bulk of a school’s staff will outweigh the unintended consequences. Yet, retrospective analyses of such dramatic interventions have concluded that the resulting logistical challenges, political fallout, and loss of organizational culture make such interventions prohibitive (Mathis 2009; Dowdall 2011). Finding enough qualified personnel to refill vacant slots in reconstituted or turnaround schools has proven difficult. In some cities, for example, districts found themselves swapping principals from one SIG-funded school to another. In Louisville, more than 40 percent of the teachers hired to work in turnaround schools were completely new to teaching.³ Another study showed how hiring difficulties forced many reconstituted schools to begin the school year with high numbers of substitutes (CEP 2008).

Like many district-specific studies of effectiveness, turnaround studies advocate for schools to focus on the technical dimensions of reform that are presumed to yield quick boosts in test scores: curriculum alignment, test preparation, and a sharp focus on test-based student achievement goals. But the presumed boost from such reforms is only weakly supported by rigorous, long-term empirical research (Trujillo, forthcoming). These recommendations echo those of the earlier school and district effectiveness studies almost word for word. One possible exception to these patterns might be found in the current IES Turning Around Low-Performing Schools studies, whose preliminary results suggest these conventional technical strategies are most helpful when implemented in conjunction with multiple interventions, including strategic teacher recruitment and intensive professional development (Sparks 2012). Nevertheless, the overall similarity across the literature raises questions about the degree to which the knowledge base on turnarounds has evolved conceptually and theoretically in the years since those studies were conducted.

Our review of the research on turnarounds revealed that authors continue to focus primarily on the within-school factors that may shape the potential of schools to turn around test performance, in place of research that situates schools within their broader socio-political and normative contexts.⁴ By concentrating primarily on technical issues around hiring and firing, curricular changes, and the like, this emerging field seems to be developing along the same lines as the previous generations of school and district effectiveness research. It also appears to be perpetuating the same narrowly framed debates about public education that consider changes inside of schools in isolation from schools’ broader institutional conditions – federal and state funding arrangements, etc.

One minor exception to this pattern of de-contextualization can be seen in the literature’s treatment of community engagement with the reforms. Most analyses advise leaders to solicit community input. Yet they recommend doing so in order to generate support for the turnaround. Most analysts are silent on the potential broader purposes of community engagement (Johnson et al. 2011).⁵ This literature generally fails to recommend soliciting input into the specifics of the turnaround process, facilitating more democratic decision making in public schools, or advancing notions of the public good. The result, as in the school and district effectiveness literature, is a set of proposals that discount the powerful influence of social, political, and other contexts in shaping school reforms.⁶

RECOMMENDATIONS

We outline six recommendations that are intended to guide federal, state, and local policymakers toward more equitable, democratic turnaround processes. Each recommendation stems from the provisional lessons that are emerging from current SIG-inspired turnarounds, from research on earlier efforts to improve school and district effectiveness, and from pockets of promising community- based practices that are developing at local and national levels.

Recommendation 1. Increase current federal and state spending for public education, particularly as it is allocated for turnaround-style reforms.

• Increase and equitably distribute federal and state education funding based on districts’ and schools’ demonstrated needs (based on poverty levels, communities’ economic and racial isolation, etc.).
• Maintain these spending arrangements in order to ensure that basic levels of financial capacity exist across all schools and districts. Federal accountability outcomes, regardless of local capacity.

Recommendation 2. Focus school turnaround policies on improving the quality of teaching and learning rather than on technical-structural changes.

• Outline a set of options for schools and districts focused on improving the quality of teaching and learning through efforts to systematically recruit and retain qualified teachers in turnaround schools, which historically tend to be difficult to staff.
• Provide guidelines for ongoing, cumulative professional development that deepens teachers’ knowledge of pedagogy and of the community in which their schools are embedded.
• Grant schools and districts greater autonomy to determine the details of each school’s turnaround plans. Provisions such as these would give schools and districts the authority to implement intense, dramatic improvements without undercutting the democratic nature of their efforts.

Recommendation 3. Engage a broad cross-section of schools’ communities – teachers, students, parents, and community organizations – in planning and implementing turnaround strategies that are tailored to each school and district context.

• Require school and district leaders to solicit and incorporate teachers’ professional expertise as well as parent, student, and community input into decisions.
• Specify the required timelines, financial and non-financial resources, and accountability structures for meaningful community engagement.
• Offer school, district, and state leaders training on authentic community engagement and models of best engagement practices at the federal, state, and district levels.
• At the school level, develop a representative oversight body that can solicit teachers’ professional judgments and the community’s ideas, concerns, and shared values and vision about what they want their schools to look like.
• Use parent surveys and hold multiple, accessible meetings (i.e., meetings held at times and locations that parents can attend and that provide free childcare and simultaneous translations) for community input.
• At the district level, establish a SIG advisory committee for stakeholders from multiple school sites to share experience and wisdom on school turnaround.

Recommendation 4. Surround struggling schools with comprehensive, wrap-around supports that stabilize schools and communities.

• Help struggling schools and districts sort through the SIG guidance by identifying existing community resources that can be integrated into the improvement process. For instance, provide specific examples of community-based organizations that can partner with districts and schools to provide non-academic supports related to health, nutrition, and other social services.

Recommendation 5. Incorporate multiple indicators of effectiveness – apart from test scores – that reflect the multiple purposes of schools.

• Develop indicators of schools’ progress in setting and working toward other academic, social, and democratic goals for their students.
• Measure students’ preparation for long-term academic success by tracking access to highly credentialed teachers and college-preparatory and/ or advanced courses. Track English Learner re-classification, graduation and college-enrollment rates. Disaggregate these indicators by race, family income, and language status, as well as by students’ access to highly credentialed, experienced teachers.
• Measure schools’ development of students’ social skills and awareness by assessing students’ work in groupbased learning tasks, problem-based projects, and curricula that relate directly to students’ communities. Track suspension and expulsion rates. Disaggregate these indicators by race, family income, and language status, and access to highly credentialed, experienced teachers.
• Measure schools’ democratic effectiveness by tracking the degree to which schools engage members of the public in school governance and improvement planning. Also examine whether schools make transparent certain information and decisions about schools’ budget, resources, and programs.
• Track these indicators longitudinally to assess whether outcomes and conditions for particular groups of students and schools are improving over time.
• Commission a diverse panel, composed of educational experts and practitioners from SIG sites, to select and define these broader indicators.
• Support SIG schools to track their progress toward non-test-based goals in order to bring energy and resources to bear on those student and community outcomes that are not easily monitored through standardized tests but that nonetheless represent meaningful goals for public education and equity-oriented reform.

Incorporating these other conceptualizations of effectiveness is another means by which the federal policy can promote more democratic norms and processes in turnaround schools, in place of narrowly market-oriented ones.

Recommendation 6. Support ongoing, systematic research, evaluation, and dissemination examining all aspects of turnaround processes in schools and districts.

• Solicit and fund research and evaluations that incorporate multiple points of view – teachers, students, and parents – to better understand what schools gained and where they experienced challenges when attempting to turn themselves around.
• Complement these more complete perspectives with information from classroom observations that reveals how these reforms are associated with different forms of instructional quality – beyond those reflected in standardized test scores.
• Support long-term research that illuminates the evolution of school and district turnarounds, including the rich historical and social legacies that aid successful turnarounds or thwart them, and that considers how such patterns unfold at the state, district, school, and community levels.
• Disseminate research and evaluation findings in formats useful to those leading turnaround efforts (e.g. accessible reports, guides, case studies, webinars, clearinghouses, and presentations).