Mosaic Home | EDC Home

Studying the Evolution of a 15-Year Reform Effort

The Researching the Sustainability of Reform project analyzes school reform initiatives over decades rather than years

"People usually ask: how do reforms change schools? Here we examine the reverse: how schools change reforms."

—David Tyack and Larry Cuban in
Tinkering Toward Utopia¹

In 1999, Jeanne Century and her colleagues in EDC's Center for Science Education set out to explore Tyack and Cuban's provocative question in unprecedented depth. They identified elementary science programs in nine districts across the country that had been in place for a decade or more. The goal of the Researching the Sustainability of Reform (RSR) project was to study the evolution of the science programs and understand the factors that contributed to or inhibited their sustainability.

Like many studies of school reform, RSR capitalizes on the power of hindsight—looking backward to understand the implementation and impact of a specific program. What's unique about RSR is the length of its hindsight; the project investigated the development of programs over 10–30–plus years versus the typical 3–5–year time span.

By selecting programs that had survived for so many years, RSR researchers hoped to unlock some of the keys to successful sustainability. Perhaps not surprisingly, the research revealed much more complex stories. Some of the programs seemed to have been sustained in name only, given the distance they had traveled from their original incarnation. Others appeared to sustain themselves out of inertia rather than through building a proven record of success. On the other hand, some programs that appeared to have failed actually exerted powerful though subtle influences on classroom teaching long after their supposed demise.

The value of the extensive RSR data—gathered through extensive interviews, observations, surveys, and document analysis—may be in the questions it raises about the concept of sustainability: What does it mean for a school reform effort to succeed? What is the difference between succeeding and simply surviving? And what kinds of changes does a program need to go through in order to meet the needs of a diverse and ever–changing community?

The story of the SHOW program in the Lakeview school district is a case in point.² On the surface, it is the story of a well–designed program that attracted outside funding, tapped the expertise of a local university, and built strong community support over 15 years. However, through interviews with designers of the program, supporters, critics, those who were there at its inception, and those who encountered it years later, RSR staff also found a story of frustration and failed expectations.

More than anything, it is a story that underscores the complexity of effective, long–term school change-how difficult it is to meet the evolving needs of various constituents while maintaining the quality of each component of a program (curriculum, professional development, student assessments, etc.).

The Program

In 1985, two science professors had a vision for improving science education in Lakeview, their local, urban school district. They wanted elementary school students to learn science not from a textbook, but by doing it—just as their graduate students learned by working as scientists in a laboratory, and just as leading scientists and researchers have recommended as a model method for K–12 science education. Their plan involved implementing an inquiry–based curriculum, in which students would learn science concepts through hands–on explorations over several weeks. The teacher would serve as a facilitator and a coach, helping to guide students to deepen their analysis and understanding of evidence gathered in the hands–on activities and experiments. The curriculum would be accompanied by extensive and long–term professional development for teachers, and a collaboration with scientists from the local university.

The Timeline

1985: The two professors use their professional networks to identify model kit–based science programs that have been implemented in other urban districts. They select units from a well—established program and create materials to guide its implementation in Lakeview. They present their program—called SHOW (Science the Hands–On Way)–to the district, and receive support to pilot–test the program at one school. The district hires a "hot shot" language arts teacher to serve as the coordinator. The professors help to secure some funding from the university and other partners.

1985-87: The pilot program at the Green School officially lasts for two years. The field–test teachers are overwhelmingly enthusiastic about "the science" that is happening in their classrooms, crediting the rich content of the units and the professional development provided by the scientists and their graduate students.

1988: The professors write grants to support expansion of the program, and the district commits more than $75,000 to bring SHOW to five new schools. During this phase of the pilot test, principals of all schools in the district are involved in mandatory monthly training on the program.

1990-94: In response to a proposal written by the professors, the National Science Foundation awards the district nearly $700,000 to expand the SHOW program to all 25 elementary schools in the district. The grant funds four resource teachers, who operate like coaches, teaching demonstration lessons, sharing resources, and assisting teachers with students who are having trouble.

1995: NSF extends the grant for three more years and provides an additional $300,000 in funding to support the expansion of the program to grade 6.

1998: NSF funding ends. The district must find funds to maintain core staff for the program. Only two resource teacher positions are maintained, despite the increasing number of new teachers responsible for teaching SHOW units.

1998-2000: The state issues new science frameworks, which are better suited to textbook learning than hands–on, inquiry–based science programs like SHOW. The state also places nearly all of its emphasis on reading and mathematics, rather than science.

2000: A mathematics professor spearheads an attack against SHOW, charging that the program lacks rigor and takes up resources that would be better spent on reading programs. At a contentious school board meeting, nearly 40 teachers and community members provide a passionate defense of SHOW. They speak of the value of having students learn science through inquiry and experimentation rather than from textbooks, but they lack hard data showing how the program has improved student learning. The program survives on a 3–2 vote, though the board stipulates that the district must develop a better accountability system to measure the impact of SHOW; the board also provides funds for the purchase of science textbooks to supplement the program.

The Dynamics of Longevity

Looking back over the 15–year history of SHOW, EDC researchers found some divergent explanations for the longevity of the program. Critics of the program saw its sustainability as a sign of educational inertia: Teachers resist change. They want to continue doing what they've been doing. Supporters saw the opposite: Maintaining SHOW, they said, had been extremely hard work—a constant struggle to create change in the first place, to help it grow, and to defend it against attacks. Here are some of the critical challenges the program has faced throughout its tenure:

• Funding. Securing grants from NSF and other sources wasn't easy in the first place, but making up for the loss of those funds proved even more difficult. During the height of the funding, some departments began to resent the resources devoted to the science program. When SHOW had four resource teachers, for example, the mathematics and reading programs had none. And when the NSF grant ran out, the district had to allocate its own funds to sustain the program—which directly pitted science against all other departments.
• Standards and testing. The Lakeview district science standards were based on the national standards rather than the state standards. The state standards emphasized understanding of terms and specific facts; the national and district standards placed more emphasis on concepts and the scientific method. That conflict helped pave the way for the combative school board meeting of 2000. But the larger issue was that both the state and the district had become less concerned about science and much more concerned about literacy and mathematics.
• Assessments and data. The state's ninth grade science assessment is poorly aligned with the content and philosophy of SHOW, according to district leaders. Each SHOW unit includes assessments, but the district has long been interested in measuring learning across the entire program. In the early 1990s, the district hired an outside assessment expert who worked with a district team to design new measures. However, the process was fraught with disagreements, many teachers thought that the initial assessments were overly complex, and the work was never completed. The lack of hard data on student learning has become the weakest link in the SHOW program.
• Professional development. With high teacher turnover rates and fewer resource teachers, fewer and fewer teachers feel well equipped and committed to teach the SHOW units. In a recent informal survey, little more than half of the responding teachers in the district reported that they were teaching the required four kits per year, and only about a fifth of those reported teaching the kits from start to finish. Half the teachers using kits said that they are picking and choosing the parts they teach.
• Leadership. Throughout its 15–year tenure, SHOW has survived the departures of many key leaders and advocates-but it hasn't always prospered. Many supporters believe that stronger leadership in the late 1990s would have resulted in the development of an assessment process that would prove the positive impact of the program. And many sources–supporters and critics–believe that too much of the leadership came from the university partners. Program advocates should have spent more time cultivating support in the district office and the community, they say.

Wider Implications

The fact that SHOW survived at all during periods of weak leadership highlights one of the subtler factors RSR researchers saw in several districts. Century and colleague Abigail Jurist Levy refer to it as "philosophy." In the districts that sustained a science program for several years, there was an embedded set of core values and beliefs about what good science instruction looks like—even when those beliefs weren't clearly articulated or consistently championed by district and school leaders. The philosophy was evident in the multiple interviews Levy and Century conducted with diverse groups in the districts. Over and over, they heard the same sort of phrases and assumptions about the value of hands–on science education repeated by teachers, administrators, and even parents.

"What's striking is how important philosophy was even in the absence of an intentional plan for sustaining a particular science program," comments Levy. "Think what could be done if districtand school leaders were more intentional about how they implemented and supported these programs."

To Century, one of the other key lessons from the Lakeview story is the importance of a strong accountability system. "Just because you sustain a program doesn't mean it has quality, or a positive effect on student learning," says Century. "Without an accountability system of some sort, you have no data to evaluate the impact and quality of the program you are trying to sustain." She and Levy hope that the findings of their study may help to lay the foundation for more programs that are both high quality and sustainable.

¹ Tyack, D. B. and Cuban, L. (1997). Tinkering Toward Utopia: A Century of Public School Reform. Cambridge, MA: Harvard University Press.

² As school districts participating in the RSR study have requested anonymity, Lakeview and SHOW are pseudonyms.

For questions or comments, contact mosaic@edc.org.

Copyright 2000-2003 Education Development Center, Inc. All Rights Reserved.

What is the difference between succeeding and simply surviving?

Some programs that appeared to have failed actually exerted powerful though subtle influences on classroom teaching long after their supposed demise.

"Think what could be done if district and school leaders were more intentional about how they implemented and supported these programs."