San Jose Unified School District, 2010–2013: Building a Culture of Evidence-Based Practice around College Readiness

The author would like to recognize the efforts of the San Jose Unified School District CRIS Team, whose three years of hard work on the CRIS project is reflected in this article.

In 2012-2013, leaders and staff of the San Jose Unified School District (SJUSD) focused on accomplishing the district’s new mission: to aggressively pursue solutions to close the opportunity gap and ensure that all students leave SJUSD with twenty-first-century skills, prepared to participate in a global society. The district’s participation in the College Readiness Indicator Systems (CRIS) initiative¹ presented an opportunity to place the CRIS work within the context of the district’s 2012–2017 strategic plan, Opportunity21. The district positioned the CRIS work as a way to establish, validate, and model a process for broad implementation of the district’s Key Performance Measures (KPMs) – a set of connected data metrics that signal how well the district is accomplishing its goals (see the sidebar for a list of KPMs).²

This positioning of CRIS has ensured a coherent K–12 effort by our three CRIS schools – Lowell Elementary, Hoover Middle, and Lincoln High, which together create a college readiness pipeline throughout one feeder cluster of schools – and created a model for districtwide rollout of college and career readiness indicators.³ CRIS teams working on indicators at the school and district (setting and systems) levels⁴ successfully implemented what we refer to as data intervention cycles, which use data to identify struggling students, match the students with supports, and then evaluate the effectiveness of the supports. The district has used these cycles to begin building a culture of evidence-based practices districtwide.

CRIS as a districtwide model of evidence-based practice and data intervention cycles

​SJUSD’s strategic planning process in 2012-2013 included reflecting on past practice and on what the district has done well, looking around the country at models of effective practice, and building an ambitious but achievable plan to deliver on these strategic frameworks.For many years, SJUSD has met with instructional and programmatic success and is often referenced for showing innovative leadership, particularly in the areas of data use at the school level and in setting standards for students. For instance, SJUSD students will graduate having taken the courses necessary for entry to University of California and California State University systems (A–G standards).⁵

However, gaps in achievement continue to exist, particularly for San Jose’s Hispanic students. As the district worked toward closing the opportunity gaps among our students, there were several strategies that the organization leveraged to help propel work forward. While all of the elements of the strategic plan are important, there are a vital few that will help to jumpstart the efforts to accomplish our new mission. CRIS work, specifically, supported the district’s commitment to data-driven decision making. Our plan was to build on existing, good systems to reinforce behaviors that channel more resources and support to our students in most need. These include targeted academic programs, curricula, and interventions before, after, and during school for students who are struggling with core subjects and literacy; individualized adult support for students with behavioral and socio-emotional difficulties; and college advising in the areas of navigating the application and financial aid process for college-eligible students who might otherwise not receive support in overcoming these barriers to college entry and enrollment. This was driven primarily by the publication of our Key Performance Measures (KPMs), which both guided the organization and held it accountable for results. The CRIS program was critical to allowing us to validate our measures and to pilot systems of action based on these indicators.

During 2010 to 2012, the SJUSD CRIS team put in place these key evidence- based structures, practices, and processes:

• created a CRIS District Team, thereby ensuring system-level supports for college readiness focus and work;
• created school-level teams at the feeder pattern schools Lowell Elementary, Hoover Middle, and Lincoln High, to implement data intervention cycles that support student success at the setting level;
• conducted ongoing retreats to share out best practices;
• established biweekly meetings of the District CRIS team and each site team; and
• shared our work with other partner districts in the CRIS network.

Building a culture of evidence use through CRIS site teams

The CRIS site teams used data intervention cycles, which involved using data to:

• identify a struggling group of students, using indicators in three dimensions of college readiness: academic preparedness, academic tenacity, and college knowledge (see the article by Borsato, Nagaoka, and Foley in this issue of VUE);
• design interventions to support these students and improve their success;
• evaluate the effectiveness of the interventions; and
• scale them out to the rest of the school.

These cycles and the CRIS school teams have become a model of evidence-based practice that the district subsequently planned to scale out in 2012-2013 and beyond through cross-functional groups called (OP- STAT) teams⁶ that represent different departments, roles (principals, teachers, counselors, district administrators), and levels (elementary, middle, and high), and engage in deep inquiry around one of the eleven KPMs. In the 2012-2013 school year, SJUSD launched three OPSTAT teams that focused on increased AP/IB participation and performance, socio-emotional learning, and graduation rates and completion of A–G requirements. Three to four additional OPSTAT teams per year will tackle other KPMs in the coming school years, starting with early literacy in 2013-2014.

During year one of work (2012-2013 for the first wave of teams), OPSTAT teams engaged in intense and focused data analysis to identify the thresholds to be used in developing different tiers of supports for students. The teams also examined and selected the most effective strategies to improve student success on the particular KPM. The teams published reports and/or shared findings with the rationale behind the KPM and its significance, data examined, evidence-based processed implemented, recommended interventions, and implementation and monitoring processes for school sites to put these supports to action.

In 2013-2014 and beyond, school-level teams that are CRIS-like will continue the work of each KPM’s OPSTAT team at their sites, to ensure roll-out and sustainability of the evidence-based practices and beliefs. SJUSD is building out a district accountability system based on reports that track schools’ KPM progress. Reports will be automated and sent out on alerts through the new Data Warehouse and Business Intelligence Tool to which the district is transitioning in 2013-2014. KPM reports will be customized for different levels within the organization, including senior leadership, site leadership, and teachers. Site and district OPSTAT teams will use these reports, with support from the district office, to monitor the success of interventions at the setting and system levels.

School data teams, modeled on CRIS and OPSTAT teams, will conduct annual inquiry cycles in accordance with the following projected timeline:

• research and investigate thresholds for struggling students: first and second quarter of the school year;
• identify and plan for effective interventions and supports for struggling students: third quarter; and
• implement interventions and supports: fourth quarter and summer.

Throughout this process, the teams will consistently communicate their work to other site staff to elicit buy-in and build capacity for implementation of the selected strategies.

Ongoing indicator development and implementation

The district CRIS team and the three CRIS schools continued refining indicators of academic preparedness, academic tenacity, and college knowledge at the individual and setting levels. Teams continued to meet on a regular basis to assess progress, share challenges, and make mid-course corrections based on implementation feedback and evidence. Retreats took place at the beginning of each school year for the school and district CRIS teams to share indicator development and data collection results for each chosen indicator and to cement lessons learned for K–12 expansion of CRIS work.

SJUSD also made advances in the selection of indicators and the implementation of a CRIS at the elementary and middle school levels, with the goal of informing and influencing a K–12 trajectory of college readiness beginning with the early grades. SJUSD partnered with the Everyone Graduates Center at Johns Hopkins University. This partnership, through direct collaboration around the socio-emotional learning OPSTAT team, helped us to identify when off-track indicators emerged in the academic tenacity trajectory of our students at the elementary and middle school levels and what interventions from grade to grade could be most effective to bring students back on track. Our Hoover Middle School CRIS team, in particular, has been implementing a micro-process intervention strategy since August 2011.

Looking ahead

We are confident that SJUSD will continue making advances on the most persistent challenges to developing and scaling a CRIS. The district will take advantage of a new user-friendly data warehouse and reporting tools to position CRIS work and its offshoots (like OPSTAT) so as to increase data capture of online learning technologies related to college readiness in the years to come. SJUSD will continue to scale out CRIS work and processes across school sites and the central office through OPSTAT district and site teams.

Efforts are also in place to validate our high schools’ selection of indicators and how they map to indicators at the elementary and middle school levels, as well as to explore the development of predictive models around the CRIS work. We will also forge ahead with efforts to map out supports and interventions and track these consistently through emerging data system capabilities. The district is poised to leverage the success and lessons learned from CRIS efforts to date in order to create a scalable framework that will guide overall KPM efforts in data collection and reporting; data-based decision-making (use of data and goal-setting based on measures’ cut points); implementation strategies for interventions and supports based on the data analysis; and the extension of CRIS work to other sites in conjunction with the roll-out of KPMs.