Measuring Academic Tenacity: New Visions for Public Schools

Over the past three years, the College Readiness Indicator Systems (CRIS) project has afforded New Visions for Public Schools an opportunity to reflect upon our college readiness systems and practices.¹ Through our involvement in the project, we have been able to explore various strategies for ensuring that our students graduate from high school ready for college and careers. Before taking readers through our CRIS experience, however, it is important to note two key ways in which New Visions is unique among the CRIS team members.

The first is that we are the only CRIS site that is not a formal school district. As a nonprofit Partnership Support Organization, New Visions is responsible for providing a number of district-like supports and services to the roughly seventy-five New York City public high schools in our network, although we do not have authority over school-level decision making or practices.

The second difference lies in New Visions’ early development and adoption of a college readiness indicator, preceding similar activities by our CRIS district colleagues. In 2005, New Visions created an “On-Track to College Readiness” metric to support our schools with student- and school-level planning and assessment. The metric divides students into four performance groups – on track for college; on track for graduation; almost on track; and off track–based on their credit accumulation, attendance, and New York State Regents test pass rates after each semester.

Our on-track metric has been extremely useful in helping our schools identify which of their students are at risk of not graduating ready for college. But because it is based solely on academic indicators, it only paints a partial picture. Given the growing recognition in both the academic literature and policy circles of the importance of students’ noncognitive skills development (known as academic tenacity by the CRIS project) for college and career readiness, New Visions has been interested in finding ways to measure – and to incorporate such measures – into our college-readiness metric. The CRIS project enabled us to focus on this increasingly important work.²

Linking academic tenacity to the common core

Early on in the project, we concluded that beyond utilizing subjective student self-assessment tools like surveys or other very rough proxies such as student attendance, no one had yet found any convincing measures of academic tenacity. Around the same time, New Visions had just begun work on developing Common Core–aligned curriculum modules in math and English, thanks to funding from both the federal government and the Bill & Melinda Gates Foundation. As we engaged in these two strands of work, we began to see a convergence between the two efforts.

We proposed that true mastery of the Common Core will require students to demonstrate academic tenacity, as the standards help students to understand the skills and knowledge that they need and support them in self-regulating their progress to mastery. For example, Common Core writing standard #5 for English language arts for ninth- to tenth-graders requires students to “develop and strengthen writing as needed by planning, revising, editing, rewriting, or trying a new approach.”³ Research also suggests that students are more effective at self-regulating their progress when they deeply understand the knowledge and skills that they are required to master (Sadler 1989; Nicol & Milligan 2006; Rust, Price & O’Donovan 2003). So we hypothesized that engaging students with a Common Core–aligned rubric would help them to understand at a very granular level their performance on various skills and how they can improve.

To test this, we conducted focus groups of both higher- and lower-performing students in four New Visions schools that were part of the same initiative to design and implement Common Core–aligned curriculum. Our initial findings suggested that students were able to articulate the Common Core skills they were required to master, and they were able to pinpoint their strengths and weaknesses and describe how they could improve on their deficits.

Although these student focus groups provided us with promising initial evidence to support our hypothesis, we are limited in our ability to put it to further testing until the Common Core is widely adopted across our schools, and it is premature to operationalize this work. New Visions will continue exploring the relationship between tenacity and the Common Core as the standards are rolled out across our network schools.

Supporting comprehensive programming to boost the academic tenacity

While we await the rollout of the Common Core standards, New Visions has shifted our focus towards supporting comprehensive programming to boost academic tenacity, specifically through a program called iMentor, a school- based mentoring program that matches public high school students with college-educated mentors in one-to-one relationships. Each student receives a mentor who augments existing guidance and college counseling programs at their school. Mentor-mentee pairs at New Visions schools are matched for all four years of high school, over which time they exchange weekly emails and meet in person on a monthly basis.⁴

The iMentor program seeks to develop seven core noncognitive skills: social capital development, utilizing a growth mindset, perseverance, critical think- ing, help-seeking and self-advocacy, optimism and excitement about the future, and curiosity and love of learning. Mentors also help their mentees develop college knowledge, such as how to identify the best college match, apply for financial aid, meet application deadlines, write effective essays, and integrate into college life – all of which allows students to reach important milestones on their path to college graduation.

Over the next several years, each entering student at eight New Visions high schools will be paired with a mentor for four years, as part of a six-year study of these schools by the Research Alliance for New York City Schools, which will be completed in 2018; the first annual report will be released in fall 2014. Approximately 2,500 students will be mentored over the life of the project.

The future direction of college readiness at New Visions

The CRIS project has enabled New Visions to explore new ways of improving upon our existing college readiness indicators and supports, focusing on the promotion and development of academic tenacity in particular, and has created the space for us to think deeply upon the challenges and opportunities we face. Three years later, as the CRIS project comes to an end, we have identified some promising paths forward and are excited to continue with this highly important work.

Going forward, New Visions will continue to explore the convergence of the Common Core with the development of academic tenacity and ways to support comprehensive school programming aimed at developing students’ noncognitive skills. We are also developing a data warehouse that will allow us to connect many formerly isolated data sets and produce sophisticated and nuanced data analysis to help our schools make informed decisions about programming and resource allocation. We have also begun to examine longitudinal college readiness data, tracking the four-year high school trajectory of individual students at each school in our network.

The data warehouse concept was facilitated by the CRIS project in that it awarded New Visions’ Deputy Director of Research, Brad Gunton, a data fellowship, which enabled him to participate in a number of data convenings. Brad’s learnings through these convenings helped him to understand the critical role that a data warehouse could play in New Visions’ ability to more quickly identify and address school- and student-level challenges to success. The data warehouse will enable New Visions to make comparative data accessible to our schools, unite traditionally separated data strands, and conduct sophisticated analysis over time. Beyond our current capabilities, the warehouse will facilitate the integration of data from multiple sources to encompass academic information, behavioral history, teacher assignments, scheduling and program practices, grades and curricula, and budget and resource allocations. We also will be able to create meaningful tools and user interfaces that coherently aggregate this data to meet the diverse and complex day-to-day needs of New Visions’ organizational staff, school staff, and the external providers that deliver services to our students.

Another promising strand of work around college readiness is the tracking of individual student progress over time, what we refer to as Stock and Flow mapping, developed by New Visions’ director of research and organizational learning, Susan Fairchild.⁵ These maps show how a cohort of students move between the various on-track performance groups over their eight semesters in high school. Large movement, or “flows,” of students from a lower to higher category at a particular point in time can provide evidence that a particular intervention is working. Conversely, large flows of students from higher to lower categories might suggest current programming or structures are not adequately serving students. We have only begun to scratch the surface of what these maps can tell us, and we believe that they have the potential to become powerful tools for sustainably getting more students college ready.