Engagement
   Student engagement is measured by four outcomes in this section: attendance, discipline, district leavers, and in-district transfers. High attendance rates, low discipline rates, and low mobility rates are indicators of a positive school culture. Pilot School outcomes in these areas are better than BPS averages.
Pilot School students at each level have a median attendance rate that is higher than the BPS student rate, with the distance between the rates increasing with each school level. Pilot high school students attend school on average two weeks more than their BPS counterparts.
The out-of-school suspension rate is an indicator of individual student engagement as well as of the culture of a school. Pilot elementary schools have a suspension rate that is one third that of BPS schools; Pilot middle schools have suspension rates that are slightly lower than BPS schools; and Pilot high schools have a rate that is about one half that of BPS schools.
While student background characteristics are important determinants of a school’s mobility rate, up to half the variability of high school turnover rates can be attributed to school characteristics, including teacher:student ratio, quality of teachers, class size, and average daily attendance.2 High student mobility has been “highly associated with a low level of student performance” at all levels.3 Therefore, school characteristics influence mobility rate, and a high mobility rate leads to lower student engagement and performance.
The district leaver indicator includes students who transferred out of the district or dropped out of school. The in-district transfer rate includes all students who transferred from one school in the district to another for any reason other than completing the highest grade available at a given school. The district leaver and in-district transfer rates speak to the holding power of a school and the stability of the school population; taken together, they serve as proxies for a school’s mobility rate. On average, Pilot Schools have a lower district leaver rate than BPS, and the Pilot School in-district transfer rate is one third that of BPS.

Performance
   Student performance is measured by four indicators: grade-level retention, post-secondary education plans, post-secondary education participation, and MCAS scores. As a whole, Pilot School students are performing better than the BPS average on all four performance indicators. As with the engagement indicators, the largest differences between the two types of schools are at the high school level. Two of these indicators, college-going rates and MCAS scores, are described here.
   A measure of success for any high school is the number of its graduates who are in post-secondary education. The numbers reported below represent the percentages of graduates who were in post-secondary education—including four-year colleges, two-year colleges, and technical or vocational programs—one year after graduation. The proportion of Pilot School graduates enrolled in post-secondary education one year after graduation is 18% higher than that for BPS graduates. Five out of six Pilot high schools are at or above the BPS average; the sixth school serves students who have previously dropped out or been unsuccessful in BPS high schools. The school with the same average as BPS is Boston Community Leadership Academy (BCLA), a new conversion Pilot School whose 2003 graduates experienced only the transition year to Pilot status.

Massachusetts Comprehensive Assessment System (MCAS) exams are criterion-referenced tests administered by subject to students across the state.* The MCAS is used as one measure of student performance. It is a high-stakes test in the 10th grade: students must pass the 10th-grade exams in order to graduate from high school.
All tests administered for reading, English/Language Arts (ELA), and mathematics in 2004 were analyzed using two criteria: percent achieving advanced/proficient status and percent passing. Advanced/proficient equals the proportion of students at either of those achievement levels, and passing equals the proportion of students in the advanced, proficient, and needs improvement categories.
Pilot School students outperformed BPS school students at all grade levels on all tests, and at both the passing and advanced/proficient levels. As a whole, the proportion of Pilot School students in the advanced/proficient categories ranged from 10 to 34 percentage points higher than BPS students. The proportion of Pilot School students passing each exam ranged from 3 to 27 percentage points higher than BPS students. The following graphs show MCAS results for grade 8 math and grade 10 ELA.

Summary
The research findings show that Pilot School students are faring well on a wide range of indicators of engagement and performance. The data confirms previous reports and demonstrates consistently positive findings for Pilot School students for the years 1997–2004.4

* For a full report see Progress and Promise: Results from the Boston Pilot Schools, January 2006, Center for Collaborative Education, http://www.ccebos.org/pubslinks.html

* This calculation excludes one outlier school. Boston Day and Evening Academy students attend school four days a week, for an average of 300 minutes per day because they are older students who have other responsibilities such as work and parenting.

* MCAS Position Statement, Board of Directors of the Center for Collaborative Education: “The Center for Collaborative Education affirms that the current MCAS (Massachusetts Comprehensive Assessment System) is a test and not a comprehensive assessment system, that a single score on a test should never stand as the sole measure of a student’s knowledge, understandings, performance, and intellectual habits, that the use of a test for high stakes decisions is not educationally defensible, and that more appropriate accountability systems are possible. With this in mind, the Center should begin working with a cross section of schools and districts to develop a more comprehensive system of assessment to demonstrate the richer possibilities for supporting good education and accountability.” http://www.ccebos.org/ mcasposition.html

Click here for printable page