Assessment Results Minus Admin Spin

Guest contribution by Arthur Green

Your school district made two important press releases this week touting the academic achievement of our students.  While champagne corks pop at the SAU office, let’s look at the results in context.

Here is what the district disclosed about 11th Grade SAT scores:

The Timberlane Regional School District is proud to announce that the percentage of Grade 11 students who met or exceeded benchmarks on state accountability testing in ELA increased from 59% in 2015 to 66% in 2016, and from 37% to 40% in Mathematics. Timberlane outperformed this year’s state average in Mathematics and surpassed several surrounding towns in Mathematics and English Language Arts.

Keep in mind:

  • In 2014 Grade 11 students wrote NECAP examinations
  • In 2015 they wrote Smarter Balanced tests
  • In 2016 they wrote revised SAT exams.

This means that there is no apples to apples comparison for academic progress for the last three years.

It is not even meaningful to compare this year’s SAT scores with those of previous writers of the SAT, because the SAT exam was totally revised for Common Core and with knowledge that it would be used as a universal standard rather than a measure of college readiness.

Since the evaluation of “proficiency” uses different methods from year to year, we can’t just look at 66% of students proficient this year compared to 59% proficient last year – the measuring stick has changed.  However, we can try to calibrate the yardstick by looking at the proficiency rate compared to state average for that year.

So for example, 65% of Timberlane Grade 11 students rated proficient or better in English Language arts, compared to a state average of 66%.  So Timberlane’s proficiency score was 98% of the state average.  By comparison, last year’s rating of 50% proficient (TRSD press release claimed 59% – see end note) compared to the state average of 59%, so Timberlane’s proficiency rate was 85% of the state average.  We can graph Timberlane’s performance compared to the state average:

proficiency-as-percent-of-state-average

This begins to put context to the results just announced – the scores have rebounded from a two-year decline, and almost come back to the high scores of 2013.  For most of this period, Timberlane has been below the state average.

Another way of visualizing this history is to plot it on a “scatter” chart.  English scores are plotted on the x-axis, and math scores on the y-axis.  Each axis represents the state average.  Here’s the history of Timberlane’s grade 11 assessments:

timberlane-proficiency-history

The lower left quadrant is below state average in both English and Math.  In 2013, the students scored above average in both.  The 2016 results are a definite improvement.

Now let’s look at Timberlane results in context with similar districts.  There are 9 other districts in NH which have the same general scale of enrollment as Timberlane, and which manage all grades pre-K through 12, each with a single high school and its feeder schools.  Those districts are shown on this scatter chart:

scatter-chart-of-2016-assessment-10-districts

Three districts – Bedford, Merrimack and Londonderry, exceed state averages in both English and Math.  Four districts – Concord, Keene, Dover and Rochester, fall below state averages in both.  Timberlane, Hudson and Salem exceed the state average in Math, but fall below in English.  So in context of similar districts, there are certainly those with clearly weaker results, but there are also districts that are much stronger.

This scatter chart showed only the 2016 result.  What about the history of the comparable districts?  The chart would become unreadable if it had a similar “scribble” of historic results as we saw above for Timberlane.  So here is a chart showing two data points for each district – the 20-16 assessment result, and the prior 6-year average:

chart-of-assessment-history-10-districts

The triangle represents the average, and the box represents the same 2016 result we saw on the previous chart.  So we would like to see movement from the lower left toward the upper right.  Three districts show that favorable movement – Timberlane, Bedford and Londonderry.  Rochester and Concord both show the opposite, unfavorable movement on both English and Math assessments.  Merrimack, Salem, Hudson, Keene and Dover have shown some improvement in Math but are losing ground in English.  And Merrimack, despite losing some ground, is still above state average in both areas.

Now that we have reviewed the results, what about the resources that these districts have drawn from their communities?  As it happens, Timberlane’s Student Teacher Ratio of 11.4 is the second-highest of all the districts on this list except Keene at 11.3.

The three districts in the upper right quadrant all have lower staffing than Timberlane:

  • Bedford:  14.1 students per teacher
  • Londonderry:  12.8
  • Merrimack:  12.1

When we have the 2016 Cost Per Pupil figures later this fall, we will pull that into the analysis.

We all agree that we want our students to have a strong education.

We all agree that we want to see improved academic results.

We all agree that the assessment results for 2016 show improvement.  We desire that improvement and we want more of it.

In the context we have shown above, the results do not merit a standing ovation.  And more specifically, we do not view this year’s modest improvement as a vindication of the excessive staff level and budget.  Other districts are clearly capable of delivering better results at a lower staffing level and lower cost.

NOTE: The Timberlane district press release states that the percentage of students meeting or exceeding proficiency on the 2015 Grade 11 ELA assessment was 59%.  I have used the figures from the NH Department of Education website, which has at least two independent pages showing the correct number is 50%, not 59%.  As it happens, 59% was the state average that year.  If I made that mistake I would be keelhauled by the administration and its supporters.

Advertisements

1 Comment

Filed under Sandown Issues

One response to “Assessment Results Minus Admin Spin

  1. Pingback: Timberlane Test Scores Nothing to Hoot About | timberlaneandsandown

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s