Ranking the states on NAEP – while taking socioeconomics into account

In their reporting on the 2013 NAEP results, State Impact Florida and redefinedonline.org have tried to take into account the differences in socioeconomic status of students in Florida and the NAEP leaders such as Massachusetts.  The percentage of Florida public school students who are eligible for free or reduced lunch was 56.0% in 2010-2011 (the last year for which I could find a national compilation) while in Massachusetts the corresponding number was 34.2% (the national rate was 48.1%, which I find staggering).  State Impact Florida and redefinedonline.org have tried to illustrate this by quoting numbers on the results for different student groups and different states.

I thought I’d try something different – a parameter that allows a direct comparison between states, and a ranking of the states.  I’ve dubbed it the Socioeconomic NAEP Achievement Parameter (SNAP), and it is based on the idea that the higher the affluent student population is, the higher the percentage of students attaining proficiency in a subject tends to be.  So the parameter is the quotient of the percentage of students achieving proficiency on NAEP and the percentage of students not eligible for free or reduced lunch (or 100 minus the percentage of students eligible for free or reduced lunch).  I multiply the quotient by 100 to give a whole number instead of a decimal less than one.

As an example, consider Florida and Massachusetts on the NAEP 4th grade reading exam.  On this exam, 39% of Florida students achieved proficiency, while 48% of Massachusetts students did so.  However, Florida’s free and reduced lunch percentage is 56.0%, while for Massachusetts it is 34.2%.  So Florida’s SNAP for 4th grade reading is given by

SNAP(FL; 4th grade reading) = 100*39/(100-56.0) = 89

while the corresponding value for Massachusetts is

SNAP(MA; 4th grade reading) = 100*48/(100-34.2) = 73

By this metric, Florida is doing a much better job than Massachusetts in 4th grade reading, if socioeconomics are considered.  In fact, Florida is ranked first in the nation in 4th grade reading according to this metric.  Massachusetts is ranked tenth.

Now let’s look at 8th grade math.  Florida has a proficiency rate of 31%, while the Massachusetts rate is 54%.  The SNAP results for Florida and Massachusetts on this exam are 70 and 82, respectively.  Massachusetts not only has the nation’s highest proficiency rate for 8th grade math, but also the highest SNAP score – they are doing as great a job with their student population in 8th grade math as Florida is in 4th grade reading.  The allegation that Massachusetts has great results in 8th grade math because their students are affluent is simply not true.  Florida’s ranking in 8th grade math is 12th – the state’s lowest SNAP ranking.

In 8th grade reading, Florida ranks third on SNAP.  The state’s 4th grade math ranking is 10th.  These results reflect Florida’s strong focus on reading during the last fifteen years (“Just Read, Florida!”) and its relative lack of emphasis on math.  (Let’s not even talk about science here)

While Florida is ranked first in 4th grade reading and Massachusetts is tops in 8th grade math, the District of Columbia is first in 4th grade math and Kentucky is number one in 8th grade reading (go figure).

I will note for the record that in devising SNAP I was anticipating that Florida would be at or close to the top in 4th grade reading and Massachusetts in 8th grade math.  When the results came out this way, I was pleased.  Education insiders may find this amusing given our state’s recent history.

Here is the full spreadsheet.  Sort away!


This entry was posted in Uncategorized. Bookmark the permalink.