As the Orlando Sentinel’s Leslie Postal pointed out in her article about the FSA exam dilemmas facing Florida’s State Board of Education, setting cut scores and passing rates is a delicate dance that requires fortitude but also a feel for the political situation. Make the exam scoring too easy, and students will not progress. Make the scoring too difficult, and a New York State-style rebellion will break out among parents, teachers and voters (as new Acting US Secretary of Education John King knows all too well).
But Leslie also pointed out a notable success story – Kentucky. Like New York State, Kentucky dove into Common Core-based exams early, but carefully managed its exam results to push students relatively hard while keeping the lid on parent and teacher discontent.
Aside from Kentucky’s political success, the American Institutes for Research (AIR) reported in April that the state’s students had made greater progress since the implementation of the Common Core than they had before implementation. The AIR determined this by examining the state’s ACT results since that test became required for Kentucky 11th graders in 2007 (a dozen states now require that all 11th graders take the ACT).
Coincidentally, the achievement profile of Kentucky’s students in math (as measured by NAEP) is quite similar to that of Florida’s students. In the 2013 NAEP math assessment, 41% of Florida’s 4th graders and 31% of Florida’s 8th graders were “proficient” or better. The corresponding proficiency rate for Kentucky 4th graders was identical to Florida’s – 41%. NAEP said that 30% of Kentucky’s 8th graders were proficient – only an indistinguishable 1% below Florida’s result.
Kentucky’s cut score strategy is illustrated in the graph below, which compares 2014 passing rates for that state’s standardized math exams to the 2014 passing rates for Florida’s FCAT and the passing rates Florida’s Commissioner of Education has recommended for this year’s FSA math exams. The graph also shows the 2013 NAEP math proficiency rates for 4th and 8th graders in Florida that Jeb Bush’s education foundation has set as target passing rates for Florida’s state exams.
There are several things to point out in the graph. First, the Commissioner’s recommendations are very close to the 2014 FCAT passing rates. Commissioner Stewart is basically recommending the status quo. Second, the reader should ignore Florida’s 8th grade test scores, since the strongest third of Florida 8th graders do not take the 8th grade math FSA (nor took the 8th grade math FCAT) because they take (or took) the Algebra 1 or Geometry end-of-course exams. Finally, Kentucky’s passing rates run between 3 and 12 percentage points below Florida’s (if you ignore 8th grade).
So how can Florida take advantage of Kentucky’s successful experience? Perhaps by adopting this “Kentucky Plan”:
- Adopt the ACT as Florida’s standardized high school test for federal accountability purposes so that every 11th grader takes it, as in Kentucky and eleven other states. This way, it wouldn’t be just the psychometricians who can make sense of Florida’s high school test scores. Most parents understand that the ACT is one of the two national yardsticks for student performance. And voters could easily compare the performance of Florida’s students to the achievement of students in other states that require the ACT.
- For 2015, adopt FSA passing rates (at least in math) that are five percentage points below the Commissioner’s recommendations at every grade level. This would be a stopgap measure for one year to bring Florida approximately in line with Kentucky’s passing rates. Why is a stopgap scoring system necessary? Because Florida’s “expert” process for devising scoring systems is compromised. And fixing it is my recommendation number 3…
- For 2016, run the cut score process again, except this time bring out-of-state experts in to write the Achievement Level Descriptions that drive the rest of the process. The Achievement Level Descriptions (ALD’s) for this year’s FSA exams were written last spring by a committee of Florida teachers – teachers who are being evaluated partly on the basis of the test results. A panel of outside experts would almost certainly write more rigorous ALD’s, which would automatically result in the lower pass rates that the Bush education foundation and some State Board of Education want, without the ad hoc tinkering described in recommendation 2.