Why some Florida Board of Education members are upset with the recommendations for FSA test scoring

Why are some Florida State Board of Education members convinced that the FSA scoring scales proposed by two panels – one made up of teachers (the “educator panel”) and the other made up of representatives from business and higher education (the “reactor panel”) – are too soft on students? (Reports from the Orlando Sentinel and News Service of Florida)

Here is why, at least for math.  The plot below compares the percentage of students who would pass the FSA math tests for grades 3-8 under the scoring scales proposed by the educator and reactor panels to the percentages of students who achieved at the proficient level or above on the 2013 National Assessment of Educational Progress (NAEP), which measures students at grades 4 and 8 and is considered the gold standard.

To summarize the plot:  The scoring scales proposed by the FSA educator and reactor panels would label between 50% and 60% of Florida’s elementary and middle school students as proficient, while NAEP says that only 41% of Florida’s elementary students and 31% of the state’s middle school students are proficient or better.

Florida’s NAEP proficiency rates are at or below the national averages, so the state has ground to make up to be competitive with the top tier states.  And excusing Florida’s students from the national (and international) competition, as the educator and reactor panels are proposing, will not help.

So would it make sense for the State Board of Education to simply recommend lower passing rates for the FSA?  That is problematic, and here is why:  The FSA scoring scale recommendations produced by the educator panel and endorsed by the reactor panel came out of a process that is built to resemble the rigorous process used to set scoring scales for Advanced Placement exams and other national and state exams.  I’ve been involved in such “standard-setting” processes for both Advanced Placement and Florida exams.

So when Commissioner Stewart comes to the Board with her recommendations next month, they are certain to be close to the soft scoring scales proposed by the educator and reactor panels, and she is certain to say that she feels obligated to those scales because they are the result of a research-based process.  If the Board overrules Stewart, it will look as if the Board members are prioritizing their personal agendas over good education policy.

But here is what is wrong with the process being used to set FSA scoring scales:  The first and most important step in this process was compromised by a conflict of interest.  This first and foundational step, which was conducted at the end of April, is for a committee to specify “Achievement Level Descriptions” that describe what students at the different FSA scoring levels (1, 2, 3, 4 or 5) can do.  All of the scoring done by the educator panel is based on these Achievement Level Descriptions.  If the Achievement Level Descriptions are not ambitious enough, then the scoring scale is certain to be too soft.  And the FSA Achievement Level Descriptions are written by Florida teachers – who with their colleagues are going to be evaluated by the FSA.  It is a classic conflict of interest, with those writing the Achievement Level Descriptions having little reason to make them appropriately ambitious.

So what should be done by the State Board of Education members who are concerned about the FSA scoring scales that have been recommended by the Educator and Reactor Panels?  Propose scrapping the work that has been done this year.  Then start again next year, with an Achievement Level Description panel that is composed of national experts instead of Florida teachers.  A panel of national experts would almost certainly produce a much more ambitious product than this year’s panel did.  And the resulting FSA scoring scales would reflect the Board’s interest in pushing Florida’s students toward the nation’s top tier.