On testing and accountability, Florida should adopt the Kentucky Plan

As the Orlando Sentinel’s Leslie Postal pointed out in her article about the dilemmas facing Florida’s State Board of Education regarding FSA exams, setting cut scores and passing rates is a delicate dance that requires fortitude but also a feel for the political situation.  Make the exam scoring too easy, and students will not progress.  Make the scoring too difficult, and a New York State-style rebellion will break out among parents, teachers and voters (as new Acting US Secretary of Education John King knows all too well).

But Postal also pointed out a notable success story – Kentucky.  Like New York State, Kentucky dove into Common Core-based exams early, but carefully managed its exam results to push students relatively hard while keeping the lid on parent and teacher discontent.

Aside from Kentucky’s political success, the American Institutes for Research (AIR) reported in April that the state’s students had made greater progress since the implementation of the Common Core than they had before implementation.  The AIR determined this by examining the state’s ACT results since that test became required for Kentucky 11th graders in 2007 (a dozen states now require that all 11th graders take the ACT).

Coincidentally, the achievement profile of Kentucky’s students in math (as measured by NAEP) is quite similar to that of Florida’s students.  In the 2013 NAEP math assessment, 41% of Florida’s 4th graders and 31% of Florida’s 8th graders were “proficient” or better.  The corresponding proficiency rate for Kentucky 4th graders was identical to Florida’s – 41%.  NAEP said that 30% of Kentucky’s 8th graders were proficient – only an indistinguishable 1% below Florida’s result.

Kentucky’s cut score strategy is illustrated in the graph below, which compares 2014 passing rates for that state’s standardized math exams to the 2014 passing rates for Florida’s FCAT and the passing rates Florida’s Commissioner of Education has recommended for this year’s FSA math exams.  The graph also shows the 2013 NAEP math proficiency rates for 4th and 8th graders in Florida that Jeb Bush’s education foundation has set as target passing rates for Florida’s state exams.


There are several things to point out in the graph.  First, the Commissioner’s recommendations are very close to the 2014 FCAT passing rates.  Commissioner Stewart is basically recommending the status quo.  Second, the reader should ignore Florida’s 8th grade test scores, since the strongest third of Florida 8th graders do not take the 8th grade math FSA (nor took the 8th grade math FCAT) because they take (or took) the Algebra 1 or Geometry end-of-course exams.  Finally, Kentucky’s passing rates run between 3 and 12 percentage points below Florida’s (if you ignore 8th grade).

So how can Florida take advantage of Kentucky’s successful experience?  Perhaps by adopting this “Kentucky Plan”:

  1.   Adopt the ACT as Florida’s standardized high school test for federal accountability purposes so that every 11th grader takes it, as in Kentucky and eleven other states.  This way, it wouldn’t be just the psychometricians who can make sense of Florida’s high school test scores.  Most parents understand that the ACT is one of the two national yardsticks for student performance.  And voters could easily compare the performance of Florida’s students to the achievement of students in other states that require the ACT.
  2. For 2015, adopt FSA passing rates (at least in math) that are five percentage points below the Commissioner’s recommendations at every grade level.  This would be a stopgap measure for one year to bring Florida approximately in line with Kentucky’s passing rates.  Why is a stopgap scoring system necessary?  Because Florida’s “expert” process for devising scoring systems is compromised.  And fixing it is my recommendation number 3…
  3. For 2016, run the cut score process again, except this time bring out-of-state experts in to write the Achievement Level Descriptions that drive the rest of the process.  The Achievement Level Descriptions (ALD’s) for this year’s FSA exams were written last spring by a committee of Florida teachers – teachers who are being evaluated partly on the basis of the test results.  A panel of outside experts would almost certainly write more rigorous ALD’s, which would automatically result in the lower pass rates that the Bush education foundation and some State Board of Education want, without the ad hoc tinkering described in recommendation 2.
Posted in Uncategorized

What percentage of Florida’s students are on track for 21st century career options? Isn’t that the question the FSA should be answering?

Commissioner Stewart’s recommendation that between 50% and 60% of Florida’s students pass the FSA math exams (except for 8th grade – ignore that for now and I’ll get back to it) has set her up for a confrontation with Jeb Bush’s education foundation and some State Board of Education members, who believe that the percentage of students who pass the exams should be closer to the 30-40% range.  That latter range corresponds to the percentages of Florida students deemed “proficient” on the 2013 National Assessment of Educational Progress (NAEP).

If it were up to me, I’d ask a different question:  What percentage of Florida’s students are on track to have a full range of college and career options when they graduate from high school?  In answering this question, let’s not just focus on the students who should be prepared to successfully pursue bachelors’ degrees in engineering, physical sciences and computer science – and the entire top third of high school students should be prepared for these options.  Instead, let’s broaden our view on this:  What percentage of high school graduates are prepared to be successful in an associate degree program in engineering technology?  Students who graduate from such programs earn starting salaries in the $40,000’s (which most state university system grads can only dream of).

According to the American Society for Engineering Education, the proper high school preparation for an associate degree program in engineering technology includes a Precalculus course (as well as chemistry and physics, but I’m writing about math today).  If Florida’s K-12 schools were running on all cylinders, two-thirds of high school grads would be properly prepared to pursue an associate degree in engineering technology – and that would mean that two-thirds of Florida’s high school grads would have taken Precalculus.

What’s the actual percentage of Florida high school grads who have taken Precalculus?  To estimate that, we divided the number of students enrolled in a Precalculus course in the Spring 2015 by the number of 12th graders at that time.  And the answer was (wait for it) 25%.  To put on my Captain Obvious (@CaptainObvious on Twitter) hat for a moment:  25% is not close to two-thirds.

So what should policy-makers (which in this case is the State Board of Education) do about FSA cut scores?  For the moment, let’s set aside the technical problems that I believe have undermined the process used by the Florida Department of Education to recommend cut scores to the Board.  By my reckoning, teachers, parents and students (who collectively decide who takes Precalculus) believe that only 25% of high school graduates are properly prepared for an associate degree program in engineering technology.  If passing the FSA math exam means that a student is on track for such a program, then passing 50% of 6th and 7th graders – as Commissioner Stewart is recommending – probably doesn’t make any sense.  In this context, the NAEP result that only 31% of Florida 8th graders are proficient in math starts to seem like a more reasonable FSA passing rate for Florida’s middle school students.

Back to the 8th grade FSA math exam:  Close to one-third of Florida’s 8th graders take Algebra 1 (30.4% of Algebra 1 end-of-course exam-takers in Spring 2014 were 8th graders), and a smaller percentage takes Geometry (7.2% of Geometry end-of-course exam-takers in Spring 2014 were 8th graders).  These students take the end-of-course exams for those subjects and do not take the 8th grade math FSA.  So more than one-third of 8th graders – and that’s the strongest one-third – do not take the 8th grade math FSA.  Therefore, the passing rate on that test should be lower than the 6th and 7th grade exams.

Posted in Uncategorized

Precalculus in Florida’s high schools: Are your district’s students being prepared properly for 21st century careers?

With all the debate over what percentages of Florida students should be allowed to pass the state’s new FSA standardized exams, we keep forgetting why we are testing them in the first place.  The reason we test our students is to see if they are on track to have a full range of college and career options when they graduate from high school.  Or at least that should be the reason.

And in the modern economy, the best career options are those in the science and engineering fields.  Sure, not every student is cut out for a bachelor’s degree in engineering, physics or computer science.  If our educational system is doing its job properly, then every student in the top third of our high school graduating classes should have the preparation necessary to succeed in a college major in one of those fields.  And according to the American Society for Engineering Education (ASEE), that means students should have successfully completed courses in chemistry, physics and precalculus – and preferably calculus.

But what about the other two-thirds of students?  Most of those students have access (or should have access) to associate degree-level programs in engineering technology and programming, which yield starting salaries in the mid-$40,000’s.  They don’t need physics and precalculus, do they?  Think again:  Here’s the full text of the statement by the ASEE Board of Directors on the subject:

The ASEE Board endorses the recognition that academic preparation for high school students hoping to complete a college degree in engineering or engineering technology should include a full year each of chemistry, physics, and mathematics. The year of mathematics should be at least to the pre-calculus level; a year of calculus is preferred. [Emphasis mine]

The ASEE Board included engineering technology in its statement.  That is, if we are preparing students for associate-level degrees in engineering technology – and we should be doing so for all students except those with significant learning disabilities – then that group needs chemistry, physics and precalculus.

Precalculus should not be a capstone math course given to a minority of students.  Instead, it should be the standard senior math course for the majority of Florida’s high school students.

At present, it’s not.  In the spring of 2015, 47,476 Florida public high school students were taking a precalculus course (Honors Precalculus, IB Precalculus, AICE AS Math, and dual enrollment college precalculus).  There were 189,459 12th graders.  So about only a quarter of Florida high school students presently graduate with precalculus.  Therefore, Florida is doing poorly even in just preparing students for associate degree-level STEM fields. (Kudos again to the Florida Department of Education for doing the best job in the nation in providing access to data like that used here.)

But what’s even more striking is that there is tremendous variation in the emphasis that Florida’s school districts put on getting students into Precalculus classes.  This remarkable variation is shown here in a ranking of the districts by Precalculus-taking rates, which we define to be the number of students enrolled last spring in the Precalculus class divided by the number of 12th graders:


Now consider this:  This ranking is topped not by the usual Florida math and science powerhouses like Brevard, Leon and Seminole Counties.  Instead, four of the top five districts are rural:  Glades, Hamilton, Taylor and Liberty.  Of course, there are plenty of rural counties at the bottom as well – Washington, Lafayette, Jefferson and Calhoun Counties did not have a single student enrolled in a precalculus course.

It’s worth noting that several Florida districts steer large numbers of students into trigonometry courses instead of precalculus courses, which generally include trigonometry as well as additional skills having to do with graphing functions on the xy plane and working with transcendental functions.  Most of the districts that steer large proportions of their students into trig courses are small – Calhoun, Gulf, Highlands, Holmes, Liberty, Monroe, and Washington.  Several of these districts dual enroll students in the college level Analytic Trigonometry course, which is only two credit hours and thus would be a very leisurely one-year high school course.

If Florida’s schools, students and parents are deciding that only a quarter of the state’s students should be taking Precalculus, then we can also conclude that three-quarters of the state’s students are not properly prepared for the 21st century economy.  And addressing that reality should be a high priority for the state’s school districts, no matter what the cut scores for the FSA math tests are.

Posted in Uncategorized | 1 Comment

Does a school district’s success in teaching math depend on socioeconomics? Sure, but there’s more to it than that.

Yesterday’s release of FSA data (posted at School Zone and Gradebook) in which the numbers of students in each school and district scoring in each of Florida’s quartiles shows there was considerable variation from school to school and district to district.  The correlation between student achievement and socioeconomics is well known and can be seen clearly in the plot below, in which the percentage of each district’s students scoring in the state’s top two quartiles (or the state’s top 50%) on the grade 3-8 math exams is graphed against the percentage of students eligible for free and reduced price lunches.


The highest scoring district in the graph is also the state’s only truly affluent district, St. Johns.  The lowest scoring district – with fewer than 20% of its students making it into the state’s top half – is Jefferson County.

But it’s worth noting that even this simple graph demonstrates that demographics is not destiny.  Little Dixie County, with its free and reduced prince lunch rate (FRL) near 100% gets almost half of its students into the state’s top half.  Another rural district, Union, gets 61% of its students into the state’s top 50% despite a 64% FRL.  Miami-Dade County, with an FRL of 75%, has 48% of its students in the state’s top half.  Citrus County, with FRL=66%, has 57% of its students in the state’s top 50%.

Posted in Uncategorized

Why the Biology end-of-course exam requirement is bad for Florida high school students

Under the No Child Left Behind Act, the federal government requires that high school students take one statewide standardized test in science.  In Florida, that test is the Biology end of course examination.  Yesterday, I argued that the Biology end of course examination should be replaced in Florida by the ACT Science Section, and in fact that the ACT should be Florida’s statewide standardized test for federal accountability purposes.  The ACT is presently used this way by a dozen states, and its science section includes physical sciences, which are presently excluded from Florida’s high school testing program.

The plot below shows one reason why Florida should dispose of the Biology end of course exam and instead shift its high school science emphasis toward the physical and computer sciences – salaries.


According to the Georgetown Center on Education and the Workforce, biology bachelors’ degree recipients actually earn less than the average for all degrees (not just all STEM degrees, but all degrees).  Chemistry majors earn significantly more than both biology majors and the all-major average.  Physics majors do considerably better – in fact physics is ranked 15th among all college majors in the Georgetown salary report.  And of course, this ignores engineering, which dominates the Georgetown rankings (17 of the top 25 majors are engineering majors) and which is built on a foundation of physics.  However, fewer than 25% of Florida students take physics in high school – a rate that is far below the national average of 39%.

Then there is computer science, which is the elephant in the STEM room.  Computer science is 11th in the Georgetown salary rankings, but the demand for computer science graduates is astronomical.  At least at FSU, two semesters of physics are required for a bachelor of science degree in computer science.

Does high school course-taking really matter in career decisions?  Of course it does.  A student who doesn’t take a high school physics course is not going to major in physics, and is quite unlikely to major in engineering (and is going to run into headwinds if she or he chooses to major in engineering despite the lack of high school physics).

So…Florida should adopt the ACT as its standardized high school test for accountability purposes.  The need to shift the state from its high school science emphasis from life science to the physical sciences is just one reason to do so.

Posted in Uncategorized

Commissioner Stewart sticks with panel recommendations (within an error bar) for FSA cut scores and passing rates, setting up confrontation with State Board members


As expected, Florida Education Commissioner Pam Stewart has recommended cut scores and passing rates for the FSA exams in English and math that are very close to those recommended by the Educator and Reactor panels (see the School Zone report here and the FLDOE media release here).  Her recommendations now position her in conflict with several members of the State Board of Education and Jeb Bush’s Foundation for Excellence in Education, who want higher cut scores and therefore passing rates more in line with the proficiency rates on the National Assessment of Educational Progress (NAEP; see graph above).

In justifying her recommendations, Commissioner Stewart cited the work of three panels – the Achievement Level Descriptions panel that sets expectations for how well students at different achievement levels should perform generally, the Educator Panel that applies the Achievement Level Descriptions to the exams given last spring and comes up with cut score recommendations, and the Reactor Panel composed of state leaders who essentially comment on the Educator Panel results.  Both the Achievement Level Descriptions panel and the Educator Panel are composed of Florida teachers who, along with their colleagues, are evaluated partly on the basis of the exam results.

The State Board of Education has final say on setting cut scores, and the Vice Chair of the board has made clear his desire to toughen the grading of the exam.  The State Board will vote on cut scores at its January meeting.

Posted in Uncategorized | 1 Comment

Why Florida should adopt the ACT as its official high school accountability test

With Senators Gaetz and Legg looking at the possibility of changing high school test requirements (see the Tampa Bay Times report), it’s worth briefly reviewing why Florida should make the ACT the official state test for accountability purposes:

  1. The ACT and the SAT are the tests the world looks at when making judgments about how strong a student is.  And the world doesn’t care whether the ACT and SAT perfectly reflect Florida’s state standards in English and math – even though those tests are awfully close to doing so.
  2. If 100% of Florida’s students took the ACT (and 79% already do), parents and voters could very easily compare the performance of Florida’s students to that of students from the other dozen states where the ACT is required.  Even though not all of Florida’s students take the ACT, it’s illuminating to see where the 79% of students who do take that test stand in comparison to the 13 states where the high school graduating class of 2015 was required to take the ACT:  tied for first in reading (with Colorado and Montana); but behind ten of those states (Colorado, Illinois, Kentucky, Louisiana, Michigan, Montana, North Dakota, Tennessee, Utah and Wyoming) on writing; behind seven of those states (Colorado, Illinois, Michigan, Montana, North Dakota, Utah and Wyoming) in math; and behind nine of those states (Colorado, Illinois, Kentucky, Michigan, Montana, North Dakota, Tennessee, Utah and Wyoming) in science.
  3. For all the concern about how well the ACT aligns with Florida’s standards, it’s worth noting that the official high school science test in Florida is the Biology end-of-course exam.  This exam certainly aligns with the high school biology standards written by the state’s science standards committee back in 2007-2008 (I was a member of that committee).  But it ignores the other two-thirds of the high school science standards – those in physical and Earth/space sciences.  The ACT covers those other science fields, so it is accurate to say that the ACT reflects Florida’s high school science standards much better than our state’s present official high school science test for accountability.  One more thing:  Why have I focused on the ACT instead of the SAT for items 2 and 3?  Because the SAT doesn’t have a science section, and the ACT does.  Only the ACT can save Florida from the Biology end-of-course exam.
Posted in Uncategorized | 1 Comment