In this age of word-counting and laughter-tabulating for clicks, there are worse things you could do than assess the reading level of a president’s State of the Union address. Politico is pushing a study that shows that President Obama’s three SOTU speeches were written at an 8th-9th grade reading level, based on the Flesch-Kincaid standard. Aside from deciding what conclusions to draw from this, is this measure any less arbitrary than word-counting?
President Obama’s 2012 State of the Union address again rated at an 8th grade comprehension level on the Flesch-Kincaid readability test — the third lowest score of any State of the Union address since 1934.
The University of Minnesota’s Smart Politics conducted an analysis on the last 70 State of the Union addresses and found that President Obama’s three addresses have the lowest grade average of any modern president. “Obama‘s average grade-level score of 8.4 is more than two grades lower than the 10.7 grade average for the other 67 addresses written by his 12 predecessors,” they conclude.
The study also notes that writing to a lower grade level isn’t necessarily a bad thing. The average American reads at an 8th or 9th grade level, right in the President’s SOTU wheelhouse, so as the audience for the address has shifted over time, from Congress to the American people, attenuating the speech to that audience is not only wise, it’s egalitarian (although it still leaves out the entire audience of Wheel of Fortune).
You could also draw the conclusion, if you were so inclined, that the President is talking down to people. If this result had been for George W. Bush‘s SOTU addresses, his detractors would doubtless have concluded he wasn’t capable of more.
On its own, however, the entire exercise seems arbitrary: (empasis mine)
“The Flesch-Kincaid test is designed to assess the readability level of written text, with a formula that translates the score to a U.S. grade level. Longer sentences and sentences utilizing words with more syllables produce higher scores. Shorter sentences and sentences incorporating more monosyllabic words yield lower scores,” the University of Minnesota’s Eric Ostermeier explains.
By this measure, the song Supercalifragilisticexpialadocious would probably clock in at around Stephen Hawking’s grade level. By contrast, The Art of War is written in very short sentences, with few syllables, but I wouldn’t want to hand it to a third-grader. The study itself notes some of the ways its score can be skewed:
Dragging down Obama’s Flesch-Kincaid score are the series of short sentences written – perhaps for dramatic effect – on a number of policy issues.
On the state of the American auto industry:
“We bet on American workers. We bet on American ingenuity. And tonight, the American auto industry is back.”
On the need to rebuild the nation’s infrastructure:
“So much of America needs to be rebuilt. We’ve got crumbling roads and bridges. A power grid that wastes too much energy.”
On the importance of passing a payroll tax break extension:
“There are plenty of ways to get this done. So let’s agree right here, right now: No side issues. No drama. Pass the payroll tax cut without delay.”
On remembering the mission to kill Osama Bin Laden:
“All that mattered that day was the mission. No one thought about politics. No one thought about themselves.”
You get the idea.
By themselves, tools like this scale, or word-counting/clouding, are of little, if any, value. The true effect of a speech like this is difficult to measure, and even harder to convey.
Have a tip we should know? firstname.lastname@example.org