Every content vendor now advertises a reading grade. It is the easiest number to produce and the easiest to game, and it measures something narrower than almost anyone using it believes.
What a readability score measures
The common formulas are arithmetic on surface features: how many words per sentence, how many syllables per word. Shorten both and the score improves. Nothing in the calculation inspects meaning.
This makes the score genuinely useful as a smoke alarm. A document scoring at university level is almost certainly failing the patient in front of you, and that is worth knowing cheaply.
It makes the score close to useless as a certificate. Passing it tells you the sentences are short. It does not tell you the reader understood the diagnosis, the plan, or the thing that should send them to hospital.
What it misses
Consider the sentence ‘Your ferritin is low.’ Three words, two syllables in the longest one, and an excellent readability score. It is also incomprehensible to most patients, because the difficulty lives in the concept rather than the word length.
Now consider ‘Iron is stored in your body, and the blood test shows those stores have run down.’ Longer sentence, worse score, dramatically better comprehension.
Optimizing the number therefore pushes writing in exactly the wrong direction at the point where it matters most: explaining an unfamiliar concept usually requires more words, not fewer.
The simplification trials are the right experiment
A clinical trial registered as NCT07505719 is comparing original patient educational materials against AI-simplified versions, measured on patient comprehension and health literacy rather than on a formula.
Separately, the AI-CARE protocol describes a double-blind, crossover, non-inferiority randomized trial in which primary care providers and patients compare human- and AI-generated digital health messages.
These are the questions worth funding. Not ‘can a model shorten sentences’, which is settled, but ‘does the shortened version leave the patient knowing what to do’, which is not.
Four things that raise comprehension without moving the score
Put the action before the explanation. Patients read the top of a page and abandon the rest, so the instruction should not be waiting in paragraph four as a reward for persistence.
Use absolute numbers rather than relative ones. ‘Reduces risk by half’ means very different things at a baseline of two in a hundred and twenty in a hundred, and patients cannot recover the difference.
Name the concept before you use it, and describe warning signs as things a person can observe rather than as clinical categories. None of these four changes will improve a readability grade. All of them change what the reader walks away knowing.
How I would use the score
Run it. Treat anything above roughly an eighth-grade level as a prompt to look at the document properly. Then ignore the number entirely while you actually revise.
The revision question is not ‘how do I shorten this sentence’ but ‘what does this person need to be able to do tomorrow morning, and does this page make that obvious’.
A document can pass every formula in use and still leave a newly diagnosed patient with no idea which symptom means call an ambulance. That failure will never show up in the metric, and it is the only failure that matters.
References used in drafting
The registered trial NCT07505719, comparing original patient educational materials with AI-simplified materials on comprehension and health literacy, and the AI-CARE study protocol describing a double-blind crossover non-inferiority comparison of human- and AI-generated digital health messages, informed the discussion of current evidence.
General guidance on readability targets for patient materials, and standard cautions on relative versus absolute risk communication, informed the practical recommendations.
