One of the presenters at
Metricon 5.0 was comparing IT security to other fields
in various aspects of metrics and monitoring.
I mentioned I thought she was giving far too much green for good to
the field of medicine.
This provoked repeated back and forth later.
My point was that 150 years after the invention of epidemiology
and 100 years after the discovery of bacterial transmission of disease,
in medicine application of known preventive measures is so low that
Atul Gawande of Harvard
has gotten large (on the order of 30%) reductions in deaths from complications
of surgery in many hospitals simply by getting them to use checklists for
things like washing hands before surgery.
I have an elderly relative in a nursing home who can’t take pills whole
due to some damage to nerves in her neck.
Again and again visitors sent by the family discover nursing home staff
trying to give her pills whole without grinding them up.
Why?
They don’t read instructions about her, and previous shifts don’t remind
later shifts.
This kind of communication problem is epidemic not only in nursing homes
but in hospitals.
I found my father in a diabetic coma because nurses hadn’t paid any attention
to him being a diabetic and needing to eat frequently.
Fortunately, a bit of honey brought him out of it.
Even nurses readily acknowledge this problem, but it persists.
I can rattle off many other examples.
To which someone responded, yes, but medicine has epidemiology,
and Edward Tufte demonstrated in one of his books that that goes
well beyond checklists in to actual analysis, as in a physician’s
discovery of a well in London being he source of cholera.
I responded, yes, John Snow, in 1854:
that was the first thing I said when I stood up to address this.
But who now applies what he learned?
One-shot longitudinal studies are not the same as ongoing monitoring
with comparable metrics to show how well one group is doing compared
to both the known science and to other groups.
Many people still didn’t get it, and kept referring to checklists
as rudimentary.
So I tried again.
If John Snow were alive today, he wouldn’t be prescribing statins for
life to people with high blood pressure.
He would be compiling data on who has high blood pressure and what
they have been doing and eating before they got it.
He would follow this evidence back to discover that one of the main
contributors to high blood pressure, heart disease, and diabetes
in the U.S. is
high fructose corn syrup (HFCS).
Then he would mount a political campaign to ban high fructose corn syrup,
which would be the modern equivalent of his removal of the handle from
the pump of the well that stopped the cholera.
To which someone replied, but there are political forces who would oppose that.
And I said, yes, of course.
Permit me to elaborate.
There were political forces in John Snow’s time, too, and he dealt with them:
Dr Snow took a sample of water from the pump, and, on examining it under a microscope, found that it contained “white, flocculent particles.” By 7 September, he was convinced that these were the source of infection, and he took his findings to the Board of Guardians of St James’s Parish, in whose parish the pump fell.
Though they were reluctant to believe him, they agreed to remove the pump handle as an experiment. When they did so, the spread of cholera dramatically stopped. [actually the outbreak had already lessened for several days]
Snow also investigated several outliers, all of which turned out to involve
people actually travelling to the Soho well to get water.
Still no one believed Snow. A report by the Board of Health a few months later dismissed his “suggestions” that “the real cause of whatever was peculiar in the case lay in the general use of one particular well, situate [sic] at Broad Street in the middle of the district, and having (it was imagined) its waters contaminated by the rice-water evacuations of cholera patients. After careful inquiry,” the report concluded, “we see no reason to adopt this belief.”
So what had caused the cholera outbreak? The Reverend Henry Whitehead, vicar of St Luke’s church, Berwick Street, believed that it had been caused by divine intervention, and he undertook his own report on the epidemic in order to prove his point. However, his findings merely confirmed what Snow had claimed, a fact that he was honest enough to own up to. Furthermore, Whitehead helped Snow to isolate a single probable cause of the whole infection: just before the Soho epidemic had occurred, a child living at number 40 Broad Street had been taken ill with cholera symptoms, and its nappies had been steeped in water which was subsequently tipped into a leaking cesspool situated only three feet from the Broad Street well.
Whitehead’s findings were published in The Builder a year later, along with a report on living conditions in Soho, undertaken by the magazine itself. They found that no improvements at all had been made during the intervening year. “Even in Broad-street it would appear that little has since been done… In St Anne’s-Place, and St Anne’s-Court, the open cesspools are still to be seen; in the court, so far as we could learn, no change has been made; so that here, in spite of the late numerous deaths, we have all the materials for a fresh epidemic… In some [houses] the water-butts were in deep cellars, close to the undrained cesspool… The overcrowding appears to increase…” The Builder went on to recommend “the immediate abandonment and clearing away of all cesspools — not the disguise of them, but their complete removal.”
Nothing much was done about it. Soho was to remain a dangerous place for some time to come.
John Snow didn’t shy away from politics.
He was successful in getting the local politicians to agree to his first
experiment, which was successful in helping end that outbreak of cholera.
He even drew his biggest opponent into doing research, which ended up
confirming Snow’s epidemiological diagnosis and extending it further
to find the original probable source of infection of the well.
But even that didn’t suffice for motivating enough political will to
fix the problem.
From which I draw two conclusions:
-
Even John Snow is over-rated.
Sure, he found the problem, but he didn’t get it fixed longterm.
-
Why not?
Because that would require ongoing monitoring of likely sources of infection
(which sort of happened) compared to actual incidents of disease (which does
not appear to have happened), together with eliminating the known likely
sources.
Eliminating likely known sources is what Dr. Gawande’s checklist is about,
150 years later, which was my original point.
And the ongoing monitoring and comparisons appear not to be happening, even yet.
As someone at Metricon said, who will watch the watchers?
I responded, yes, that’s it!
One-shot longitudinal studies can create great information.
That’s what John Snow did.
That’s what much of scientific experiment is about.
But even when you repeat the experiment to confirm it,
that’s not the same as ongoing monitoring.
And it’s not the same as checklists to ensure application
of what was learned in the experiment.
What is really needed is longitudinal experiments combined
checklists, plus ongoing monitoring, plus new analysis derived
from the monitoring data.
That’s at least four levels.
All of them are needed.
Modern medicine often only manages the first.
And in the case of high fructose corn syrup (HFCS),
until recently even the first was lacking,
and most of the experiments that have happened
until very recently have not come from the country
with the biggest HFCS health problem, namely the U.S.
A third of the entire U.S. population is obese, and another
third is overweight, with concomittant epidemics of
heart disease, diabetes, and high blood pressure.
And the medical profession prescribes statins for life
instead of getting to the root of the problem and fixing it.
Yes, I think the field of medicine gets rated too much green for good.
And if IT security wants to improve its own act, it also needs
all four levels, not just the first or the second.
-jsq
