From Furious of Feltham

[lathany]

One of this evening's news headlines is a story about a mother whose conviction for the murder of her children was overturned. As a result of this, the BBC website produced a page commenting on the suitability of one of the witnesses, namely Sir Roy Meadow.

Now, Sir Roy belongs to a class of people who really annoy me. Being responsible for convicting an innocent woman ? No (well, actually yes, but not specifically what I'm complaining about). It is because he is a non-statistician claiming to be an expert on statistics.

This happens... far, far too often. Doctors are particularly guilty of it because the use of their own data in the field of medical statistics makes a vocal minority think they know how to interpret all the results. But the lack of statistical training really, really shows. And it gives those of us who actually work in the field a bad name.

Grrrrr.

Comments

Stats Blindness

[bateleur.livejournal.com]

Agreed.

I mentioned the statistical fallacy condign was laughing at in his law school blog (see here (http://www.threeyearsofhell.com/archive/000341.html) if you missed it) to a group of work colleagues.

I got a mixture of blank looks and general snorting about the evils of homophobia, but was surprised to note that the underlying fallacy was not immediately obvious to anyone there.

Re: Stats Blindness

[bateleur.livejournal.com]

(Footnote: This is actually a response to dr_bob's post below... he must've spotted a typo and redone it !)

Re: Stats Blindness

[verlaine.livejournal.com]

Not sure I understand myself - why should it be permissible to discriminate against homosexuals as a higher risk group if discriminating against higher-risk black donors (presumably a "racist" act) is out of bounds?

Re: Stats Blindness

[bateleur.livejournal.com]

That's not the matter under discussion. The question is not whether the treatment of homosexuals in this case is right or wrong (which is politics, not statistics) - it's: Can you spot why the statistical argument given is bogus ?

If someone has a serious point to make, they damage their case significantly by trying to use a technical argument and getting it wrong.

Re: Stats Blindness

[onebyone.livejournal.com]

But you said So we're looking at the population "all heterosexuals".

I don't think that covers it. The statistical claim compares homosexuals with African-Americans, and is indeed bogus as stated.

However, let's look deeper. Accepting the stats quoted, and using the fact that 13% of Americans are black plus the estimate that 2-5% of the population are practising homosexuals we see that yes indeed, "homosexual blood" is at least (42/5) / (33/95) = 24 times more "dangerous" than "straight blood". However, "African-American blood" is still roughly (54/13) / (33/95) = 12 times more dangerous than average "straight blood" (including straight African-Americans).

"Dangerous" here means "likely to be from a donor who is a recent case of HIV infection".

So, assuming all the things that need to be assumed about the independence of being gay, being black, and infection rates in the various intersections, I think it's reasonable to question the criteria by which "high-risk" groups are defined.

However, someone else on the law blog has questioned whether infection mechanisms are similar in the ways required. They say that in fact most black HIV infections would be excluded from giving blood for other reasons, or just don't give blood. In the absence of that extra information suggesting that "black blood" is safer than it looks, there is a case to answer.

Re: Stats Blindness

[onebyone.livejournal.com]

Sorry, I meant "times as dangerous" in each case, not "times more dangerous".

Re: Stats Blindness

[http://users.livejournal.com/_alanna/]

Yep, I think there's a further issue that both homosexuals and black Africans would be less likely to make it through our blood donation system than heterosexual whites for reasons completely unrelated to HIV, e.g. Hepatitis, malaria etc.

Re: Stats Blindness

[onebyone.livejournal.com]

True. I don't think either the original argument or my calculation concludes the case, but both illustrate the point that much more information is required before we can accept the NHS position that "homosexuals are too dangerous a group to take blood from".

Re: Stats Blindness

[bateleur.livejournal.com]

Although you're out by a factor of 2-ish since not all homosexuals are male. And then there's the fact that 2-5% is a Kinsey-style figure, so in fact includes bisexuals.

None of which invalidates your basic observation that it's a "Where do we draw the line" question... Still, if I wanted to touch on the politics as well as the stats I'd throw in socio-economic group as a factor and then there'd be something to argue over !

Re: Stats Blindness

[verlaine.livejournal.com]

As far as I could see the comparison was between homosexuals (a high percentage of AIDS cases despite coming from a numerically small subsection of the population) and black people (a high percentage of AIDS cases despite coming from a numerically small subsection of the population). The fact that someone on a cursory glance might think that there is no greater prevalence of AIDS in the homosexual than the heterosexual population is fairly irrelevant if that's not the issue under discussion.

I didn't see any claim or implied claim that a given homosexual in the catchment area is scarcely more likely than a given heterosexual to have AIDS, but maybe I didn't investigate the relevant documents closely enough. That's the claim that would get my hackles up.

[dr-bob.livejournal.com]

I reckon this kind of thing happens right across the range of professions. I am constantly irritated by people making ill-conceived judgements about scientific stuff; GM foods, biowarfare, MMR etc, where people with a basic grounding in science would tell what was tosh from what is reasonable. Unfortunately, most people in the media don't have a clue, so never pick up on the fallacies and errors that get propagated this way. Just like in court, there is often no-one who can pick up on inadequate knowledge or understanding.
Dr Meadow qualifies for annoying bloke in my book, beacuse as well as not understanding stats, he also clearly hasn't got a clue about genetics. It is likely that the cot death kids in the cases described all had genetic (and probably also environmental) factors that predisposed them to cot death.
And right across bioscience, statistics is a woefully under-taught (and in my case ill understood) part of data analysis.

[cuthbertcross.livejournal.com]

Hang on, Furious of Feltham, what about all those perfectly good doctors who get a bad name from the few bad apples who bandy around statistics?

(small rant follows, not to be taken the wrong way; I'm just trying to explain that it's frustrating when all "doctors" get lumped with the same brush)

I am in no way defending the deliberate misuse of emotive "statistics" that appears to have occurred in this case. However, I am constantly told that my professional duty lies in providing information to enable patients to make "an informed choice". I am very often asked for figures by my punters to help them decide what they want to do e.g. have an operation, try one particular treatment or another. I would find it really difficult to say "I can't give you the mortality rate for this operation unless I can first explain to you the full data that gave rise to this mortality rate, the number and premorbid condition of the patients involved in obtaining the figures, the confidence intervals of the actual result, and the applicability of this figure to your good self". Quite aside from the fact that many people would drift off before the end of the sentence, how can I be expected to convey such complicated concepts in ten minutes?
Answer; I can't ; and neither can most doctors. The patient asking me doesn't want to understand detailed statistics; they want a rough chance in terms they can understand.

Here is a real example, based on what I'd tell a patient with an 8cm aortic aneurism who was otherwise fairly well, and asked advice on whether he should get it repaired or not:
OK, There is a 9 in 10 chance that if your aneurism bursts, it will kill you. If we try to fix it by replacing the damaged artery with a graft, the operation has a 1 in 10 chance of killing you. However, this means that 9 in 10 people do get better after the operation. What you have to balance is whether you prefer to just wait and see what happens knowing that you could have up to 40 years of health before the aneurism bursts (but it mught be less than that, and no-one can predict how much less) , or you can take the 1 in ten risk of an operation, and if it does work then you won't have an aneurism to burst anymore. Have a think about it, talk to your wife, talk to a friend and come and see me again next week.

Does that sound too complicated?

[lathany.livejournal.com]

what about all those perfectly good doctors who get a bad name from the few bad apples who bandy around statistics?

<grin>

I appreciate that it ain't just statisticians that get annoyed at Sir Roy. Obviously other doctors do too.

I am in no way defending the deliberate misuse of emotive "statistics" that appears to have occurred in this case.

Unfortunately this wasn't the issue. It wasn't a deliberate misuse of emotive stats, but simply that he didn't know his a*se from his elbow when it came to calculating probabilities. What he did was to say that something had a vanishingly small chance of being the case when it was actually pretty large. I.e. suggesting that a accidental death was 1 in several million when it was actually a large two digit percentage.

A little like him telling the patient of your example that they had a 1 in 1,000,000 chance of dying if their aneurism burst when the actual figure was 9 in 10 - to compare on a rough scale.

I am very often asked for figures by my punters to help them decide what they want to do e.g. have an operation, try one particular treatment or another.

And you have my sympathy. Nor would I be annoyed at a doctor who got his figures wrong in this instance.

What I'm cross about is this Sir Roy has taken the stand for a number of serious court cases (murder, for god's sakes) not to give medical evidence (for which he is qualified) but to give wrong statistics.

[condign.livejournal.com]

Oh dear god...

Did you have to start a discussion on the misuse of statistics and expert testimony a day before my Torts exam?

Trust me, my frustration mirrors your own. Indeed, it is growing to immense proportions, and may dwarf the sun soon...

[cuthbertcross.livejournal.com]

OK, I take yr point! Though I would argue that emotive figures were used (whether deliberately or not); standing up and telling a jury of non-medical non-statisticians that the chances of this happenning were teeny tiny teeny does strongly imply the unsaid so she probably bumped them off, didn't she..." . I doubt that the bloke thought she was innocent. if you ever want to have a discussion about Munchausen By Proxy, invite me round for tea and we'll whisper so the beebies don't need therapy...