Medical Statistics

LEARNING OBJECTIVE

This workshop will outline the basic principles on medical statistics made easy for dummies by key-experts in the field.

Basic statistics for dummies – Dominique Benoit

Notes/Discussions

Dominique Benoit opens with a pleasing statement: “I will not talk about the maths, but more of the philosophy of statistics.” Phew!

He starts by breaking down the myths of the infamous P-value. “It does not prove anything, it just states that it is probably true”. He illustrates this point with slides showing how a normal distribution of two populations can differ and at what point they become ‘statistically significantly different’ with a p-value < 0.05. Next up, he discusses Type 1 errors and how the pharmaceutical industry use power calculations to make sure that the costs of a trial are minimised whilst providing sufficient data to avoid a type 1 error (ie. a false negative conclusion).

Dominique then discusses the utilisation of confidence intervals (a tool I prefer when writing for The Bottom Line). Depending on whether the confidence interval is applied to an absolute difference or a relative difference (either a risk ratio or an odds), the significant point may be whether the interval crosses zero or one (I.e. a 95% CI for an absolute difference of -5 to -1 will also have a p-value of <0.05 as the interval does not cross zero – the point of null effect).

Leaving the stats behind, Dominique discusses the design and conduct of RCTs and how various biases can be introduced. A series of powerful slides illustrate these nicely. To finally demonstrate the biases from poor design and conduct, and the failure to interpret statistics correctly, Dominique discusses the trials investigating activated protein C (rAPC).

Summary/Key points

• A p-value does not prove anything but defines the probability that two groups are not the same.
• A poorly conducted small randomised-controlled trial is not really an RCT as biases will distort the truth leading to an unacceptable conclusion
• “We have to take care that we do not simplify reality to such an extent that we become experts in nothing.”
• “Medicine is learned by the bedside and not in the classroom. Let not your conceptions of disease come from words heard in the lecture room or read from the book. See, and then reason, and compare and control. But see first…” Osler 1849–1919

The fragility index – Manu Malbrain

Notes/Discussions

Manu opens with a video trailer for a TV programme called “Fragile” and a definition – fragility is defined as “the quality of state of being easily broken”.

So how do we apply that to medical statistics? Manu uses my favourite GraphPad Quick Calcs to run a Fisher’s analysis on data. Through a case study he further demonstrates how Quick Calc can be used to sequential move one positive outcome to a negative outcome until Fisher’s Exact test demonstrates non-significance (I.e. P > 0.05).

Summary/Key points

• Many high impact trials have fragility index of zero, one, two or three, despite having many hundreds of participating patients – beware of accepting the outcome as a true positive conclusion.
• Use GraphPad Quick Calcs (https://www.graphpad.com/quickcalcs/) or Paul Young’s Fragility Index website (http://fragilityindex.com) to calculate FI

Logistic regression for dummies – Dominique Benoit

Notes/Discussions

“It is nearly impossible to publish today without logistic regression, but if you read a trial paper it is also nearly impossible to understand what the authors did!”

Logistic regression allows analysis of association between two variables – an x against a y.

Dominique talks us through a worked example investigating the probability of traffic accidents against measured variables such as drugs, alcohol and speed. We see that alcohol and drugs have strong correlation but speed has a much greater relevance than drugs as speed is continuous (with increasing probability per each additional 1 km/hr) whilst drugs is a dichotomous true or false.

The talk goes deeper with Dominique explaining to us the difference between independent, partially interchangeable and completely interchangeable variables. Examples are gender vs APACHE 2, age vs APACHE 2 and SAPS 2 vs APACHE 2 respectively. Due to the way inter-dependence effects logistic models, this must be carefully considered.

After some complex ideas on logistic modelling, Dominique brings it back to something basic. Think about the sequence of variables in the model. For example, if the weather changes it may lead to children eating ice cream. The alternative model might be that children eating ice cream can bring out the sunshine. Although this is obviously wrong, it isn’t so obvious when this error is made in medical research.

Dominique illustrates, with a discussion on body mass index (BMI), how some variables are not linearly associated with a chosen outcome variable. A low or high BMI is associated with a better survival, so categorisation of this variable into cohorts must be performed before applying a logistic regression model.

Summary/Key points

• The author will often work with up to 90 different models upon the data set, but will only publish one. Ask why the author has chosen that model. Is it the best fit for the theory?

Sense and nonsense of medical statistics: a critical analysis – Luciano Gattinoni

Notes/Discussions

Luciano opens with “I hate statistics. If you require statistics to prove something, then it probably isn’t true.”

It’s a brilliant lecture talking us through the concepts of research and scientific reasoning. Inductive reasoning leads to theory, to hypothesis, to experiment and finally a conclusion – a process that is often not considered when designing, conducting, reading and interpreting clinical trials. Luciano talks us through a variety of trials that have tested hypothesis that at disconnected from the inductive reasoning and theory, such that the outcome is entirely predictable and of no clinical application. Regarding ECMO, a 1979 trial by Zapol was wrongly concluded in the context of the underlying theory, leading to ECMO being under-researched for 30 years. It wasn’t the intervention (ECMO) that was wrong, it was the hypothesis and theory that should have been re-examined. Luciano also discusses the PROWESS trial investigating APC, suggesting the story is not over yet!

Trial after trial, Luciano explains to us that if the trial demonstrates no difference (I.e. disproves the deductive hypothesis), we must question whether the underlying premise (theory) is reasonable. Should we go back to the drawing board, rather than designing a bigger trial with greater power?!

Luciano finishes on the topic of simvastatin in sepsis. He exclaims: “Why!? Why should a drug we invented help a syndrome we invented? There’s no reasonable premise to base this upon.”

Summary/Key points

• “Trials could be better designed if we sit for ten minutes over a beer to discuss the inductive reasoning and premise, before then designing a trial hypothesis.” Luciano Gattinoni