Thursday, February 19, 2015

Don't trust the abstract; read the whole paper


Above are the results and conclusion from an abstract of a paper called "Single-Incision Laparoscopic Cholecystectomy: Will It Succeed as the Future Leading Technique for Gallbladder Removal?" It appears online in the journal Surgical Laparoscopy, Endoscopy & Percutaneous Techniques.

It is another great example of why you need to read the entire paper and not just the abstract.

The methods section says that the study involved 875 patients with prospectively collected data. Don’t be fooled by prospectively collected data. This research is retrospective.

Here are some issues.

How were patients selected to have single-incision (SILC) or standard 4-port (LC) laparoscopic cholecystectomy? They certainly were not randomized or it would have been mentioned. The easier cases were chosen for SILC. How do I know that? Well, 17.4% of the SILC patients had acute inflammation of the gallbladder compared to 35.5% of the LC patients. The statistical significance of this was omitted from the abstract, but the chi square p value is 0.0001.

Also not mentioned in the abstract was that 82% of the SILC patients had elective surgery compared to only 57.2% of the LC group, p <0 .001.
 
I have talked about the statistical trick of using a “straw man” before. [Type straw man in the search field to your right on this blog site.] A straw man occurs when researchers set up an artificial control and the find better results with their intervention.

The straw man in this case is the average operative time of the LC patients, 79 minutes which is significantly longer than the 71 minutes it took to perform SILC. The problem is that as shown in many other studies, standard LC can usually be done in 40 to 45 minutes.

If you look at some of my other posts on this subject, you will see that a straw man is particularly likely to appear in papers about SILC.

Will SILC succeed as the future leading technique for gallbladder removal? Maybe, but we are certainly going to need to see better evidence than this.

11 comments:

artiger said...

Who is the journal editor on this nonsense, Andrew Wakefield? Are they just putting out this stuff hoping to get published in the NY Times, so that patients will see it and request it (think: DTC marketing)?

The ergonomics of SILC are still in question from some stuff I've read recently.

Skeptical Scalpel said...

A surgeon in Florida claims the ergonomics are no problem. Everyone else is just doing it wrong. http://www.generalsurgerynews.com//ViewArticle.aspx?ses=ogst&d=In+the+News&d_id=69&i=February+2015&i_id=1148&a_id=29499

Anonymous said...

Totally agree. Exposes the journal for being of poor quality. For what it is worth, the Impact Factor is not even 1, so fortunately nobody is really taking that much notice of it. If I were the editor in chief of this journal, I would be embarrassed.

Michael-Burchett said...

The length of stay is rather long for both procedures.

Moose said...

This is one of the reasons why I (and others) are pushing for science research to not be published behind pay walls. Too much misinformation is being spread due to abstracts that tell you what the researchers wanted the conclusion to be, not what they actually found.

Skeptical Scalpel said...

Michael, you are correct. The lengths of stay are long and typical of what is apparently the standard in Europe. I didn't mention it because it is not relevant to the situation in the US.

Moose, I agree. I don't understand why journals allow authors to get away with "anything goes" in the abstract. It's an example of the sorry state of peer review in medicine.

Skeptical Scalpel said...

Anon, sorry I forgot to acknowledge you comment which is a good one. Not only is the impact factor 0.94, the journal is ranked 112th of 379 surgery (all types) journals by SCImago, a website that measure journal visibility in the Scopus database.

William Reichert said...

One big issue is how the media reports research articles. Recently, the media reported a study by researchers at Yale ,suggesting that
internists were harming older people by trying to control the blood sugar too tightly in elderly diabetics. What the researchers actually did was take the position and assume that tight control in elderly diabetics was always harmful and then surveyed a group of physicians and showed that many tried to use tight control in elder diabetics But what they did not find or even study in the paper was whether this tight control caused actual harm in these patients,They just assumed tight control caused harm ( which MAY be true) and found that lots of doctors practiced tight control. This is a subtle but BIG real difference. The media often does not report the difference
accurately, probably because there writers don't know how to closely read medical research papers.

Skeptical Scalpel said...

William, I agree. The media can be a real problem. In fact I was alerted to this paper because it was mentioned in an article without any analysis of its results in the current issue of General Surgery News.

DavidP said...

Sadly normal members of the public like me are usually unable to look beyond the abstract because of the pay walls.At least this abstract showed that age, gender, BMI and AIG were significantly different, making it reasonably obvious the results were meaningless.

Skeptical Scalpel said...

David, I agree that paywalls are a problem, but they may disappear in a few years. Meanwhile, I'll try to point out things like this when I can.

Post a Comment

Note: Only a member of this blog may post a comment.