Is This Study Flawed and Why?

[Uncle Buck]

http://aje.oxfordjournals.org/content/160/10/929.full


Guns in the Home and Risk of a Violent Death in the Home: Findings from a National Study

I am hoping you'll look at this and help me explain why it is (Or isn't) flawed.

It seems to me that excluding know factors from a survey, or over-sampling a specific group, would skew the results towards what you are looking for.

I have a friend who has cited this study, but after reading it, it seems like they set out an agenda and used only the data that supported their presumptions.

Thanks in advance. I know we have Doctors, Lawyer, statisticians and many of very knowledgeable people here who could explain this better to me and my friend. Thanks.

 

[Brian Pfleuger]

Well, they're STARTING with homes where a homicide or suicide occurred and then checking to see how many of those homes contain a firearm. This automatically excludes the tens of millions of homes that contain firearms that DON'T have suicides or homicides.

I'm speculating here but I would guess that attempted suicides and homicides are more likely to be "successful" if the weapon of choice is a firearm so looking only at "successful" events would automatically bias toward firearms being present.

At the very least, I would think that attempted homicides and suicides would have to be included, otherwise you're automatically biased by the "effectiveness ratio" of firearms versus other methods.

Seems like the only true measurement would be to know the total number of homes that have and do not have firearms and the total number of both "attempted/failed" and "successful" suicides and homicides in each set.

I wouldn't be surprised to find that homes with firearms have a higher rate of "successful" homicides and suicides but I'd be willing to be that the ATTEMPT ratio is the same.

 

[Doyle]

The very first rule you learn in any good statistics class is NEVER confuse correlation with cause and effect.

 

[olddav]

It's flawed in as much as owning a car increase your risk of vehicular homicide and/or vehicular suicide. The presence of any tool increases your risk of injury as a result of you or anyone else using the tool.

But that's just my way of thinking!

 

[Booone]

Also, I am no statistician but those are some pretty wide confidence intervals. I don't have time or access to the data to really look at it, but even if you think that the correlation here indicates causation, you really need to look at both the statistical significance and the real world significance of these numbers.

 

[Metal god]

The presence of any tool increases your risk of injury as a result of you or anyone else using the tool.

^^^^^^^THIS^^^^^^^^^

COMON-MAN . It's like saying people with pools are more likely to have a drowning in there pool then people that don't have a pool , duh . I'm not sure what the point of the study is .

 

[Evan Thomas]

The authors acknowledge the obvious limitations on their data and their methodology. However, the study is 9 years old, and it relies on data from 1993. Given that homicide rates in the early 90's were about double what they are now, while the percentage of gun owners in the population is about the same, one wonders how valid their conclusions, at least about homicide, are today. (Suicide rates don't show the same trend -- and have risen dramatically since the economy tanked.)

In general, the results they report don't surprise me. And they draw no conclusions about the desirability or otherwise of firearms ownership: they point out that their study couldn't assess whether the benefits of having a firearm in the home outweigh the risks. In principle, that's an empirical question, but in practice, it's difficult to answer.

However, the question of whether the public health risks of firearm possession justify government intervention isn't an empirical one -- it's one of political philosophy, and I think people who believe that it can be settled by appealing to statistics need to be reminded of that.

 

[ballardw]

The ICD codes exclude police shootings. Also there is no place in the ICD coding scheme for "justifiable" homicides. The codes only deal with cause of death. So any case of self-defense.

I note that they oversampled Blacks and under 35. Typically an oversample is used to increase the chance of getting events that may be less common in specific sub-populations.

Another interesting phrase is "deaths in the home" not "deaths of residents of the home".

They cite the Kellerman study which is well known to be terribly flawed. Kellerman is the one that is the basis for the “43 times more likely” to be murdered in their own home if they own and keep a gun in their home then to defend themselves. He defined defense only as killing an intruder. Also the author has since withdrawn the statement after years of papers challenging his methodology and conclusions. Which raises some question in my mind of the methods used in this study as well.

I find where Table 6, which shows higher homicides odds ratios for households with all guns locked up twice that of households with one or more guns not in locked storage, to be very interesting. Especially as there is no discussion of a that difference.

 

[Romeo 33 Delta]

Maybe for the same reason that homes with bath tubs will have a higher incident of drownings than homes with only shower stalls. Stay safe ... don't bathe?

 

[scrubcedar]

Ok I used to take these things apart as a part of my work.
Here are the most obvious flaws.
1) 20 year old data only applies to twenty years ago when describing social behavior.

2) Statistics based off of voluntary self reporting mean absolutely nothing about the public as a whole, cannot be verified, and only apply to the small minority motivated to return the survey. What is their motivation?

3) CDC is not an accepted scientific source for social questions they are politically driven.

4) The study admit's for itself "To produce more reliable estimates, Blacks, persons less than 35 years of age or older than age 100 years, and persons who died from external causes of homicide, suicide, and unintentional injury were over-sampled in this survey. Considering what percentage of African-
Americans (approximately 84% in the 1990 census) live in cities, you have now over-sampled urban areas. There are many more legal guns per capita in Rural areas. Which leads us to the next point.

5)No differentiation in the survey between lawful gun owners and criminals in possession, illegally, of guns. This was the big card they palmed that clearly shows their intent as it isn't addressed at all. They pay lip service later to some of the other factors they didn't control for, Location, Socio-Economic status etc. but they very carefully do not mention this.

You take already unreliable 20 year old data, from when the murder rate from gang violence was at it's peak, over-sample ethnic minorities under 35 involved in the gangs, then don't differentiate between lawful gun owners and criminals and tally it up.
Amazing! guns in the home are the cause of violence!

I was going to call them idiots, but they were very clever in how they manipulated this data.

 

[1-DAB]

reading stupid studies is the cause of lots of smacked foreheads.

and now .gov wants the CDC to do more 'gun' studies. pass.

 

[2damnold4this]

Look here

This type of study treats firearms like cigarettes or radon exposure and works backwards from homicide victims and tries to compare them to similar people who were not homicide victims. No one thinks to themselves that they are going to take up smoking in order to give their roommate emphysema in thirty years. Not many people take up smoking or live in a house with high radon levels because they want to get cancer and die. But someone might buy a gun for protection from a roommate or rival drug dealer, to murder a roommate or rival drug dealer or to commit suicide.

 

[scrubcedar]

Just a note. I've looked at literally hundreds of medical studies doing "research based practice" checking. In other words, are we treating the patients the way the latest research says is the most effective?
A certain percentage of the studies were about medical devices of one sort or another.
Normally the studies for these were paid for by the manufacturer of the device itself, so the research was sometimes, very heavily even, biased. I had to learn to pick holes in the research based on what they didn't mention or correct for.

This study, even after all that, is the most intellectually dishonest study I've seen.
Out of date, tainted,even admittedly inaccurate, data was used. The data was not controlled for ANY of it's obvious flaws.
If I had used this study to illustrate a point or recommend a change in the "Nursing Practice Council" that I presented my recommendations to I would have been ripped to shreds within the first five minutes, and demoted from my position within ten minutes.
I'm not exaggerating even a little people, this one is.. just.. Wow.

 

[scrubcedar]

Linda L. Dahlberg, Ph.D., is the associate director for science in the Division of Violence Prevention at the U.S. Centers for Disease Control and Prevention. In her current position, Dr. Dahlberg serves as one of the senior advisers on matters of science and policy to the director of the Division of Violence Prevention. She also coordinates international research and programmatic activities for the division and serves as a subject matter expert and consultant on a number of international scientific planning committees and advisory boards.

In other words if this crap isn't true there is no reason for any of her jobs.

Robin M. Ikeda, MD, MPH, is the Deputy Director for the Centers for Disease Control and Prevention and Director for Noncommunicable Diseases, Injury and Environmental Health. In this position, she is responsible for providing guidance and leadership to the CDC′s scientific and programmatic portfolios. She previously served as Acting Director for the National Center for Injury Prevention and Control (NCIPC) from January to November 2010, while also serving as Deputy Director.

Originally a Clinton Appointee, then promoted under Obama, same reasons as above for Dr. Dahlberg apply.

Marcie-jo Kresnow
Centers for Disease Control and Prevention
Publications: 35 | Citations: 434
Fields: Psychiatry & Psychology, Pharmacology, Immunology
Collaborated with 79 co-authors from 1994 to 2011 | Cited by 1270 authors

A little harder to get your arms around this one, but her studies are almost exclusively about backing up Left leaning political causes. Another CDC hack.

Everyone here has a huge financial stake in the findings, clear, identifiable bias.

 

[Hal]

The authors acknowledge the obvious limitations on their data and their methodology. However, the study is 9 years old, and it relies on data from 1993.

Exactly......

I could rig a study that proves no connection between texting while driving and car accidents using data collected from 1993....

 

[44 AMP]

give me a couple hundred grand grant money and I'll write you a great study prooving that texting while driving has NO effect on the accident rate, using data from 1893!

 

[Rainbow Demon]

People who keep a gun in the home specifically for home defense are those who live in areas where home invasions are commonplace, so those homes are more likely to be targeted by home invaders than a home in a town with a very low incidence of home invasions.

Suicides with firearms are more likely in a home where a firearm is present, well DUH.
People who want to kill themselves will use any means at hand, from driving into a bridge abutment at 90, or drinking drain cleaner, to hanging themself from their kids swingset. Taking the gun in the home out of the picture won't prevent suicides. Theres far too many means of self destruction around the home.

Personally I'd never pick a firearm as a means of suicide, any round big enough to be absolutely certain of a clean death, rather than existing for years as a drooling vegetable, would make an open casket funeral very expensive if not impossible.

 

[JimmyR]

Ahh, I thought I got away from social science journals when I finished grad school. Ah, well...

First off, Scrubcedar nailed a lot of what stuck out to me. I was especially curious about why oversampling Blacks and <35's made a lot of sense, especially since they gave no reasoning other than "improved reliability."

The suicide is a moot point. Working in mental health, I have heard more plans for suicide than I would care to think about, but I can tell you a few trends- People in south-central cite "jumping off a bridge" less than folks from Louisville, KY (right across the Ohio River from me); kids and adults who don't drive almost never cite CO2 poisoning; and people who don't know how to get a gun don't plan on using one. Further, when you look at effectiveness of different suicide plans, you notice that guns are pretty reliable, compared to overdosing, hanging, strangulation, cutting, etc., because you can often get a person medical attention after the person attempts.

 

[ballardw]

I believe Lott et al. looked into some of this data and the guns used in the non-suicide shootings were often not from the household counted. That was implied or assumed by the writers..

 

[Dwight55]

ANY TIME you look at figures, numbers, statistics, polls, etc, . . . keep in mind two famous sayings:

"There are three kinds of lies, . . . lies, damned lies, and statistics"

"Figures never lie, . . . but liars can sure figure"

Just like history is written to show the writer's point of view: polls, stats, etc. are written to show the same.

May God bless,
Dwight