Science

[Khornet]

Please indulge me, fellow TFLers. It seems to come up again and again that strong claims are made on the basis of 'scientific proof' of one thing or another. I feel compelled to point out that most people just don't understand how research works. I'm no scientist (and I don't play one on TV either), but every day on the job I'm required to use the results of other people's science, and I have to decide whether what they claim is beleivable.

Turns out that a lot of what we 'know' just ain't so. Case in point: the notorious Kellerman study showning how we're a billion times more likely to be massacred with our own gun than to be defended by it. By now most people here understand the methodologic flaw in that study: the counting of only killings or shootings, disregarding the much more common scenario of assailants being frightened off by the sight of the intended victim's gun.

Apply that lesson to most of the science you hear reported. If possible, try to find out the method of the study. If you see bad methods, ask yourself why. Why would a university prof not think of the existence of non-lethal deterrence with a gun? Because of the bias inherent in the prof's approach to the issue.

BIAS is the bugbear of research. It can creep in at the very beginning of a study, in the nature of the question it seeks to answer, as in studies designed to assess the effects of global warming on frogs. Such a study assumes that GW exists, which is a big assumption indeed.

Bias can arise from selection of the test subjects. That's what I meant when I said EMS Guy suffered from selection bias: the population from which he derived his conclusion is made up of the people calling 911. That's not a representative sample of much of anything, and it can lead you to overstimate the prevalence of disease.

Bias can creep in after all the data is gathered. The researcher can make math errors. Or he can use an inappropriate statistical test. Or, all too common, he can draw a conclusion which is just not warranted by the data.

So you've got to be very, very careful in designing your study, interpreting the result, and reporting your conclusion, that you're not deceiving yourself and making the facts fit the picture you have in your head. It's very easy to make a mistake even with honorable intent.

Then it gets worse, because the study is reported by people who have no idea how to judge research, and who have biases of their own. That's how Arming America got a free pass in the press.

I once heard of a study of the correlation between human birth rate and stork populations in a Dutch town. It showed 100% correlation: more storks, more babies. Ergo, the researcher said, storks bring babies. He was trying to illustrate bad research: of course, storks nest in chimneys in Holland, and when birth rates are up, more houses are built, and more nests for storks.

I'm not saying we should abandon science. I'm just saying that we humans are very good at b**llsh**ting ourselves, and we really have to watch it. Especially when we propose to use science to justify abridging anyone's rights.

Thanks for listening. This has been a public service of Khornet.

 

[spacemanspiff]

how about we post examples of scientific studies that seem to hold some water? i'll go first.

in the june issue of discover magazine, an article discussed the theory of physicist john wheeler, who seems to have proven that the universe reacts to our observations of it. the future and present, as well as the past are determined by how we observe its workings. for example, a test was conducted of light particles, since light has a dual nature. sometimes light behaves like a compact particle, or a photon, and sometimes it acts like a wave spread out in space. in the experiment, light, a stream of photons, shines through two parallel slits and hits a strip of photographic film behind the slits. the experiment is run two ways: with photon detectors right beside each slit that allow physicists to observe the photons as they pass, or with detectors removed, whicha llows the photons to travel unobserved. when physicists use photon detectors, the result is unsurprising: every photon is observed to pass through one slit or the other. the photons act like particles. but when the detectors are removed, a patter of alternating light and dark stripes appear on the film. such a pattern could only be produced if the photons are behaving like waves, with each individual photon spreading out and surging against both slits at once.

the end results developed the theory that the universe didnt exist until we started looking at it. and when we look at it, it APPEARS like its was there 10 billion years ago.

 

[Dave B]

Good post, but...

So what's the answer?

What are we to believe, what do we discard, how can the courts or the gummint or your neighbors make informed decisions if science is suspect?

You seem to be advocating 'believe the studies that support your preconceptions'.

db

 

[Khornet]

No, Dave.....

I'm asking that we employ the same common-sense skepticism about science that we do with other everyday matters. I'm asking that we recognize that much of today's research is ideologically driven, and we must watch out.

A study is not any longer credible just because it appears in the New England Journal of Medicine or comes out of a respectable institution, e.g. Emory.

We are skeptical of gun-control research because we know a lot about guns. We're insufficiently skeptical about research in other areas, and can be just as dogmatic and blind about that research as gun-grabbers are about Kellerman.

If we don't have the truth we have nothing.

 

[griz]

Amen Khornet.
I get very tired of someone telling me "I know it’s true because I heard it on the news". Your TV news is an absolutely terrible source of scientific information. They are in the entertainment business and tend to say things that sound interesting instead of being as accurate as possible.

 

[Torquemada]

Or the "researcher" can't or won't make the distinction between cause and correlation.

Ex: It always rains right after my knee aches. Does the rain (falling atmospheric pressure) cause my knee to ache? Does my knee cause the rain to fall? Does it ALWAYS rain, or do I just notice when it does? No, no, no, and yes. It's a correlation.

Ex: There's a chance of rain under certain atmospheric conditions. Temperature x, humidity y, low and high pressure fronts z, etc. all of which can be categorized. It absoulutely CANNOT rain if one or more factors is wrong/not present. It is a cause.

Guns do not "cause" crime, but they do correlate with crime. Eliminate guns and there still is crime, eliminate crime and there still are guns.

Oh yeah, the other big test of a "scientific" study is that the results are reproducible by others through independent research, which is where cold fusion has failed.

 

[Dex Sinister]

Just because you're not part of the problem doesn't mean that you're part of the solution...

...sometimes you're just part of the precipitate.

Dex }:>=-

 

[40ozflatfoot]

...much of today's research is ideologically driven...

Not only is this true, but a lot more of today's research is driven by ego. Few of today's scientists are interested in the advancement of science nearly as much as they are interested in using science to advance their own career, reputation, and ego.

 

[Justin]

I think 40ozflatfoot is right. I think a lot of research is funding or grant driven.

In other words, gov't agency x proposes a study be done. However, said agency, deliberately or accidentally tips their hand as to what sort of results they want.

Scientist y is recruited to complete the study proposed by x, all the while realizing that if his research doesn't support a pre-conceived outcome that his funding will be yanked. The scientist, not wanting to end up on the street or teaching undergrads, is happy to oblige and gives x the results they want.

 

[MeekAndMild]

Torquemada, there is a hilarious and obscene discussion about the laws of cause and effect in the great Sci Fi classic "Venus on the Half Shell" by Kilgore Trout (Phillip Jose Farmer). I can't repeat it here for fear of being perminently banned, but you should read it!

 

[Skibane]

Or the "researcher" can't or won't make the distinction between cause and correlation.

Reminds me of the report about a scientist who had succeeded in training a flea to jump whenever it heard a bell ring. The scientist then proceeded to remove one leg at a time before ringing the bell. The flea continued to jump, until its last leg was removed, at which point it remained motionless. The scientist's conclusion: removing all a flea's legs causes it to go deaf.