1.The Natural Why-Quirer (story map)

2.GM corn squabble

3.Nature pulls a U-turn

4.New news on old experiments

5.Madness in the method?

British physicist Arthur Eddington proved Einstein's theory of relativity. Or so he claimed. Courtesy AIP, Emilio Segrč Visual Archives.

Good. That means we'll have some arguments. But something else is at play, says Trevor Pinch, who studies the process of science and technology at Cornell University. "Scientific experiments depend on a lot of craft skills."

The scientists and technicians who can pull off ultra-precise measurements, Pinch says, are said to have a "golden hand." Because their clever techniques often cannot be fully described in scientific journals, scientists intent on repeating an experiment may sometimes visit colleagues for training.

Although the scientific paradigm requires that all elements of an experiment be controlled, it can be difficult to know what to control. Take the standard psychological rat-in-maze test. Obviously, to eliminate variables, the rats must be the same breed, the same age, eat the same food, and run the same maze. But must the experimenter be the same person? Rats are excellent sniffers -- so what if the technician starts wearing perfume? Might one rat follow a scent trail left by another?

Edifying story
To understand the true story of how scientific advances occur, Pinch and co-author Harry Collins, a British sociologist of science, have studied contemporary descriptions of some historic experiments. The results, they say, do not jibe with streamlined, modern views of the experiments.

Take, for example, the famous 1919 test by British physicist Arthur Eddington, of Einstein's prediction that gravity bends light. Eddington wanted to track starlight passing near the sun, and he used the sun's approximate mass to calculate the deflection predicted by Einstein.

Let's look at the experimental situation:

The measurements had to be precise, since the calculated deflection was a hair-thin 1/3600-degree.

Since the sun's gravity would be bending the light, Eddington had to measure a star almost directly behind the sun.

Because the sun's glare would normally obscure the star, he had to work during a solar eclipse, meaning he had to set up a small, field telescope in a part of the world under a total eclipse.

Because the comparison pictures (those not affected by the sun's gravity) had to be taken at night, Eddington had to factor in the temperature-induced size changes in the telescope. That tiny increment was about equal to the gravity-induced deflection being sought.

Faced with these complications, it's perhaps not surprising that Eddington's two expeditions returned with some data that confirmed Einstein -- and data that confirmed Newton.

What to do? Eddington had done the experiment because he was partial to Einstein's theory, Collins and Pinch write, so he was predisposed to believe the data backing Einstein. (These days, "gravity lenses" provide new evidence for the same phenomenon.)

Albert Einstein was a patent office drone when he grasped that gravity would bend light waves. He didn't figure out how to test his theory... Courtesy W. F. Meggers Gallery of Nobel Laureates, AIP, Emilio Segrč Visual Archives.

Right, right?
Eddington may have been right, but was he scientifically right? Not according to the standard "it's the experiment, stupid" view of science. His conclusion, as seen by Pinch and Collins (see "The Golem:..." in the bibliography), depended as much on interpretation as data-gathering.

After looking at other experiments at the frontier of science, Pinch describes them as similar to Eddington's Einsteinian edification. "They are difficult, demanding, they have a small signal, it's quite near the noise level, and when you repeat the experiment, you often don't find the same thing."

When, as in the case recounted, the data do not agree, Pinch asks, "who screwed up? Maybe both groups, maybe one or the other."

For more refutation of the stereotyped, "to do science, just do an experiment" paradigm, Collins and Pinch recount Louis Pasteur's debunking of spontaneous generation, the 19th century view that life did not necessarily descend from other life. In these experiments, Pasteur sterilized a closed container with heat, and the gook inside was not decayed by microbes. But his results were murky, and luck played a larger role than popular histories recount.

Experimenter beware
If we want to understand the process of science, retrospect is a difficult vantage point. At the research frontiers, Pinch says, things are confusing, and you don't know how to interpret conflicting data. "You don't have hard-and-fast criteria, you don't know if the phenomenon you're looking for is really there."

Think of following a new recipe, Pinch says. "Suppose you watch a cook on TV making a fancy recipe... and when you try it at home, all those exotic mushrooms turn to mush. You're faced with dilemma in principle like an experimenter at the research frontier."

Did the TV chef cut a corner? Are you too clumsy? From the evidence at hand, it's impossible to know what went wrong. At this point, social factors like trust, reputation and experience may enter the picture. "You can say the TV chef is much more skilled... or it could go the other way," Pinch says. "You could say you're a darn good cook, maybe this recipe doesn't work, maybe the TV chef cheated."

This culinary example demonstrates what Pinch and Collins call "experimenter's regress." When the data conflict, what do you believe? Prestige, certainly, plays a major role: as any scientist knows, papers from well-regarded researchers and well-known institutions get taken more seriously -- whether or not they are more accurate or inherently important.

And as the Eddington example illustrates, even expectations can play a role when the scientific field is wide open. Finally, for better or worse, articles in highly regarded journals, like Science and Nature, are accorded unusual respect and validity.

History, Pinch argues, shows that science is not a purified, hyper-logical pursuit, but rather a form of human expertise that's subject to some of the same limitations affecting other areas of expertise.

When the moon blocks the sun, you can see nearby stuff.Eddington
used this blockage to see a star so near to the sun that its light was
bent -- veeeery slightly! NASA.

Only in it for the money?
Ironically, although Albert Einstein was working in the patent office when he cooked up his theory of relativity, he never did the obvious thing -- apply for a patent. These days, patents and money threaten the notion that the scientific process is a pure, honest pursuit of truth. Naomi Oreskes, an assistant professor of history at the University of California at San Diego, says the role of money is "Huge now, more than before. It's not that scientists never had patents, but the scale has changed."

She points to biotech (the realm of the GM corn controversy) as a place where money and patents threaten the free flow of information on which science depends. "A lot of scientists have intimate consulting arrangements with private industry, or have created their own companies," she says. "Now there are all kinds of complicated problems. If I'm doing research in the lab, and have a company, will I publish all the details if it's patentable? Do I encourage grad students to publish if it's patentable? There's a direct conflict with open exchange of information, which is critical to evaluating the legitimacy of work."

Secrecy also threatens the essential "peer review" process, which precedes publication in most reputable science journals. Editors send manuscripts to experts for comments, criticism and anonymous hazing (scientists can be quite direct). If the authors revise the article adequately after the review, the article is published. Articles that fail peer review are rejected.

Ideally, peer review assures readers and editors that the article is worthwhile and probably accurate. But peer review, as Nature's tardily-rebuked GM corn story demonstrates, is fallible.

To Pinch, Nature's disowning-but-not-retracting of the article "is tantamount to acknowledging that something went wrong with the peer review process. It's kind of having things both ways -- backing both horses."

In fact, Oreskes claims that much of the review process "is perfunctory today. Almost every scientist can give examples of completely preposterous things that came out in their area. It's almost impossible to get a reviewer who will check math or review a computer model, check the code ... you're almost asking somebody to do the work all over again."

Testing, testing
You'd think nobody would want to do anything all over again. But then you think about replication, another critical scientific check-and-balance. Theoretically, replication is simple, direct and powerful. Your scientific article describes your process so thoroughly that any colleague can repeat ("replicate") your work and find out if you were on the level -- or on the bevel.

In practice, however, there's no glory in saying "Look at me! I found the same durn thing!" so, as Collins has pointed out, exact replication is rare.

However, doing something similar may get you published, and therefore be worthwhile, notes Oreskes, "So replication is a kind of complicated morphing from A to B to C. That's often productive, but it creates problems because it could be the case that the original phenomenon was false, but you never really test it."

I see somebody looking over your shoulder.

There are 1 2 3 4 5 pages in this feature.
Bibliography | Credits | Feedback | Search

©2002, University of Wisconsin, Board of Regents.