A new technique that involves powerful computer calculations is greatly enhancing the statistical analysis of problems in virtually all fields of science.
The method, which is now surging into practical use after a decade of refinement, allows statisticians to determine more accurately the reliability of data analysis in subjects ranging from politics to medicine to particle physics.
The technique also enables statisticians to pull more information out of data than any statistical method previously developed, making it possible to solve some problems that had proved intractable, leading statisticians say. The method addresses the key problem in statistics: how to infer the truth from a sample data that is by no means complete.
"There's no question but that it's very, very important," said Frederick Mosteller, a statistician at Harvard University. Because the method replaces standard, and simplifying, assumptions about data with extensive computer calculations, it can be a powerful tool, Dr. Mosteller said.
"It gives us another way to get empirical information in circumstances that almost defy mathematical analysis," he said. "It's a nice way of letting the data speak almost for themselves."
Jerome H. Friedman, a Stanford statistician who has used the new method, called it "the most important new idea in statistics in the last 20 years, and probably the last 50." He added, "Eventually, it will take over the field, I think."
The new method was invented in 1977 by Bradley Efron, a statistician at Stanford University, to take advantage of the high-speed power of computers. Conventional statistical methods were developed for the most part between 1800 and 1930, when computation was slow.
The standard methods were therefore forced to rely on statistical measures that can easily be calculated by mathematical equations. In contrast, the new method uses the number-crunching power of computers to perform statistical analysis that lie beyond the reach of mathematical equations but can be dealt with through millions of numerical calculations.
"If statistics had evolved at a time when computers existed, it wouldn't be what it is today," Dr. Efron.
The old and the new methods are both designed to determining how accurately a sample of data, such as people who are asked whom they will vote for, represents the whole array of possible data.
For example, a national survey typically draws a sample of 500 to 1, 000 people, Dr. Friedman said. The hope is that the group is representative of the population, but in fact it is only a small sample of 150 million people. The basic question of statistics is, if a different group of people had been selected, would the survey results come out the same?
The conventional method determines the accuracy of a sample by comparing it in a theoretical exercise, with additional hypothetical samples that are generated mathematically. These artificial samples are based on the assumption that all the data fall into a "normal distribution" that conforms to a bell shaped curve, with a few low values, a few high values and a great mass of middle values.
Dr. Efron's new method allows statisticians to construct artificial data sets without making any assumptions about bell shaped curves. Instead the computer constructs new data sets by randomly picking data from the original set.
If the data set were 10 measurements of the effect of a new drug
on high blood pressure, these 10 measurements would be entered
into the computer. The computer would randomly pick a measurement
from among the 10 and use that number to start a new data set
while leaving it as well in the original data set. In the same
way, the computer would pick a second number, a third and so on
until it had picked ten numbers for the first set of artificial
data. The computer could repeat this process 100 times, for example,
creating 100 new data sets. In each data set, some of the original
10 numbers could, by chance, be chosen more than once and others
might not be chosen at all.
Simple but Powerful
By analyzing how these data sets vary from the original one, statisticians can determine the reliability of the inference they drew form the original 10 numbers. The method, conceptually simpler than the conventional methods, requires too many calculations to be done without a computer.
Dr. Efron calls his method "the bootstrap" because the data, in a sense, pull themselves up by their own bootstraps by generating new data sets through which their reliability can be determined. His method is the most prominent of several that rely on the power of computers to generate artificial data sets.
Persi Diaconis, a statistician at Harvard University, said Dr. Efron's technique is "very simple but remarkably powerful." He added, "You can use it across a broad spectrum of problems."
The method is now coming into practical use, after having been argued and discussed by theoretical statisticians for several years.
"It was a practical idea, but it went thorough a big theoretical development with statisticians tearing it in a whole bunch of directions," Dr. Diaconis said. "Now there has been a synthesis and it boils down to simple recommendations on how to use it in the most sensible way in day-to-day situations."
Dr. Efron said new users of the method include paleontologists, physicists, economics and psychometricians, who study the results of psychological testing.
Dr. Efron's idea allows statisticians to use unorthodox methods to see patterns in data because now, for the first time, statisticians can assess how accurate those methods are.
When the Curve Isn't Correct
In recent studies applying his method to data collected by others, Dr. Efron has found that a cholesterol-lowering drug may be more effective than previously suspected and that a subatomic particle may decay in an unexplained way.
In some cases, the bell-shaped curve assumption of classical statisticians is correct, and then the bootstrap method gives the same answer as the standard methods. But Dr. Friedman said, "Data don't always follow bell-shaped curves, and when they don't, you make a mistake" with the standard methods. In fact, he added, the data frequently are distributed quite differently than in bell-shaped curves, although "no one knows" how often this occurs.
Using the new method, Dr. Efron recently analyzed data on the decay of tau particles, which are produced in particle accelerators and studied by physicists who want to understand the fundamental nature of matter. The data were supplied to him by Martin L. Perl, a Stanford physicist, who was puzzled by the traditional statistical analysis, which indicated that there might be a mysterious new way that the particles could decay.
The new analysis gave a more precise estimate of the accuracy of the physics data and indicated more firmly that some decay of the tau particles is not explained by the traditional theories, Dr. Efron found. He added that the new analysis gave more definitive, and believable, evidence to the theoretical physicists.
In another recent study, Dr. Efron obtained data form a study of cholestyramine, a cholesterol-lowering drug, from Dr. John Farquhar and Dr. Daniel Feldman of Stanford Medical School. The data were from a clinical trial sponsored by the National Heart, Lung and Blood Institute more than a decade ago. The data showed that the more cholestyramine the patients took, up to the maximum daily dose, the more their blood cholesterol dropped. The standard method of analysis indicates that there is a straight-line relationship between the amount of cholestyramine a patient takes and the amount the blood cholesterol is lowered.
But a different way of analyzing the data seemed to give a more accurate picture, Dr. Efron found. The reliability of that method could be assessed only by the new method. The new analysis indicated that when patients took more of the drug they did better than what would have been expected by the straight-line relationship.