When people approach me after a speaking engagement, they often express great excitement and joy over the new scientific evidences I present for the Christian faith and for the inspiration and inerrancy of the Bible. However, many of these same folks are reluctant to use these evidences in their personal witnessing to non-Christian family, friends, and associates. I’ve found that one reason for their hesitation is an inability to evaluate the certainty of the results that I present.
Whenever I express a value for a particular result, I consistently append the error bars. If possible, I also add corroborating results from independent means of investigation. It was my wife, however, who pointed out to me that knowing how to define error bars and understanding the importance of multiple independent means of investigation are unfamiliar to most lay audiences. Thus, my hope is that this article will help lay readers gain a better understanding of how to evaluate scientific results and help scientifically savvy readers better communicate that understanding to others.
Error Bars
An example of an error bar notation used by scientists would be a recent measurement of the cosmic expansion rate based on H II regions (star-forming gaseous nebulae) in galaxies: 74.3 ± 3.1 kilometers/second/megaparsec.1 (A megaparsec = 3.258 million light-years.) The “± 3.1” is the error bar or what statisticians call the standard deviation. This error bar means that there exists a 68 percent probability that the true cosmic expansion rate lies between 71.2 and 77.4 kilometers/second/megaparsec. It also means that there is a 32 percent probability that the true cosmic expansion rate lies outside that range of values.
Sometimes scientists will quote a range of possible values based on two or three standard deviations. For the above measurement a two standard deviation range of possible values would be between 68.1 and 80.5 kilometers/second/megaparsec. In this case, there is only a 4.6 percent possibility that the true cosmic expansion rate lies outside of that range of values. A three standard deviation range of possible values would be between 65.0 and 83.6. Here, there is only a 0.3 percent chance of the true cosmic expansion rate falling outside the range of values.
The above error bars account only for possible random or statistical errors. Random errors describe the statistical scatter in values the scientist observes in his measurements. The above result of 74.3 ± 3.1 kilometers/second/megaparsec was based on measurements of 69 nearby galaxies containing H II regions that could be measured accurately. The mean, or averaged value, of all the measurements was 74.3. The 3.1 error bar describes the scatter about the mean manifested in the measurements.
Random errors directly impact the certainty of a published result. An indirect but equally important factor is what scientists term systematic errors. Systematic errors result from instrumental effects, data selection effects, and assumptions about the phenomenon being measured that skew all the measurements to either higher or lower values.
The team that produced the 74.3 ± 3.1 kilometers/second/megaparsec result for the cosmic expansion rate exhaustively examined all possible instrumental problems and biases as well as all the assumptions about the properties of H II regions that conceivably could be mistaken or, for example, change if one were to sample more distant galaxies. This examination led them to conclude that in addition to the ± 3.1 random error in their cosmic expansion rate determination, there also existed a ± 2.9 systematic error.
The ± 2.9 systematic error means that there exists a 32 percent probability that the ± 3.1 random error centers on a value for the cosmic expansion rate outside of the range between 71.4 and 77.2 kilometers/second/megaparsec. As with random or statistical errors, increasing the range of values based on multipliers of the systematic error yields greater certainty in the result. For example, there is only a 0.3 percent possibility that the ± 3.1 random error centers on a value outside the range between 65.6 and 83.0 kilometers/second/megaparsec.
The most accurate published value for the cosmic expansion rate is 69.6 ± 0.7 kilometers/second/megaparsec.2 Are this value and the one based on HII regions discordant? Not at all. If the two sets of measurements faithfully reveal the true value for the cosmic expansion rate based on the error bars, then there is high probability (a little greater than 32 percent) that the two values will differ from one another by 6.0 kilometers/second/megaparsec.
Real Effect or Environmental Noise?
Sometimes scientists have to detect the signal of a real effect amid a lot of environmental or instrumental noise. How do they, or we, know if their detection is real or just an artifact of the noise? They do so by measuring how much the signal rises above the noise, specifically by determining the signal-to-noise ratio. Barring systematic effects, if the signal-to-noise ratio is two, then the probability that the signal is a real effect is 68.3 percent. A signal-to-ratio of three raises the probability to 95.4 percent, of four to 99.7 percent, of five to 99.994 percent, and of six to 99.99994 percent.
In the physical sciences a measured effect is not considered to be a real detection unless the signal-to-noise ratio exceeds five. Thus, physical scientists are not permitted to declare a discovery unless they can demonstrate a signal-to-noise ratio that exceeds five. In some life science disciplines it is often not possible to achieve signal-to-noise ratios greater than five. Therefore, the standard for publication frequently is much lower for journals that focus on those areas.
Advice for all readers, whether they be scientists or lay people, is that before one determines how much confidence to place on a scientific finding always look at the signal-to-noise ratios and both the random and systematic error bars.
Finally, before putting any long-term confidence in a scientific discovery look for corroboration. Has the result been confirmed by other independent research teams using different detection equipment or different detection methods or both? Is the result confirmed by both observations over time and experiments? Is there a theory that successfully integrates and explains all the observations and experiments?
Personally, I do not put a lot of confidence in a scientific result unless it is established by experiments, observations, and theory and unless I see the consistency among all the observations and experiments becoming progressively better as the error bars, both random and systematic, shrink. What pleases me about the biblical creation model we are developing at Reasons to Believe is how the march of scientific advance over the past century has yielded an improving consistency among the theories, observations, and experiments that undergird our model. The improving consistency emboldens me, and hopefully all believers, to declare to an unbelieving world the reasons for our hope of eternal life through Jesus Christ.
Will Myers
Intelligent Design and Biblical Scripture 