validated. The consequences of the violation of these assumptions and their impacts are listed below.
1. FBIR is not a representative and random collection of samples from a well-defined population of B. anthracis samples.
1,059 samples do not appear to satisfy assumption 1. They were obtained in response to a request from the FBI. No information is available on samples in the population that were not submitted. In fact, the “target population’’ seems not to have been defined. It could be the population of all unique preparations of B. anthracis Ames in the United States, or in the world, or from selected institutions. The absence of a definition of “well-defined population” makes it difficult to assess representativeness of the collection. The elimination of samples that had “inconclusive” results on assays also appears to be nonrandom, as some institutions had many more “inconclusive” assays than others.
2. The 1,059 samples in the FBIR are not independent.
FBI submitted to the committee a table of known transfers of samples between institutions. Hence, the second assumption is violated. Thus, the results of the chi-squared tests for independence of the mutations that are calculated in the report are not meaningful. Further, the confidence interval for the proportion 8/947 is not appropriate. The correct denominator for this proportion is likely not 947. A more accurate numerator and denominator might refer to the number of known independent preparations rather than the number of samples, but such information may not be possible to obtain.
3. Violation of assumptions renders invalid the inferences from the statistical analyses.
the FBIR is not a representative and random collection of independent samples, the results on the assays from the repository may be biased. Virtually all statistical procedures assume that the units on which measurements are made comprise a random, representative collection from the target population. (The effects of biased sampling on inferences have been well documented; see, e.g., Freedman et al., 2007). Without an appropriate model that characterizes the nonrepresentativeness and the degree of dependence among the samples, it is not possible to calculate a meaningful measure of “statistical significance” in the results.