Approximated 95% Confidence Intervals calculation by jackknife

Since the measured congruence is dependent on the particular sample taken from the population, there is variability in the estimated values relatively to those of the true population (Pinto et al., 2008). The use and interpretation of the congruence measures can be improved by estimation of suitable confidence intervals (CIs). However, for most of the congruence measures, there is no analytical formula for CIs calculation.

To better understand how statistically significant is the data set under analysis, and how the sampling can affect results, resampling techniques can be used to estimate CIs. We recently compared the use of different resampling techniques for CI calculation. Our results shown the CIs estimated by jackknife pseudo-values matching, in some cases, and outperforming, in other cases, the CIs calculated by bootstrap (Severiano et al, 2011). Therefore, we recommend the use of jackknife pseudo-values CI when an analytical formula is not available. To calculate the jackknife pseudo-values confidence interval, click in "95% CI estimation by resampling" bellow the results' table.

• 9.1. The jackknife pseudo-values approach

  
  

  The delete-one jackknife relies on resamples that leave out one entity of the sample at a time, where entities are those individuals that are randomly sampled from the population. A pseudo-values approach was used to calculate the jackknife CIs. For an estimator S, the i^th pseudo-value of S was calculated as ps_i=NS-(N-1)S_i where S_i is the estimator value for the sample with the i^th data point deleted. The jackknife CI was then calculated as

  
  

  where

  and

  
  

• 9.2. Previous versions

  
  

  Two bootstrap methods were initially made available for the 95% confidence interval estimation: Percentile method and Bias Corrected and accelerated method (BCa). In this later version of the website, we no longer present the confidence intervals calculation by this bootstrap methods. According to our results, the jackknife pseudo-values method outperforms both Bootstrap methods, with faster computation times. Although we recomend the use of jackknife pseudo-values for the CI estimation, you can use this link to calculate bootstrap BCa and percentile CIs to confirm previous results.

  You can also expand this text to see further information about the display of bootstrap distributions.

  • 9.2.1 Interpretation of Bootstrap results

  By clicking on the confidence interval on the results tables you can see the bootstrap distribution, the median, the analytical value and intervals.

  In Figure 9.1, we have an example of the bootstrap distribution of Adjust Rand (Blue/Green bars) for PFGE Sma80 vs T type (microbial typing test data set). The analytical value for the Adjusted Rand is represented in Red and the median value for the bootstrap distribution in Black. The 95% confidence interval calculated by the percentile method is represented in Light Blue/Green. As we can see the bootstraped median value for the Adjusted Rand is very similar to the analytical one, indicating that there is little or no bias in the distribution.

  
  Figure 9.1
  
  

  In Figure 9.2 the Wallace coefficient bootstrap distribution for PFGE Sma80 -> T type is represented. Here we can compare the analytical 95% confidence interval with the approximated bootstrapped confidence interval by the Percentile Method. Once again, the median value is very similar to the analytical one, indicating that there is little or no bias in the distribution and the marginal difference between the approximated CI's calculated by the different methods reflects the little skewness in the distribution.

  
  Figure 9.2
  
  

  However, in same cases, the resampling distribution might be biased. This frequently happens when there is little congruence between partitions. Also, resampling methods should not be applied for very small samples. We are currently evaluating the applicability of bootstrap confidence intervals given a sample and we do not recommend the use of bootstrapped CIs when the distribution is very biased. In Figure 9.3 an example is presented in which, for a very low value of Wallace coefficient (no congruence), the bootstrap distribution is very biased although it still includes the Wallace coefficient for the sample. We are working on a solution regarding those cases.

  
  
  
  Figure 9.3