posted on Aug, 25 2012 @ 09:50 AM
Okay, so they were right 8 times in a row. What are the statistical odds that the model holds a 9th time, since this is a statistical model? To answer
this, I used a simple on-line tool (www.mccallum-layton.co.uk...
), and entered 8 for 8, give me a
95% confidence interval. [-10.8% to 26.8%] was the result. Given that this is the Z approximation for poisson intervals, it returns a negative
percent, and the results a bit off. Most folks say it's close enough, and state this as [0% to 26.8%]. So based on 8 observations, the true value of
the predictive power of the model lies in this range with 95% confidence.
The inverse of this range is [73.2% to 100%] is how accurately the model predicts. So if the model predicts with 75% accuracy, then it could still get
8 right in a row by an acceptable random chance. Of course, the model was trained with data, and there is probably much better odds than this given
the full training set, but in practice it's been used 8 times.
I.e., it's not got a long track record for predicting binary outcomes.
edit on 25-8-2012 by CyberGarp because: Added some details.