http://blog.oddhead.com/2006/12/26/evaluating-probabilistic-predictions/ Musings of a computer scientist and Yahoo on prediction markets, gambling, and estimating the odds of everything Thu, 11 Mar 2010 12:32:57 +0000 http://wordpress.org/?v=2.9.2 hourly 1 http://blog.oddhead.com/2006/12/26/evaluating-probabilistic-predictions/comment-page-1/#comment-88 Pete Hopeland Mon, 03 Sep 2007 17:12:33 +0000 http://blog.oddhead.com/2006/12/26/evaluating-probabilistic-predictions/#comment-88 This is a common pitfall in the use of statistical data. Most people do not understand this. As a result, many professionls including advertisers and politicians use this loophole in twisting facts. This is a common pitfall in the use of statistical data. Most people do not understand this. As a result, many professionls including advertisers and politicians use this loophole in twisting facts.

]]> http://blog.oddhead.com/2006/12/26/evaluating-probabilistic-predictions/comment-page-1/#comment-87 Midas Oracle .ORG » Blog Archive » Evaluating probabilistic predictions — David Pennock Edition Tue, 20 Feb 2007 22:51:14 +0000 http://blog.oddhead.com/2006/12/26/evaluating-probabilistic-predictions/#comment-87 [...] Evaluating probabilistic predictions — David Pennock Edition David Pennock: […] So what is the “right” way to evaluate probabilistic predictions? There is no single absolute best way, though several tests are appropriate, and probably can be considered stronger tests than the calibration test. In our paper “Does Money Matter?” we use four evaluation metrics: [...] [...] Evaluating probabilistic predictions — David Pennock Edition David Pennock: […] So what is the “right” way to evaluate probabilistic predictions? There is no single absolute best way, though several tests are appropriate, and probably can be considered stronger tests than the calibration test. In our paper “Does Money Matter?” we use four evaluation metrics: [...]

]]> http://blog.oddhead.com/2006/12/26/evaluating-probabilistic-predictions/comment-page-1/#comment-86 Midas Oracle .ORG » Blog Archive » How to publish a high-quality prediction market blog Mon, 29 Jan 2007 14:37:41 +0000 http://blog.oddhead.com/2006/12/26/evaluating-probabilistic-predictions/#comment-86 [...] #12. State that evaluating probabilistic predictions is very tricky —in case readers missed it the first time. [...] [...] #12. State that evaluating probabilistic predictions is very tricky —in case readers missed it the first time. [...]

]]> http://blog.oddhead.com/2006/12/26/evaluating-probabilistic-predictions/comment-page-1/#comment-85 Midas Oracle .ORG » Blog Archive » Reality Check on TradeSports-InTrade John Delaney’s Exil Statement Mon, 22 Jan 2007 14:32:19 +0000 http://blog.oddhead.com/2006/12/26/evaluating-probabilistic-predictions/#comment-85 [...] #3. “implying infallibility of PM’s” - Nobody in the industry is claiming that. These naysayers are the leftists who had embraced prediction markets before the 2006 US elections and who are now turning against them. [...] [...] #3. “implying infallibility of PM’s” – Nobody in the industry is claiming that. These naysayers are the leftists who had embraced prediction markets before the 2006 US elections and who are now turning against them. [...]

]]> http://blog.oddhead.com/2006/12/26/evaluating-probabilistic-predictions/comment-page-1/#comment-84 David Pennock Wed, 17 Jan 2007 21:46:25 +0000 http://blog.oddhead.com/2006/12/26/evaluating-probabilistic-predictions/#comment-84 Yes, you're right: 2 & 3 will always rank predictors equivalently. (We used 3 because it's the one used by the <a href="http://probabilitysports.com" rel="nofollow">ProbabilitySports</a> contest that we were comparing against.) Yes, you’re right: 2 & 3 will always rank predictors equivalently. (We used 3 because it’s the one used by the ProbabilitySports contest that we were comparing against.)

]]> http://blog.oddhead.com/2006/12/26/evaluating-probabilistic-predictions/comment-page-1/#comment-83 Foolish Jordan Wed, 17 Jan 2007 20:44:45 +0000 http://blog.oddhead.com/2006/12/26/evaluating-probabilistic-predictions/#comment-83 Aren't metrics 2 and 3 mathematically equivalent? That is, can you find two sets of predictions x_1 .. x_n vs. y_1 ... y_n where #2 prefers x_i and #3 prefers y_i? Aren’t metrics 2 and 3 mathematically equivalent? That is, can you find two sets of predictions x_1 .. x_n vs. y_1 … y_n where #2 prefers x_i and #3 prefers y_i?

]]> http://blog.oddhead.com/2006/12/26/evaluating-probabilistic-predictions/comment-page-1/#comment-82 Oddhead Blog: Prediction Markets, Gambling, Electronic Commerce, Artificial Intelligence: David Pennock: Yahoo! Research Fri, 05 Jan 2007 00:58:59 +0000 http://blog.oddhead.com/2006/12/26/evaluating-probabilistic-predictions/#comment-82 [...] Evaluating probabilistic predictions [...] [...] Evaluating probabilistic predictions [...]

]]>