Sunday, May 4, 2008

The reluctance to give probabilities

Coin Scenario: I am about to flip a fair coin. What is the probability it will come up Heads?

Ignorant Fight Scenario: The World's Greatest Boxer fights the World's Greatest Wrestler. Will the wrestler win? (Assume you have no reasonable methodology, other than the Principal of Insufficient Reason, for deciding who to bet on.)

Howard Raiffa argued that in both cases, you should assign a subjective probability of 0.5. However, most people are more comfortable assigning a probability in the Coin Scenario. In the Ignorant Fight Scenario, most people would rather leave it unspecified, or give a "probability of probabilities." Why? Here are some (overlapping) possible reasons.

1. Weight of Evidence: When you look at the larger model that probability assessments come from, you can sometimes attach different weights to various probability assessments. The significance of the weights is that if you find that two different probability estimates conflict, then the estimate with more weight should shift less than the estimate with less weight. Leaving a "blank space" in your probability estimate might be a reasonable way for an agent with limited memory storage to remember that the weight is close to zero. (This paper's explanation is probably isomorphic to this theory.)

2. Ease of Gaining Additional Evidence: You leave a "blank space" for now if you don't see a use for an initial estimate, but believe it would be easy to gain a more refined estimate later, "if needed." The human equivalent of "lazy evaluation."

3. Avoiding Looking Foolish: The more certain you sound about the "validity" of your estimate, the more foolish you will look when (not if) others, in hindsight, decide after the bout that there was evidence that you should have considered, but failed to, when making your probability assessment. This is the explanation that I lean toward.