PredictionBook is now read-only ( read more ).

I will change my view on whether PredictionBook should have us mark our predictions “right” or “wrong” or use different language instead in the next 48 hours

Created by WilliamKiely on 2016-02-06; known on 2016-02-08; judged right by WilliamKiely on 2016-02-07.

  • WilliamKiely estimated 20% on 2016-02-06
  • WilliamKiely said “My current view is that of Philip Tetlock. From his book Superforecasting: “If a meteorologist says there is a 70% chance of rain and it doesn’t rain, is she wrong? Not necessarily. […]” https://imgur.com/uG3rgGU” on 2016-02-06
  • WilliamKiely said “So my current view is that different language should be used because the mere fact that a prediction is “wrong” doesn’t mean that the probability estimate is wrong, or vice versa.” on 2016-02-06
  • JoshuaZ said “So, to finish Tetlock’s argument: if the meteorologist keeps getting 70% and when they say that when it happens it only rains 55% then they are clearly wrong. ” on 2016-02-06
  • JoshuaZ said “We could say “and the predicted event happened” or not, but that seems extremely wordy. Do you have a preferred term for us to use? ” on 2016-02-06
  • WilliamKiely said “I didn’t identify a preferred term, no. My view that it should be changed is based entirely on this perceived problem with calling the predictions “right” or “wrong.”” on 2016-02-06
  • PseudonymousUser said “It seems to me that issue here is that the predictions themselves are supposed to be about the values that a random variable takes on. The “estimates” are supposed to be the probabilistic part.” on 2016-02-07
  • PseudonymousUser said “E.g., your Meteorologist example, if put on PredictionBook, should have “it will rain” as the “prediction” and then the estimate should be “70%”. ” on 2016-02-07
  • PseudonymousUser said “When the event does not occur, PredictionBook says we “judge this prediction” right or wrong, i.e., judge whether the event transpired.” on 2016-02-07
  • PseudonymousUser said “I.e., PredictionBook’s usage of “right” and “wrong” does not refer to the quality of the probability estimates. ” on 2016-02-07
  • PseudonymousUser said “This is true even though the judgments do inform us about the error rate of the probability estimates.” on 2016-02-07
  • WilliamKiely   judged this prediction wrong on 2016-02-07.
  • WilliamKiely   judged this prediction right on 2016-02-07.
  • WilliamKiely said “Okay, so I just changed my view to believe that it’s fine (and there’s no bettet phrase I could think of) to say “I judge this prediction right” to mean that the event transpired.” on 2016-02-07
  • WilliamKiely said “So then I clicked “Wrong” to record my judgment of the prediction, since I didn’t think that I w going to change my view (80% likely). But then I re-read my prediction and realized that I had stated it as the negative of what I ..” on 2016-02-07
  • WilliamKiely said “…remembered. So in fact the stated event did come to pass. So ky prediction was “right”. So I changed my judgment to “right”. But then after seeing that I was confused by this I think I might agree with my initial view.” on 2016-02-07
  • WilliamKiely said “Nevermind on the last comment. My view has definitely changed: I believe using “right” and “wrong” as PredictionBook does is the best option. So my prediction was right.” on 2016-02-07
  • WilliamKiely said “Although I am now confused about whether my 20% probability estimate was wrong or if Imdon’t know if it’s right ir wrong, or what.” on 2016-02-07
  • PseudonymousUser said “Was I the one that convinced you? Well let’s say there is some probability distribution that describes the outcomes of future events. Your “20%” was an attempt to estimate a certain part of that distribution. ” on 2016-02-07
  • PseudonymousUser said “Probability estimates like that are real-valued, so it’s pretty unlikely that you’d get their value exactly right (for real-world things).” on 2016-02-07
  • PseudonymousUser said “You could think of the probability you estimated at “20%” as the probability that someone would come along and change your mind.” on 2016-02-07
  • PseudonymousUser said “It’s hard to know the true probability of stuff like that. That’s why it’s hard to assess how close to the “true probability” your 20% estimate was. It’s easier to make those kind of assessments in aggregate.” on 2016-02-07
  • PseudonymousUser said “So I don’t know whether your probability estimate was right or wrong—or how close it was to being accurate—but you could still talk about whether your estimate being equal to the “true probability”. ” on 2016-02-07
  • PseudonymousUser said “And so yeah, we would have a notion of right and wrong for probability estimates. But often it’s not easy to assess those estimates empirically.” on 2016-02-07
  • WilliamKiely said “Tapetum-Lucidum, you helped change my mind yes. After reading JoshuaZ’s comment I thought further about it and almost changed my mind, but your comment made me change my mind definitively.” on 2016-02-08
  • WilliamKiely said “I see the point you are making; Tetlock made it too. However, it seems to me that there is another possible way to look at it: Suppose that determinism is true; in this case, all “true probabilities” would be either 100% or 0%. So…” on 2016-02-08
  • WilliamKiely said “…so we would only be able to assess the “reasonableness” (or something?) of a probability estimate rather than the correctness (since technically all probability estimates other than 0% and 100% would be wrong).” on 2016-02-08
  • PseudonymousUser said “Ah, that’s an important point. My idea of a “true probability distribution governing future events” is flawed in that regard. ” on 2016-02-09
  • PseudonymousUser said “My argument was heavily “frequentist”, and assumed things that happen only one time could’ve happened differently (“counterfactual definiteness”). ” on 2016-02-09
  • PseudonymousUser said “If determinism is true, then we have to have some other theory for distributions—e.g., Bayesianism still works—but it becomes more difficult to discuss “true” probabilities in a Bayesian framework. ” on 2016-02-09
  • PseudonymousUser said “Because you have to work relative to someone’s priors.” on 2016-02-09
  • PseudonymousUser said “And yeah, reasonableness seems like a good notion, though hard to define formally. People on PredictionBook talk about “calibration” with others’ estimates, consistency within their own. All attempts at reasonableness :)” on 2016-02-09