Prior Analytics
9 July 2010, 19:23 UTCIncoherent Prediction Markets?
I have known about prediction markets for some time, and find botboth the theoretical arguments and the evidence convincing. But only today did I register for one and look around. I found something odd.
read more...
22 June 2010, 2:15 UTCWho Knows Best?
Ann Nicholson just pointed me to an excellent discussion of two approaches to behavioral economics: nudge vs. regulate.
read more...
22 June 2010, 1:13 UTCEmacs Calendar & pdfLaTeX
How to print emacs calendar landscape using pdfLaTeX.
read more...
13 April 2010, 1:12 UTCRiding the E+ Sun Ray EZ Recumbent
6 April 2010, 18:46 UTCPerformance-Weighted Opinions
27 March 2010, 1:00 UTCYudkowsky on Priors
2 February 2010, 19:28 UTCFundamental Law
14 January 2010, 2:39 UTCPhoto in The Fairest
19 November 2009, 21:47 UTCProbability Words
6 November 2009, 15:12 UTCExplosives Divining Rod
6 November 2009, 10:58 UTCConsistency
31 October 2009, 0:57 UTCPHIL 721 Advanced Seminar: Causation
22 October 2009, 3:29 UTCBNs for Intelligence Analysis
21 October 2009, 23:59 UTCBNs for Initiative Assessment
6 October 2009, 13:33 UTCNetbooks vs Thinkpad X40
14 September 2009, 19:12 UTCpasswd:
13 August 2009, 0:37 UTCPay Too Much For Food & Power
6 November 2008, 16:51 UTCGelman on the Election
9 October 2008, 3:32 UTCRail vs. Bus
5 August 2008, 12:12 UTCBicycle Helmets & Pedestrian Casualties
9 July 2008, 14:58 UTCReston to GMU Bike Route -- Updated
1 July 2008, 21:45 UTCCorrelation & Causation
18 June 2008, 20:38 UTCThe Crowd Within
16 June 2008, 19:40 UTCAudiophiles, Significance Tests, Greenspun's Tenth Rule
10 June 2008, 13:27 UTCDemocracy
13 March 2008, 0:24 UTCKrugman on Interstellar Trade
6 March 2008, 1:33 UTCScience & Magic
29 November 2007, 1:58 UTCFixed Links
10 October 2007, 16:41 UTCAviation collision experiments
22 September 2007, 14:52 UTCPay for free software
"Very good chance" means anything from 50% to 90%. Here is a chart from the 1977
Handbook for Decision Analysis, that I have seen reproduced in other places.
The data come from an experiment
conducted by NATO intelligence analysts who were concerned about the problem of communication. Several different sentences were constructed in the following manner. "It is highly likely that the Soviets will invade Czechoslovakia," or "It is almost certain that the Soviets will invade Czechoslovakia," or "We believe that the Soviets will invade Czechoslovakia." The basic structure of all the sentences remained constant; only the verbal qualifiers changed.
Twenty-three officers, ranking from squadron leader to lieutenant general, served in the experiment. All participants were familiar with reading intelligence publications. They were asked to indicate the probability (in percent) that they would attribute to each message if they were to read it in that form in an intelligence article."
The shaded boxes show Sherman Kent's suggested ranges for probability words. I added the bold in the quotation to emphasize the actual sentences that were used in the experiment.
As a colleague (Tod Levitt) points out, it's actually much worse than this. In some contexts, an informal "highly Likely" could easily be less than 10%. Luke Hope and I ran into this problem when using his software Verbal Reckoner to have people estimate probabilities for a Bayesian network. When there are many options and none is favored, the natural phrase to use was "even chance", or even "50-50". This of course made a global assignment incoherent. The best we could hope for was consistency within a particular Conditional Probability Table.
I have seen this diagram in various places. Here it comes from the
Handbook for Decision Analysis. As part of trying to understand the history of Bayesian methods in Intelligence Analysis, I have a copy of the handbook on loan from David Schum. However, I am happy to say I found it online (for $45) at
Storming Media and (for $free) at ERIC (link above). Storming Media shows that DDI had many reports between 1979 and early 1986. Relevant to
Apollo, several are co-authored by Buede and Sticha.
This is from the DDI Handbook, pp. 67-68, Fig. 3-5. The handbook was written for the Cybernetics Technology Office of DARPA (then, ARPA). It is written as a handbook, and has no references. Does anyone know the original NATO study?