Our MapScore paper is now in press at Transactions in GIS! From the abstract:
The MapScore project described here provides a way to evaluate probability maps using actual historical searches. In this work we generated probability maps based on the statistical Euclidean distance tables from ISRID data (Koester, 2008), and compared them to Doke’s (2012) watershed model. Watershed boundaries follow high terrain and may better reflect actual barriers to travel. We also created a third model using the joint distribution using Euclidean and watershed features. On a metric where random maps score 0 and perfect maps score 1, the ISRID Distance Ring model scored 0.78 (95%CI: 0.74-0.82, on 376 cases). The simple Watershed model by itself was clearly inferior at 0.61, but the Combined model was slightly better at .81 (95%CI: 0.77-0.84).
In the summer of 2006, Rick Toman (Massachusetts State Police) and Dan O'Connor (NewSAR) organized a sweep width experiment and summit called "Detection in the Berkshires" at Mount Greylock in Massachusetts. In addition to the sweep-width experiment, Perkins & Roberts provided search tactics training for several teams, and the summit provided a chance for us to explore similarities and differences between formal search theory and formalized search tactics. It was an important the chance to meet many key people, compare notes, and discuss ideas. I wish I had been more diligent about following up. Many thanks to Rick & Dan for organizing the event, and to many others listed at the end. However, this post is mostly to provide a reference for the sweep width experiment.
Syrotuck's main study is his 1976, with N=242. But he gives much more detail about distance travelled in his 1975 paper, breaking distance down every 0.2 miles. Unfortunately, he only reports probabilities, not numbers, and doesn't even report total N. We know he got more data between 1975 and 1976, but didn't know how much. Is the 1975 breakdown representative of the 1976 data? Unfortunately, no one has Syrotuck's original data. But we re-created it. (Spreadsheets available!)
We have been collecting data on land SAR incidents in Australia since 2000. In November 2003 we wrote a draft report that was presented at the NATSAR council but not generally released. It was styled after the U.K. report. In June 2006 we released the final version (see below), which has evolved its own style.
Our data was collected using the form available below (or an earlier version thereof). The form itself helps define what is meant by each term or category, and is essential to interpreting the data. We have also prepared a definition key in the report, to explain our terms more precisely. The UK report gave us the idea, but there are some differences in definition.
Early 2003: Charles Twardy plans to reanalyze the Virginia data, correcting for some problems in last year's run. In February, we will analyze the Australian data for the draft report.
Dec 2001: In preparation for the Australian data, Adam Golding analyzed the Virginia data. Cluster analysis revealed only 4 or 5 types of lost person, assuming Gaussian (bell-shaped curve) types.
Adam Golding and Luke Hope then tested several machine-learned models, Syrotuck's model, and a simple model estimated by Rik Head. There were strong differences in predictive accuracy, but negligible differences in a more meaningful score, information reward. The most recent presentation of this work was in Charles Twardy's presentation to the NASAR 2002 conference in Charlotte, NC (June 2002).
NOTE: This is OLD software. It was only a proof-of-concept. It was NEVER operational. Needless to say, it is completely without warranty, etc. Continue reading