The SARBayes MapScore server has been running for a month now at http://mapscore.sarbayes.org.  It's a portal for scoring probability maps, so researchers like us can measure how well we are doing, and see which approaches work best for which situations.  Take a look.  (And if you have a model, register and start testing it!)

What is MapScore?

The idea is that you take a probability map like this computer model:

or this Subjective Consensus:

and score them given the (secret) actual find location.

Repeat the test for many cases, and we hope to get reliable measures of model performance.

VASARCON Slides

We presented our work at VASARCON 2012 a few weeks ago.  The slides are here.  Many thanks to our 8 participants for a very active discussion, and for staying around for an extra 30 minutes of live tabletop exercise and discussion.  We like the suggestion to allow people (or teams) to provide subjective estimates on pre-drawn regions.  Any searcher (or team) could rank themselves on the leaderboard, get some feedback, and contribute to research.

Eric managed the consensus mapping on-the-fly, but it may be even easier if we can learn MapSAR, described in yesterday's post.)

Update (2012-10-30): The original images were lost when we moved to WordPress.  I've replaced them with the models from the VASARCON Mt. Rogers Case, and a recent leaderboard screenshot.

Author: ctwardy

Charles Twardy started the SARBayes project at Monash University in 2000. Work at Monash included SORAL, the Australian Lost Person Behavior Study, AGM-SAR, and Probability Mapper. At George Mason University, he added the MapScore project and related work. More generally, he works on evidence and inference with a special interest in causal models, Bayesian networks, and Bayesian search theory, especially the analysis and prediction of lost person behavior. From 2011-2015, Charles led the DAGGRE & SciCast combinatorial prediction market projects at George Mason University, and has recently joined NTVI Federal as a data scientist supporting the Defense Suicide Prevention Office. Charles received a Dual Ph.D. in History & Philosophy of Science and Cognitive Science from Indiana University, followed by a postdoc in machine learning at Monash. View all posts by ctwardy