The SARBayes MapScore server has been running for a month now at http://mapscore.sarbayes.org. It's a portal for scoring probability maps, so researchers like us can measure how well we are doing, and see which approaches work best for which situations. Take a look. (And if you have a model, register and start testing it!)
What is MapScore?
The idea is that you take a probability map like this computer model:
or this Subjective Consensus:
and score them given the (secret) actual find location.
Repeat the test for many cases, and we hope to get reliable measures of model performance.
We presented our work at VASARCON 2012 a few weeks ago. The slides are here. Many thanks to our 8 participants for a very active discussion, and for staying around for an extra 30 minutes of live tabletop exercise and discussion. We like the suggestion to allow people (or teams) to provide subjective estimates on pre-drawn regions. Any searcher (or team) could rank themselves on the leaderboard, get some feedback, and contribute to research.
Eric managed the consensus mapping on-the-fly, but it may be even easier if we can learn MapSAR, described in yesterday's post.)
Update (2012-10-30): The original images were lost when we moved to WordPress. I've replaced them with the models from the VASARCON Mt. Rogers Case, and a recent leaderboard screenshot.