MapScore public leaderboard, from October 2012.

MapScore: A Portal for Scoring Probability Maps

The SARBayes MapScore server has been running for a month now at http://mapscore.sarbayes.org.  It's a portal for scoring probability maps, so researchers like us can measure how well we are doing, and see which approaches work best for which situations.  Take a look.  (And if you have a model, register and start testing it!)

What is MapScore?

MapScore public leaderboard, from October 2012.

The idea is that you take a probability map like this computer model:

DELL computer model for the Mt. Rogers case, generated May 2012. DELL is the average of Distance, Elevation, Linear Features, and Land Classification

or this Subjective Consensus:

Subjective consensus for the Mt. Rogers case, from participants at VASARCON 2012.

and score them given the (secret) actual find location.

Repeat the test for many cases, and we hope to get reliable measures of model performance.

VASARCON Slides

We presented our work at VASARCON 2012 a few weeks ago.  The slides are here.  Many thanks to our 8 participants for a very active discussion, and for staying around for an extra 30 minutes of live tabletop exercise and discussion.  We like the suggestion to allow people (or teams) to provide subjective estimates on pre-drawn regions.  Any searcher (or team) could rank themselves on the leaderboard, get some feedback, and contribute to research.

Eric managed the consensus mapping on-the-fly, but it may be even easier if we can learn MapSAR, described in yesterday's post.)

 

Update (2012-10-30): The original images were lost when we moved to WordPress.  I've replaced them with the models from the VASARCON Mt. Rogers Case, and a recent leaderboard screenshot.

 

One thought on “MapScore: A Portal for Scoring Probability Maps

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>