MapScore: A Portal for Scoring Probability Maps

The SARBayes MapScore server has been running for a month now at  It's a portal for scoring probability maps, so researchers like us can measure how well we are doing, and see which approaches work best for which situations.  Take a look.  (And if you have a model, register and start testing it!)

What is MapScore?

MapScore public leaderboard, from October 2012.

The idea is that you take a probability map like this computer model:

DELL computer model for the Mt. Rogers case, generated May 2012. DELL is the average of Distance, Elevation, Linear Features, and Land Classification

or this Subjective Consensus:

Subjective consensus for the Mt. Rogers case, from participants at VASARCON 2012.

and score them given the (secret) actual find location.

Repeat the test for many cases, and we hope to get reliable measures of model performance.


We presented our work at VASARCON 2012 a few weeks ago.  The slides are here.  Many thanks to our 8 participants for a very active discussion, and for staying around for an extra 30 minutes of live tabletop exercise and discussion.  We like the suggestion to allow people (or teams) to provide subjective estimates on pre-drawn regions.  Any searcher (or team) could rank themselves on the leaderboard, get some feedback, and contribute to research.

Eric managed the consensus mapping on-the-fly, but it may be even easier if we can learn MapSAR, described in yesterday's post.)


Update (2012-10-30): The original images were lost when we moved to WordPress.  I've replaced them with the models from the VASARCON Mt. Rogers Case, and a recent leaderboard screenshot.


This entry was posted in MapScore: A Portal for Scoring Probability Maps and tagged , , , , . Bookmark the permalink.

One Response to MapScore: A Portal for Scoring Probability Maps

  1. ctwardy says:

    See the Project Page for our 2012 slides and posters from WASAR, NASAR, and MORSS. Eric presented alongside BYU's Ron Zeeman at WASAR & NASAR. Charles presented the poster at MORSS.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>