The MapScore project described here provides a way to evaluate probability maps using actual historical searches. On a metric where random maps score 0 and perfect maps score 1, the ISRID Distance Ring model scored 0.78 (95%CI: 0.74-0.82, on 376 cases). The Combined model was slightly better at .81 (95%CI: 0.77-0.84).
Our MapScore paper is now in press at Transactions in GIS! From the abstract:
The MapScore project described here provides a way to evaluate probability maps using actual historical searches. In this work we generated probability maps based on the statistical Euclidean distance tables from ISRID data (Koester, 2008), and compared them to Doke’s (2012) watershed model. Watershed boundaries follow high terrain and may better reflect actual barriers to travel. We also created a third model using the joint distribution using Euclidean and watershed features. On a metric where random maps score 0 and perfect maps score 1, the ISRID Distance Ring model scored 0.78 (95%CI: 0.74-0.82, on 376 cases). The simple Watershed model by itself was clearly inferior at 0.61, but the Combined model was slightly better at .81 (95%CI: 0.77-0.84).
A logical extension of the Distance Rings model is to fit a smooth function to the distribution of data found in ISRID. Examining the Euclidean Distance data for different categories, it was found that a lognormal curve roughly captured the shape of the data. The Log-Normal (LN) is a two parameter distribution which assumes that the logarithm of your data follows a normal distribution. The probability density function of the LN curve is given by, where are the mean and standard deviation of the logarithm of distance.
Thanks very much to summer intern Jonathan Lee (@jonathanlee1) for many MapScore fixes. Jonathan is a keen Python programmer with extra geek points for running Linux on his Macbook Air and having an ASCII-art avatar. He learned his way around Django in no time and brought us a slew of features and code refactoring including: Continue reading "MapScore Updates Summer 2014"
The SARBayesMapScore server has been running for a month now at http://mapscore.sarbayes.org. It's a portal for scoring probability maps, so researchers like us can measure how well we are doing, and see which approaches work best for which situations. Take a look. (And if you have a model, register and start testing it!)