UAVs, MH370, Prediction Markets

In which I discuss two-and-a-half approaches to crowdsourcing Search & Rescue, and invite you to try one -- namely, mine: http://bit.ly/1eBBMkq

In which I discuss two-and-a-half approaches to crowdsourcing Search & Rescue, and invite you to try one -- namely, mine.  (SciCast)

UAVs & MicroTasking for SAR

Patrick Meier at iRevolution has several great articles on UAVs and crowdsourcing for disaster response. Here are two:

The “x” in xUAV refers to expendable.

Every year, the Patterdale Mountain Rescue Team assists hundreds of injured and missing persons in the North of the Lake District. “The average search takes several hours and can require a large team of volunteers to set out in often poor weather conditions.” So the University of Central Lancashire teamed up with the Mountain Rescue Team to demonstrate that UAV technology coupled with crowdsourcing can reduce the time it takes to locate and rescue individuals.

Once the UAV uploaded the photo with the target, it took their crowd 69 seconds to identify the (simulated) lost person.

However, in a larger search, there is the problem of false positives.  The crowdsourced imagery site tomnod.com reports that on a (successful) search for a downed plane, they had 17,775 "wreckage" tags.  All on the actual plane?  Unlikely.  In the crowdsourced image search for Steve Fosset, they found a lot of flying planes, unrelated items, and to their credit, some actual unsolved air crashes.  But not Fosset's plane.

In the now ongoing search for Malaysian Air Flight MH 370, the TomNod servers have had trouble keeping up with demand.  And people are finding things, because, well, there are things to find.  Here's their current list of finds:

MH370 sightings count from TomNod.  All these sightings are probably false positives.
MH370 sightings count from TomNod. All these sightings are probably false positives.

But... right now we haven't found the aircraft yet.  It's now almost certain the plane flew for over 6 hours and is probably nowhere on this map at all.  (The best lead is now off the southwest coast of Australia.)  So as Steve Simmons says, all marked items are false positives.

Prediction Markets for SAR?

For the past three years, my day job at George Mason University has been to run a large crowdsourced forecasting project.  For two years we were the DAGGRE geopolitical market, and at the sponsor's request, we are now the SciCast Science & Technology market.  We think the S&T domain will make better use of our "combinatorial" features, especially conditional probabilities.  Probability of Detection (POD) is a conditional probability.  It's the probability you would have detected the target given that the target was in the area to begin with. But today let us talk of Probability of Area, POA.

In WiSAR, we often create an initial probability map (when we bother at all) by asking a bunch of experienced planners to rate the various search regions, and then averaging. This is the initial "consensus".  It's not really a consensus, but it is a form of crowdsourcing or model averaging, for a small crowd.  What if you had a large crowd?  You could have everyone complete their map map online, and average.  (Liz Sarow at ESRI has created a demo for this, but I don't know if it's still running.)  Effectively, the usual approach is a survey with a map interface.  It's simple and familiar, but as the crowd increases, we're getting a lot of redundant information.  We could be using the crowd more efficiently.

Another option is to show everyone the current consensus, and ask them to find the one region which is most wrong, and fix it.  "Edit out of outrage".  If the crowd is large enough, this will be the same as the survey, but it will take far less time per person.

But one region is too restrictive, so instead give them a finite number of points, and tell them to use those points wisely, to make the best improvement they can.  The bigger a change they make to some number, the more points it costs.  They can choose to make a big change to few regions, or small adjustments to many.  Or just leave it alone, if it's fine.

If you reward people for increasing the probability of (what turns out to be) the actual find location, you have created a prediction market, a forecasting tool which encourages frequent updates as new information becomes available, and gradually gives more influence to more accurate forecasters.

MH 370 Prediction Market

Doing this right will require a nice map interface.  I don't have one right now, but nevertheless I have created three linked questions on MH 370: http://bit.ly/1eBBMkq .  Anyone can view the current probabilities, but you have to register to edit or  comment.  (Registration is free, and open to anyone over 18.)  The questions are:

  • What happened?  (The Scenario)
  • Where is it? (Original version, all regions close to Malaysia)
  • Where is it? (Extended version, from Kazakhstan to the Southern Ocean)

You can adjust any probability directly, or adjust a conditional probability like:

  • Chance of being in the original search region given pilot incapacity or error.
  • Chance of being in the original search region given a deliberate diversion.

Current scenario weights, based on 56 forecasts (including mine):

Scenario chances on 21-Mar based on 56 forecasts.
Scenario chances on 21-Mar based on 56 forecasts.

The extended search region is using this map:

Extended search region map (from http://i.imgur.com/zNgnicG.png?1).

Regions are the Northern and Southern satellite arcs, the original regions around Malaysia, Other land, and Other water.  And "Not found by 30-APR".  (In a prediction market, we want all questions to be resolvable, so often we put a "use-by" date on them.)  The extended search region question is logically linked to the original search region question.

If our funding continues into next year, we may get a chance to build actual map interfaces.  In the meantime, please explore this experiment in continuously-updated, scenario-driven, probability "maps".

 http://bit.ly/1eBBMkq 

Author: ctwardy

Charles Twardy started the SARBayes project at Monash University in 2000. Work at Monash included SORAL, the Australian Lost Person Behavior Study, AGM-SAR, and Probability Mapper. At George Mason University, he added the MapScore project and related work. More generally, he works on evidence and inference with a special interest in causal models, Bayesian networks, and Bayesian search theory, especially the analysis and prediction of lost person behavior. From 2011-2015, Charles led the DAGGRE & SciCast combinatorial prediction market projects at George Mason University, and has recently joined NTVI Federal as a data scientist supporting the Defense Suicide Prevention Office. Charles received a Dual Ph.D. in History & Philosophy of Science and Cognitive Science from Indiana University, followed by a postdoc in machine learning at Monash.

2 thoughts on “UAVs, MH370, Prediction Markets”

Comments are closed.