Syrotuck's main study is his 1976, with N=242. But he gives much more detail about distance travelled in his 1975 paper, breaking distance down every 0.2 miles. Unfortunately, he only reports probabilities, not numbers, and doesn't even report total N. We know he got more data between 1975 and 1976, but didn't know how much. Is the 1975 breakdown representative of the 1976 data? Unfortunately, no one has Syrotuck's original data. But we re-created it. (Spreadsheets available!)
How to turn a USGS map or satellite photos into vector data? A couple of links, and some questions.
Back when Adam Golding and I prototyped Probability Mapper we already had an algorithm that could give the probability based on distance, terrain, vegetation, and other factors. But we were working with raster images, so we only had distance. If you had vector layers for terrain & vegetation, you'd be set.
~Originally 9 June 2008
Don Ferguson's paper shows how they used ESRI GIS software on an actual search. Obviously he's developed some SAR-specific layers. Imagine if the probability map could be auto-generated...
Yes, I'm getting in touch with him.
We have been collecting data on land SAR incidents in Australia since 2000. In November 2003 we wrote a draft report that was presented at the NATSAR council but not generally released. It was styled after the U.K. report. In June 2006 we released the final version (see below), which has evolved its own style.
Our data was collected using the form available below (or an earlier version thereof). The form itself helps define what is meant by each term or category, and is essential to interpreting the data. We have also prepared a definition key in the report, to explain our terms more precisely. The UK report gave us the idea, but there are some differences in definition.
(Written 2004 and updated 2007. Exhumed and posted to this WordPress site Feb. 2013. Please pardon the formatting. In Feb. 2013, the Internet Archive still had the original 2004 post and the updated 2007 post.)
Comments on O'Connor 2004
Back in 2004, I commented on a piece by Dan O'Connor. At the time, some people found it useful. Had I written it now, it would have been a blog post. So, here it is again.
Things have moved on a bit since then. In the summer of 2006, Rick Toman and Dan O'Connor organized sweep width experiments in the Berkshires, and provided a "Camp David" style setting for ISAR and MSAR folks to meet. (Unfortunately, it shamefully took me a over a year to finish analyzing that and get the spreadsheet back to Rick. I think I still haven't posted it here.) As I note below, there have been developments in adding MSAR ideas to new versions of standard of SAR software. But, as an exercise in blogging, here is the original post again, with minimal edits to get it displayed properly here.
Daniel O'Connor submitted a white paper for commentary. The paper is titled "Controversial Topics in Inland SAR Planning". The February 2004 draft is available from:
These are my comments on that draft.
The paper argues that Inland SAR (ISAR) is not just a makeshift beast, but "in the family tree spawned by Koopman," and that ISAR methods are"adaptations to the practical realities of moving from the two-dimensional plane of the ocean to the multi-dimensional land search." Daniel O'Connor argues that some of the techniques from Maritime SAR (MSAR) are "flatly [no pun intended?] inappropriate".
I very much admire this effort and think the final version will play a valuable role in the discussion. I think, however, that this draft overstates the case, and hope my comments will be helpful. I apologize for the critical nature of many comments --- it is easier to notice things I disagree with.
Perhaps O'Connor takes MSAR more narrowly than I do. It sometimes seems that he sees MSAR as wedded to current drift models and the specifics of looking for distressed boats. Clearly he thinks MSAR is wedded to grid searching techniques. No wonder he's suspicious of it!
Rather, I take MSAR to be mathematical search theory more generally, which provides optimal resource allocation given an POA map and detection profiles. How you update POA in absence of search is of course task dependent. In MSAR it is current and leeway. In ISAR, it will be wandering.
I often wish Jack Frost would write shorter comments. Sadly, I find I've written a long batch. So here is the quick summary:
Detailed comments below, grouped by sections of the paper.
I think open and closed systems are equivalent. So O'Connor is right to argue that ISAR can keep ROW if it wants. However, advocating that we need ROW and open systems is a red herring. One may be more convenient, but they are both equivalent.
It is trivial to go back and forth between open and closed systems.
If you start with a closed system, and later discover that you were
wrong, you have to expand your closed system. So you re-scale the
initial POA estimates, and update from there. This takes 3
steps. (1,p.9) So "closed" systems can do what "open" systems
do. They don't have to have all areas pre-defined.
On the other hand, the extreme case of open system is that you start
with 1 area: ROW. Then you subdivide it into Here and ROW, and search
Here. With more resources or time, you can subdivide ROW again, just
as you can divide Here. So ROW is really just a convenient way to
make your open system a closed system.
Also, if you initially left 5% to ROW, and you start expanding your
search area on the boundaries, should you really confine yourself to
5%? No. Usually you expand not because you have so thoroughly searched
the initial areas, but because you have a new scenario. That means you
are re-estimating your initial POA. Just like for a closed system.
ROW does provide a convenient way to track search success: when ROW
has a POA greater than the rest of the search, it may be time to go
home. But if ROW starts at 5%, that's equivalent to considering home
when POS is 95%. Just as convenient.
I'd like to make a few other points relevant to the table on
p.8:
The comparative example: the point of course is that grid searching is
unrealistic in ISAR. Yes. And MSAR techniques generally presume grid
searching. Yes, they have. And although there is no reason they must
(I can model hasty searches fine), there is also less reason to use
them until grid searching is necessary. It's not a limit of theory,
it's a limit of pragmatics: why bother dragging out the computer if
you're still doing hasty searches? POD on a hasty search is near 100%
(along the feature only), and the darned subject might wander back
onto them after search anyway. MSAR will only be useful here when
software can start quickly, has GIS maps with paths noted, and
ideally, has moving-person models. (But then, the same complaints have
been made about invoking ISAR theory at this stage.)
A couple of notes on the comparison though.
Dan, your MSAR example is unrealistic if the search object is really a
vessel. MSAR would never assign such a tight track spacing, because it
makes the coverage absurdly high, and wastes effort.
But if the target were a person in the water, you would have a
coverage of 0.8 and a POD of 67% (in ideal search conditions), because
the measured sweep width of a fixed-wing aircraft for a person in the
water is 0.1 NM. Numbers from (3, appendix I, esp. Table I-5 and
Figure I-12.).
The same search over clear flat land would also waste effort, because
the sweep width is 0.5, giving a coverage of 4, and a POD above 98%.
Of course, that drops quickly as you add vegetation, hills, haze, etc.)
Now your comparison goes through: grid searching large areas is not
feasible for grunts. But then, ISAR rarely has to search 100 square
miles. If we did, your calculations suggest we should use aircraft.
Consider: even in poor visibility and dense vegetation, it looks like
16 aircraft could get a 70% POD in those 5 hours.(note 1) Granted, the correction factors in
IAMSAR may not be severe enough, but clearly if we had to search such
large areas we would use other resources. And we do.
But the fairer comparison uses our real initial POA map, because if
we're dealing with ground searchers, we probably aren't covering such
large areas. (Or if we are, as in the shuttle search, we accept that
it's going to be a lot of people for a long time.)
Our initial POA map tells us our subjects will be close in, or on/near
navigational aids. Likewise, MSAR initial POA maps also constrain the
area to far less than a "theoretical search area" given a boat
travelling at cruise speed!
Given a POA map and a set of resources, MSAR is applicable. However,
one could argue that in most land cases, it is
unnecessary. Unnecessary because we don't need a computer to track
POS when our POD along paths is 100% and our initial search area is so
constrained that we can cover it quickly. But then, that's just when
we don't need CASIE either.
No problems with the first page. Definitely POD does not overrule
POA. Likewise, your conclusion not to use priority schemes that bias
towards POD over POA. Have to use both. (By the way, what schemes
emphasize POD over POA?)
On the second page you make a stronger claim. As applied to God-given
POA and POD values, it does not make sense, but I think you are
applying it to normal estimates, esp. with current practice. You
should make that clearer.
Your claim is: given choices of two assignments with equal POS, we
should always prefer the one that had higher POA. I think I see your
point. I think you have one good reason and one bad one. Also, since
you have not mentioned search time, I'm presuming them equal. So let's
look at your two reasons.
| Note: Things have happened since I wrote this in 2004. Most notably, in 2006 & 2007, O'Connor & Lovelock (and probably others) have created a Windows version of CASIE (WinCASIE III), that incorporates some optimal resource allocation algorithms, including one from Washburn. Also, Search Manager is moving in that direction, and has some decent GIS support. --crt [2007] |
Actually, the reason MSAR proponents don't advocate CASIE or Search
Manager is that the resource allocation algorithms are notably
sub-optimal.
The one in CASIE wouldn't be too bad if the POD values could be
trusted. But since the ones used aren't related to effort expended,
they are suspect. Not entirely CASIE's fault. If they could be
trusted, and you had enough computer time, CASIE would eventually find
the best solution available that didn't split (or combine?) resources.
Using stuff from Washburn, you can do better. Either you can allow
splits and get truly optimal OPOS, or you can use that true optimum to
greatly speed the search for the best CASIE-style optimum. CASIE uses
a brute-force search.
Search Manager used to use some very dodgy hacks for search
priority. I know Martin put a lot of work into revising those, but I
haven't looked at the new version. My fault, I should.
Dan: who has been advocating going back to pencil-and-paper? I get the
feeling we're not talking to the same MSAR people!
If the initial probabilities had to be locked down, you might be right
here. But as noted above, and in (1,2), they can easily be revised on
the fly as you get new scenarios, new experts, etc.
My ideal system would do this:
That is all part of MSAR.
False sense of precision: yes, always a risk in using computer
models. We can ameliorate it by entertaining and tracking multiple
scenarios, and can track uncertainties, but it is always a problem
when you run a computer model. Same argument applies to using CASIE
though.
Maybe better to say POS=32%, meaning that of our original 40
units, we've accounted for 32, leaving 8.
(I don't know if it's clearer, but you could also do it the
other way. Since, the original probability of missing the
subject was 20%, after searching, there is a .2 x .4 = 8%
chance the subject is still there.)
In MSAR, "sweep width" is not something you can decide.
You should use "track spacing" here instead. Likewise,
"lateral range" is not just half the track width, so don't use
that term, even later when describing ISAR grids.
The phrase "POD or coverage" suggests they are
identical. Maybe "high coverage (hence high POD)".
Note 1: Here, in addition to the correction factors for bad weather etc., I have used the exponential curve for "poor" detection, which should be applied whenever navigation is not perfect, or generally conditions are in any way bad. It's lower than the main curve in Figure I-12. The corresponding chart in IAMSAR gives it. So do many of Frost's publications. That's the main point of this footnote.
For mathies, the "poor" value is easy to compute: [ 1 -exp(-C) ], where C is coverage. In this case, coverage is easy to compute. It's just sweep width divided by spacing. Sweep with is 0.5 NM times a bunch of correction factors making it 0.02. Spacing is 1/8 NM. So we get C = 0.02 x 8 = 0.16.
That gives POD = 0.14. But if we had 8 planes, we multiply coverage by 8. Then we have [ 1 - exp(-1.28) = 72% ].
You're still reading this footnote? Wow. You must be really interested. Well, here, let's spot a slight fudge above. If I'm using the "poor" search curve, it's likely that navigation is not perfect. So in theory I shouldn't use the "easy" formula for coverage, since our tracks may be wobbly. But I don't think it's going to matter for this example. Please, go back to the text now.
The Australian Centre for Field Robotics is doing a lot of good work on Bayesian search theory and sensor fusion, for multi-agent robotic systems.
Igor Carron briefly describes one of their recent technical papers in this short blog post . The abstract begins: This paper describes a decentralized Bayesian approach to coordinating multiple autonomous sensor platforms searching for a single non-evading target. Unfortunately, the article has restricted access.
However, I recommend spending awhile at Igor's blog. Follow links about Jim Gray, Bayes, SAROPS.
Thanks to Bob Koester for the reference.
"Now," said Rabbit, "this is a Search, and I've Organized it------".
"Done what to it?" said Pooh.
"Organized it. Which means---well, it's what you do to a Search, when you don't all look in the same place at once."
The following paper creates landmine probability maps almost precisely in the way that I have wanted to create lost person behavior probability maps. (I'll add some more notes on it later.)
(This is recovered from an old history sidebar for SORAL. The code was updated in 2008 for the AGMSAR package.)
Version 2.0 release scheduled for 28 Feb 2003.
Jan. 2003: The code is now fully redocumented using dOxygen!! The public interface can be seen here. The private interface is available to developers upon request and right now is available here. In both cases the Developer's Manual is available under the "Related Pages" link in the header. Thanks to André Oboler!
Early 2003: Charles Twardy plans to reanalyze the Virginia data, correcting for some problems in last year's run. In February, we will analyze the Australian data for the draft report.
Dec 2001: In preparation for the Australian data, Adam Golding analyzed the Virginia data. Cluster analysis revealed only 4 or 5 types of lost person, assuming Gaussian (bell-shaped curve) types.
Adam Golding and Luke Hope then tested several machine-learned models, Syrotuck's model, and a simple model estimated by Rik Head. There were strong differences in predictive accuracy, but negligible differences in a more meaningful score, information reward. The most recent presentation of this work was in Charles Twardy's presentation to the NASAR 2002 conference in Charlotte, NC (June 2002).
