SARBayes -- Comments on Cooper et al

Home	Blog	Projects	Methods	Downloads	Credits	Links

Notes on Cooper et al 2004: Compatibility of Land SAR Procedures with Search Theory

Comments by Charles R. Twardy

About a month ago, Dan O'Connor posted a draft White Paper examining the differences between traditional land SAR theory and what he called Maritime SAR theory [2]. I quickly wrote a reply [3]. Shortly thereafter, Cooper et al released the compatibility report [1]. I said that I would provide a short summary and review. I have had a hard time making it short, and also took the time to get comments from Dan O'Connor and Jack Frost: many thanks for their help.

Some more context: the compatibility report is supposedly written for the broader SAR community, but appears to be written ``at'' its authors most vehement opponents. Consequently, most any land SAR reader may feel attacked. I hope here to provide a summary and evaluation that helps readers get beyond that feeling.

The recent compatibility report is generally right, and is a valuable reference, but so negative and overkill that I fear it may actually prevent people from seeing the merits of its method. The main message would go something like this:

Land SAR theory stemming from Syrotuck, Kelly, and Wartes failed to properly re-invent modern mathematical search theory.

Fair enough. But we might be so annoyed by the finger-wagging tone that we miss some very interesting things (for example: Wartes may not have been talking about POD after all!). The authors may have personal justification for this tone. But I hope that this is the last document to bear the scars of past battles.

Also, many have the impression that the authors want to remove hasty searches, containment, and other sensible tactics, and replace them all with ``grid'' searching. The authors have strongly denied this elsewhere recently: they do not discuss hasty searches etc. simply because those are fine. The report concerns the way to allocate resources when the search has extended beyond the reflex tasks -- the ones where you really do need to crunch numbers.

First, two acronyms. O'Connor used ``MSAR'' for ``Maritime SAR theory.'' Let's keep the acronym but call it ``Mathematical Search Theory,'' for two reasons:

The theory itself knows nothing of sea or land.
No one is suggesting we adopt maritime-specific tactics.

O'Connor contrasts MSAR with ``ISAR'' or ``Inland SAR Theory''. Because ISAR is a NASAR course, I'll use ``LSAR'' for ``Land Search Theory''.

Brief summary of the critique

On page 85, the report gives a concise and clear summary of the key shortcoming, and why it could cause problems.

The ultimate problem, and all others pale in comparison, is that the land SAR community has had no standard method for relating probability of detection to effort density. That is, POD estimates are not based on any estimate of ``detectability'' (sweep width) or any detection function that relates POD to the level of effort and size of the area over which it was expended. A land search manager could easily assign two equally matched search teams for equal times to two segments equivalent in all respects except that they were significantly different in size. Then, according to current (highly subjective) procedures, the same POD could be assigned to both without anyone even being conscious of the inconsistency. This situation is the reason all previous efforts by land SAR practitioners to develop valid optimal effort allocation procedures have failed.

[1, 85]

Now, maybe you wouldn't do that, but I hope it is at least clear that if it did happen, it would be wrong. And the report demonstrates that as written, LSAR could allow or even encourage that.

So what to do? There is a nice bullet-point summary on page 98. I encourage you to start there.

The report is a good reference. it combines several earlier critiques in one place, including a glossary and bibliography. The LSAR literature survey is also valuable, if dismissive.

Key constructive points

So what did Syrotuck et al get right? And how can we interpret (or reconstruct) their work inside mathematical search theory?

Multiple vs Single (68-70):

Full marks for not only dismissing a persistent error, but for also showing what was right behind people's intuitions about multiple low-POD searches. Most of us learned that for the same level of effort, multiple low-POD passes gave better POD than if we combined them into a single (high-POD) pass. That is not true. But what is true in typical cases¹ is that our POS goes up faster if we cover the whole segment at low density first. And we want our POS to go up as fast as possible. Our final POD is the same, for the same effort.

The big surprise? It seems that this is actually what Wartes said!

Spacing vs. optimal allocation (74-75):

Much LSAR theory was about spacing within a region, and not about the optimal allocation of effort among regions. Only that distinction got lost. Wartes (1974, 75) comes off well again here, but was misinterpreted. Thus people suggest even high-priority segments get low initial coverage, not to be re-searched until later. Oops.

I think this insight goes a long way towards understanding the debate. Another way to make the distinction: best way to search a segment given your known resources, versus how many resources to assign to the segment.

Searcher vs team (58):

Wartes (and others) does not examine detection on a per-searcher level. Right! Another key difference: MSAR assumes the individuals are the detectors. But LSAR treats the team as a detector. They want to know how to adjust the detection profile of the team to match what is needed. The only parameter they really have is spacing. Well, does it work? Not yet. Maybe it can't, but I haven't seen the disproof yet.

Critical separation (65, 73):

Although they do it as a critique, I think the authors have done a good job reconstructing critical separation in MSAR terms. Specific claims of critical separation may well be wrong, but I think on page 65 the authors give a valid reconstruction of what C.S. is saying in terms of a detection function, that is, a lateral range curve. ``Wrong'' is different from ``incoherent''. In particular, as formulated one can ask ``how wrong'' and estimate an answer. My rational reconstruction² is that we can imagine LSAR folks saying, ``Gee, we don't have sweep width. But let's write down a reasonable lateral range curve as a first approximation, and work from there.'') I hope to write more separately.

POD & Searcher Spacing

I misread the report as saying POD was unrelated to searcher spacing. Now, Joe searcher knows that a line with searchers every meter will have a higher POD than one with searchers every 10 meters. So it's hard to take the authors seriously if you think they're denying that. They're not.

Once we know effort, POD is unrelated to searcher spacing. If Joe thinks about it, he also knows that if we merely decreased the spacing of the original crew, but gave them no more time, the segment POD would not go up. But I think LSAR folk tacitly assume that tighter spacing means more time in the field, or more people, or both. On those assumptions, tighter spacing means more effort, hence more coverage, so it does mean more POD.

POD and spacing are related. Nevertheless, it is more accurate to express things in terms of ``effort''.

Quibbles and overstatements

Information theory:

As already noted on sar-l, information theory is not ``essentially incompatible'' with search theory. In fact it can be used to derive many of the results independently. Jack Frost graciously agreed after reading the same paper that convinced me. But I agree that Ken Hill's paper on that topic was way off-target.

Proportional POA:

The authors disparage non-numeric versions of the Mattson consensus because they do not preserve ratios. In theory, good point. In practice, I think numerical Mattson consensus is unlikely to fare much better because

People are demonstrably bad at estimating probabilities.
We probably can't expect accuracy of more than 10% anyway.

Jack Frost suggests, in correspondence, an Olympic-style rating system. I think that's a good idea, and hopefully we can get some time to work on it, using the literature on probability elicitation for expert systems.

Segments vs. Regions:

``Segments'' are the portions of the map you block off for searching. ``Areas'' are regions of probability, usually regions with uniform probability density. The authors claim it is wrong to use segments as the initial areas for the Mattson consensus. I'm not yet convinced.

One argument is that ``there is no logical connection between where a subject is likely to be and how a segment can be searched.'' (45) I disagree. Segments easily searched by ground searchers are the same segments easily traversed by the subject.

Another argument is that we typically have many segments, and not nearly enough information to distinguish that many probability regions. I concede that may be true.

The authors also complain that guidelines for segmenting are vague: they do not specify the level of coverage nor the size of the search team. OK, but if a bunch of search managers gave roughly the same segmentation, then the vagueness is only in the writeup. I suspect there's just not much variation. Any takers?

Conclusions

So should we continue using LSAR? I learned LSAR, and for a short time, helped teach it in FUNSAR. I switched to MSAR not because I thought LSAR was completely unworkable, but because it was comparatively ad-hoc: it required more epicycles and addenda to get it to work, and didn't allow the same insights. In short, I found MSAR a more unified and powerful theory.

But any new theory should explain the successes of the former theory. Until now, I think most of the discussion has been focussed on showing the flaws. Although that is necessary and worthwhile, I am glad to see some attention to addressing some correct intuitions amid the flawed (or plainly wrong) statements.

Bibliography

1: Don C. Cooper, John R. Frost, R. Quincy Robe, and Potomac Management Group.
Compatibility of land SAR procedures with search theory.
Technical Report DTCG32-02-F-000032, Department of Homeland Security, U.S. Coast Guard Operations, 2003.
Report tasked by the Research & Development Working Group of the U.S. National Search and Rescue Committee (NSARC).
Available from http://www.uscg.mil/hq/g-o/g-opr/nsarc/LandSearchMethodsReview.pdf and other places.
2: Dan O'Connor.
Controversial topics in inland SAR planning.
White paper, Northeast Wilderness Search & Rescue (NEWSAR), Needham, MA, February 2004.
Second draft for review and comments. The URL now links to the Feb. 27 version with extensive comments by Jack R. Frost.
http://newsar.org/White%20Paper.htm
3: Charles R. Twardy.
Notes on ``Controversial topics in inland SAR planning''.
Commentary, February 2004.

About this document ...

Notes on Cooper et al 2004: Compatibility of Land SAR Procedures with Search Theory

This document was generated using the LaTeX2HTML translator Version 2002-2-1 (1.70)

The command line arguments were:
latex2html -t 'Comments on Cooper et al' -split +0 cooper04

The translation was initiated by Charles Twardy on 2004-03-18

Footnotes

... cases ¹: Where the segment is basically one kind of terrain and vegetation. Those conditions lead to a uniform detectability and, if the area is not so large that distance-from-PLS comes into the picture, also a uniform Pden.
... reconstruction ²: Not, I stress, what actually happened.

This page last modified Sep 22, 2007