Some (solicited!) technology and engineering advice for the TSA


UPDATE BELOW.
I was pleased recently to receive an email from Russell Wooten, the IT Strategy Branch Chief of the TSA. His email reached me through my membership in the Maryland chapter of the Institute for Operations Research and Management Sciences (INFORMS). For the uninitiated, operations research is the discipline of applying advanced analytical techniques to help make better decisions. Mr. Wooten was soliciting input on these questions:

Do you have any ideas on how the Transportation Security Administration (TSA) can improve its capabilities in utilizing its current security technology, upgrading its security technology, or improving its security processes?

What can/should we do to improve these technologies and their processes?

AIT, Advanced Imaging Technology
AT, Advanced Technology
BLS, Bottled Liquid Scanner
EDS, Explosives Detection System
ETD, Explosives Trace Detector

My response read:

Dear Russell Wooten,

Thank you for asking for our input about improving the TSA’s security technologies and processes. This is a topic I have taken a keen interest in over many years, and one that frequently appears in both my public lectures and as classroom examples. I’ve also presented my findings in a Congressional briefing last May and on the web at tsanewsblog.com.

Bottled Liquid Scanner (BLS): The biggest question mark surrounding the bottled liquid scanner is why it hasn’t been deployed at checkpoints to clear large containers of liquids and eliminate one large headache for travelers who’ve had their items confiscated. Instead, the bottled liquid scanner has been deployed against coffee drinks purchased after the checkpoint, which only confuses passengers who don’t realize that airport vendors are bringing large quantities of unscreened supplies into the secure side. Perhaps it would be best not to emphasize that vulnerability posed by airport vendors – all the more reason to use the bottled liquid scanner at checkpoints instead.

Explosive Trace Detection (ETD): The ETD technology is designed to detect materials used in explosives, though identical chemical compounds could be detected in medications, lotions, fertilizers, and other innocuous sources. As I understand the TSA’s procedures (though of course the public is not permitted to know what the TSA’s procedures are), when an ETD alarms that passenger is flagged for a secondary search in a private room. However, there is an extremely low base rate of passengers trying to bring explosives through a checkpoint – let’s use 21 in 10 billion, though of course it should really be 2 in 10 billion, since the weapons used on 9/11 were boxcutters rather than explosives. Using Bayes’ Rule, we can calculate that even if explosives detection technologies were nearly perfect: catching every actual explosive and only falsely alarming on one in every ten thousand passengers, then only one out of five million positive test results actually indicates presence of an explosive device. The psychological impact of processing millions of false alarms without a single real hit guarantees that screeners implementing the secondary screening process will assume that every positive is a false positive, undermining the effectiveness of any secondary screening process that could possibly be designed.

In similar fashion, screener psychology defeats secondary bag searches. Passengers report that when a bag is flagged at the checkpoint, screeners search the bag looking for a prohibited item – and stop looking when one is found, even when multiple prohibited items are present. This is reported to be how one passenger was allowed to fly with a clearly labeled 5-pound block of C4 in his carry-on.

What to conclude? ETD and other technologies that flag passengers for extra screening based on any test can never be made to be effective, because the base rate fallacy dooms even a near-perfect flagging system. The false positive rate would need to be something like an unattainable 1 in 1 million chance of a false alarm on a non-threat passenger in order to achieve a sustainable ratio of false alarms to real alarms. Resources being spent on ETD should be directed elsewhere.

Advanced Imaging Technology (AIT), or colloquially, body scanners: The only thing we need to know about the Rapiscan backscatter X-ray machines is the conclusion of the Committee to Assess Health Risks from Exposure to Low Levels of Ionizing Radiation, National Research Council, National Academies, which said after a comprehensive review of the available data that: “the risk of cancer proceeds in a linear fashion at lower doses without a threshold, and the smallest dose has the potential to cause a small increase in risk to humans.” Further, “The committee has concluded that there is no compelling evidence to indicate a dose threshold below which the risk of tumor induction is zero.“

While there are no similarly compelling data to suggest health risks from millimeter wave scanners, the AIT scanners have been publicly exposed as ineffective, most recently by blogger Jon Corbett who defeated the machines with a sewing kit, but also on German television by a man who set off a rather large explosion using the items he brought undetected through an AIT scanner. Also, a peer-reviewed publication pointed out that low Z explosives would be indistinguishable from the human body, so moldable explosives in the shape of a pancake would likely be undetectable. For that matter, any object of appropriate size could be hidden under a bag of water in any biologically plausible shape. Other obvious weaknesses are trivially exploitable: threat items in the mouth, under feet, or within an obese passenger’s folds of skin.

Also, AIT systems slow passenger throughput at the checkpoint, so using AIT will certainly make passengers less safe. A recent RAND study of airport vulnerabilities at LAX concluded that “small, portable explosives have been the most likely and most lethal means of attacks at airports” and that “The greatest risks for casualties for most types of attacks are in the high-density areas passengers encounter before reaching the security checkpoint, particularly lines for ticketing and for passing the security checkpoint.” Thus, AIT is not only ineffective, it is actually dangerous because it leaves passengers waiting in long lines vulnerable.

But surely the most glaring weakness of the AIT system is that one can plan one’s flights to avoid AIT scanners, as I have been doing ever since they came into use. Thus, we can be utterly certain that AIT technology has never played even the slightest role in discouraging an attack, since any adversary who feared AIT might discover his plot could simply choose, say, Reagan’s terminal A or Fort Lauderdale’s Southwest terminal for his departure. From the Congressional Research Service’s recent report, we know this wide-open door for anyone to fly sans AIT will remain open: “Even at full operating capacity, not all airports and not all screening lanes will be equipped with AIT under TSA’s plan.” Only innocent travelers will ever be screened with AIT – terrorists can evade it easily.

Neglected risks: The TSA appears to have blind spots for some threat vectors. The insider threat looms large after a number of high-profile arrests of screeners for smuggling drugs through the checkpoint. Bribing a screener to speed explosives through a checkpoint (perhaps unwittingly) would clearly be a winning strategy for a terrorist. Many passengers also report seeing large pallets of food and merchandise bound for secure-side vendors being waved through the checkpoint or delivered through unscreened corridors.

I’ve highlighted these two risks because there appear to be no fundamental barriers to addressing them. One way to address insider threat is to hire fewer part-time screeners, both because they may have a more tenuous sense of loyalty than full-time screeners, and because fewer screeners means fewer opportunities to find a compromisable person. Another good practice would be to search screeners arriving for their shifts, in checked baggage rooms as well as at passenger checkpoints. Searching vendor supplies is as straightforward as it sounds. In contrast, perhaps the TSA chooses not to address the risk of ground-launched missile attacks on planes because of a dearth of effective countermeasures.

The TSA has also entirely failed to consider the risk that TSA procedures will divert would-be flyers onto the roads. Blalock, Kadiyali, and Simon found that a decrease of 1 million emplanements leads to an increase of 15 driving-related fatalities. If negative publicity and the accumulated impact of negative passenger experiences at the checkpoint causes only a 1% drop in emplanements, then the TSA is responsible for 100 or more deaths each year. While it’s true that the TSA will probably avoid blame for these deaths, it is simply unacceptable engineering practice to neglect this obvious side effect of the TSA’s screening choices.

I would be happy to meet with you and/or collaborate on a more detailed analysis of any of these issues. I am deeply concerned about the ways that the TSA makes us all so very much less safe.

Sincerely,

Sommer Gentry
INFORMS member, Maryland chapter

UPDATE: Sommer Gentry reports that it’s been two months since she sent this letter. She has yet to receive an acknowledgment from the TSA, let alone a considered reply. -Lisa Simeone

(Photo: Bitman/Flickr Creative Commons)

Published by

Sommer Gentry

Sommer Gentry is a mathematician who specializes in Operations Research. Operations Research is the discipline of applying advanced analytical methods to help make better decisions, including optimal allocation of resources and rational responses to risks. She has a B.S. and M.S. from Stanford University and a Ph.D. from MIT, and is a tenured Associate Professor of Mathematics.

  • R

    Once again Sommer,

    Stupendous.

  • I just checked up on Russell Wooten. He is on LinkedIn. And as far
    as I’m concerned, he ALREADY KNEW the answers he was going to get. Now my question for Mr. Wooten is, did you know about this presentation in May and if so why did you not attend?

    Executive Management, IE, OR/MS, PM, Modeling & Simulation, Analytics

    Current:

    US Department of Homeland Security (DHS) /
    Transportation Security Administration (TSA),
    Washington Academy of Sciences,

    Washington DC Council of Engineering and Architectural Societies (DCEAS)

    Previous:

    US Department of Homeland Security (DHS) /
    Transportation Security Administration (TSA),
    McDermott Technology Inc., R & D Division,
    Akron General Medical Center

    Education:

    Master’s Degree, Major:

    Operations Research and Management Science at Cleveland State University;

    College of Business Administration

  • RB

    UPDATE: Sommer Gentry reports that it’s been two months since she sent this letter. She has yet to receive an acknowledgment from the TSA, let alone a considered reply. -Lisa Simeone

    There was no reply because TSA didn’t really want any input from anyone. TSA knows it all!

  • the actual purpose of TSA procedures is Terrorism Theater.

    With that in mind, recommendations should be of the sort that maximise fear and intimidation in the passengers.

    Eg. there should be more and varied useless tests and many more false alarms – each false alarm should be greeted with much shouting and waving of deadly weapons, etc. (.. without explicitly stating the additional tests would be useless and alarms obviously false, recommendations of this type that extend terrorism theater would have got a positive response from TSA)

  • Fisher1949

    This is a technical tour de force on screening systems and procedures that will almost certainly exceed Mr. Wooten’s mental capacity given that he is a TSA worker which usually equates to being from “bottom of the barrel”.

    TSA’s arrogance will undoubtedly lead them to dismiss Dr. Gentry’s thoughtful and well reasoned recommendations and pursue the ham handed methods that these intellectual midgets prefer.

    An excellent piece of writing Sommer.

  • I’ve got to hand it to you Sommer, you have the science to back up the
    obvious, and you delivered it in a perfectly polite fashion.

    In
    the best case, if Russell Wooten had printed out your response and
    personally delivered hard copies to his bosses and talked about it, I
    seriously doubt it went down well. I fear for Russell Wooten – he had
    the good sense to find the right
    person who can address these questions, and he comes out of an
    organization that more closely resembles a combination madhouse/reform
    school. I wonder if he is
    still employed by TSA. He is the polor opposite of Blogdad Bob.

  • What a great response, powerfully written and elegant in its succinctness. You’re to be congratulated, Sommer.

    When you report that “ETD and other technologies that flag passengers for extra screening based on any test can never be made to be effective” it’s hard to think of any recommendation short of “well, then, stop doing things that are expensive, invasive, and don’t work.” Unfortunately, our intuitive response is to double-down with more scanners, more technology, more “layers” of security.

    After $60 billion and a decade of ever-increasing restrictions, it’s going to take some real profiles in courage to say “we were wrong.” While Vindictive Me celebrates Congressional hearings in which the TSA is publicly lambasted, Rational Me says we need to find a way for the agency to save face while quietly dismantling some of its silliest sacred cows.

    Perhaps as a start (and it’s only a start) we could re-purpose the agency’s PR and lobbying budget (including very public faces like Blogger Bob and eighteen Twitter accounts) to explaining to America that while, statistically, airplanes are infinitely safer than their own bathtubs, it is impossible to eliminate all risk.

    We still drive cars, despite 33,000 traffic deaths a year, because the economic utility of driving far outweighs its risk. Precisely because a single terrorist act (or even threat) is far more spectacular than a collection of individual car crashes, it will be an uphill battle to suppress our intuitive desire to over-react, particularly with the now-entrenched interests of the security-industrial complex (I’m talking to you, Rapiscan) fighting to maintain the status quo.

  • Sommer, your letter is so comprehensive, so well-written, and so founded on facts, logic, and statistical evidence. My sincere thanks to you. As Chris says, please keep us posted on its impact, which I hope will be significant and profound.

    • “…so comprehensive, so well-written, and so founded on facts, logic, and statistical evidence….” which is why it is difficult to get Congress’s attention.

      Great job by Sommer, as always.

  • 1amWendy

    Brava, Sommer – once again you explain, in easily understandable and logically irrefutable terms, the basic unemotional and scientific reasons why current TSA policies are folly.

  • Susan Richart

    Chris, did you mean to say “if the TSA responds”?

  • Excellent analysis. Please let us know how the TSA responds.

    • Unfortunately, I can report that two months after sending this, I haven’t received so much as an acknowledgement, let alone a response.

      • Sommer, none of us can be surprised. The entire TSA is window-dressing; therefore, so are their PR and “outreach” efforts.

      • TSAisTerrorism

        Well, you didn’t write a letter telling them that everything they did was right, so really (to them) your opinion isn’t valid. I’ve never seen a more insular, solipsistic, and selfish group of “people” in my entire life.

        Though you should know that it is a very excellent, very well written letter!

        • *Sigh* This is so true. TSA only seem to listen to the suck-ups. I was especially angered by the representative for the flight attendants (at the hearing the other day): she parroted the whole “TSA makes us safer/makes us feel confident about flying” B.S., and noted some example of how the metal detectors of yesteryear didn’t catch a big old knife that some guy walked through the checkpoint with. Obviously this woman was oblivious to the fact that metal detectors have sensitivity settings, and those have been dialed up considerably since 9/11. I know this because when I flew to NY a week or so after the airports were re-opened, not only could I not wear a belt without alarming, the underwire of my bra set off the beepers, and one of the pre-TSA guards had to run the wand over me to make sure I wasn’t carrying a weapon!

          There are few things that I despise more than disingenuous shills–with ulterior motives–twisting facts and assuming everyone is stupid. Grrr.