Gallup TSA poll biased?

by Bill Fisher on August 9, 2012

A recent Gallup poll was reported in Politico and other sources saying that most Americans believe that the TSA’s procedures are effective.

From Politico:

About 54 percent of Americans say the Transportation Security Administration is doing an excellent or good job while another 30 percent said TSA is doing a fair job, according to a Gallup poll released Wednesday afternoon. Twelve percent of respondents said the security arm is doing a poor job.

Ironically, this comes on the day when two more TSA screeners in Atlanta were indicted for drug trafficking, bringing the total number of TSA workers charged with smuggling contraband through security to 12 in 20 months.

Interestingly, in a recent interview, TSA Administrator John Pistole acknowledged that a Wall Street Journal poll indicated that “customer satisfaction with his agency, according to a recent poll, is 38 percent, although some airline travelers may think that is on the high side.” The article also states:

The negative perception is 43 percent with respondents, according to The Wall Street Journal, mentioning a perception of ‘TSA incompetence and overstepping its authority.’ Anybody who flies regularly has seen both.

But there’s plenty of nuance that these articles neglect. Reading the Gallup version of the report and the supporting detail provides a different view.

A favorite phrase of TSA spokesmen is “The screeners have to succeed every time, the terrorists only once.” This statement implies that that the structure of the questions may have led to some bias in the results, since, according to TSA’s own philosophy, this is a pass-fail proposition. If the results are recast using this premise, then they can be distilled into two categories: Reliable and Not Reliable. The Extremely  and Very Effective results would qualify as being in the Reliable category, and the Somewhat, Not Very, and Not Effective would be in the Not Reliable category. (There’s no explaining how the ubiquitous No Opinion people can be so oblivious to reality that they can’t form a thought, but that’s for another discussion.)

From the Gallup detail:

56% 44%
How Effective do you think TSA’s screening procedures are at preventing acts of terrorism on U.S. airplane?
Reliable Not Reliable No Opinion
Extremely 9
Very 32
Somewhat 44
Not Very 8
Not 5
No Opinion 4

When these results are aggregated into logical categories, the majority — 51 respondents of 98 with an opinion — found these procedures Not Reliable. Put another way, only 41 of 98 respondents, or 42%, found TSA procedures Reliable, while 58% of respondents found them Not Reliable.

With the question regarding whether TSA is doing a good job, the results are slightly more favorable but still less than stellar for an agency that told Congress that “everyone loves us” only last week. Applying a similar pass-fail analysis, the recast results from the Gallup poll are not as rosy as the articles would imply. Instead of reliability, the standards would simply be Acceptable or Unacceptable. Both Excellent and Good are Acceptable, and the remainder are Unacceptable.

From the Gallup detail:

Do you think TSA is doing an excellent, good, only fair or poor job?
Acceptable Unacceptable No Opinion
Excellent 13
Good 41
Fair 30
Poor 12
No Opinion 4
TOTAL 54 42 4

In this breakdown, 54% find the procedures Acceptable, while 44% find them Unacceptable. Given that the poll allows a 4% margin of error on a sample of 1,014 respondents, the poll indicates that Americans are largely divided on whether the agency is doing a good job or not.

The article also implies that frequent fliers are equally impressed with TSA’s performance, but the details do not support this conclusion. The article states “Frequent fliers and sporadic travelers offered similarly positive views about TSA, as did fliers with or without young children, often a sore subject in the news media.”

A review of the results, however, reveals that a whopping 75% of respondents made less than two round trips in the past year, and nearly half have not flown at all. Only 12% would remotely qualify as frequent fliers, making 5 or more round trips in a year.

How many round trips have you taken in last 12 months?
None 48%
1-2 27%
3-4 13%
5 or more 12%

Other questions also remain unclear. The poll failed to ask if people had a favorable or unfavorable opinion of the TSA workers they encounter. Anecdotal evidence, eyewitness testimony, and comments on published TSA stories would predict that the answer to this would have been overwhelmingly negative.

Gallup didn’t disclose who sponsored the poll or if the TSA or one of its vendors had any input on it. As we all know by now, pollsters can get any result they want by controlling the way questions are asked and the group that is polled. This would explain why other Gallup polls, particularly those on political issues, are often at odds with other equally respected polls.

The fact remains that no matter how many fluff pieces or favorable polls get published on behalf of the TSA, many Americans are sick and tired of this agency and its corrupt workers.

(Photo: Flickr Creative Commons/Michael Kreil)

Previous post:

Next post: