Drug testing: results not all they seem

CALGARY -- Suncor Energy, Total E&P Canada and Canadian Natural Resources Ltd. recently announced a pilot study of randomized drug testing as part of the Drug and Alcohol Risk Reduction Pilot Project. Workers will be randomly tested for alcohol and drugs such as marijuana, cocaine, and methamphetamine. The aim is improving worker safety and reducing accidents on the job.

The Communications, Energy and Paperworkers Union of Canada (CEP) shares this aim but has labelled the drug-testing pilot "an affront to the privacy and dignity of an individual," calling for "broadly based programs focusing on education and prevention" instead. The pilot project points out that the programs CEP prefers are ineffective. CEP notes the same is true for random (and mandatory) drug testing. There is talk of a court challenge.

The accuracy of drug tests is central to this debate. Testing accuracy is more important to our lives than many realize, impacting things such as pipeline integrity, roadside breathalyzers, medical tests, employee bonus evaluations and, of course, employee drugtesting. Grappling with the issue requires some statistical reasoning, but it’s worth it.

The accuracy of any test depends on two factors: sensitivity and specificity. Sensitivity is the test’s ability to detect something — a drug in this case. The higher the sensitivity, the lower the false-negative rate (undetected drug users) and the more effective the testing program. Companies want high sensitivity.

Specificity is the probability the test correctly identifies the absence of a drug. The higher the specificity, the lower the false-positive rate (detecting something that isn’t there) and so the fewer people falsely accused. Workers want high specificity.

 Employee drug-testing accuracy is claimed to be 99 per cent or better. This actually refers to specificity. (Empirical studies estimate specificity as 75 per cent to 99.8 per cent, but let’s use 99 per cent for ease of presentation). This high level is achieved using a two-stage testing process to reduce the number of false positives.

With a 99 per cent accuracy, it seems reasonable to think only one per cent of employees will be wrongly accused of doing drugs. Not so, because of a little thing called the base rate: If 10 per cent of workers are actually on drugs (the base rate), a test 99 per cent accurate will falsely accuse people eight per cent of the time, not one per cent. That means almost one out of every 10 people testing positive will be innocent.

How does that work? Here’s the math. Assume we test 1,000 people. Of these, 100 are intoxicated (10 per cent base rate) and our test will identify 99 of them. Nine hundred are not intoxicated but the one per cent error rate in the test means nine of them will be falsely identified as intoxicated. So in all, 108 people will test positive for drugs, of which nine (or 8.3 per cent) are innocent.

These calculations apply to all tests. The good news? If you just tested positive for a rare form of cancer (one in 10,000 occurrence) using a test that is 99.99 per cent accurate, there is only a 50/50 chance of actually having cancer. (Go to http://www.converge-group.com/?p=562 for an online calculator to test other scenarios.)

The calculation above assumes sensitivity and specificity are equal at 99 per cent. In practice, they rarely are. Empirical studies of employee drug testing puts sensitivity between 38.2 per cent and 68.9 per cent. So, roughly, there is only a 50 per cent chance a drug test will actually detect someone on drugs. Add in avoidance techniques (switching urine, etc.) and the probability of actually finding someone through drug testing drops significantly further. This is why employee drug testing has such a poor record in preventing accidents — it’s just not that effective at detection.

This means companies may not be getting what they think they’re buying with drug-testing programs, and the confusion may lead to a false sense of security. Companies may think the drug-detection problem solved and reduce other detection/prevention programs in response. This produces more accidents. This same confusion can lead to overconfidence with things such as pipeline-integrity testing, leading to disaster.

The uncertainties and probabilities involved in testing mean no single detection program will be effective. Neither will a loose collection of programs. Combine programs the right way, however, so the probabilities are improved, and both false positives and negatives melt away.

For example, drug-testing an employee only after being identified by a fellow employee increases the effective base rate in our favour. If employees correctly identify intoxicated workers 50 per cent of the time, the proportion of false positives from a drug test in our original calculations falls from eight per cent to one per cent.

Not perfect, but not bad either.

Robert Gerst is a partner in charge of operational excellence and research & statistical methods at Converge Consulting Group Inc.

— Troy Media

Republished from the Winnipeg Free Press print edition September 22, 2012 J1