Counter-IED Analysis: Hidden Dangers, Hidden Answers
A tactic for saving soldiers from roadside bombs in Iraq and Afghanistan lay buried in data.
In 2004, an ever-growing number of coalition forces were falling prey to a hidden danger in Iraq. Concealed behind guardrails and road signs, under piles of debris, even within animal carcasses, improvised explosive devices (IEDs) had killed 145 coalition forces in just the first half of the year. Gen. John Abizaid wrote in a memo, “IEDs are my number one threat in Iraq. I want a full court press on IEDs.” Within months, CNA would join that effort with a major analytical push to find solutions in the data—solutions as deeply concealed as any camouflaged explosive.
That summer, three CNA analysts were visiting a Marine Corps base in Arizona when, Ed Michlovich recalls, an officer “practically grabbed us by the collar.” His fellow Marines were getting blown up every day in Iraq, and he pleaded for analysis to demonstrate whether a secret counter-IED effort was saving lives. The tactic was expensive and at risk of being cancelled.
“It was that moment of perfect clarity one sometimes gets,” says Michlovich. “I thought to myself, ‘this is why I joined CNA.’ So I pummeled him with questions for two hours.”
Statistical analysis to counter an IED conundrum
The officer had four months of data on the dates and times of IED detonations with and without the defeat tactic, but no clear answer as to whether it had made any difference. Other experts had dismissed the idea that any analysis of such data could confirm his hunch that the preventative tactic helped. “You can’t prove a negative,” they had argued.
If anything, hearing of those naysayers steeled Michlovich’s determination to take on the challenge. This was CNA’s reason for existence: to tackle the problems that were too complex, too ill-defined, or simply too hard for others to solve.
Michlovich returned to CNA’s headquarters to mull over a spreadsheet of “sketchy” data, most notable for what it was missing. How would he determine how this intervention compared with no intervention at all? It was as if he had the results of a medical experiment in which all the data on the control group had been deleted. “There was a hidden meaning in that spreadsheet,” Michlovich believed, “if I could just find the key to unlock it.”
He figured that key lay in statistical analysis. Michlovich had used it for his physics Ph.D. dissertation, to compare concentrations of rare elements in meteorites. Now, he wanted to use it to find the “signal” of effectiveness in the “noise” of war. This required developing a suitable statistical approach for these unique circumstances, and the way to do this was not obvious. For weeks, the question followed him everywhere. It swirled through his head as he lay in bed.
At last Michlovich thought of a method for creating his control-group data. On the computer, he plotted the dates and times of all detonations, with each IED explosion represented by a dot on a grid. Then he stripped out the data for all times when the secret counter-IED tactic was in effect, and let the computer resample the remaining data to fill in the dots. Each resampling would show a probable array of detonations that would have taken place without any intervention.
It took nearly a minute for the software to run one iteration, and Michlovich wanted 1,000 for a reasonable distribution of probabilities. He stared at his monitor for the first several, as the dots on the grid were translated into numbers of probable detonations in the absence of any intervention. Over and over again, the iterations came up with numbers that were higher than the actual detonations under the influence of the counter-IED tactic — the first solid evidence that the tactic was actually protecting soldiers. “Good Lord,” the analyst muttered to himself. “This actually is a real thing.”
Briefing counter-IED research to the military brass
The military leadership was so hungry for any insight into the counter-IED fight that Michlovich would soon find himself briefing these results to the Chief of Naval Operations, to Brig. Gen. Joseph Votel — director of the Joint IED Defeat Task Force at that time — and finally to Gen. Abizaid, commander of United States Central Command.
Abizaid appeared almost as a silhouette when Michlovich presented his analysis in the darkened briefing room on MacDill Air Force Base in Florida. The general and his deputy commanders were silent throughout. “Mostly what I noticed was the glistening of all the stars on their shoulders, reflecting the dim lights above them,” he says. The analyst finished with a recommendation that the defeat tactic should not be cancelled, but rather expanded — and held his breath for the reaction. Finally, Abizaid thundered: “Why hasn’t anyone shown me this before?”
In Iraq, the counter-IED tactic was tripled in scope. And at CNA, the demand for analysis of the counter-IED fight took off. “The more people saw the power of analysis, the more they realized they needed it,” Michlovich says.
CNA’s Counter-IED Analysis Team Effort
Along with manager Maureen Wigge, he began assembling a team that would include Ciro Lopez, Charles Nickerson, David Broyles and Mary Lauer among its key members. Field representatives embedded in Iraq and Afghanistan improved data collection and added on-the-ground impact to the work at CNA headquarters. When Votel transformed his task force into the Joint IED Defeat Organization, he asked CNA to lead its “defeat the device” analysis effort. Dozens of CNA analysts would contribute to the counter-IED fight over the next several years.
The pressure on the analysts built as the number of IEDs targeting coalition forces in Iraq skyrocketed through 2005 and 2006, to more than 3000 a month. U.S. soldiers were dying at much higher rates than they had in the original invasion. Field reps worked feverishly, and headquarters analysts frequently labored seven days a week and into the night, sometimes leaving the office after their colleagues in Iraq had started their next day of work. “It was brutal,” says Michlovich, “but days meant lives.”
Again and again, insurgents adapted to counter-IED tactics, improving their technology. U.S. forces answered back with new counter-tactics, each demanding analysis to learn which were most effective. “The enemy was changing at such a rapid rate,” says Nickerson. “We had to keep up.”
Soon the challenge shifted from too little data to too much. “Everyone was like — Boom! — Get this guy data,” says Michlovich. Computer models now had to accommodate such details as GPS data on the movements of vehicles in a region.
In one particularly complex statistical analysis, the first iteration was taking so long that the team calculated their standard 1,000 iterations would demand thousands of years of computing time. “We had to do some on-the-fly software development to quicken the process,” says Michlovich. “Those were tense days.”
One reason for the deluge of data was the input from CNA field representatives in Iraq and Afghanistan. This was the first extended conflict in history in which headquarters analysts had continuous contact with the front lines over secure channels. Some of the first CNA deployers supported Joint CREW Composite Squadron One (JCCS-1), set up to help equip military vehicles with jammers to block the radio signals used by insurgents to detonate most IEDs. Field reps rushed relevant data back home and relayed results to commanders and troops. They also conducted their own on-site analyses to support the command’s operations.
Geographical data became invaluable as guerillas constantly shifted the location of their IED activities. So CNA’s Mary Lauer taught herself to use geographical information system (GIS) software to map the locations of discovered IEDs in Iraq. Lauer, whose four deployments to the region put her on the ground for well over a year in total, was later deployed with a U.S. Army Explosive Ordnance Battalion in Afghanistan. She helped planners use GIS to predict where they should concentrate their IED searches as certain routes became more or less dangerous.
CNA’s counter-IED work began to gradually wind down from 2010, but recognition of its significance would linger. The Military Operations Research Society awarded Michlovich the 2007 Rist Prize for the best operations research to influence major decisions, based in part on a recommendation from Gen. Votel. But the real reward for the analysts who committed themselves to the counter-IED fight was the knowledge that they had made an impact. It was the awareness that they had performed, in the words of Votel, “exceptional and life-saving work.”