Performance to Plan: Big Data Keeps Fighters Flying

Performing to a Plan

Even before the 80% target was set, data scientists at CNA were deeply involved in the initiative that would catapult the revival: Performance to Plan, or P2P. The idea was to use predictive analytics to project future performance, and to identify the drivers that can generate improvements. “The secretary of defense was tired of unwarranted optimism, tired of looking at graphs that drew a straight line between where we are and where we want to be, without analysis,” says Dr. Gregg Schell, who led the Super Hornet P2P analysis for CNA.

CNA assembled historical maintenance, supply and personnel data, incorporating it into a new model to forecast the percentage of mission-capable Super Hornets with 97% accuracy. It identified two drivers out of the 20 in the model that had the greatest impact on readiness: the quantity of ready spares produced at the two major Fleet Readiness Centers and the number of mechanics and technicians at the squadron level. Navy aviation began to focus on those issues.

But the model also came to a sobering conclusion. Even if the Navy could match its best historical performance in spares and staffing, it would fall well short of the 80% goal. The whole maintenance process needed to be fixed. That finding triggered an effort to enlist private-sector expertise. The Navy adopted a suite of best maintenance practices from commercial airlines, while CNA monitored the impact of each change. CNA’s Data Science team had to develop increasingly sophisticated models using massive datasets that tracked the status of each individual aircraft.

Precision Targeting

The P2P process required quarterly meetings of leaders from the Pentagon and Navy aviation, where Schell would report on the latest refinements of the models and their findings. “When they first learned that they were going to be the guinea pigs for P2P, they were not excited,” recalls Schell. “But over time, they went from ‘this is impossible’ to ‘we can definitely do it.’”

One of the most significant findings highlighted the value of a novel measurement called the Aviation Maintainer Experience score, or AMEX. An AMEX score quantifies the experience of each member of a maintenance crew, by their qualifications and their specialty and paygrade. With big-data analysis of AMEX scores, CNA’s identification of the drivers of readiness went from broad (“maintainers at the squadron level”) to precisely targeted (“supervisors in safety equipment aviation structural mechanics”). One of the most important findings from AMEX modeling was that because the Navy deployed its most skilled mechanics and technicians to sea, chronic maintenance problems piled up at air stations ashore. A modest balancing of the distribution of experienced maintainers from sea to shore could have an outsized impact on readiness. “AMEX is helping us put the right person with the right skillset in the right place,” said Adm. Miller, who retired in 2020 as the Navy’s “Air Boss.”

The success of P2P with Super Hornets inspired the Navy to set up a series of P2P projects. Today CNA’s Data Science Division is supporting P2P efforts to improve performance in shipyards, submarine maintenance and human resources, in addition to the aviation P2P, which has expanded to other aircraft.

Schell gives much of the credit for the turnaround in Super Hornet readiness to the sailors on the line, who rapidly adapted to new approaches suggested by data — and who are an important source of that data. “I was not on a flight line turning a wrench; I wasn’t personally involved in the process change,” he says, “but we were certainly a partner with the Air Boss in identifying drivers of the change to get you to 80 percent.”