A fifth-generation fighter takes more than 1,500 measurements a second over every conceivable aircraft parameter. Yet the most important part of the fighter, the pilot, doesn’t have a single measurement recorded during flight. This is in a day and age when nearly anyone can record a half-dozen physiological data streams in his quest to become fitter or healthier, including a log of alpha rhythms to diagnose sleep quality. For an elite athlete or corporate executive, the sky is the limit in terms of quantified physiological parameters.
The Defense Department has flirted with the concept of human performance monitoring and augmentation over the last several decades, but the idea has never become a central tenet in either manpower or acquisition planning. Now, as the services prepare to reduce force strength over the next four years, they must find a way to do more with less.
One solution is to take a page from the growing performance quantification in the civilian world to develop the vision of a “quantified warrior.” From preventing mishaps to increasing mission efficiency and effectiveness and spurring acquisition reform, the revolution this technology can realize is manifold. In this article, we lay the foundation for a sense-assess-augment framework for human performance augmentation and demonstrate how the DoD can realize its mission impact in both the near- and far-term.
The Sense-Assess-Augment Paradigm
Data has become a weapon in its own right in the 21st century, with the potential to harm friend and foe alike. In a drone attack that killed 23 Afghan civilians in 2010, the primary cause of the accident cited by Air Force and Army officials was information overload. Operators, who were monitoring the drone’s video feeds while simultaneously engaging in dozens of instant-messages and radio exchanges with troops on the ground, failed to mentally account for the children they acknowledge they saw in the video feeds. Additional research by the Army Research Laboratory showed that when soldiers operated a tank while simultaneously monitoring video feeds, they failed to see targets right around them.
The danger of information overload can only be expected to increase. It’s also clear more manpower won’t be forthcoming, nor can technological advances in autonomy be expected to solve the problem, as we detailed in this journal in October 2011 (“The Autonomy Paradox”). The question is: Can human performance augmentation fill the gap? The Air Force Research Laboratory recently led a workshop on the topic. Participants noted that autonomy research has shifted some tasks from man-in-the-loop to man-on-the-loop, allowing the human to perform multiple tasks while still presumably providing supervisory control. This has been critical to cyber defense strategies and control of multiple unmanned aerial systems by a single pilot, for example.
What’s needed now is to “close the loop,” where the physical and mental states of the operator are fed back into the weapon system, making the human a more seamless part of the overall system. Thus, a sense-assess-augment framework was developed to guide the application of the human performance augmentation into systems engineering across the services.
Step 1: Sense
Historically, real-time physiological monitoring wasn’t necessary. The most sophisticated piece of machinery a Marine took into battle was his rifle. The metric for successful flight operations was time of useful consciousness. But as mechanical systems have been replaced by information systems as today’s weapons of choice, the sensing of executive function, rather than mere consciousness, becomes central to mission success.
Sensing is now the most mature piece of the paradigm, thanks to considerable commercial investment in athletics, health care and productivity. Sensors exist or are in development that can measure a huge range of parameters, such as brain activity, eye movement, skin temperature and biological performance markers (for example, blood glucose levels or molecules that indicate the onset of fatigue). To measure an individual’s capacity to multitask, problem-solve and reason, suites of sensors will be required. Measures might include sleep quality and duration, levels of stress biomarkers such as corticosteriods, heart rate, or even self-reported changes in dietary consumption or mood. In concert, sensor suites can act like a kind of “check engine light” for individual soldiers and operators.
This is why the pulse oxygen sensor recently introduced to help diagnose what’s happening in the F-22, while a step in the right direction, will likely be disappointing. It’s not enough information alone to pinpoint the problem for investigators, which may in the end have a collection of causes. Without real-time data during the incidents, investigators can only infer a cause based on the symptoms described — a laborious process. In addition to measuring blood-oxygen levels, a sensor suite could capture breathing and heart rate, gas mixtures in exhaled breath, glucose monitoring and eye-tracking. This would provide a much better complement to the aircraft flight measures being used to aid investigators.
Step 2: Assess
Of all the steps, the ability to interpret data from multiple, individual sensors and merge it into actionable information in a timely manner is the most difficult. The first challenge is one of time scales. For example, sleep quality and duration are measured on a daily basis, while heart rate is reported as beats per minute. To get an accurate picture of executive function using these measurements, how do we integrate that information across minutes and days? What does real-time monitoring mean with this kind of data mismatch? The second challenge stems from the fact that different tasks will seek different outcomes. Some tasks will focus on efficiency, while for others, the main concern may be safety. These outcomes will in turn drive different types of assessments, suggesting that mass production of performance augmentation across missions will not be feasible. Instead, it will be designed for each task and desired outcome. Researchers will need to work closely with operational units to make progress, which could be difficult given current operational tempos.
Performance assessments need to be quantified relative to an individual baseline collected over time. To say a soldier is tired or injured doesn’t reveal how likely it is he will complete or impede the mission. But if it were possible to know, for example, when a soldier’s ability to accurately shoot a target was decreased by 25 percent, a better decision as to how to address the symptom of fatigue could be made. Nor should these assessments occur only at the tactical level. Those higher in the chain of command are just as likely, if not more so, to be suffering from lack of sleep and exercise, poor nutrition and information overload that can impair decision-making.
Finally, the results of such assessments should be objectively interpreted and incorporated into the relevant training, tactics and procedures. In essence, units would now be able to quantify operational availability and readiness for the human component as part of an overall assessment of the weapon system. A good deal of research, looking at both model and data-driven algorithms, is necessary before this is possible.
Step 3: Augment
Human performance augmentation isn’t new, but our understanding of how to apply it has advanced beyond merely creating exoskeletons for extra strength. For example, Stanford University researchers discovered that muscles don’t fail due to a lack of fuel such as glucose; instead, they get overheated. The Defense Advanced Research Projects Agency gave them money to create a glove for special operations units that pulls a slight vacuum while cooling the veins of the hand, what the researchers refer to as the “radiator of the body.” The vacuum prevents the veins from constricting while exposed to the cold temperatures, rapidly cooling the blood returning the heart and lungs. This rapid reduction of core body temperature after exertion provides a jolt of rejuvenation, allowing users to instantly double or triple their performance threshold. Tests showed that use of the glove provided performance enhancements that were the equivalent of taking human growth hormone for 18 months, without the undue side effects.
The challenge is to create augmentation systems that don’t impinge on the ability to execute the mission. Many troops are already maxed out in terms of weight or power requirements. The glove for special operations forces, for example, is about the size of a coffee pot. This means the device cannot be worn continuously, but must be unpacked for use when needed. Given the kinds of situations where one might anticipate its greatest benefits, such as in the midst of battle, it might not be worth taking along on deployments.
The possibilities for augmentation extend beyond physical enhancements. DARPA has explored the concept of “nutritional armor,” where food supplements improve resiliency and prevent suicides. Mild brain stimulation is being tested for its ability to accelerate learning and reduce training times, potentially allowing maintainers to morph into cybersecurity specialists within weeks. Electroencephalogram technology can pick up when intelligence analysts detect an object of interest, even before they are consciously aware of it, allowing them to process images more quickly.
It sounds far-fetched, until you hear about USA Cycling’s covert operations in 1996 to improve performance in preparation for the summer Olympic Games. Nicknamed “Project 96,” it isolated and optimized eight crucial components of cycling performance. Some were features of the bike itself, but the majority concentrated on the rider, examining diet, psychology, aerobic capacity and lactic acid tolerance. What was revolutionary about the program was that it isolated and quantified each component individually, then optimized the set of parameters for each athlete.
The result? The team won more medals than it had since 1984, a time when competition was reduced due to the Olympic boycott. Ten years later, one of the team members had traded the life of an elite athlete for that of stressed-out CEO. He began experiencing a range of health issues and was informed by his doctor he was at high risk for heart attack. Using the same sense-assess-augment strategies employed in Project 96, he not only reversed his health issues but in the process set a world cycling record at the age of 35 — a feat previously thought biologically impossible due to declining testosterone levels.
The corollary for military acquisitions and operations should be clear. Stressed manpower, increased complexity and sustained operations mean the services can’t solve their problems through technology alone. A weapon system can no longer be evaluated or enhanced in isolation from its human operator. In the earliest stages of the acquisition cycle we need to start testing and designing for the critical parameters of joint human and machine performance.
The Way Ahead
Although the concept of human performance augmentation has been technically feasible for some time, it’s failed to deliver more than modest gains. Without the paradigm described above, the sensing and augmentation communities have largely worked independently and the assessment piece has lacked a research leader to make significant progress to bridge them. If there is one lesson from the decades of dabbling in human performance augmentation, it is the necessity and interdependence of the three pieces of the paradigm.
Sensing without assessment is frustrating. It is, in fact, one of the most common complaints of consumers trying to make sense of the athletic, health and productivity data they are collecting. Many ask: What does the data mean and how do I alter my performance accordingly? Augmentation without the sensing and assessment components is not only potentially dangerous, but breeds distrust among the public and policymakers. For example, the Air Force pilots responsible for the friendly fire deaths of Canadian troops in Afghanistan in 2003 implicated “go pills” as the cause of the accident. Although the official investigation found no contribution of the drug to the outcome, the public and media were not persuaded. Physiological monitoring and assessment might have provided objective proof that the cause was poor judgment by the pilot, not a side effect of a widely used drug.
What the sense-assess-augment paradigm offers is a data-driven feedback loop that can improve mission performance and inform personnel and acquisition decisions. In the near term, the DoD should use existing commercial sensors to explore operational assessment in defined and relatively controlled environments. Training missions offer the best access and control, without jeopardizing critical mission outcomes.
In the far term, what’s needed is a concerted strategy by the services that looks beyond sensing or augmentation in isolation. To do that, we suggest the following:
Position the DoD laboratories as “owners” of the assessment piece through existing scientific communities of interest. The DoD can rely on industry for the sensing piece and potentially buy the augmentation piece, but assessment is uniquely a DoD mission. Existing communities of interest, established by the Office of the Secretary of Defense in areas like autonomy, should develop a strategy that sets broad technical milestones and allocates collaborative resources, both people and dollars, to achieve them. Given the constrained resource environment, this will likely require redirecting resources away from some existing sensing and augmentation projects, while building better collaborations with industry and academia on these topics.
Create a recursive, systems engineering view of sense-assess-augment in a weapons system demonstration that views human performance as an integral system parameter — not separate and foreign as it is done today. As already stated, assessment is task specific and can’t happen inside the laboratory alone. Ideally, the scientific communities of interest would identify acquisition programs that were at appropriate levels of technical maturity and stood to gain the most from human performance augmentation. Spiral developments of unmanned aerial systems, which are used by all the services, may serve as an appropriate test system.
Conduct a legal and policy review early to examine the ethical boundaries of this construct for U.S. armed forces, especially given the potential breadth of possibilities. In 2003, the President’s Council on Bioethics authored a report on biotechnology, saying, “Even in moments of great crisis, when superior performance is most necessary, we must never lose sight of the human agency that gives superior performance its dignity.” This is the crux of the concern for many. While it is deemed acceptable to heal the sick and wounded, the support for “beyond therapy” outcomes is more varied, especially in a military environment where voluntary participation is questionable. The DoD must embrace legal and ethical discourse and transparency early in the process to avoid public-relations nightmares that disparagingly equate such technology with turning a soldier’s body into “just another piece of equipment.”
The challenges to human performance augmentation appear significant: declining budgets, stressed ops tempo and a history of negative public opinion. However, the world, and with it the nature of war, is changing at an ever-increasing pace. It is unlikely we can accelerate our acquisitions to effectively counter more nimble adversaries without investing in and augmenting the most critical weapon system in the inventory: the human.
Jack L. Blackhurst is the director of the Air Force Research Laboratory Human Effectiveness Directorate. Jennifer S. Gresham is a visiting research scholar at the Florida Institute for Human and Machine Cognition. She previously served 16 years in the Air Force and is a reservist for the Air Force Office of Scientific Management. Morley O. Stone is the chief scientist of the Human Performance Wing at the Air Force Research Laboratory.