The human challenge of technology-intensive military systems
During the major combat operations phase of Operation Iraqi Freedom (OIF), Army Patriot air and missile defense units were involved in two fratricide incidents. In the first, a British GR-4 Tornado was misclassified as an anti-radiation missile and was engaged and destroyed. The second incident involved a Navy F/A-18 Hornet that was misclassified as a tactical ballistic missile and was also engaged and destroyed. OIF involved a total of 11 Patriot engagements by U.S. units. Of these, nine resulted in successful tactical ballistic missile engagements; the other two (18 percent) were the fratricides.
Although significant in and of themselves, these fratricides opened the door for a unique look at the human performance problems introduced by increasing system and operational complexity in a major weapon system. The initial assessment was followed by a multiyear effort focused on remediating the problems identified during the incident investigation. Lessons and observations from both the initial fratricide assessment and follow-on mitigation work set the stage for of the discussion to follow. The focus of the discussion is the nature of the human challenge posed by technology-intensive military systems and the scope of the personnel transformation that must parallel the introduction and successful use of such systems. The discussion centers on Army systems and practices, but the lessons and observations apply to other services as well.
Given that Patriot has been in the Army’s inventory since the early 1980s, what do lessons from the weapon tell us about system complexity and its impact on future Army systems and on future military operations? In some respects, Patriot provides a glimpse into the future of military systems and operations. As Patriot has evolved over the past 25 years, it has acquired features and characteristics that are more typical of systems the Army will employ in the future than those in the current inventory. This is not to suggest that future systems or their performance demands will be identical to Patriot. Rather, many future systems may result in an operator and crew performance environment similar to Patriot’s. At a conceptual level, many of the performance demands placed on individual soldiers and crews will be similar to those currently found in Patriot.
THE INITIAL FRACTICIDE ASSESSMENT
In early 2004, a team from the Army Research Laboratory (ARL) began looking into the OIF Patriot fratricides and the more general issue of Patriot human-system performance at the request of the then-chief of air defense artillery Maj. Gen. (now Lt. Gen.) Michael A. Vane. Vane was interested in operator and crew performance issues as they relate to the effectiveness of automated air defense battle management systems. He was particularly concerned by what he termed a “lack of vigilance” on the part of Patriot air battle management crews along with an apparent “lack of cognizance” of what was being presented to them on situation displays with a resulting “unwarranted trust in automation.” The project team spent most of the summer and fall of 2004 reviewing the OIF fratricide incidents — reading documents, interviewing knowledgeable personnel, and observing Patriot training and operations. An initial report of findings was delivered to Vane in October 2004.
The ARL team’s assessment report was organized around two central themes, denoted (1) undisciplined automation during Patriot development and (2) automation misuse on the part of Patriot crews. The first contributing factor, undisciplined automation, is defined as the automation of functions by designers and subsequent implementation by users without due regard for the downstream consequences for human performance. Undisciplined automation tends to define the operators’ roles as byproducts of the automation. Every function that can be automated is automated. Operators are left in the control loop to monitor the engagement process and override that process when it is determined that the system’s engagement logic is not accurate. Research and operational experience indicate that this is a difficult role for operators to perform adequately.
Before the first missile round was fired during OIF, the stage was set for the second primary contributor termed automation misuse — specifically, automation bias on the part of Patriot crews. Automation bias is defined as unwarranted over-reliance on automation. It has been demonstrated to result in monitoring failures (vigilance problems) and accompanying decision biases usually involving unwarranted and uncritical trust in automation. In essence, control responsibility is ceded to the machine.
During the initial assessment report, the ARL assessment team was careful not to place too much blame for the OIF fratricides on the Patriot crews involved. In the team’s view, the roots of the crews’ apparent performance shortcomings can be traced back to systemic problems resulting from decisions made years earlier by concept developers, software engineers, procedures developers, testers, trainers and unit commanders. In hindsight, the most surprising aspect of the team’s fratricide assessment is that there really were no surprises. The OIF Patriot air battle management crews did what they had been trained to do, what Patriot’s culture and command climate emphasized and reinforced, and what the literature on human-automation integration suggests might happen under such circumstances.
RECOMMENDATIONS
The ARL team’s report recommended two primary actionable items to address the problems identified during the fratricide incident assessment:
1. Re-examine air defense battle management automation concepts to emphasize effective operator control: Look into ways to mitigate situation awareness problems resulting from undisciplined automation of Patriot control functions.
2. Develop more effective air battle management teams: Re-examine the level of expertise required to employ a system such as Patriot on the modern battlefield.
A later report on Patriot system performance during the second Gulf War prepared by the Defense Science Board reinforced the ARL team’s recommendations concerning the importance of effective operator control and improved operator and crew proficiency for complex systems like Patriot.
From the assessment results, it was clear that Patriot’s primary human performance shortfall was loss of effective control. In a system like Patriot, effective control means that operators and not the automated system are the ultimate decision makers in engagement decisions. Decisions to shoot or not to shoot must be made by crews having (1) the technical potential for adequate situation awareness and (2) the expertise to understand the significance of the information available to them. Situation awareness, the key to effective control, involves far more than a simple display of icons on a screen.
The ARL team’s second recommendation concerned re-examining the level of expertise required of Patriot operators and air battle management crews. In phrasing this recommendation, the team deliberately chose to use the term expertise rather than training. In present usage, the term expertise refers to moving beyond rote battle drills and being able to think critically about an emerging tactical situation making use of an extensive technical and tactical knowledge base. Developing expertise takes job-focused training, appropriate on-the-job experiences, mentoring by expert job performers, and, above all, time. The team’s position, backed by extensive research and direct observation, is that traditional Army training does not provide Patriot crews with the expertise required for effective use. The Army’s own post-fratricide board of inquiry criticized Patriot training for emphasizing rote battle drills over critical thinking and problem solving. In spite of the view in some quarters, the extensive use of automation in battle command does not eliminate the need for operator expertise. One lesson of the Patriot fratricide assessment is that inadequately trained operators coupled with extensive automation can result in a de facto fully automated system.
Delivery of the fratricide assessment report was not the end of the ARL team’s involvement with Patriot. Rather, the report led to a five-year follow-on effort working with the air defense community to implement selected aspects of the team’s recommendations. In keeping with the two actionable items listed previously, implementation work proceeded in two separate areas: (1) reinforcing effective operator oversight of Patriot engagement operations and (2) training improvement.
Reinforcing effective operator oversight. One of the ARL team’s observations during the fratricide assessment and later during testing was that Patriot’s battle management problems had more to do with lack of a well-defined and properly functioning work system to accomplish the objectives of air battle management than with the design of individual equipment items. Patriot’s air battle management work system includes all of the command and control components organic to the unit itself plus any external command and control components having an impact on engagement decisions. Another way of characterizing the battle management work system is the term “kill chain.” As Patriot evolved over its 25-year life, the system’s kill chain expanded to include additional components, both internal and external to the unit. Air battle management became more complex. One of the team’s primary activities during the follow-on effort was to assist in upgrading and then vetting through experimentation and field testing the operational concepts; supporting tactics, techniques, and procedures (TTP); and training essential to proper kill-chain functioning.
A second aspect of reinforcing effective operator oversight involved implementing a control modification termed Weapon Safe. Following the OIF fratricides, it became standard practices in some Patriot units to place their launchers in standby to prevent automatic engagements. Crews apparently wanted a “second look” before permitting an engagement. Placing launchers in standby was not a good solution to the problem of preventing automatic engagements. Consequently, the system was enhanced with the Weapon Safe feature. This had the same operational effect as placing launchers in standby, but without the downside. Under Weapon Safe, operators have to explicitly consent to a track engagement. Weapon Safe represents a step back from fully automated engagement operations.
A mild surprise associated with the ARL team’s design-related work directed at reinforcing effective operator oversight was a continuing need to enhance operator training. Design changes precipitated changes in system usage concepts and TTP, and thus impacted training. Design and training were inextricably linked, and the linkage was stronger than initially assumed. Well-designed materiel still requires well-trained soldiers for effective use.
Training improvement. The Army’s post-fratricide board of inquiry remarked that the Patriot system is too lethal to be placed in the hands of crews trained to such a limited standard. Lack of expertise also showed up in subsequent operational tests involving Patriot software upgrades. To address the expertise issue, the ARL team partnered with the Air Defense School on a project intended to demonstrate (1) what expertise-focused training for Patriot operators and crews would look like, (2) how well it would perform, and (3) how it would be received. Patriot has an embedded training capability and an associated suite of stand-alone trainers used primarily to support institutional training. Unit-level embedded training is supplemented by a supporting facility that provides the external connectivity needed to train all components of the system’s kill chain; that is, to train like Patriot fights.
Personnel at the supporting facility frequently remarked that Patriot crews coming to the facility often required extensive remedial training before being able to make use of the facility’s advanced training capabilities. Training in their home units did not properly prepare them for that critical next step in the training sequence. The ARL team had observed and reported that Patriot’s embedded training environment did not provide a suitable setting for expertise-focused training. That observation was particularly true for the middle segments of the Patriot training sequence where the skills required to take advantage of the supporting facility’s advanced training features were supposed to be developed.
To address this deficiency, the ARL team in concert with school personnel developed a demonstration training capability to illustrate expertise-focused training for Patriot. This capability consisted of training equipment, prototype instructional modules, and military expert job performers to serve as instructors. The most important aspect of the demonstration was implementing a deliberate practice instructional model. A deliberate practice instructional regimen has been shown to be critical to the development of expertise as that term is used here. The demonstration itself was conducted using participants from an operational Patriot unit. Results from the demonstration project were positive with respect to each of the evaluation criteria listed previously. Following the demonstration project, the air defense community warmed considerably to the idea of using stand-alone training equipment to supplement embedded training in Patriot units. Units are now able to acquire this supplemental training equipment. However, there has been little follow-up on the ARL team’s push to restructure air and missile defense battle management training using a deliberate practice instructional model. The team considers this latter action to be the most important aspect of Patriot training reform. Training reform gained some traction in the materiel arena, but the “soft” changes necessary to make more effective use of that new materiel did not occur. This lack of follow-through on the soft aspects of training is unfortunate. More than 50 years of training research across the DoD has shown that instructional design issues often trump issues pertaining to training equipment and simulator fidelity. The bottom line for this body of work is best captured in the statement, “It’s not how much you have but how you use it.
In a training-focused report prepared in 2001, the Defense Science Board warned of an increasing risk that “training failure will negate hardware promise.” A follow-on report issued in 2003 continued this theme and remarked that the future will require more of our people to do new and more complicated things. Meeting this challenge amounts to “a qualitative change in the demands placed on our people that cannot be supported by traditional training practices.” That same report concluded that training reform will be slow in coming because the bureaucracy will not let it happen. Unfortunately, these cautions and conclusions summarize the ARL team’s experiences with Patriot training reform in the five years following the initial fratricide assessment. Old thinking tends to dominate training concept formulation, development and conduct. We keep doing what we have always done. This is an unfortunate reality because the whys and hows underlying the development of expertise are generally known. The challenge going forward is applying this theory and practice in the contemporary training environment.
A related challenge concerns routine personnel administration practices and the Army’s formal personnel system. Over the course of the fratricide assessment, the ARL team observed that Patriot units were very turbulent places personnelwise. Some of this personnel turbulence was attributable to the need to support two ongoing wars; some was attributable to routine personnel transitions; but a considerable amount was attributable to normal workings of the Army’s formal personnel administration system. Patriot and other technology-intensive systems require considerable operator and crew expertise for effective use. Research indicates that developing these levels of expertise requires several years of full-time effort just to reach the journeyman level. In spite of the best intentions with respect to training and job preparation, the Army’s formal personnel system makes it difficult to keep operators and crews in one position long enough to reach necessary levels of on-the-job competence. Unsupportive personnel practices can undo the best laid training plans and practices.
FIXING THE PROBLEM
So what has to be done to meet the challenge summarized above? Generalizing from the ARL team’s five-year Patriot experience, major changes are required in three areas.
First, it is necessary to temper the Army’s near exclusive preoccupation with hardware and technology during system development. The Army’s own post-fratricide board of inquiry criticized Patriot developers and users for their “fascination with and blind faith in technology.” In the ARL team’s view, this preoccupation with technology over soldier performance contributed significantly to Patriot’s problems during OIF. Technology can amplify human expertise, but not substitute for it. A good step forward toward this goal would be to require that the unit of interest for analysis, testing, costing and the like be the manned system in its organizational context. Looking at the hardware alone without crews and outside its planned organizational context provides a biased picture of a system’s eventual cost and performance.
Second, system developers and users must recognize the requirement for expertise in technology-intensive systems. Current training concepts and practices will not meet this challenge. To emphasize this point, consider a comment by DoD’s Director of Operational Test and Evaluation, J. Michael Gilmore, concerning results from the most recent Patriot operational test: “The Limited User Test highlighted the growing complexity of the Patriot system which requires a higher level of operator expertise that exceeds the current Army training standard.”
Institutional and unit training must be reorganized to support expertise-focused training for systems requiring high levels of user competence. This will require that institutions and units be properly resourced to support this type of training. Old practices will have to be changed. In addition, the focus of training development within the Army must shift away from a growing preoccupation with training technology and equipment — analogous to the technology preoccupation encountered in the acquisition arena. This shift will require a renewed focus on designing human-centered training systems that support the acquisition of essential skills.
Third, the Army must make a concerted effort to modify its personnel practices to support rather than impede the development of individual, crew and unit expertise. Increases in system complexity and the concomitant need for highly competent operators and crews require performance-driven personnel practices rather than practices driven by administrative convenience or high-level bureaucratic concerns. At present, the tail appears to be wagging the dog in this respect. Training and personnel practices both impact soldier performance and must be addressed in tandem.
A theme sometimes encountered in the economic development literature is that technology is skill-biased. Technology favors and even requires highly skilled users for its benefits to be realized. As a consumer of technology embodied in the systems they procure, the military services cannot escape this reality. High-tech systems require highly skilled people for effective use. Current concepts for system development, training and personnel administration were developed in an earlier time for use with simpler systems. They must be modified to provide the highly skilled personnel contemporary systems now require. It is a mistake to believe that desired performance levels will be achieved by widgets alone.
JOHN K. HAWLEY is an engineering psychologist at the Army Research Laboratory Human Research and Engineering Directorate at Fort Bliss, Texas. The views expressed here are the author’s own and do not necessarily reflect those of the Army.
Recent Comments