John K. Hawley’s letter in the January/February issue requires clarification, particularly regarding “automation” vs. humans in the loop [“Controlling armed robots”]. The Patriot missile fratricide of Pennsylvania National Guard troops was not an “oops moment” of automation, but was caused by incompetent human programmers, engineers and management. If I recall the facts correctly, the “sophisticated” system properly detected and tracked the incoming live Scud warhead accurately. However, that particular warhead was performing slightly outside the parameters of the programmer-installed database for that type of Scud. The humans who programmed the database and the basic system said that since the parameters did not match, it must be an error of the target and showed the operators a completely blank screen.
Rule One of designing combat equipment or sensors is that you must design for the basic function first, not the fancy sophistication or databases (which constantly change in warfare and the real world). In the case of a defensive radar/missile system, the No. 1 function is to tell the customer about anything that is incoming. If the target does not fit the database, tell the operator that an “unknown bogey” is coming in, or something is out there and inbound, but never ignore it.
These are not “automation vs. human control” issues. These are issues with arrogant and incompetent programmers and managers who concentrate on using mega amounts of lines of code and “sophistication” for the sake of justifying their salaries and existence vs. making the systems as simple, foolproof and functional as possible.
If programmed correctly, the robots will not do anything stupid or dangerous without first asking for a human intervention decision. We need to instill that into the programmers and managers.
Recent Comments