British FAA: Cockpit Computers Unfit For Human Consumption 2
Hal beats Dave at chess
Sam: "Why show me that?"
John: "The British FAA documents that while some airlines tried better screen prompts or improved training to fix these problems, each failed. Pilots never beat the Computer."
Sam: "Unbelievable."
John: "Dr. Lisanne Bainbridge explains, 'Designers put computers in planes because computers remember more and make quicker decisions, but there's no way a pilot can check if the computer is following its rules correctly. So if the computer is there because human judgement and reasoning are inadequate to keep up, which decisions is the human to accept? Pilots have no way to check if what the 'smarter' machine is doing is acceptable; designers have given pilots an impossible task.'"
Sam: "A brain-twister; have an example?"
John: "Check out this simulation similar to the XL Airways/Air New Zealand debacle -- this A320 computer was quite happy to murder its humans."
Sam: "Horrifying."
John: "As they crashed into the Med, the Computer told its pilots . . ."
Sam: "Dreadful."
John: "We don't know what these computers are gonna do, they don't know what they're gonna do, and are indifferent to human life."
Sam: "Understand."
John: "Drs. Sydney Dekker and Judith Orasanu find cockpit computers so craplex, the smallest of failures can jeopardize a flight. They write that the mere presence of craplex designs produces extra workload -- under everyday operations -- that can make pilots lose track of what the Computer is doing to the plane."
Sam: "Do you have an example?"
John: "This A320 murdered 143 when the Computer commandeered the pilots' minds and plane during an autothrottle go-around that looked similar to this."
Sam: "Horrid; how could that happen?"
John: "It's how our brains are hard-wired. Imagine waking up to a hand shoved on your mouth and a butcher knife at your throat. Adrenalin surges, neurons firecracker . . ."
John: ". . . your brain's amyglada goes into sensory overload . . ."
John: ". . . the prehistoric reptilian part of your brain takes charge . . ."
John: ". . . your perception of time and space become warped. Your brain goes into overload as your lizard-brain shuts down the higher regions you need to solve the riddle. You get tunnel vision . . ."
John: "Cockpit computers can be a supermassive black hole that sucks your logical brain right outta your head, leaving you unable to solve the riddle."
John: "Another Airbus autothrottle computer incinerated 264 at Nagoya."
Sam: "Holy cow; you've explained the British finding that pilots often just sit there and watch the computer mess up -- or do the wrong thing and make the situation worse."
John: "Uh-huh; those Nagoya pilots were just being normal lizards."
Sam: "This explains the British finding that pilots can't beat the Computer, such as last years' Air France 447 disaster, the Continental Buffalo crash, or that Boeing 737 debacle at Amsterdam you showed me last time, right?"
John: "Doc Perrow explains, 'Even if we try very hard, do everything possible, have the best talent, computers are bound to fail if they are interactively complex; computer errors interact in unexpected ways and if they are tightly interconnected, we can not slow them down or shut them off' because your mind's neurons firework, rendering you helpless to stop the coming catastrophe."
Sam: "Stunning; what else did the Brits discover?"
John: "That the Computer isn't a team player, makes independent decisions in secret, then hides them from its pilots. People assume that computer traps are a cuddly Christmas puppy who peed on the rug. Not so; computer traps can be this guy."
Sam: "Holy smokes."
John: "I've saved the best for last; the Brits find that computers can make pilots see . . . well . . . ghosts."
Sam: "Ghosts?"
John: "Apparitions -- pilots 'see' display indications that were never there. The captain during the Cali crash chased a ghost for three minutes until . . . the ghost won."
Sam: "Chilling; how can you guys tell what's real and what's a ghost?"
John: "We can't; this is a negative human trait based on millions of years of evolution no amount of training will ever de-program."
Sam: "Yikes. So . . . what's the solution?"
John: "The Brits don't see one; they predict that craplex computers will skulk around cockpits until 2035, forcing pilots who've forgotten how to fly to mud-wrestle these craplex designs. Add the new Boeing 787 and Superjumbo Airbus A380? We're looking at decades of computercides."
Sam: "Shocking; where does that leave us? Pretending to be horrified by more and more 'normal' computercides?"
John: "Seems so; MIT's Nancy Leveson says that it's common to blame plane crashes on pilots rather than investigate the computer system design that caused the crash. She adds that even as plane crashes increase, society is unlikely to abandon risky new computer systems that represent technological progress, yet that won't stop the media from jumping on the pilot error bandwagon . . . 'You know those people.'"
John: "NASA's Dr. Key Dismukes writes that pilot error is a myth: it's a myth that an accident crew made mistakes other pilots handle routinely. It's a myth that a crew was deficient -- they lacked skill, had a bad attitude, or just didn't try hard enough. He writes, 'Truth is, the most skilled, conscientious expert in the world can do something perfectly 100 times, then bungle the 101st. Bungling is normal in medicine, music, mountain climbing, along with flying.'"
Sam: "You gotta publish this stuff."
John: "Doc Dismukes lectures, 'Identifying pilot error as the cause of crashes is dangerous because it encourages the aviation community and public to think something was wrong with the pilots and that the problem is solved because the crew is dead."
Sam: "We're happier if our pilots are dead?"
John: "MIT's Nancy Leveson agrees that plane crashes can result from interactions across perfectly-functioning computers in craplex systems. The crashes occur because Designers fail to account for all interactions. There are computer-generated snakes on these planes."
Sam: "Yikes; is there no solution?"
John: "Doc Leveson explains, 'The best way to protect people is to keep them out of harm's way.' Vera Lynn explains what we gotta do."
To be continued
Read all John Halliday's columns here: www.huffingtonpost.com/john-t-halliday/