Rational Accidents

Reckoning with Catastrophic Technologies

Look inside
Paperback
$50.00 US
| $66.00 CAN
On sale Jan 16, 2024 | 344 Pages | 9780262546997
An unflinching look at the unique challenges posed by complex technologies we cannot afford to let fail—and why the remarkable achievements of civil aviation can help us understand those challenges.

Nuclear reactors, deep-sea drilling platforms, deterrence infrastructures—these are all complex and formidable technologies with the potential to fail catastrophically. In Rational Accidents, John Downer outlines a new perspective on technological failure, arguing that undetectable errors can lurk in even the most rigorous and “rational” assessments of these systems due to the inherent limits of engineering tests and models. Downer finds that it should be impossible, from an epistemological viewpoint, to achieve the near-perfect reliability that we require of our most safety-critical technologies. There is, however, one such technology that demonstrably appears to achieve these “impossible” reliabilities: jetliners.

Downer looks closely at civil aviation and how it has reckoned with the problem of failure. He finds that the way we conceive of jetliner reliability hides the real practices by which it is achieved. And he shows us why those practices are much less transferrable across technological domains than we are led to believe. Fully understanding why jetliners don't crash, he concludes, should lead us to doubt the safety of other “ultra-reliable” technologies.

A unique and sobering exploration of technological reliability from an STS perspective, Rational Accidents is essential reading for understanding why our most safety-critical technologies are even more dangerous than we believe.
John Downer is Associate Professor in Science and Technology Studies at the University of Bristol.
ACKNOWLEDGMENTS ix

INTRODUCTION: PURPOSE, SCOPE, AND STRUCTURE xv

I THE AVIATION PARADOX I
I CATASTROPHIC TECHNOLOGIES: THE RISE OF RELIABILITY AS A VARIABLE OF CONSEQUENCE 3
2 FINITISM AND FAILURE: ON THE LOGICAL IMPLAUSIBILITY OF ULTRAHIGH RELIABILITY 17
3 THE AVIATION PARADOX: ON THE IMPOSSIBLE RELIABILITY OF JETLINERS 29

II CONFRONTING ULTRAHIGH RELIABILITY 39
4 ORGANIZING AVIATION SAFETY: RELIABILITY REQUIREMENTS AND LOGICS 41
5 WHEN THE CHICK HITS THE FAN: TESTING AND THE PROBLEM OF RELEVANCE 53
6 THE SUM OF ALL PARTS: MODELING RELIABILITY WITH REDUNDANCY 71
7 RATIONAL ACCIDENTS: ON FINITISM’S CATASTROPHIC IMPLICATIONS 89

III MASTERING ULTRAHIGH RELIABILITY 109
8 PARADOX RESOLVED: TRANSCENDING THE LIMITS OF TESTS AND MODELS 111
9 DESIGN STABILITY REVISITED: CONTEXT, CAVEATS, COMPOSITES, AND CONCORDE 131
10 SAFETY COSTS: THE STRUCTURAL FOUNDATIONS OF ULTRARELIABLE DESIGN 151
11 INCENTIVES IN ACTION: ON DEFICIENT 737s AND NEGLECTED SURVIVABILITY 173

IV RECKONING WITH CATASTROPHIC TECHNOLOGIES 191
12 BURDENS OF PROOF: THE HIDDEN COSTS OF POSITIVISM 193
13 THE MYTH OF MASTERY: ON THE UNDERAPPRECIATED LIMITS OF TECHNOLOGICAL AMBITION 211
14 FUKUSHIMA REVISITED: REAPING THE WHIRLWINDS OF CERTAINTY 223

NOTES 235
REFERENCES 255
INDEX 293

About

An unflinching look at the unique challenges posed by complex technologies we cannot afford to let fail—and why the remarkable achievements of civil aviation can help us understand those challenges.

Nuclear reactors, deep-sea drilling platforms, deterrence infrastructures—these are all complex and formidable technologies with the potential to fail catastrophically. In Rational Accidents, John Downer outlines a new perspective on technological failure, arguing that undetectable errors can lurk in even the most rigorous and “rational” assessments of these systems due to the inherent limits of engineering tests and models. Downer finds that it should be impossible, from an epistemological viewpoint, to achieve the near-perfect reliability that we require of our most safety-critical technologies. There is, however, one such technology that demonstrably appears to achieve these “impossible” reliabilities: jetliners.

Downer looks closely at civil aviation and how it has reckoned with the problem of failure. He finds that the way we conceive of jetliner reliability hides the real practices by which it is achieved. And he shows us why those practices are much less transferrable across technological domains than we are led to believe. Fully understanding why jetliners don't crash, he concludes, should lead us to doubt the safety of other “ultra-reliable” technologies.

A unique and sobering exploration of technological reliability from an STS perspective, Rational Accidents is essential reading for understanding why our most safety-critical technologies are even more dangerous than we believe.

Author

John Downer is Associate Professor in Science and Technology Studies at the University of Bristol.

Table of Contents

ACKNOWLEDGMENTS ix

INTRODUCTION: PURPOSE, SCOPE, AND STRUCTURE xv

I THE AVIATION PARADOX I
I CATASTROPHIC TECHNOLOGIES: THE RISE OF RELIABILITY AS A VARIABLE OF CONSEQUENCE 3
2 FINITISM AND FAILURE: ON THE LOGICAL IMPLAUSIBILITY OF ULTRAHIGH RELIABILITY 17
3 THE AVIATION PARADOX: ON THE IMPOSSIBLE RELIABILITY OF JETLINERS 29

II CONFRONTING ULTRAHIGH RELIABILITY 39
4 ORGANIZING AVIATION SAFETY: RELIABILITY REQUIREMENTS AND LOGICS 41
5 WHEN THE CHICK HITS THE FAN: TESTING AND THE PROBLEM OF RELEVANCE 53
6 THE SUM OF ALL PARTS: MODELING RELIABILITY WITH REDUNDANCY 71
7 RATIONAL ACCIDENTS: ON FINITISM’S CATASTROPHIC IMPLICATIONS 89

III MASTERING ULTRAHIGH RELIABILITY 109
8 PARADOX RESOLVED: TRANSCENDING THE LIMITS OF TESTS AND MODELS 111
9 DESIGN STABILITY REVISITED: CONTEXT, CAVEATS, COMPOSITES, AND CONCORDE 131
10 SAFETY COSTS: THE STRUCTURAL FOUNDATIONS OF ULTRARELIABLE DESIGN 151
11 INCENTIVES IN ACTION: ON DEFICIENT 737s AND NEGLECTED SURVIVABILITY 173

IV RECKONING WITH CATASTROPHIC TECHNOLOGIES 191
12 BURDENS OF PROOF: THE HIDDEN COSTS OF POSITIVISM 193
13 THE MYTH OF MASTERY: ON THE UNDERAPPRECIATED LIMITS OF TECHNOLOGICAL AMBITION 211
14 FUKUSHIMA REVISITED: REAPING THE WHIRLWINDS OF CERTAINTY 223

NOTES 235
REFERENCES 255
INDEX 293