Originally published 1996. Author is psych professor at Bamberg.

Focuses on failure modes in complex systems.

Much of the book is based on two groups of experiments:

In the first group subjects become dictators of fictional towns or countries. Not clear how the results of their actions are decided - seems to be much room for experimenters to influence which behaviors are successful. The analysis is post-hoc and mostly quantitative. Probably best to think of these as exploratory studies.

In the second group subjects are asked to make predictions about or attempt to control simple dynamical systems. These seem better controlled, since the behavior of the system is clear defined. Again, the analysis is post-hoc and mostly quantitative. Would be interesting to see if these have been repeated at scale.

Still, many of the failure modes in the book seem worth considering even without knowing their prevalence.

Systems are prone to human failure when they are:

Parts of decision-making:

  1. Formulate goals
  2. Formulate models and gather information
  3. Predict and extrapolate
  4. Plan and execute actions
  5. Review effects

Ways to fail in goal-setting:

Ways to fail in model building and review:

Features of complex, dynamical systems: Feedback. Buffering. Critical vs indicator variables. Delayed response.

Ways to fail in planning

Efficiency diversity - if planning all the way to the goal is infeasible, focus on plans which lead to states where there are many actions available (diverse) which have high chances of success (efficient). (Analogous to one-armed bandit approach to playing go.)

I didn’t really like the structuring of the book. Can we categorize these problems more helpfully?