The ACH Methodology and Its Purpose

From The Open Source Analysis of Competing Hypotheses Project

Analysis of Competing Hypotheses (ACH) is a simple model for how to think about a complex problem. It is an analytic process that identifies a complete set of alternative hypotheses, systematically evaluates data that is consistent and inconsistent with each hypothesis, and rejects hypotheses that contain too much inconsistent data.

ACH takes you through a process for making well-reasoned, analytical judgments. It is particularly useful for issues that require a careful weighing of alternative explanations of what has happened or is happening. ACH can also be used to provide early warning or help you evaluate alternative scenarios of what might happen in the future. ACH helps you overcome, or at least minimize, some of the cognitive limitations that make prescient intelligence analysis so difficult; it helps clarify why analysts are talking past one another and do not understand each other's interpretation of the data. ACH is grounded in basic insights from cognitive psychology, decision analysis, and the scientific method. It helps analysts protect themselves from avoidable error and improve their chances of making the right call.

This software provides a structured process for breaking a complex analytical problem down into its component parts--a full set of hypotheses (i.e., alternative explanations or outcomes), evidence and arguments that are useful in assessing these hypotheses, and judgments about the consistency or inconsistency of each item of evidence with each hypothesis. The software steps you through a process that helps you question your assumptions and gain a better understanding of the issue. The value of ACH is measured by the extent to which it helps you see an issue from alternative perspectives, prods you to look for additional evidence you had not realized was relevant, helps you question assumptions, identifies the most lucrative future areas of investigation, and generally stimulates systematic and creative thinking about the issue at hand.

The hypotheses, evidence, and analysis of the evidence are entered into a matrix that becomes a record of your thought process in analyzing a given topic or problem. This written record of your thought process is what helps you deal with the complexity inherent in most analytical problems. The software also allows you to sort and compare evidence in a variety of analytically-useful ways.

How Is ACH Different?

ACH differs from conventional intuitive analysis in three important ways. Each of these differences is discussed in greater detail elsewhere in this Tutorial.

• ACH requires that you identify and analyze a full set of alternative hypotheses rather than a single most likely conclusion. This ensures that less likely but possible hypotheses receive fair treatment.
• You proceed by trying to refute or eliminate hypotheses, whereas conventional intuitive analysis generally seeks to confirm a favored hypothesis. The correct hypothesis is the one with the least--or no--inconsistent information.
• Instead of looking at one hypothesis and weighing the evidence pro and con for that hypothesis, you look at each item of evidence, one at a time, and assess whether that evidence is consistent or inconsistent with each of the hypotheses. This enables you to determine the "diagnosticity" of the evidence. An item of evidence is diagnostic when it helps you determine that one hypothesis is more likely to be true than another hypothesis. An item of evidence that is consistent with all hypotheses has no diagnostic value.

Refuting vs. Confirming Hypotheses

A fundamental precept of the scientific method is that one should proceed by rejecting or eliminating hypotheses, while tentatively accepting only those hypotheses that cannot be refuted. No matter how much information you have that is consistent with a given hypothesis, you cannot prove that hypothesis is true, because the same information may be and often is consistent with one or more other hypotheses. On the other hand, a single item of evidence that is inconsistent with a hypothesis may be sufficient grounds for rejecting that hypothesis. A classic example is a criminal suspect who has a solid alibi. A natural human tendency is to give more weight to information that supports our favorite hypothesis than to information that weakens it. This is unfortunate, as we should do just the opposite.

The scientific method obviously cannot be applied in toto when working with ambiguous and incomplete information, but the principle of seeking to disprove hypotheses, rather than to confirm them, can and should be applied to the extent possible. Instead of being content with showing that your favored hypothesis is supported by a lot of evidence, you need to refute all the possible alternatives. Any hypothesis that you cannot refute should be taken seriously. This switch in perspective forces you to ask questions and seek evidence you would not otherwise consider. This is what provides insurance against unpleasant surprise.

Original source: ACH manual by Richards Heuer