Made for intelligence analysts. Now free to the public.
Follow OpenSourceACH on Twitter

ACH, Step By Step

From The Open Source Analysis of Competing Hypotheses Project

The below steps are for all ACH practitioners, whether you're using pencil and paper or a software tool. Many Web apps are built with the expectation that users will make up their own rules and usage patterns. However, ACH was created specifically to guide researchers through a structured thought process. To insure you get the most from ACH, we suggest you follow these steps closely. For a more detailed explanation of how to use the Web application, see the User manual.

1. Define the question

Make sure that everyone involved in the process has a solid understanding of the specific question to be answered.

2. Identify the possible hypotheses

Your hypotheses should be mutually exclusive; that is, if any one of your hypotheses is true, all others must be false. Adhering to this requirement will help you discover new hypotheses. For instance, if you begin with two working hypotheses--"Person A committed the crime," and "Person B committed the crime"--ask yourself if these hypotheses are mutually exclusive. They aren't; persons A and B could have committed the crime together. Therefore, you should alter your hypotheses and add a third one:

  • Person A committed the crime alone
  • Person B committed the crime alone
  • Persons A and B committed the crime together

Use a group of analysts with different perspectives to brainstorm all possible hypotheses. Include the deception hypothesis when appropriate to ensure that you have exhausted all the possibilities. For more information on creating hypotheses, see the user manual.

3. Make a list of evidence and arguments (including assumptions and logical deductions) for and against each hypothesis.

Remember to include assumptions, logical deductions, and conclusions from other analyses--anything that affects your judgment about the likelihood of any hypothesis. Also include the absence of evidence one would expect to find if a hypothesis were true. See the user manual for more details.

4. Prepare a matrix with the hypotheses across the top and the evidence/arguments down the side.

Work horizontally across the matrix to rate each data item's consistency or inconsistency with each hypothesis. There is an option to also assess the Credibility of each item of evidence to determine how much weight it should have in the analysis. Check the diagnosticity of the evidence. An item of evidence is diagnostic if it helps you determine that an item of evidence or argument shows that one or more hypotheses may be less likely than the others.

When you're done, you should have something like this:

Samplematrix.png

In this example, Hypothesis 1 is looking pretty unlikely. (But remember, that doesn't necessarily mean that Hypothesis 2 is correct. Our goal is to refute hypotheses, not prove them.)

5. Reconsider the hypotheses

Have you learned anything that might suggest they should be modified? Check the consistency of your evidence ratings. Discard evidence that has no diagnostic value because it is consistent with all the hypotheses. Identify any gaps in the evidence and arguments that may need to be filled. Delete any evidence that has no diagnostic value because it is consistent with all the hypotheses.

6. Check the Inconsistency or Weighted Inconsistency Score and draw tentative conclusions about the relative likelihood of each hypothesis.

The most likely hypothesis is the one with the lowest score. Sort the evidence and arguments by diagnosticity to identify those few items that are most influential in driving your conclusions. Consider the consequences for your analysis if any key item of evidence or argument is wrong, misleading, or subject to a different interpretation? Sort the evidence by type of source and be alert to any indication of possible deception.

7. Compare your personal conclusions about the relative likelihood of the hypotheses with the Inconsistency Score or the Weighted Inconsistency Scores generated by the software.

If they differ, figure out why and make appropriate adjustments. See guidance in the sections on Interpreting the Inconsistency Scores. Solicit critical input from other analysts. Draw your final conclusions.

8. Report conclusions, discussing the relative likelihood of all the hypotheses, not just the most likely one.

Include discussion of the diagnostic evidence or arguments that enabled you to reject or discount the alternative hypotheses.

9. Identify indicators or milestones for future observation to monitor whether one or more hypotheses might be changing.


Original source: The Psychology of Intelligence Analysis by Richards Heuer