Historical Evidence

What is wrong with the following argument?

“Protests worked for the civil rights movement so they should work for our cause!”

This type of argument  is commonly used by charities, social movements, and other organizations to justify an action they are taking. It is a specific form of historical evidence.

Historical evidence can be a useful tool in determining what might work or what might be worth doing more thorough testing on, such as a randomized controlled trial (GiveWell’s history of philanthropy is a great example of this). But sadly there are such significant problems with most historical evidence that it is not really useful. How can we tell if it’s good evidence or bad evidence?  Some general characteristics of the good and bad historical evidence are listed below.

All these flaws are really hard to avoid. In practice, most charities’ historical evidence is about as good as a single anecdote. It’s likely that they cherry-picked evidence from a few successful examples which they knew about offhand, overstating their success and connection to the current issue. Good historical evidence is almost always written down with thorough citations and conservative claims.

Next time you read a historical claim, run through this checklist in your head to see how strong the evidence actually is. You’ll be surprised at how shaky the most confidently asserted beliefs are.