The Old Way
Last year I somewhat informally narrowed down the options until I chose to run a study on handing out educational leaflets. The rough steps I followed were to:
- Ask experts and Executive Directors of charities what studies would be valuable.
- Look at interventions with promising preliminary findings. In this case leafleting already had one study showing positive results.
- Rule out the ones that were too costly or would take too much time.
The New Way
I have recently been thinking of a different system that may work better than this. It’s basically using the heuristic of neglectedness a lot more, and would look more like the following:
- Check which animal populations are the most neglected. This can be determined by looking at total funding per animals killed. Animal Charity Evaluators (ACE) thankfully already has this data in a neat chart. This step would suggest farm animals are the first area to look at.
- Assess which broad interventions farm animal donations have supported, such as political lobbying, vegetarian advocacy, meat alternatives etc. Compare this to the broad achievements they have accomplished, like passing laws, persuading people to go vegetarian, or increasing sales of meat alternatives. This information is not easy to acquire and I am not sure what conclusion it would yield.
- Narrow the broad interventions down into more specific actions and find out how much funding goes to each. For example, in outreach this might be flyering, documentaries, events, etc. Then compare this to achievements in each of the areas. For outreach this could be acquired via surveying vegetarians about what convinced them to make the switch. This has already been done by The Humane League Labs.
- From this you would be able to get a sense of what interventions seem to have an oversized impact per cost. You could then design a randomized controlled trial that would be feasible cost/time wise and test what impact the intervention had in a much more thorough way.
At best I might have found areas that looked considerably more promising than flyering. For example, if documentaries have both a larger effect and have less funding than vegetarian flyering it would be better to test the effects of screening a vegetarian documentary instead.
It is important to look at the pros and cons of a method and here are some of the potential flaws with this system:
- I can imagine it being worth putting more time into checking out all the broad interventions. Legal advocacy, veg outreach and meat alternatives might all look promising and be worth investigating at a more specific level.
- It might be better to simply determine what areas work on average instead of figuring out the best specific intervention.
- This method might take so long that it would not be worth the time.
That being said it seems like a possibly interesting way of determining what specific interventions would be worth running a randomized controlled trial on in an area that has very limited evidence so far.