Cluster Approach Thinking Applied to Selecting Fundraising Methods

Hi, I am Kieran. I joined the Charity Science team in June 2015 after volunteering since February. Hope you're enjoying the new website.

Charity Science recently evaluated more than 20 different fundraising methods to determine which methods would be the most promising to experiment with in the third quarter of 2015. This prioritization task resulted in slightly more than a hundred pages of research about fundraising methods, which will be helpful for other groups considering a wide range of fundraising options. When researching, reviewing and evaluating these methods something that persistently proved useful was the cluster thinking approach. In essence, cluster thinking involves approaching empirical questions from numerous reference frames or mental models and synthesising this cluster of views into one’s opinion.

We attempted to apply this approach when researching fundraising methods by asking researchers to look for and explain reasoning from multiple angles. These angles included but were not limited to:

• Hard research numbers or data.
• An expected value calculation.
  
• What other charities/experts generally think of this method.
  
• The extent that other charities use this strategy and how effective Charity Science will be at competing with them. 
  
• The soft sense the researcher had for this fundraising technique.

It was not always possible to represent each angle and in different reports the same angle was weighted quite differently. For instance, in one report the expected value calculation may be based off pure estimates whereas in another it was shaped by several sources of information.

In addition to having researchers consider a number of different outlooks when completing their research they themselves also represented a set of slightly different perspectives. This stemmed from their differing levels of involvement with Charity Science, experience in fundraising and familiarity with research. However, that didn’t translate into all points proffered being equal and the conclusion reached just an average of them. This wasn’t an exercise in pure “philosophical majoritanism.” Yet nor was a view completely ignored in this decision making process either.  

One of the reasons we encouraged research informed by cluster thinking approach was because, as we have previously written,  there is no good science on fundraising. Further, there appears to be systemic issues with the available information as it is usually informal and anecdotal in nature, seemingly pervaded by publication bias and rarely acknowledges let alone attempts to answer questions of cross applicability… but there issues for another time. What matters here is that robust expected value estimates couldn’t be made.

Instead constructed expected value estimates were often heavily contaminated with uncertainty. For instance, 95% confidence intervals could span orders of magnitude without even accounting for knightian uncertainty. This didn’t prevent us from completing expected value estimates during the research, rather it caused some researchers and all reviewers to adjust the epistemic weight assigned to them. Within the cluster thinking approach this meant that expected value estimates were just one line of reasoning that was integrated into our overall conclusions about a fundraising method. For and related piece that goes into more depth about Bayesian updating in light of uncertainty in expected value calculations see why expected value estimates can’t be taken in literally even when they are unbiased.  

An advantage of treating expected value estimates in this way is that it prevented one poorly constructed or mighty powerful link in their chain from dominating our concerns. For instance, even though some fundraising methods possessed a chance for enticingly high potential returns, as in High Net Worth Individual acquisition and their stewardship, this wasn’t enough to swamp all other considerations. Similarly, possibly remarkably poor counterfactual estimates involved in other expected value estimates will not derail a conclusion that draws on many weak arguments instead of one relatively strong one.

After incorporating elements of cluster thinking at the individual research level and at the individual review stage we also sought to apply this in the final evaluation stage. To do this all Charity Science staff and some board members were invited to individually evaluate each fundraising technique. We asked that each individual didn’t didn’t express their preferences as this would likely influence the ratings of others. After all involved had reached their independent conclusions we then compared differing items and explored areas of disagreement. Throughout this process we were aware of the distinction between fox and hedgehog style thinking and attempted to emulate the fox type because evidence suggests its predictions are more accurate.

From start to finish this process went for ~2 months and wasn’t consistently worked upon. Our conclusion was that the fundraising methods most promising were legacy fundraising, niche marketing and online advertising. In future posts it’s likely we will write more about each method.