Direct mail appeals present a great opportunity to test new fundraising technologies. Earlier this year, Dataro teamed up with a large disability services provider to test whether machine learning could be used to improve the organisation’s Spring appeal. The results once again demonstrated how Dataro’s approach can reliably be used to increase net revenue and reduce appeal mail volumes.

Key results included: 

  • A huge 16.4% improvement in net revenue from the Dataro campaign;
  • An increase in gross revenue despite a much smaller mailing list; and
  • A response rate of over 50% from the top 1000 donors identified by Dataro.

Methodology

The goal behind the latest test was to compare the performance of a classic ‘segmentation’ model against Dataro’s machine learning methodology. Dataro used its ListOptimiser approach:

  1. Dataro generated propensity scores for every donor, predicting the likelihood of each individual to give in the Spring appeal. Dataro also analysed projected gift amounts, to identify donors with a propensity to give a larger gift.
  2. Dataro used these propensity scores and campaign costs (cost per piece plus postage) to calculate the ‘optimal’ campaign size for the best projected net return.
  3. Dataro’s recommended campaign list was compared with the campaign list generated by the charity itself using its normal approach. In this case, the organisation used a recency, frequency, monetary value approach to bundle donors into relevant segments and select segments to include in the campaign. This is the standard industry methodology.
  4. By comparing the two proposed campaigns, Dataro was able to identify:
    1. Donors appearing in both lists with the highest probabilities of giving (labelled in blue);
    2. Donors appearing in the RFM list that Dataro’s modelling showed had a very low probability of giving (labelled in red); and
    3. Donors appearing in the Dataro list that were missed by the RFM model but which had a higher probability of giving (labelled in green).
  5. The charity sent the campaign to the red, green and blue groups, allowing Dataro to compare results from the Dataro campaign (green + blue) versus the RFM campaign (red + blue).

Results

The results highlighted one of the main problems with the classic RFM approach: it cannot tell the difference between donors that fall in the same segment. This results in larger mail files and less accurate projections. The reason is that the standard model only takes into account the recency, frequency, and value of financial gifts, resulting in broad ‘segments’ with the same characteristics. It does not consider the relationship between the charity and the individual, communications information, details about the donor, or information about acquisition channel and other gifts.

Machine learning, on the other hand, can incorporate all of this data and more. Using these sophisticated algorithms to accurately ‘weight’ each factor, Dataro is able to apply a much more nuanced analysis to identify higher probability givers. Importantly, this method allows charities to treat donors as individuals, rather than part of a segment.

In this case, the results showed that machine learning delivered strong performance in terms of gross revenue, net revenue, and response rates.

The below graph shows how, as projected, the ‘blue’ segment performed strongest, followed by the green segment in terms of response rates. The red segment responded poorly at lower than 1%.

Revenue by Appeal Colour

Similarly, when donors were broken into ‘bands’ of 1000 based on their Dataro score, it was clear that the donors with the highest scores contributed by far the majority of gifts and the majority of revenue. 

Response Rate by Campaign Group (Dataro Rank)

Spring Appeal Response Rates by Dataro Score

Percentage Revenue by Campaign Group (Dataro Rank)

Revenue by Dataro Rank

These results, together with other Dataro case studies, demonstrate how machine learning can be used to more carefully analyse which donors are likely to participate in a direct mail appeal, resulting in better overall performance.