Testing Dataro across Australia for UNHCR direct mail appeals

Australia for UNHCR is the UN Refugee Agency’s national partner in Australia, raising funds and awareness to support displaced people across the globe. 

Direct mail appeals form a core part of Australia for UNHCR’s fundraising calendar. So in September 2021 the team embarked on a partnership with Dataro to seek better appeal returns using machine learning. 

After testing Dataro’s AI donor propensity software across four appeals, we found:

  • Dataro’s predictions were highly accurate for Core value donors 
  • Average appeal ROI from Core donors increased by 28%
  • Average appeal costs for Core donors decreased by 23% 
  • Average appeal net revenue from Core donors increased by 3%

Background

Dataro’s Direct Mail Appeals Module provides AI-generated predictive ranks and scores for warm supporters based on their likelihood of giving to a direct mail appeal. Australia for UNHCR tested these predictions in multiple campaigns including their 2021 ‘Winter Survival’ Christmas campaign, March 2022 ‘Ukraine’ campaign, Tax 2022 ‘Ukraine’ campaign, and September 2022 ‘Food Insecurity’ campaign. 

Testing for each campaign was run by the charity across their Core, Middle, and High Net Worth donor program areas. 

In order to validate the Direct Mail Appeal propensity model, the charity wanted to know:

  1. Are the machine learning model predictions accurate for predicting appeal gifts? 
  2. Is a higher ROI and Net Revenue result achieved when using Dataro’s AI targeting approach compared to segmentation alone?

Test 1 – Assessing Dataro’s donor prediction accuracy

To measure the accuracy of the Dataro predictions, the charity wanted to specifically know whether donors with a propensity score of 1% or more responded at higher rates than donors with a propensity score below 1%. 

In every campaign, the donors Dataro predicted were more likely to give to the appeal actually did give at higher rates, conclusively demonstrating the accuracy of the Dataro models. 

Amongst Core donors, which made up the vast majority of each appeal, response rates were on average 8.75 percentage points higher amongst the high propensity group (donors with a predictive score of greater than 1%) vs the low propensity group (donors with a predictive score of less than 1%). 

The same trend continued across the Middle and High Net Worth groups. Donors with a higher Dataro appeal score actually did give at a higher rate. However, as there are fewer donors in each of these cohorts, the results were less statistically significant.

Test 2 – Does Dataro’s AI targeting approach generate higher ROI and Net Revenue

In order to determine whether the accurate predictions translated into additional revenue, Australia for UNHCR considered two scenarios: 

  1. ROI and Net Revenue from donors with appeal score greater than 1%
  2. ROI and Net Revenue from donors selected via the traditional segmentation approach

Of course, there is a large overlap between the two cohorts as many donors with a high propensity score also fall into one of the segments the charity would normally include in their targeting. 

For Core donors in each campaign, the charity therefore compared the ‘Dataro selection’ against the ‘Australia for UNHCR’ selection to identify: 

  1. Donors in the Dataro group only
  2. Donors in both groups
  3. Donors in the Australia for UNHCR group only

Across all four direct mail appeal campaigns, the results were clear:

Results for the Core donor cohort showed a consistently positive impact on net revenue and ROI potential of an appeal by using the Dataro Direct Mail appeal scores.

“By focusing on the higher propensity donors we will be able to target more effectively than by using our segmentation alone,” said Jo Purcell-Jones (Senior Business Intelligence Developer).

“In reducing mailing volume to high propensity donors only, we can make significant savings in costs, without jeopardizing revenue.”

Across the Middle and High Net Worth groups, the Dataro appeal propensities also resulted in a decrease in costs and an increase in ROI. However, the potential cost savings of removing low scoring Middle and High Net Worth donors from an appeal does not outweigh the value of what those donors give. 

“Whilst it is not valuable to use the scores to reduce mailing volumes for Middle (as with Core), the DM appeal model is still able to identify which donors are most likely to respond to an appeal, which we can potentially use to be more efficient in the stewardship of these donors,” said Jo. 

For improving mid-level and High Net Worth giving, Dataro recommends using additional propensities specifically developed to predict donors more likely to become mid-level and major donors. Their higher average gift amounts will typically mean it is worth including these donors in an appeal, even if their actual probability of giving right now is low.

In Summary

Based on these results, Australia for UNHCR is now only including Core donors with a Dataro DM Appeal score of greater than or equal to 1% in their direct mail appeal selections (or a lapsed donor score of greater than 1%). 

The charity will maintain mailing volumes for Middle donors and High Net Worth donors using segmentation, but will add in any additional donors with a Dataro DM Appeal score of greater than 1% to ensure they don’t miss any donations from these valuable cohorts. Australia for UNHCR is planning further testing of Dataro’s mid-level donor propensity model, and use of the DM appeal scores throughout the Middle and High Net portfolios will be reviewed periodically.

What they said

“We have seen impressive results from the Dataro DM Appeal scores. We went through a thorough testing process to ensure that the scores were predicting accurately, and that by using them we were adding value to our mail appeals, without jeopardizing any potential revenue, and the results were clearly positive on both fronts.”

Jo Purcell-Jones

Senior BI Developer

More Case Studies

Helping Amnesty Donors Leave a Legacy that Protects Rights

Helping Amnesty Donors Leave a Legacy that Protects Rights

Amnesty International Australia has a loyal and passionate supporter base of human rights activists. Their “Leave a Legacy” initiative is an important opportunity for Amnesty donors to express their values and leave a legacy that continues to uphold human rights. In August 2021, Amnesty partnered with Dataro to

Read more
Greenpeace uses Dataro Predict to retain 531 monthly donors & fight for a greener planet.

Greenpeace uses Dataro Predict to retain 531 monthly donors & fight for a greener planet.

Greenpeace Australia Pacific is the leading the charge fighting for our planet. AUNZ independent campaigning organization fighting for a green and peaceful future for our planet.  To help them in their cause, Greenpeace Australia Pacific has used Dataro’s AI modelling software since 2019 to inform their data strategy

Read more
How UNICEF Australia use AI to raise more for children in need

How UNICEF Australia use AI to raise more for children in need

UNICEF is a leading global non-profit working to improve the lives of children in need. As an organization that is 100% donor-funded, UNICEF Australia relies on the generosity of individual donors to fund their work. In 2020 UNICEF Australia partnered with Dataro to improve their direct mail appeal

Read more
Erskine donors leave a lasting legacy for Scottish veterans

Erskine donors leave a lasting legacy for Scottish veterans

Charitable income from legacies has quadrupled over the past 30 years and was forecast to top £4 billion per annum by the end of last year, experts say. It is little wonder that legacies (Gifts in Will) have become a key focus for charities that have one eye

Read more