Testing Dataro across Australia for UNHCR direct mail appeals

Australia for UNHCR is the UN Refugee Agency’s national partner in Australia, raising funds and awareness to support displaced people across the globe. 

Direct mail appeals form a core part of Australia for UNHCR’s fundraising calendar. So in September 2021 the team embarked on a partnership with Dataro to seek better appeal returns using machine learning. 

After testing Dataro’s AI donor propensity software across four appeals, we found:

  • Dataro’s predictions were highly accurate for Core value donors 
  • Average appeal ROI from Core donors increased by 28%
  • Average appeal costs for Core donors decreased by 23% 
  • Average appeal net revenue from Core donors increased by 3%

Background

Dataro’s Direct Mail Appeals Module provides AI-generated predictive ranks and scores for warm supporters based on their likelihood of giving to a direct mail appeal. Australia for UNHCR tested these predictions in multiple campaigns including their 2021 ‘Winter Survival’ Christmas campaign, March 2022 ‘Ukraine’ campaign, Tax 2022 ‘Ukraine’ campaign, and September 2022 ‘Food Insecurity’ campaign. 

Testing for each campaign was run by the charity across their Core, Middle, and High Net Worth donor program areas. 

In order to validate the Direct Mail Appeal propensity model, the charity wanted to know:

  1. Are the machine learning model predictions accurate for predicting appeal gifts? 
  2. Is a higher ROI and Net Revenue result achieved when using Dataro’s AI targeting approach compared to segmentation alone?

Test 1 – Assessing Dataro’s donor prediction accuracy

To measure the accuracy of the Dataro predictions, the charity wanted to specifically know whether donors with a propensity score of 1% or more responded at higher rates than donors with a propensity score below 1%. 

In every campaign, the donors Dataro predicted were more likely to give to the appeal actually did give at higher rates, conclusively demonstrating the accuracy of the Dataro models. 

Amongst Core donors, which made up the vast majority of each appeal, response rates were on average 8.75 percentage points higher amongst the high propensity group (donors with a predictive score of greater than 1%) vs the low propensity group (donors with a predictive score of less than 1%). 

The same trend continued across the Middle and High Net Worth groups. Donors with a higher Dataro appeal score actually did give at a higher rate. However, as there are fewer donors in each of these cohorts, the results were less statistically significant.

Test 2 – Does Dataro’s AI targeting approach generate higher ROI and Net Revenue

In order to determine whether the accurate predictions translated into additional revenue, Australia for UNHCR considered two scenarios: 

  1. ROI and Net Revenue from donors with appeal score greater than 1%
  2. ROI and Net Revenue from donors selected via the traditional segmentation approach

Of course, there is a large overlap between the two cohorts as many donors with a high propensity score also fall into one of the segments the charity would normally include in their targeting. 

For Core donors in each campaign, the charity therefore compared the ‘Dataro selection’ against the ‘Australia for UNHCR’ selection to identify: 

  1. Donors in the Dataro group only
  2. Donors in both groups
  3. Donors in the Australia for UNHCR group only

Across all four direct mail appeal campaigns, the results were clear:

Results for the Core donor cohort showed a consistently positive impact on net revenue and ROI potential of an appeal by using the Dataro Direct Mail appeal scores.

“By focusing on the higher propensity donors we will be able to target more effectively than by using our segmentation alone,” said Jo Purcell-Jones (Senior Business Intelligence Developer).

“In reducing mailing volume to high propensity donors only, we can make significant savings in costs, without jeopardizing revenue.”

Across the Middle and High Net Worth groups, the Dataro appeal propensities also resulted in a decrease in costs and an increase in ROI. However, the potential cost savings of removing low scoring Middle and High Net Worth donors from an appeal does not outweigh the value of what those donors give. 

“Whilst it is not valuable to use the scores to reduce mailing volumes for Middle (as with Core), the DM appeal model is still able to identify which donors are most likely to respond to an appeal, which we can potentially use to be more efficient in the stewardship of these donors,” said Jo. 

For improving mid-level and High Net Worth giving, Dataro recommends using additional propensities specifically developed to predict donors more likely to become mid-level and major donors. Their higher average gift amounts will typically mean it is worth including these donors in an appeal, even if their actual probability of giving right now is low.

In Summary

Based on these results, Australia for UNHCR is now only including Core donors with a Dataro DM Appeal score of greater than or equal to 1% in their direct mail appeal selections (or a lapsed donor score of greater than 1%). 

The charity will maintain mailing volumes for Middle donors and High Net Worth donors using segmentation, but will add in any additional donors with a Dataro DM Appeal score of greater than 1% to ensure they don’t miss any donations from these valuable cohorts. Australia for UNHCR is planning further testing of Dataro’s mid-level donor propensity model, and use of the DM appeal scores throughout the Middle and High Net portfolios will be reviewed periodically.

What they said

“We have seen impressive results from the Dataro DM Appeal scores. We went through a thorough testing process to ensure that the scores were predicting accurately, and that by using them we were adding value to our mail appeals, without jeopardizing any potential revenue, and the results were clearly positive on both fronts.”

Jo Purcell-Jones

Senior BI Developer

More Case Studies

How Save The Children used AI to engage more mid-value donors to support children in need

How Save The Children used AI to engage more mid-value donors to support children in need

Save the Children is the world’s leading aid and development agency dedicated to protecting children and helping them reach their full potential. In 2022, Save The Children Netherlands partnered with Dataro to see if they could convert more of their mass-market donors (single giving, recurring giving, multi-donors) to

Read more
$831k in Extra Funding Secured in One Year with AI Donor Predictions

$831k in Extra Funding Secured in One Year with AI Donor Predictions

A leading international nonprofit dedicated to improving the lives of people with disabilities in developing countries partnered with Dataro seeking better direct mail appeal returns. This charity relies on the success of its direct mail appeal program to fund its important work and implements approximately 12 campaigns per

Read more
Engaging the Right Mid-Value Prospects and Donors to Support Displaced People Worldwide

Engaging the Right Mid-Value Prospects and Donors to Support Displaced People Worldwide

Australia for UNHCR is the UN Refugee Agency’s national partner in Australia. It raises funds and awareness to support displaced people worldwide. Since late 2021, the charity has used Dataro’s AI donor predictions to improve fundraising ROI in its appeals program. In 2023, the UNHCR team wanted to

Read more
Transforming the Future of Pancreatic Cancer with AI Donor Predictions

Transforming the Future of Pancreatic Cancer with AI Donor Predictions

Pancreatic Cancer UK is working to transform the future for everyone affected by pancreatic cancer. Through the generous support of their donors, they fund research breakthroughs, campaign for changes to treatment and care, and provide support to those affected by pancreatic cancer. In 2021, Pancreatic Cancer UK partnered

Read more