Attribution vs incrementality: is your model telling the truth about ROI?

Every cross-channel report gives credit to something. The email, the direct mail pack, the paid social ad, the search click. The harder question is whether that credit reflects real impact or just the order in which touches happened.

This matters most at the moment you shift budget. If you move spend from one channel to another based on attribution alone, you are betting that correlation equals causation. Often it does not.

Short answer: attribution shows correlation, not causation

Definition: Attribution assigns credit for a gift across the channels a donor interacted with. Incrementality measures the gifts that would not have happened without a channel.

Attribution answers "which channels were present when this donor gave?" Incrementality answers "which channels actually caused the gift?" Those are different questions, and only the second one tells you the truth about ROI.

Attribution models are useful for spotting patterns and reporting on the donor journey. They are not proof of impact, because they can only describe touches that already correlate with giving. They cannot show what would have happened in the absence of a channel.

Why attribution can mislead your budget decisions

Attribution has three built-in blind spots.

It rewards presence, not influence. A channel that shows up near conversions gets credit even if the donor would have given anyway. Branded search is the classic example: it captures demand rather than creating it.

It ignores the counterfactual. No attribution model measures the gift that would have arrived without the touch. Without a control group, there is no baseline to compare against.

It is shaped by tracking, not behavior. Cookie loss, offline gifts, matched giving and cross-device journeys all distort the data. The model reports what it can see, not what actually happened.

The result: a channel can look efficient in your attribution report and still add little real revenue.

How the four attribution models compare

Each model applies a different rule for splitting credit. None of them prove causation. They simply make different assumptions about which touches matter.

The pattern is clear. Moving between models changes which channel looks good, but it never changes the underlying limitation. You are reallocating credit inside a closed system that cannot see the counterfactual.

What incrementality testing adds

Definition: Incrementality testing, or lift testing, compares a group exposed to a channel against a held-out control group that was not. The difference in giving is the incremental lift the channel produced.

This is the step attribution skips. By holding out a randomized control group, you create a baseline for what would have happened anyway. The gap between the two groups is the real, causal contribution of the channel.

Common approaches include:

• Holdout tests: suppress a channel for a random subset of donors and measure the difference in giving

• Geo tests: run or pause a channel in matched regions and compare results

• Ghost or placebo tests: log the audience that would have been targeted, then withhold the touch and track outcomes

Incrementality does not replace attribution. Attribution describes the journey and helps you form hypotheses. Incrementality validates whether those hypotheses hold before you commit budget.

Attribution and incrementality: how to use both

The two methods answer different questions, so use them in sequence.

Attribution generates the hypothesis. Incrementality tests it. Budget decisions should rest on the test, not the report.

Practical recommendations

1. Treat attribution as a map, not a verdict. Use it to understand journeys and generate questions, not to justify budget shifts on its own.

2. Test before you reallocate. Run a holdout or geo test on any channel before you move meaningful spend into or out of it.

3. Start with your biggest line items. Test the channels where a wrong call costs the most, such as paid media and high-volume mail.

4. Measure net revenue, not credited revenue. Incremental lift minus cost is the number that reflects true ROI.

5. Build a cadence. Impact changes as audiences and costs shift. Retest on a schedule rather than assuming last year's result still holds.

6. Protect the control group. A test is only valid if the holdout is randomized and left untouched for the full measurement window.

The bottom line

If your attribution model is the only thing telling you a channel works, you do not yet know if it works. Attribution shows correlation. Incrementality proves causation.

Use attribution to see the journey and form a view. Use lift testing to confirm real impact before you move budget. That combination turns a report you have to argue about into a decision you can act on and defend.

At Dataro, we help fundraising teams measure what actually drives net revenue, so budget moves are grounded in evidence rather than credited touches. The loop is simple: predict, act, measure, repeat.