Incrementality in Mobile Retargeting: How to Measure Campaign Lift

Incrementality is one of those terms that sounds more intimidating than it really is. Fortunately, the concept is simple. Once you’ve learned the ropes, measuring whether your mobile retargeting campaigns are generating real business value becomes much more straightforward.

In this article

1. Why Incrementality Matters in Retargeting
2. How Does Incrementality Testing Work?
3. Getting Incrementality Testing Right
4. Common Incrementality Testing Methods
5. How Do You Run an Incrementality Test?
6. Best Practices
7. Tooling: MMP-Native and Independent Solutions
8. FAQ

Why Incrementality Matters in Retargeting

Unlike user acquisition, app retargeting targets people who already know your product. Many of those users – especially loyal or paying customers – may have returned on their own without seeing an ad.

That creates a measurement challenge. If you pay to bring back users who would have returned organically, your ROAS may look impressive while actual business growth remains unchanged. The real question isn’t “Did users return?” It’s “Did they return because of the ad?”

That’s exactly what incrementality measures: the additional users, conversions, and revenue generated by your retargeting campaigns beyond what would have happened organically.

How Does Incrementality Testing Work?

At its core, an incrementality test is simply an A/B experiment. One group of users is exposed to your retargeting campaign, while another similar group is intentionally excluded. Comparing the two groups reveals the true impact of your advertising.

Group A – Control: Users who don’t see retargeting ads. Their behavior establishes the organic baseline for returns, conversions, and revenue.

Group B – Treatment: Users who are eligible to see your retargeting ads. Comparing their performance against the control group reveals the campaign’s incremental lift.

Getting Incrementality Testing Right

A successful incrementality test measures more than just returning users. Start by comparing return rates between the treatment and control groups to quantify re-engagement lift. Then compare revenue generated by each group to measure true financial impact. Finally, analyze results by audience segment and over longer time periods – such as 7, 14, or 30 days – to understand whether the campaign created lasting value or simply accelerated activity that would have happened anyway.

The framework sounds simple – and it is – but execution determines whether the results are trustworthy. Here are the key principles to get right:

• Group size: The control group should be statistically meaningful without limiting the DSP’s ability to reach enough users in the treatment group. Most experiments reserve 10–20% of the audience for the control group.
• True reach: Measure lift only among users who were actually exposed to an ad. Users assigned to the treatment group but never served an impression shouldn’t be counted as treated.
• Sanity check (A/A test): Before launching the experiment, verify that both groups are equally reachable. If one group is systematically harder to reach, the results will be biased from the start.
• Frozen audiences: Test audiences are fixed when the experiment begins. As time passes, some users naturally return for unrelated reasons, so choose an evaluation window that minimizes this effect.

Incrementality testing isn’t a one-time exercise. As your user base, product, and programmatic re-engagement strategy evolve, running regular tests provides a much clearer picture of campaign effectiveness than ROAS alone.

The growing demand in incrementality testing has led us to launch IMPACT, Persona.ly’s complimentary incrementality measurement platform, which gives customers access to the results of their incrementality tests for campaigns run by Persona.ly.

Connected directly to our programmatic bidder, IMPACT gives marketers visibility into incremental results across audiences, campaigns, and segments. Metrics such as incremental revenue and incremental return on ad spend (iROAS) help answer practical questions: Where is the campaign adding value? Which audiences deserve more investment? Where should we adjust?

Common Incrementality Testing Methods

1. Intent-to-Treat (ITT) / Holdout Testing

How it works: Users are randomly divided into treatment and control groups. Only the treatment group receives retargeting ads.

Best for: Most advertisers. This is the standard methodology supported by leading MMPs such as AppsFlyer and Adjust.

“The most common mistake we see in Singular is designing the test around basic conversion lifts rather than around a clean holdout or control methodology. Incrementality tests need to isolate the users who were exposed from a comparable group who were not exposed, and we have tools today to automate the process accurately – define the audience, treatment/control split, success metric, and test duration upfront, and then use spend and results to interpret the results once the test is complete.”

Eran Friedman
CTO & Co-Founder at Singular

2. Ghost Bidding (Ghost Ads)

How it works: The DSP evaluates both groups equally but suppresses bids for the control group, creating a highly comparable baseline without changing auction dynamics.

Best for: Large-scale advertisers seeking the most accurate measurement possible.

3. Public Service Announcement (PSA) Ads

How it works: Both groups receive ads, but only the treatment group sees the actual retargeting creative. The control group receives a neutral public service announcement (PSA) or filler ad.

Best for: Eliminating bias caused by showing no ad at all.

How Do You Run an Incrementality Test?

Running an incrementality test is straightforward if you follow a structured process:

1. Define the Goal & Audience and select select a distinct, homogeneous audience segment: Decide what you are measuring (e.g., in-app purchases, churn prevention, or subscription renewals) and for which segment (e.g., active buyers or lapsed users).
2. Split the Audience: Randomly divide your target audience into two groups: a Treatment Group that receives your mobile retargeting ads, and a Control Group excluded from seeing your ads.
3. Execute the Campaigns: Run the retargeting campaign for a statistically significant duration, usually 2 to 4 weeks depending on user volume.
4. Measure and Compare: Calculate the conversion rate for both groups.

“When it comes to incrementality testing, advertisers tend to ‘test the waters’,  running a small, parallel campaign just for the sake of the test, instead of measuring what they’re already spending on. But here’s the catch: incrementality only measures the extra impact on top of what’s already in motion. A small test campaign overlaps with existing efforts, leaving almost no ‘extra’ to find — just a sliver of performance mistaken for the whole story. Our advice? Go top-down. Test your biggest campaigns and channels first, as that’s where the real growth drivers show up.” 

Niv Reshef Klein
Head of Product, AppsFlyer

Best Practices

Like any experiment, an incrementality test is only as good as its design. Following these best practices will help you find the sweet spot between statistical confidence and practical decision-making.

1. Statistical Significance: Ensure your sample size and conversion volume are large enough for the results to be statistically reliable rather than driven by chance.
2. Use MMPs or third-party incrementality measurement solutions: Mobile Measurement Partners offer built-in incrementality tools that automate audience splitting, measurement, and reporting.
3. Avoid Cannibalization: Keep treatment and control groups isolated from other marketing campaigns to avoid contamination and ensure you’re measuring only the impact of the retargeting campaign.

Tooling: MMP-Native and Independent Solutions

Most major MMPs now productize incrementality: AppsFlyer offers built-in incrementality measurement, and Adjust offers holdout-based lift reporting – meaning audience splitting, exposure validation, and reporting no longer require a data-science team.

Beyond MMP-native tools, a category of independent measurement vendors focuses specifically on the mobile app market. INCRMNTAL provides always-on, experiment-free incrementality measurement built on causal inference – no SDK, device IDs, or user-level data required – and reports campaign-level incrementality and cannibalization scores; it’s used by mobile-first advertisers including Supercell, DraftKings, and eToro (mobile app solution overview). MetricWorks (Polaris) pairs media mix modeling with automated geo-lift experiments in an MMP-style interface, purpose-built for mobile gaming advertisers (used in production by studios such as Kabam and FunPlus), and outputs daily incrementality down to campaign and creative level without IDFAs or GAIDs.

The practical distinction: MMP-native tools excel at audience-split holdout tests within your existing attribution stack; independent vendors excel at continuous, cross-channel incrementality (including channels where user-level splits are impossible) and at auditing the attribution story your other partners tell you. Mature teams often run both – holdouts for campaign decisions, independent measurement for budget-level truth.