Persuadable. An uplift experiment

The allocation problem

Who should we show the ad to?

Most targeting picks the people most likely to visit. That is the easy question. The hard one is narrower: who visits because of the ad? We built a model for that, ran it on a real advertising test, and this page shows what it actually did.

Criteo Uplift v2.1 · 13.98M customers · holdout 99,984

How to read this page

  1. The three numbers above are extra visits at the default budget. They compare the uplift model, a response model, and random mailing, all spending the same money.
  2. The worked example takes two real customers and shows why one is targeted and the other is refused.
  3. The chart then follows the same comparison as we target further down the list. The black line is uplift, the red line is the response model, and the dotted line is random.
  4. The agent on the right answers questions about which policy to run. It calls the tools on this page and quotes their results.

A worked example

Here are two real people from the holdout. For each one, the model asks two questions: how likely are they to visit without the ad, and how likely with it? The gap between those answers decides whether we spend.

Why the win is narrow

The ad raises almost everyone’s chance of visiting by about the same amount. When an effect is that even, “will visit” and “will visit because of the ad” describe nearly the same people, so the response model is hard to beat. Uplift still wins by finding the few people the ad moves most, and that is where the edge comes from.

Here is how the holdout breaks down.

Set a budget

Tell us your budget and we will spend it on the people the model expects to move, highest uplift first. If the good candidates run out before the money does, we stop. We would rather leave a slot empty than pay to put someone off.

Look closer

Change the budget to see how the answer moves, score the model against random, or open one customer and read their numbers.

Who we’re skipping

These customers score worst, so they stay out even when there is budget left. Here is what the ad does to each one, and why we skip them.

The fine print

Model. An S-learner treats “shown the ad” as an input. We ask it twice — once as if everyone saw the ad, once as if no one did — and the difference is the uplift. We also trained a T-learner challenger; it scored lower, so we kept the S-learner.

Groups. We sort people by their predicted chance of visiting and cut at . Above the cut on both sides is a “sure thing” — they visit either way. Below it on both sides is a “lost cause.” Low without the ad and high with it is “persuadable” — the ad changes their mind. A negative score is “do not disturb” — the ad puts them off.

Data. Criteo Uplift v2.1 is a public advertising incrementality test with customers. A random part of the population saw the ad and the rest were held back, and visits were tracked afterwards. It is public, it is cited, and it is not our own campaign. Purchases were too rare to lead with, so the headline outcome is a visit.