Incrementality School, E1: What is Incrementality?

Incrementality. 

What’s the first thing you think of? 

It might be testing, it might be something related to attribution or marketing measurement, it might be something associated with experimentation or statistics or advertising spend but you’re not quite sure how to connect the dots. Or maybe you’ve heard the word thrown around, feel like it’s something you should be familiar with, but aren’t really sure what it means or how to use incrementality experiments in practice. If so, you certainly wouldn’t be alone.

That’s why we’re here: to not only help explain this concept, but to walk you through how to use incrementality experiments in marketing and business decision-making. We have a strong viewpoint on this stuff – so much so that we’re launching a new series about it. 

Welcome to Incrementality School.

A snack-sized, tactical guide to incrementality  

Congratulations, you’ve stumbled upon the very first piece in our series aimed at helping marketers, CFOs, and other stakeholders wrap their heads around this funny phrase that online dictionaries have yet to recognize (sidenote: we’re working on this). Together, we’ll cover:

• What incrementality is (insert "you are here" marker)

• What you can test with incrementality and what the consequences of not testing might look like
• How brands measure incrementality today (and the approaches that don't measure incrementality)
• Who actually needs incrementality testing 
• The difference between incrementality experimentation types (such as geo-testing, conversion lift testing, and natural experiments)
• What the output of an incrementality test looks like and how to act on the data
• How to foster a culture of incrementality experimentation at your own organization 

… and maybe even a thing or two more. These aren’t meant to be 201s or 301s – this is incrementality 101 for you, the marketer or financial stakeholder who’s heard the word and needs a primer that's deeper than surface level, but not scientist-dense. Snack-sized, if you will.

Get ready for class – you’re in good company.

Three incrementality experts, one shared understanding

Let's start with the basics: Haus' Head of Strategy Olivia Kory describes incrementality by likening the concept to randomized control trials in healthcare:

‍When you’re rolling out a new drug, you’re going to give one group of people a placebo drug (the control group) and you're gonna give another group of people – statistically indistinguishable from the control group – the drug (the treatment group).

Then, you observe the difference in behavior between those two groups to validate the efficacy of that drug – that’s incrementality testing. 

With Haus, we have a counterfactual to understand what would have happened anyway in the absence of this marketing intervention, whether it’s search, or video, or YouTube, or OOH. What was that group going to do anyway? And that’s fundamentally what we mean when we talk about incrementality testing.

Haus Principal Economist Phil Erickson takes us one step further, transitioning us from the concept of incrementality to how we might use in incrementality testing in a practical sense:

Incrementality measures how a change in strategy causes a change in business outcomes. For example, how would my revenue increase if I increased my ad budget by 10%? Or how many more units would I sell on Amazon if I moved 50% of my ad spend from YouTube to Google PMax?

Lastly, chew on this – from our no-nonsense Head of Science Joe Wyer:

Attribution and incrementality are two frameworks for understanding marketing impacts. Attribution frames impact as "crediting" each customer event to some strategy or tactic. Incrementality frames impact as the change in customer events caused by a change in the application of strategy or tactic.

The thing about "incrementality" is that it's just the non-scientist-friendly way of saying "causality.”

Causality. Incrementality = causality. 

Here’s an open secret: Whereas traditional marketing measurement solutions are rooted in correlational or observational data (yes, even traditional MMMs), incrementality is rooted in causation. Actual causation – that enables you to know exactly what’s working instead of what may-or-may-not be working.

And that, dear readers, is where we’ll leave things for now (told you: snack-sized). Tune in next week to learn more about what kinds of things you can measure with incrementality testing – and the potential real-world business consequences of punting on experimentation.