Do YouTube Ads Perform? Lessons From 190 Incrementality Tests

As a recovering growth marketer, I’m aware of the tendency we have to idealize certain advertising channels. It’s a lot more fun to envision creative concepts for a TV campaign than it is to write copy for a paid search ad. It also happens to be a lot more difficult to prove the impact of the TV ad. 

I’m also aware that most marketers aren’t blessed with the best toolkit to measure such channels. Their impact is typically delayed and eludes cookies, pixels, or whatever else we use to stalk consumers as they traverse the web. Even when we see a purchase attributed to such a channel, we’re left to wonder, “Did they even see the ad? Are these impressions actually doing anything?”

It’s an unending thought loop that typically results in dipping our toes into the channel, but never feeling like we’ve quite cracked it. “Are we lighting money on fire or sitting on a gold mine? Your guess is as good as mine, dear CFO.”

We endearingly label this group of channels “upper funnel”, though a shrug emoji would probably be more apt. Not quite brand, not quite performance, upper-funnel channels are stuck in growth marketing purgatory. Held to both brand creative standards and performance measurement standards, they’re often doomed from the start. 

One such channel is YouTube. 

YouTube’s measurement migraines

You may have heard of it — it’s the world’s second-largest social media platform and search engine. It also happens to be the most popular platform in the world to watch on your TV or to listen to music on, regardless of the device. 

Yet, despite its unmatched versatility, YouTube faces a massive monetization gap relative to peer advertising behemoths – its $10.5 billion in Q4 ad revenues remain nowhere close to Meta’s $46.8 billion over the same period. Meta does have a larger user base, but it also monetizes that user base much more efficiently. 

There are many reasons why this is true, but we’re here to talk about one in particular: measurement. 

YouTube advertising is uniquely difficult to measure because:

• Click-through rates are generally low.
• Google search cannibalizes much of its platform-reported attribution.
• It serves ads across a wide range of devices — from televisions to smartphones — raising the question of whether it should be measured as a so-called "direct-response" channel like Meta or an "upper-funnel" channel like TV.

To the third point: The correct answer is neither, but that makes us growth marketers very uncomfortable, so we force-fit it into our existing framework. We rationalize the incredibly low multi-touch ROAS or question the heavily view-through-biased conversions we see in Google Ads. But neither approach gives us enough confidence to scale it or to call it a loss and reset. 

However, there is another way — a type of measurement that does not bias towards click or view impact, that does not need pixels to track users, that can track the impact of YouTube across all your sales channels, and can tell you exactly how long it takes this impact to be felt at each level of your funnel. 

The solution we’re talking about is incrementality testing. 

Over the last 16 months, 74 brands using Haus have run at least one geo-based incrementality study on YouTube, providing us with 190 data points to better understand the channel from all angles — and, yes, we’re going to dive deep. 

With a highly creative-dependent channel like YouTube, there is no replacement for testing it yourself, but we hope the data below inspires you to take another look at a channel that is often misunderstood — as you’ll see, your gut belief in the channel probably wasn’t that misplaced after all. 

Insights from 190 YouTube incrementality tests

Understanding the data

Before we get into it, here are a few notes on the test data and terminology you’ll see below:

• The average YouTube test in this analysis was run for 21 days with an average post-treatment observation window (PTW) of 13 days for a total test duration of just under five weeks. 
• An incrementality factor (IF) is the ratio between the amount of incremental sales a channel drives relative to the sales reported in-platform. 
• iROAS = Incremental Return on Ad Spend
• Unless otherwise noted, we will refer to conversion-optimized campaigns only, such as Video Action Campaigns (VAC) and Demand Gen
• Unless otherwise noted, we will refer to the final test results including the PTW. 

Now, onto the fun stuff…

Google Ads severely underreports YouTube’s impact

Across brands in the study, YouTube drove 3.4x more incremental lift against DTC sales than was reported in Google Ads at the conclusion of the study. In other words, the performance you’re looking at day-to-day may be undercutting the channel’s true value by 70% or more — and if you’re looking at YouTube performance in a multi-touch attribution (MTA) tool, that haircut is likely even greater. 

And that doesn’t even account for the omnichannel halo effects…

YouTube drives significant halo effects across physical retail and Amazon

For brands in the study who sell across DTC and at least one other channel (e.g. physical stores, Amazon), YouTube contributed an average additional sales lift of +99% beyond its impact to DTC sales. In other words, your already conservative platform-reported number is probably a super-conservative number if you sell beyond DTC. 

For reference, the same set of brands measured an average omnichannel halo effect of +46% across all other channels they tested. Not too shabby, but it’s less than half the halo effect they saw from YouTube. 

It can take time to feel YouTube’s impacts

As we’ve already seen from the data above, YouTube can be a highly incremental channel. However, it doesn’t necessarily drive that impact overnight – it can take time to unfold. 

Across all experiments in the study, YouTube’s iROAS improved by an average of 79% during the PTW, indicating both a significant ramp-up and a long-tail effect. Furthermore, the average PTW was just under two weeks long — with a longer timeframe, it’s likely this number would be even higher. 

See below for an example of a customer who saw a persistent upward trend on lift two weeks after the experiment concluded. This is not an atypical finding for YouTube and is the callsign of a channel that has full-funnel effects. 

YouTube is an effective new customer acquisition channel

This last data point is for teams that are oriented around new customer acquisition. In a post-iOS 14 world, it’s impossible to ensure that your campaigns exclusively target prospects, but YouTube seems to do a pretty good job of finding them. 

Across the experiments we analyzed, YouTube drove 85% higher lift against new-customer KPIs than repeat-customer KPIs, with new customers responsible for 76% of its DTC sales impact. 

Tactic-level insights

Demand Gen > VAC? 

Google’s track record with rolling out new campaign types has had its ups and downs — we’re looking at you, PMax – but based on early signals it seems to have landed the plane with Demand Gen, the successor to VAC. 

In head-to-head tests at similar daily spend levels, Demand Gen outperformed VAC on DTC iROAS by an average of 7% by the end of the treatment period and 27% by the end of the PTW, indicating at least comparable direct-response impact and potentially stronger full-funnel effects. 

If these results hold up over time, it’ll be a major coup for both Google and advertisers, who will be forced to transition to Demand Gen in the coming months.

Reach campaigns can be used for direct-response

Not so long ago, this notion could be considered sacrilege, but growth marketers are waking up to the fact that spamming high-intent audiences isn’t the only way to drive incremental sales lift — and for good reason. 

Across clients who tested Video Reach Campaigns (VRC) against VAC, DTC iROAS was just 12% lower for VRC on the same average daily spend. With more time, it’s possible that the performance gap would’ve closed even more – VRC iROAS improved by 16% relative to VAC during the PTW. 

However, without validating its incrementality, investing in VRC does require somewhat of a leap of faith — its incremental impact exceeded in-platform attribution by an average of 10.0x, with some customers seeing an IF over 30x. 

If your brand is DTC-only, YouTube TV may not be a great fit

The ability to target connected-TV (CTV) inventory programmatically out of the Google Ads UI is an exciting convenience, but brands who tested it head-to-head against VAC found that it drove a 65% lower iROAS against DTC sales on the same average daily spend. The gap in short-term performance was even greater, but iROAS for YouTube TV improved by 18% relative to VAC during the PTW. 

That said, for brick-and-mortar KPIs, there was not a significant difference in iROAS across these campaign types. 

Exclusively targeting Shorts comes up…short

While YouTube Shorts inventory is eligible to be included in VAC and VRC, you currently can’t break out its reporting separately, which may be discomforting for advertisers seeking maximum control over their placements. 

However, a word to anyone considering splitting out Shorts as a placement — advertisers who tested this strategy saw 20% lower DTC iROAS on 55% lower daily spend compared to VAC or Demand Gen, in addition to a 32% increase in CPMs. 

What you should do with this data

It’s tempting to take insights like those shared above and assume they will apply to your business, but there’s a reason why we encourage brands to test for themselves as opposed to borrowing benchmarks from “similar” companies: it’s because there is no such thing as a similar company in growth marketing. 

That may sound like an audacious statement, but we have the data to back it up. Brands within a specific vertical, with similar product mix, and similar paid media portfolios will often yield wildly disparate results — it’s not rare that their IFs for the same channel are off by a magnitude of 3-4x. Ads, audiences, optimization settings: this is just a shortlist of compounding factors that will impact your results. 

With that obligatory disclaimer out of the way, here are some Haus-approved next steps: 

• With the right content and strategy, YouTube can be an effective direct-response lever that also contributes to a healthy customer acquisition funnel. If you feel tapped out on your existing channels or just feel like you can find your audience here, give it a test. 
• In-platform reporting and MTA tend to significantly underreport the true impact YouTube is having on your business, especially if you sell beyond DTC. Before you launch the channel, make sure you have an appropriate measurement framework in place to determine whether it’s working (e.g. with a geo test). 
• VAC is transitioning to Demand Gen this summer, but don’t be afraid — all signs point towards Demand Gen performing at least on par with VAC. If you’re already live on VAC, we encourage you to start testing out its successor sooner than later. 
• YouTube TV does not appear to be a strong competitor to its in-stream counterparts in terms of driving DTC impact, but if you’re a brick-and-mortar retailer who wants to dip your toes into TV, it’s worth considering as a self-service alternative to traditional linear. 
• Don’t sleep on reach campaigns — while the in-platform reporting likely won’t look great, you will find a differentiated audience from your conversion-optimized campaigns that may drive more direct-response impact than you expect. 
• There isn’t strong justification in the data for splitting out YouTube Shorts as an individual placement — in most cases, it’s likely wise to keep it in your VAC or Demand Gen campaigns. 

The power of incrementality testing

What a journey — we traversed through growth marketing purgatory, shedding unchecked assumptions along the way, and came out with a more balanced understanding of an often misunderstood channel. That’s the power of high-quality, causal data and the inventive group of customers that we’re privileged to work with here at Haus. 

If you’re interested in upgrading your measurement toolkit or just want to learn more about the work we’re doing at Haus — subscribe to our channel… err, I mean, connect with a member of our team anytime.