Episode
5
:
5 Mistakes Brands Make When Running Marketing Experiments

Joe Wyer, Head of Science at Haus, discusses five common mistakes brands make when running marketing experiments. The session covers bias and precision in experimentation, the importance of pre-commitment, aligning KPIs with customer actions, the limitations of statistical significance, and the challenges of interpreting conflicting attribution and incrementality data.

Notable moments:

  • [00:00:00] - Introduction and background of Joe Wyer
  • [00:03:51] - Discussion on the importance of considering bias and precision in experiments
  • [00:09:00] - The value of pre-commitment in experiment design and analysis
  • [00:15:33] - Aligning KPIs with customer actions and the customer journey
  • [00:22:25] - The problem with having an overreliance on statistical significance in decision-making
  • [00:28:53] - Challenges when attribution and incrementality disagree
  • [00:33:51] - Story about Jeff Bezos' commitment to causal inference at Amazon, highlighting the importance of experimentation for business success
  • [00:39:13] - Q&A session begins
  • [00:56:54] - Closing remarks and wrap-up
Oops! Something went wrong while submitting the form.

Other Open Haus episodes

Marketing Measurement in a Post-Cookie Era

Venu Sports CMO Brian Borkowski (former SVP Marketing at FanDuel) joins Open Haus to discuss the value of experimentation in marketing.

READ BLOG

Unpack Your Marketing's Impact on Amazon and Retail

Connor Dault, VP Growth and Digital Product at Caraway, joins Open Haus to talk omnichannel impacts.

READ BLOG

Growth and Measurement Trends: Perspectives from a Marketing Leader

Rob Webb, a Growth Marketing Consultant and former SVP of Growth at Tonal, joins Open Haus to discuss growth and marketing trends.

READ BLOG