Affiliate on/off test reveals the incrementality of loyalty sites for global DTC brand.

This multinational enterprise, with operations in eight countries, allocated a significant portion of its marketing budget to cash back and reward sites as part of their affiliate program. They wanted to understand and quantify the true impact of these loyalty partners. Using last click reporting from Google Analytics, it appeared to be one of their top performing channels, but they also suspected that last click reporting was overstating its impact and taking credit for sales that would have happened anyway. 

This brand was interested in running a controlled incrementality test on the Affiliate channel, but faced challenges on design due to the inability to randomize users or geos into control/exposed groups within affiliate channels like coupon sites and cash-back programs.

‍

This brand turned to Haus to overcome their measurement challenges, and determined this was an ideal scenario for a hybrid time series & geo experiment. After strategizing, the decision was made to deactivate all affiliate loyalty partners in the US for a duration of seven weeks. Haus constructed a synthetic control model using the other seven countries that remained on, allowing Haus to analyze the difference in revenue between the treatment and control, to assess incrementality.

‍

At the conclusion of the test, the analysis revealed no change in revenue between the US & comparable regions. As expected the brand did observe a sizable drop in Google Analytics affiliate share during the shutoff period. 

Haus also ran a product-level analysis to see if products that had higher affiliate shares before the shutoff fell relative to products that had lower affiliate shares. Top product SKUs with higher affiliate shares did not see a decrease in sales relative to those with lower affiliate shares.  If the affiliate channel were largely incremental, we would have expected to see products that had a higher affiliate share to be more impacted by the shutoff than products that don’t have as high affiliate share. 

Overall, this test indicated that affiliate loyalty partners were not incremental and this brand significantly lowered their US investment in the channel moving forward. In lowering their investment, they saw no material decrease in revenue and were able to reinvest the savings into more incremental channels to improve their customer acquisition costs.

‍