Attribution, Incrementality, and Impact
January 6, 2022 -
Before I begin, I’d like to credit Danial Nakhaeinia as co-author of this post, he’s who I mean when I reference my favourite data scientest about the mid-way point.
So, anyone who knows me, knows I have been ensconced in the attribution world for over a decade. Taking a step back – Attribution, we’ll keep it simple and look at it from an ecommerce perspective. Attribution is the understanding that people don’t generally see one ad, click it and immediately convert. This ‘last click’ model is what analytics had been based on for many years.
But then some super smart person realized that people may see and/or interact with many ads or content, both online and offline, before making a purchase.
Attribution models divide up revenue, using simple or complex math, to ensure every one of these touchpoints gets credit for that purchase.
A simple example – someone does a search for sneakers and sees an ad for Nike, clicks on the ad, but does not immediately buy the shoes. The next day they search ‘Nike sneakers,’ click on an ad which takes them back to the Nike website. They see the sneakers and buy them. With last click attribution (old school analytics), only the ‘Nike sneakers’ ad would get ‘credit’ for the purchase. With attribution, all touchpoints (both the sneakers and the Nike sneakers ads) would get equal credit for the purchase/revenue.
(You can read more about in this post, Defensive Attribution, Let the Models Fight it Out)
So obviously that is important. But in walks the concept of incrementality. Incrementality focuses on understanding the ‘lift’ that each marketing touchpoint has on conversions that were likely to happen anyway. To figure this out involves testing, using control and test groups – the test group is exposed to an ad and the control is not. The results may show that the conversion rate of the test group is 5% and the control is only 3%. So you infer, (enabling you to make revenue calculations for budgeting and forecasting based on this information) that this incremental lift (2%) is due to the ad.
Again, also important. Hence the great debate – which is better? Up until very recently, I would have said this should be a conversation, not a blog post, since there are way too many considerations. But then my favorite data scientist introduced the gigantic flaw(s) plaguing both of these approaches.
Suffice to say, and i quote,
“Most multi-channel attribution models (a.ka. Shapley values or DDA) are full of heuristics—decisions made by (expert) people. These attribution models are based on algorithms that people invented to reflect reality, but reality is much more complex.
Over time, you accumulate a trove of source types and chains. That makes channel/Campaign performance difficult to compare.”
But the bigger flaw is that many attribution and incrementality models are based solely on marketing/traffic sources, when really a less myopic view needs to be considered – understanding the entire behavior that drove traffic to your site AND what happened (and didn’t happen) on your site.
Enter Behavior Based Attribution.
Behavior-based Attribution Modeling using Machine Learning (ML):
Unlike single/multiple channels attribution models and custom models, advanced ML attribution models (a.k.a. Behavior-based attribution model) not only evaluate the channels/campaigns that have led the user to the web/conversion, but also analyze the behavior the user has shown on the site, and cross-references all that information to reflect the entire purchase cycle.
Therefore, attribution models with ML are not pre-designed, but continuously adjusted, and become more accurate over time, making them able to:
- Distribute value between multiple channels/campaigns (without manual algorithms and predefined coefficients).
- Process hundreds of features (behaviors) as opposed to algorithmic approaches. So in contrast with other Attribution Models, the Behavior based approach is less vulnerable to the data changes and cookies absence.
All attribution models are retrospective, meaning they tell us some statistics from the past, while ML attribution models tell us about the current performance of channels/campaigns and what to do in the future”.
Real-time (continuous) results:
Automated (real-time) attribution results enable you to see and compare the current performance of the channels/campaigns to optimize the costs and increase ROI.
I get that was a lot, I will be ‘unpacking’ all of this in the upcoming days (and introducing the concept of ‘impact’). But the takeaway is – when it comes to attribution, think more holistically not just what gets the traffic to your site – but what the traffic does once it gets there.