Heuristics! Heuristics! Heuristics!
November 22, 2021 -
Heuristics. That’s a good word – a practical way of solving a problem that produces a, potentially not optimal, solution but is good enough based on constraints.
The reason I love this word is because, back in the day (pre-machine learning), that (Advanced Heuristics) was a setting option that web analytics software ClickTracks, had and was a key point in our training.
The surprising thing was that using heuristics to do this was quite accurate. Sure, you wouldn’t want to use the web reports to reconcile sales, but it was directionally accurate nonetheless. The even more surprising thing was transforming raw data, (did I mention, in a pre-ML world) from a log file to meaningful, business-changing reports was not only just what ClickTracks did, but you could change a setting and fundamentally change the way this was calculated with a click of a button.
It astounds me – the more super smart people I meet – the more this word comes up.
Most recently a crazy smart data scientist I work with (we call him Danial) had this to say about attribution models:
“Most multi-channel attribution models (a.ka. Shapley values or DDA) are full of heuristics—decisions made by (expert) people. These attribution models are based on algorithms that people invented to reflect reality, but reality is much more complex.
Over time, you accumulate a trove of source types and chains. That makes channel/Campaign performance difficult to compare.”
I guess as much as I love the concept of heuristics as it relates to analytics, in today’s world, experts looking to the past to predict the future is no match for machine learning and artificial intelligence actually doing this.