Building an Analytics Foundation for AI (Artificial Intelligence)

In the next 5-10 years, AI will improve our standard of living in the same richter scale as did electricity and the internal combustion engine for our forefathers. Every facet of our lives, business, law, medicine, government, education, and other areas where our imagination will take us will be transformed. Yann LeCunn, Director of Facebook Artificial Intelligence Researcher (FAIR), explains that “we’ve only seen 5% of what machines can do.” Given all possibilities, it is no surprise that AI is hot.

Analytics Foundation

But amid all the excitement, it is important to note that AI itself does not guarantee success as a business—as it will soon become mainstream. Machine learning is only as good as the data you feed it, so the level of success will vary depending on your state of analytics. To achieve its full potential, businesses should invest in a long-term analytics strategy and build a robust business intelligence infrastructure. While there is no denying that businesses will have significant wins with AI, those that invest in a solid analytics foundation will differentiate themselves from the rest of the pact. To build a solid foundation, companies need to focus on three areas, a well thought out pixel tracking implementation, a solid web analytics setup (our recommendation is Google Analytics), and a data warehouse that unifies web, transaction, and other marketing data. Doing so achieves data integrity, easy accessibility, speed to data, and a 360-view of customers, all of which are important to AI. But a 360-view is particularly important to AI because it improves the ability to feature engineer – the process of creating variables for machine learning algorithms. This in turn enhances the ability to apply machine learning in more areas of the business and develop better models.

Case Study: Why an Analytics is Important

An example supporting this point includes an e-commerce retailer. Using machine learning, a targeted customer list was selected for an upcoming Christmas marketing campaign. Based on 3 years of customer transaction data, a list of few hundred thousand predicted buyers were selected from a database of millions of customers. The results were significantly better than prior RFM methodologies (Recency, Frequency, and Monetization) as revenues increased 27% YOY despite a 17% decrease in spends (see below).

Importance of Recency Variables

However, the machine learning algorithms could have been performed better. The reason is that the machine learning algorithm only used transaction data, thus there was a gap in modeling web behaviors. An interesting insight was that of the 60+ variables, purchase recency was by far the most important predictor (see below) in identifying a Christmas purchaser (see below).

Intuitively this makes sense. The more recent a customer is engaged with a brand, the more likely it is top of mind, thus increasing the probability of a purchase in the future. To visualize the importance of purchase recency, Christmas buyers and non-buyers were analyzed in a post-mortem analysis. As it turns out, Christmas purchasers had an average prior purchase around 150 days, while non-buyers averaged around 375 days since their last purchase (see below).

While the campaign was still a success, there was significant upside to increase revenues. The problem was that the company lacked a data warehouse to store both the transaction and web data. As a result, recency of a web visit and web specific behaviors were not modeled, thus missing the opportunity to identify consumers that may have been actively visiting the site with purchase intent. This is a prime example of how an analytics foundation can provide companies the data to better understand consumers and build more accurate machine learning algorithms, thus giving them a competitive advantage.