Gen AI powered Recommender Systems for Revenue Growth

What happens when customers have too many choices?

As people ponder what to buy, many brands offer recommendations. The recommender systems that enable this leverage data on what users have already liked or bought and are particularly useful when brands offer such a huge array of products that the choices can overwhelm the customer, whether that’s music, movies, accessories, or other products.

Recommender systems are integral to modern digital ecosystems. Well-executed recommendations can increase the customer's affinity for a brand and can drive revenue for brands by increasing cross-sell and upsell. But when recommender systems don't deliver truly personalized experiences, customers become disillusioned with the underlying brand, with disappointing results for the bottom line.

Traditional Recommender System Techniques

Core recommendation engine functions like handling user interactions and maintaining scalability still rely on traditional techniques, which can include:

• Collaborative Filtering relies on user-item interactions, making recommendations based on the preferences of similar users or items. It's like asking a friend with similar tastes what movies you should watch next.
• Retailers often use Association Rules (Market Basket Analysis) to identify items that frequently co-occur in transactions—the customer who buys bread might also buy butter.
• Variable-order Markov Chain Analysis predicts user actions based on their sequential behavior history. It’s akin to predicting the next word in a sentence based on the previous words.
• Matrix Factorization is a cornerstone of many collaborative filtering systems. It decomposes a large user-item interaction matrix into lower-dimensional matrices to uncover latent factors that explain observed preferences.

GenAI can’t replace recommenders.

But it can help brands fill data gaps and understand nuanced user behavior by generating synthetic data, creating personalized content, and providing context-aware insights that enhance the results that recommenders provide.

Overcoming Data Scarcity and Cold Start Problems

These traditional methods rely heavily on historical user-item interaction data. But when that data is limited or missing, they can struggle, especially with the "cold start" problem –– making recommendations to new users or introducing new items to existing users.

To address this, brands can use GenAI to create synthetic data that mimics real user interactions, then use it to enrich existing datasets, leading to more robust and nuanced recommendations. It most often does this with two techniques:

• In Generative Adversarial Networks (GAN), two neural networks compete, creating new data (like forging a painting) and judging its authenticity. Over time, the forger gets better at creating realistic data. In recommender systems, GANs can create fake user interactions for new users or unpopular items, improving recommendation accuracy.
• Variational Autoencoder (VAE) models learn to compress data into a simpler form and recreate it. They can then use this "code" to generate new data points similar to the originals. In recommender systems, VAEs can create synthetic user profiles or interactions, enriching data and leading to better recommendations.

Hybrid models can leverage both traditional methods and GenAI. For instance, they might use collaborative filtering to identify similar users and recommend items based on their preferences while simultaneously using a VAE to generate synthetic interactions for users with sparse data, enhancing the performance of the collaborative filtering model.

Unified Propensity Models

Recommender systems traditionally use propensity models to drive upsell and cross-sell, predict churn, and perform other functions. These models are accurate but can be complex and resource intensive. GenAI offers an alternative, a "decent proxy" for these models that can analyze trends and user behavior. Although not necessarily as precise, GenAI can achieve good results with significantly less effort, so long as we remember that GenAI models themselves can inherit biases from the data used to train them.

While GenAI can be more efficient , individual propensity models might still provide slightly more accurate results in some scenarios. The key lies in finding the right balance—so when the brand has a high-value customer making a key business purchase and wants to target them with a premium product offering, it might be better to use a dedicated upsell model. For broader recommendations where speed and efficiency are paramount, GenAI's unified approach can be highly beneficial.

Do you need a recommender system at all?

The null model—simply recommending popular items—can work surprisingly well. It's easy to implement, requires minimal data, and guarantees users will see well-regarded products. It can be particularly useful for new businesses or those with limited data on user preferences. Focusing on popular items can also create a sense of social proof, encouraging users to choose products others have enjoyed.

But again, GenAI can play a role. It can be crucial in determining when to choose a null model over a recommender system (assuming the brand already has one). It can also analyze user behavior, interaction patterns, data availability, and other factors to predict the most effective approach for each scenario. We're not at the point where GenAI can (or should) dictate when to switch between null models and complex recommender systems, but soon it will be able to leverage intelligent routing or agentic frameworks to make this recommendation.

Ever onward

As GenAI matures, recommender systems will transform into even more intricate tools, weaving themselves seamlessly into the fabric of digital experiences. Recommendations will not only anticipate our desires but surprise us with hidden gems. GenAI can unlock this potential, but further exploration and development are needed to bring this vision to life.

Citations

• "Why personalization isn't enough: How to deliver customer experiences that win" McKinsey & Company Report (2022)
• "How to Avoid the Personalization Paradox: Balancing Privacy and Relevance," https://www.forrester.com/blogs/category/personalization/ Forrester Research Report (2021)
• "Generative Adversarial Nets." Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., ... & Bengio, Y. (2014), Advances in Neural Information Processing Systems (pp. 2672-2680).
• "Auto-Encoding Variational Bayes." 4. Kingma, D. P., & Welling, M. (2014). Proceedings of the 2nd International Conference on Learning Representations (ICLR).