Overcoming the Potential Pitfalls of Artificial Intelligence for Pricing

A conversation with a data science expert about AI’s ability to overcome challenges associated with data, demand, and disruption

I recently sat down with my colleague Sneha Elango, Senior Analyst, Retail Innovation, to discuss the obstacles retailers face when considering Artificial Intelligence (AI) technology for pricing. She shared her perspectives on the challenges of data scarcity, intermittent demand, and collinearity, along with how AI is up to the task of helping retailers adapt to disruption. Let’s jump right in!

--

Sneha, can retailers embrace AI if they have little data to work with?

From my perspective, I don’t see “data scarcity” as a major roadblock to using AI. It takes a lot less data than you think to stand up an AI solution — it is quite good at leveraging the smallest amount of data to deliver insights.

Because of its transactional nature, retail is a data-rich industry. But depending on the kinds of questions you need to answer, you may not have all the data that you’d like. Consider all the possible data points in this age of retail digitization: channel, location, customer segments, new product development, promotions, discounts — not to mention competitive intel. There’s plenty of data, actually! It’s just that the more nuanced or granular your questions are, the less data you’re going to have to answer those specific questions.

Your largest data set is at the enterprise level. From there, you can ask the same question at the zone product level, then the zone category level, the store category level, the store product level, and so on. Using a Bayesian AI methodology, which we use at Revionics, you can model at these multiple levels of data and ask questions, using insights from higher levels to inform and gain confidence as you go deeper. This methodology helps you start close to your answer and get more precise as you drill down.

So, how does AI actually help retailers deal with data scarcity?

I see two aspects of using AI to “solve” data scarcity: what kind of question you can ask and how confidently you can arrive at an answer. There are many statistical methods that can help you boost confidence in spite of data scarcity. For example, you can use something like bootstrapping, averaging estimates from multiple small data samples to round out your data set. However, using machine learning and AI can help you be more productive and efficient in your use of analytics, including in your ability to pick up on and adapt to new consumer or market signals, which are cases of extreme data scarcity.

Transfer learning is a great example — that’s the idea of applying learnings from related ideas or experiences to the newer concept or hypothesis. Consider snow skiing. As a beginner, you could dismount from the ski lift for the very first time without any preparation. Or you could take what you know about walking, jumping, maybe even a time you water skied, and apply — or transfer — those skills as you descend the mountain.

Using AI is similar — with transfer learning, you can fine-tune models trained for a more general purpose or borrow features, allowing you to solve problems more quickly than you would starting from scratch and achieve an overall higher level of model sophistication. In this way, enterprise-level analytics and optimization can be fine-tuned to provide channel- or competitor-specific insights.

Another often overlooked aspect is how often you are refreshing your models and data strategy. Identifying the right cadence of refresh and remodel is crucial. Retraining your model doesn’t mean that the previous version is wrong; it means that you’re allowing it to learn from the most up-to-date information available. It’s also crucial to revisit your assumptions and strategic elements and make sure they are keeping up with the rapid acceleration of the trends we’re seeing now.

What problems does intermittent demand present to AI’s ability to deliver accurate price and promotion recommendations?

It’s known that consumers will buy certain items only at certain points in time. This won’t stand in the way of AI modeling; in fact, accounting for intermittent demand can yield more accurate and precise models. This is especially true for seasonal products, which have demand patterns that will fluctuate predictably.

Retailers want to understand how all relevant factors change or influence the shape of the demand curve. It’s as if you’re trying to assign a mathematical formula to the behavior of consumers — the more consistent the behavior is, the more elegant the formula will be. But when demand is choppy or intermittent, the formula becomes more complex. When applying AI to the “problem” of intermittent demand, it is important retailers have a causal demand model in play.

Revionics doesn’t employ “black box” deep learning, where you feed AI a set of data and blindly await its recommendation without insight into the levers and drivers that worked in the background. This approach doesn’t work well with intermittent demand or when new demand patterns arise. Because black box models don’t attribute value to the causal levers and drivers, they are less likely to pick up on signals that indicate those causal factors are shifting. At the same time, you have less visibility to understand what levers are at your disposal in designing your item, promotional, or competitive strategy.

A causal demand model, on the other hand, identifies the components of demand that lead to a recommendation: “If my model is telling me that demand is going to look like X next week, what are the different aspects influencing that?” This gives you room to check the model against your own intuition and expertise. When there are things coming up that your model hasn’t seen before, you can let the model know in advance and help it anticipate those changes.

If retailers aren’t careful, how might collinearity complicate their use of AI for making pricing decisions?

With collinearity, you may misattribute a causal element, or you might be considering multiple explanatory variables that are highly correlated. You may end up with imprecise results because each of those factors is creating redundant aspects in your model.

Think of Black Friday as an example. When we’re modeling demand around this shopping holiday, ask yourself how much of that demand is caused by the fact that it’s a marked seasonal change from fall to winter? How much of the demand is specific to holiday hosting or gift buying? How much of the demand is driven by sales and deep discounts? All of these will factor into demand because of the fundamental shift in consumer preferences and behavior.

To combat collinearity, you need expertise in both real-world retail scenarios and the AI methodologies available to deploy. You need broad experience in both retail and machine learning to understand how to best capture and translate real-world signals into model components and then surface insights that retailers can operationalize.

I guess what I’m trying to say is that it’s an art and a science. Revionics is in the market, day in and day out, implementing these models — we have a good sense of what works and what doesn’t. We preach the importance of testing and learning with AI, constantly investigating, which will help illuminate when collinearity may be a factor, where product cannibalization may be happening, and so on.

Can retailers trust AI to adapt to disruption, as seen during the COVID-19 pandemic, or is this where human input is still needed?

What affects one market today is going to affect another market tomorrow. We saw that when we at Revionics handled inflation in 2008. Hyperinflation is back again, and our models are more intelligent for having seen it in the past.

In an increasingly global market, everyone is a lot more sensitive to disruptions — consumers and retailers alike. With COVID-19, it wasn’t just that stores closed or that people were more reluctant to go out and shop in person, we also had a lot of supply chain issues. Shoppers’ purchasing behaviors also changed as they picked up new hobbies. I think every single disruption that we see now is going to ripple out and have a lot of second- and third-order impacts.

With disruption in mind, you can’t just upload your pricing into some machine that’s going to spit out a price and then go and execute exactly those recommendations in each one of your stores. No, the role of AI is to automatically sense the different signals coming in from shoppers and evaluate the impact of those signals on any of your pricing behaviors. In an intelligent way, AI will direct your attention to where it needs to go the most, especially during disruption.

This is especially important when you consider all the teams and vendors that touch pricing, promotions, and markdowns. During disruption, it’s important to preserve reliability in your systems, and AI will certainly do that. AI will also take on the heavy lifting of high-dimensional calculations and permutations. Think of AI as an outsourced team member freeing up your talented team members to do what they do best.

Finally, how does Revionics overcome these potential AI ‘problems’?

Revionics has a broad set of statistical tools in our arsenal that can be deployed to model each aspect of consumer demand. Beyond that, we’re set apart because of our deep understanding of the things that impact retailers, particularly as they relate to their pricing activities, and our decades of retail experience. We constantly strive to keep our AI on the cutting edge and use best-in-class data science techniques. And we want our solutions to provide the collaboration and visibility that pricing teams need today to price and promote with excellence.

There are always going to be challenges in the market and perceived “problems” that you may doubt AI’s ability to overcome. That’s why a partnership with a company like Revionics is paramount to your success.

--

Thanks, Sneha, for all this helpful context around AI and machine learning!

To learn more about taking hold of all the benefits of Artificial Intelligence for pricing, continue reading on our blog:

• Why Retailers Need AI Price Optimization
• Getting Started with AI