Facial Emotion Recognition in Retail Stores

As the integration of AI within the brick and mortar stores is increasing, the range of available technology is growing with it. Previously existing barriers to entry, such as cost and usability are eroded by the need to stay competitive and maintain the level of customer service expected by today’s clients. Sentiment analysis via social media has long been recognized as important to understand consumer mood toward brands and products. Social listening is now available as part of Microsoft Dynamics 365 Social Engagement portal, which provides a calculation of sentiment value using natural-language processing and machine learning techniques. As a result of yet another leap in technology, facial sentiment analysis is now possible in brick and mortar stores. Disguised cameras pointed toward key fixtures in the stores are now able to identify the sentiment on the shoppers' faces as they interact with the products, as well as provide an estimate of their age and gender and in some cases facial recognition. The technology tracks the seven expressions of primary emotion — joy, surprise, sadness, anger, fear, disgust, and contempt– and can identify overall sentiments such as positive, negative, and neutral. The value of this data is apparent – evaluation of the effectiveness of visual merchandising, understanding of consumer mood toward displayed products, the capture of the demographics of the client base, signaling store associates to the presence of a loyal client, are just a few possible use cases. Taken a step further, it’s possible to integrate the existing CRM system to reach out to recognized clients with an offer if they have shown interest in an item, but have not completed the purchase. Fixtures can be remerchandised with more effective product pairings based on A/B sentiment tests combined with POS data. In-store sentiment analysis can be combined with the social sentiment to evaluate overall brand perception and make just in time strategic changes that can quickly impact sales. The privacy concerns associated with this technology are clear – the information gathered can be linked to numerous other data points collected about the consumer without their knowledge. The question is whether the resulting personalized and more convenient shopping experience will compensate for the loss of privacy in the consumers’ minds.