So how do you take advantage of this opportunity and create new shopping experiences? The key to great customer experiences, whether online or in-store, is personalization. Personalization is based primarily on past buying habits and patterns (e.g., knowing when someone buys, how often, how much money was spent, etc.), and preferences (e.g., brands, colors type, style, etc.). AI in particular, can play a key role in personalization on mobile because it can:

Use more data inputs (e.g., social media data) to create micro-segmented personas.
Predict behavior using a breadth of real-time data such as moods, current events, location, and user activity.

In this blog post we’ll look at two use cases where AI and converging technologies can provide targeted content and messaging to optimize the customer’s shopping experience – both on their devices and in the store.

Natural language shopping

Imagine a user surfing the web on their smartphone, and they see a picture of a celebrity wearing a jacket they like. Next, they ‘tell’ their smartphone to purchase that jacket for an event on Saturday. Using AI, natural language processing (NLP), data collected about the event, and known user attributes such as their measurements, the app can then shop for that jacket online and present sellers that can deliver it before Saturday.

Developers could implement this on a platform like the Qualcomm Snapdragon 845 mobile platform, where its Qualcomm Hexagon DSP can perform run-time text analysis to understand the context of the words. When combined with a cloud service in the backend that gathers and reviews the user’s social media profiles, the app identifies sentiments and gains insight into attributes such as wardrobe preferences that could be used to present accessories to enhance the jacket. Here, a well-designed REST API segmented into endpoints such as AI functions, social media operations, and multimedia transfers can complement on-device functionality with rich app-to-cloud communications.

On the edge device, visual analysis of the celebrity could be accomplished using the Qualcomm Neural Processing SDK for AI in conjunction with Caffe2 or TensorFlow. For example, these components could help analyze and extract the jacket feature, and then send it to the cloud service for comparison against images of jackets from other online stores to find those that sell it. Using sentiment data about the user, the cloud service may also present similar jackets in perhaps different price points that might appeal to the user.

Using a big-data datastore, the cloud service can store this information for millions of users that the retailer can use to further re-train their AI models for additional insights. For example, after millions of purchases have been made, AI can be used for advanced insights such as predicting customer demands and inventory levels to help optimize the amount of product to stock.

In-store product-aware shopping

As a user enters a grocery store, an app on their smartphone detects coffee on their shopping list and shows an in-store map with its location. The product aisles could be equipped with mesh beacons powered by 802.15.14 (e.g., Thread and Zigbee) that are included in QCA4020 System On Chip (SoC), to broadcast information about products to the app that are in stock.

As the user approaches the coffee, the app detects nearby products broadcast by other beacons, and presents coupons based on past purchases — say, for a package of their favorite biscuits located nearby. The app also notices an upcoming appointment on the user’s calendar. Using a neural network trained with data about past appointments, the app determines that this is a dinner meeting with an important client who enjoys wines, so it recommends a bottle of wine available for purchase in the next aisle.

Pre-training neural network models for such personalized alerts can be accomplished with Caffe2 and powered by the Neural Processing SDK for AI. Here, recommendations can come just-in-time through on-device, real-time deep learning performed by the neural processing engine, while eliminating the need to send data back to the cloud for processing.

The app can also continually re-train its neural network with deep learning as different combinations of products are purchased. This can be used to continually refine the network to produce even more accurate recommendations based on additional factors like sentiments.

The retail experience of tomorrow is here today

As the use cases above have shown, there are endless ways to integrate AI and other converging technologies into retail experiences that benefit both retailers and consumers. And with more shopping being done through mobile devices than ever before, retailers can implement solutions on both mobile and IoT devices to optimize the experience and potentially drive sales.

With the holiday season approaching fast, we’d love to hear about how you are integrating intelligence into your retail systems and your customers’ shopping experiences.