Your personal style advisor available 24x7 - Designing Human like Digital Shopping Experience through AI

How multimodal AI capabilities transform online shopping experience to mimic real in-store buying experiences?

AI is ushering in massive changes to online shopping. It is the dawn of era where each of us have our Personal Style advisor.  While voice based input along with text has been in use for last few years, the inputs or searches do not persist with context. Each search is discrete. This leads to difficulty in finding the right item unless significant time is spent or choice specificity is not important.

Digital Brain along with context engineering helps elevate understanding of customer intent along with maintaining the context of the conversation across queries. This leads to more personalized recommendations.

Online retailers can start with text chat based conversational experience before upgrading to voice based conversations. 

These agents will grow to be more sophisticate and get infinite memory and infinite context. It is similar to having a style advisor that remembers all your purchases and all your interactions. The ability of companies to leverage the info will be a big differentiator in building customer advocacy as recommendations can be significantly personalized. Earlier in August, openAI released advanced speech-to-speech model yet-gpt-realtime. This will change the way online shopping is done. The cost of tokens will come down.

What multiplicative capabilities does it deliver?

• Consultative discussions leading to better recommendations
  
  • Customers can speak or chat naturally. The sentence construct or tone of the customer voice can provide insights on customer sentiments
  • Consultative discussions makes the shopping experience more engaging with recommendations shared through voice and visuals. The visuals include videos. 
  • Interactive discussions lead to instant feedback from the consumers enabling rapid tailoring of recommendations 

• Computer vision led virtual try ons 
  
  • While this functionality has been in existence for a long time, the sophistication has improved significantly with LLMs. This reduces returns by over 30%.
• Product Bundles – based on high level shopping objective eg. Going for camping or attending a friend’s marriage, recommendation engine can suggest entire set of products needed. This can be filtered on many dimensions such as price or styles. With ability to sell bundles, companies can come up with innovative pricing and increase average order value.

• Deep Product Knowledge – Shopping Assitants will be able to share details about the products which sometimes is difficult for humans to remember and highlight. Some products may have a back story which may appeal to a customer helping the saleability.

• Better discovery of unique products – Shopping assistants can suggest products based on customer preferences in a manner similar to what a human associate would do. They could highlight new launches or new brands. In the absence of conversational shopping such products may not be uncovered.

What impact will these new capabilities bring?

• Improved Conversion: Conversation based interactions lead to 35-65% higher conversion rates 
• Increased AOV: Better product discovery leads to 25-40% higher average order value 
• Reduced Returns: Accurate visual try-ons result in 30-50% fewer returns
• Customer Life Time Value: This can become 2X+

What design and technology approach needed to fulfil the promise?

Technical Challenges need to be focussed on 

1. Guardrails for the Styling agent so that she only responds to tasks in scope of shopping. Else responds politely: “Apologies, I am not aware). Grounding leverages multiple techniques such as Chain-of-Thought prompting.
2. Fine graining use case for prioritizing implementation - Take a task based approach for agentification and prove ROI before automating the entire workflow e.g. shopping journey has multiple steps including pricing, discount, cart management. In the beginning, we would want to focus just on enhancing the experience of finding the right article and making it humane and personalized.
3. AI Economics – Cost vs ROI –Cost of tokens also need to be managed.. Choice of models for different tasks need to be different. Also, there could be option for self-hosted vs public cloud hosted models.  
4. Performance – While performance is important, design can help us render the right engagement even when there is some lag. 

Design

1. The experience design for the application needs to mimic fluid agentic design
2. Delays and latency needs to be managed through appropriate design.
3. The redesign of interface for interactive shopping experience needs to be aligned with product and category managers to ensure business alignment. To ensure this, wireframes should be prepared and signed off before proceding with development for interactive experience.

Here the building blocks of  typical styling agent or a shopper assistant look like

‍

‍

‍

Why is every block important in the blueprint and what role they play? 

Frontend‍

Web / Mobile → Customers ask shopping questions and receive answers.

Admin Console → Store managers configure, monitor, and tune the advisor.

Control

Guardrails → Prevents unsafe or irrelevant recommendations.

Auth / Rate Limit → Ensures only valid users access, avoids overload.

Agents

LangGraph Orchestrator → Routes user queries to the right tools and data sources. LangGraph is an example here.  

LLM → Generates natural shopping advice from data + context. Serves as brain for an Agent helping to plan, and take actions 

Context

Session Memory → Remembers user preferences across the conversation. This differentiates this solution from standard search.

Prompt Templates → Standard query patterns for queries like product search

Context Sanitizer → Cleans and normalizes user inputs.

RAG

Retriever → Finds relevant product, review, and guide data.

RAG Assembler → Merges retrieved info with reasoning for accurate answers.

Data Layer

Vector DB → Enables semantic product and query search.

Knowledge Base → Stores FAQs, guides, and product details.

Commerce DB → Holds live pricing, stock, and order data. This is where the magic of long memory happens making the AI agent more valuable to the buyer.

Eval/Obs

Observability → Tracks advisor performance and reliability.

Eval Benchmarks → Tests product query and recommendation accuracy.

Feedback Loop → Improves results based on customer feedback.

‍

Implementation

• Both web and mobile shopping will be significantly transformed. New open weight models lowers the cost for APIs.
• With OpenAI releasing gpt-oss-20b model with 3.1b active parameters, this can be bundled with the mobile app and deployed on edge (mobile phones) especially on high end phones. Needs 6gb of memory.  Mobile quantization can lead to even smaller memory requirement and battery needs. gpt-oss-20b has 128kb as the context length. This is good enough to share 15,000 – 20,000 words
• Grounding is one of the key aspect of implementation of such an agent. Enabling the agent with Live Catalog data for search is paramount. RAG capability for product info, reviews, FAQs prevents hallucinations and allows LLMs to be used for only articulation in natural language. 
• Guardrails and business rules are essential for any agentic activity for example (e.g., no alcohol for underage buyers). Another important aspect is to let the user know the reasoning behind various recommendations and change in plans.

‍

Agent Hierarchy

• Style Advisor Agent (orchestrator)
  
  • Try On Assistant
  • Product Catalog Agent
  • Inventory Agent
  • Ordering Agent
  • Delivery Agent
    

‍

What further changes are expected in near future

The question for enterprises isn’t whether multimodal AI will become the standard for customer interaction—it’s whether they’ll lead this transformation or be forced to catch up as competitors redefine what customers expect from digital shopping experiences. This will not only impact Retail Shopping, but also insurance buying, loan disbursal and Travel booking. All consumer shopping experience will be reinvented. The power of AI will only increase. Even if there are challenges in the current technology readiness, enterprises should start taking advantage of the capability and start their implementation and adoption journey. 

OpenAI has improved the streaming interactive models. More models will be released in short order and costs will fall (currently voice output token are $40/1M tokens). This will drive voice and language to become the primary mode for online shopping. All the digital interfaces will change to align to this need and trend.