First deployment of our new chat agent helping customers with product suggestions

Overview

My first step was to understand what kind of issues customers had that would prompt them to ask the chatbot.

I analyzed all chatbot conversations from the past year, and discovered I could categorize most questions into the following

• Product Questions - When customers had questions about a specific product

• Product Suggestions - When customers did not know what product they wanted, but had a problem they wanted to solve

• Policy Questions - Questions on shipping, returns and other logistical clarifications

We needed a chatbot that could somehow learn about all the different products, answer questions about them, and talk to customers in a natural way.

Question 1: Can we modify the current chatbot to do this?

Answer: No.

Question 2: Is there a 3rd party solution that fits our needs?

Answer: There are 3rd party chatbot and AI chat apps, but none of them can answer questions effectively from custom data in the way we wanted

Based on the above, we decided to build a quick and minimal prototype to see if we had a viable path forward.

The steps I used in the prototyping phase

The results from the above method were underwhelming-

• Chat agent could not properly use URL’s from the training data to give customers links to certain products

  • The model would modify the URL, treating it as plain text

• Unable to answer simple prompts from the training data

• Model would respond with made up information

• Not consistent in answering similar questions - responses were a hit or a miss

Step 2.1 Rethinking our solution (RAG Model)

The current solution was neither feasible nor provided results as expected. OpenAI had just released a new API for developers to use, so I looked into it. This provided us with a much better alternative to manually fine tuning GPT.

“In a 2020 paper, Meta (then known as Facebook) came up with a framework called retrieval-augmented generation to give LLMs access to information beyond their training data.

RAG allows LLMs to build on a specialized body of knowledge to answer questions in more accurate way.”

-Kim Martineau, IBM

Our new solution in action

Success! Our agent is now able to answer questions, and help customers pick products to address their needs!

After our success deploying the chat agent to 2 of our E-Commerce stores, I built the whole solution as a Shopify app and published it on the App Store for shop owners to use - this would give them access to a custom “Chat-GPT” on their website that could talk to customers and help them make a purchase!