A Software Engineer's Journey to Perfect Information Retrieval Accuracy in Customer Service
In the fast-paced world of logistics, where every minute counts and accuracy is paramount, our customer service team faced a daily challenge: quickly accessing and delivering precise information about products and territory sales managers to customers. This is the story of how we built Zara, an AI agent that transformed our customer service operations, and the valuable lessons learned along the way.
The Challenge: Speed vs. Accuracy
It started with a seemingly simple requirement: our customer service representatives needed faster access to product information and territory sales manager (TSM) contact details. The manual process of looking up this information was slowing down our response times and creating frustration for both our team and customers. As a software engineer, I knew AI could solve this problem – but I didn't realize how complex the journey would become.
First Attempt: The Quick Solution
Like many engineers, I started with the obvious choice: Google Cloud Platform's suite of AI tools. The initial implementation using Google Chat and Dialogflow showed promise, especially with its ability to extract information from unstructured data like PDFs. Our customer service team was excited – finally, a tool that could quickly pull up product information!
But as with many first attempts, the cracks began to show. While the system worked well with PDFs, it struggled with our structured data stored in Google Sheets and BigQuery tables. We needed something more sophisticated.
The Multi-Agent Approach
Determined to improve, I architected a more complex solution using GCP's Agent Builder. The system consisted of two specialized agents:
- The Product Info Agent: A RAG-powered helper that could quickly retrieve product details from our documentation
- The TSM Info Agent: Designed to provide accurate territory sales manager contact information
This approach seemed perfect on paper. We had separate agents handling different types of queries, all coordinated by a main routing agent. But in the world of customer service, "seemed perfect" isn't good enough.
The Breaking Point
The limitations became clear when dealing with location-based queries. When a customer asked for the TSM information for "Akute" area, our system would return not just Akute's information, but also details for similar-sounding locations. In customer service, this kind of imprecision can lead to serious issues – imagine a customer contacting the wrong sales manager!
I tried to patch the problem by limiting results to the top three matches, but this created a new issue: the system would still provide "best guess" answers even for non-existent locations. In logistics, where accuracy can make or break a business relationship, this was unacceptable.
The Custom Solution
This experience led to a crucial realization: sometimes, to get exactly what you need, you have to build it yourself. I pivoted to developing a custom FastAPI backend that directly interfaced with Google's APIs. This gave us complete control over the information flow and search logic – no more "best guesses" when precision was needed.
Where We Are Today
Today, Zara does more than just answer queries. It sends hourly reports about truck visibility status and has become an integral part of our operations. The journey from off-the-shelf solutions to a custom implementation taught me valuable lessons about AI development:
- Start simple, but be prepared to go custom when precision matters
- Understanding your specific use case is more important than using the latest tools
- In customer service, accuracy should never be sacrificed for speed
Looking Forward
While our current implementation of Zara meets our needs, the journey isn't over. The world of AI is evolving rapidly, and so are our requirements. The next chapter will focus on expanding Zara's capabilities while maintaining the precision we've worked so hard to achieve.
As for other engineers embarking on similar journeys, remember: don't be afraid to abandon the comfortable path of pre-built solutions when they don't meet your specific needs. Sometimes, the best solution is the one you build yourself.
The story of Zara continues, and I'll be sharing more about our custom FastAPI implementation in a future post. Stay tuned!