GenAI Customer Support Agent
Reducing support ticket volume by 60% with a RAG-based conversational AI agent.
Role
Senior PM
Timeline
8 Months
Team
3 AI Engineers, 2 Full Stack, 1 UX Researcher
Context & Problem
The Context
Support costs were scaling linearly with user growth. CSAT scores were dropping due to long wait times.
The Problem
L1 support agents were overwhelmed with repetitive queries. Traditional chatbots were rigid and frustrating for users.
Opportunity & Vision
"A conversational AI that feels like a helpful expert, capable of resolving complex queries and taking action on behalf of the user."
Data & AI Opportunities
Using LLMs (GPT-4) with Retrieval Augmented Generation (RAG) to ground answers in our help center documentation and user data.
System Architecture
Vector database (Pinecone) for knowledge retrieval. LangChain for orchestration. Custom guardrails to prevent hallucinations and ensure brand safety.
Data Ingestion
Kafka / IoT Sensors
AI Processing
TensorFlow / Spark
API Layer
FastAPI / GraphQL
Client App
React / Mobile
Tradeoffs & Prioritization
We decided to use a hybrid approach: AI handles the initial triage and simple resolution, but seamlessly hands off to a human with full context if sentiment turns negative.
Execution & Collaboration
Collaborated closely with Legal and Trust & Safety teams to define boundaries. Ran A/B tests on different prompting strategies.
Metrics & Impact
Deflected 60% of incoming tickets. Improved CSAT by 15 points. Reduced average resolution time from 4 hours to 2 minutes.
Reduction
In unplanned downtime
ARR
Generated in 18 months
Accuracy
On critical failure modes
Lessons Learned
"Latency was a challenge. We optimized by caching common queries and using smaller, fine-tuned models for specific intents."