Knowledge base

Iterative AI feature improvements
Nov 2023–Mar 2024

When I joined the team in September 2023, the product was only 3 months old and still in the early stages of hypothesis validation and market fit exploration.

The Knowledge Base is the brain of every conversational AI agent and the central hub where enterprise teams organise, edit, and test the information their agents rely on to respond accurately to callers. As the most critical part of our platform, even small  changes here have a measurable impact on how effectively our AI understands and resolves customer queries.

Role
Senior product designer (before I got the promotion)
Team
Collaborated with a PM, 3 full-stack engineers, a ML engineer ,and a QA engineer
Iterations
Learned, measured, and improved the feature through 3 iterations, each spaced 1–3 months apart
Company
PolyAI builds enterprise-grade voice and chat agents that handle complex customer interactions for global brands.
Product
Agent studio enables teams to design, train, and deploy conversational AI agents through a unified, data-driven workspace.
Users
Our users are IT and CX professionals managing contact centres who rely on PolyAI to scale support and improve service quality.
Problems to solve
Business level
Users
Technical
  • Competitors have more robust platforms that leads to losing deals;
  • The size of the Knowledge base limits restrict agent performance for larger enterprises, constraining growth;
  • Slow or cumbersome knowledge creation increases onboarding time and reduces platform adoption.
  • Creating and updating data manually is time-consuming, especially for users with no technical skills;
  • Lack of hierarchy and endless scrolling reduce clarity and ease of use;
  • Large, unstructured prompt windows make content hard to read and analyse;
  • Searching and visualising each paragraph is inefficient, making maintenance frustrating.
  • Current Knowledge base design hits token limits, restricting content and complexity;
  • No RAG-powered retrieval for scaling the feature content efficiently.
  • Lack of structured information increases latency and prevents production readiness.
First ever iteration of the feature (Sep 2023)
Goals
To overcome these challenges, I set focused goals and success metrics to transform the Knowledge Base into a scalable, user-friendly, and production-ready platform.
Success metrics
In collaboration with Product Management, we defined success metrics that align user impact with business and technical outcomes.
Design exploration
To fight ambiguity, I explored everything we had: competitor benchmarks, data insights, feedback, feature requests, and our vision for the future.
Technical constraints
To ensure the solution was both elegant and feasible, I partnered with engineers early on to understand system limitations before starting the first design drafts.
Early concepts
After understanding the technical constraints and data processing flow, I created early drafts to validate direction and gather feedback from the team.
At the time, the product had no external users and only 4 in-house Dialog designers were using it. They were our primary users until the product became competitive enough to attract clients.
Change of the strategy
Rather than committing to one approach, I expanded the design exploration and reorganised the information architecture to enable a more scalable and user-friendly solution based on the information potential clients was planning to use.
New scope
The project grew beyond a single page, resulting in the design of three separate pages that improved structure and ease of use.
Design decisions
Had around 20 different versions of the layout to include and explore all future fetures and improvements.
Design artefacts
I collaborated closely with engineers to ensure design accuracy and system alignment. Every interaction state, error message, and visual detail was defined in Figma, using reusable components from the design system. This preparation reduced handoff friction, improved design-to-build consistency, and accelerated release timelines.
Final design
After several iterations and close collaboration with engineering, the final design balanced user needs with technical feasibility, setting a strong foundation for future expansion.
Outcome and impact
While early metrics and feedback confirmed strong value and usability improvements, I see this as a foundation for ongoing refinement.

Looking back, I’d involve end users and engineering leads even earlier to prototype next-phase ideas in parallel, ensuring smoother scaling later. This project reinforced the value of thinking beyond the MVP, designing with iteration in mind, not completion.
Next steps...
This release represented the first iteration of the feature that is focused on validating the core interaction model, performance, and scalability for enterprise use. Based on early adoption metrics and qualitative feedback, the next phase will focus on deepening functionality and expanding system maturity across teams.