User problem
Learners often know the concept they need but not the exact course title, module name, or vocabulary used in a catalog. Traditional search can return documents; a tutor should help someone form an answer while keeping the path back to trusted material clear.
Knowledge ingestion
Approved enterprise learning sources were prepared for retrieval through parsing, segmentation, metadata assignment, and indexing. Content identity and source metadata were preserved so results could be traced back to the learning platform.
Editable implementation placeholder: Add approved details about chunk size, embedding model, refresh cadence, and document count.
Retrieval
The system used semantic search to identify relevant learning passages for a question. Retrieval quality depended on both the vector representation and practical filters such as content status, audience, product area, and source permissions.
Prompt construction
Retrieved context was assembled with instructions that constrained the model to the supplied learning material, distinguished source content from user input, and established a useful response format. Prompt construction also had to respect context limits and avoid drowning the answer in marginally related passages.
Grounded response generation
The assistant generated an answer from selected context and retained a path to supporting learning sources. When retrieval confidence was low or sources conflicted, the safer behavior was to narrow the answer, clarify uncertainty, or ask the learner for more context.
Privacy considerations
The application was designed for enterprise knowledge. Source access, user context, and telemetry therefore needed purposeful boundaries. Private content was not used as public portfolio material, and this case study omits internal prompts, documents, screenshots, and architecture details.
Evaluation
Quality work combined offline review and product signals. Useful dimensions included retrieval relevance, groundedness, answer usefulness, source coverage, latency, and recurring failure patterns in user feedback.
Editable evaluation placeholder: Add only approved evaluation methods or results, such as a rubric, test-set size, grounded-answer rate, or improvement in successful searches.
Product integration
The production assistant ran on AWS and integrated with learning experiences through application services and APIs. Telemetry and feedback informed iteration on retrieval quality and user experience. Related work included graph data models and metadata management for content recommendations.
Lessons learned
RAG quality is a systems property. Better prompts cannot compensate for stale content, weak metadata, or poor retrieval. The most productive iteration loop connects observed user questions to retrieval diagnostics, content gaps, and product design—not only to model changes.
Technology stack
Python, AWS, retrieval-augmented generation, semantic search, Pinecone, Redis, DocumentDB, PostgreSQL, REST APIs, graph data models, metadata management, and application telemetry.