TrustMed AI: Building a RAG-Powered Healthcare Chatbot on AWS
Dec 15, 2025 · 15 min read
The Problem: Bridging Clinical Facts and Patient Language
Patients seeking health information on Type II Diabetes and Cardiovascular Disease face a frustrating reality: fragmented, unvetted sources that lack integration with authoritative medical guidance. Current systems fail to understand implicit patient intent, bridge clinical jargon with everyday language, or distinguish between patient experiences and credible medical facts. > [!IMPORTANT] TrustMed AI addresses this through a retrieval-augmented generation system that synthesizes authoritative content while understanding patient language, delivering evidence-based education with explicit source attribution and clinical safety guardrails. This project was built as part of CSE 573: Semantic Web Mining at Arizona State University. The goal was to create a chatbot that doesn't just answer questions, but answers them correctly and safely, with citations to trusted medical sources.
Project Structure
The TrustMed AI codebase is organized into distinct modules for offline data preparation and online chat serving. > [!NOTE] The separation of offline (ingestion) and online (runtime) components is a key design principle. This allows the chatbot to answer fast at runtime since all heavy data processing is done beforehand.
Building the Knowledge Base: Data Ingestion Pipeline (Offline)
A custom knowledge base was built instead of relying on a generic web search. The scope was two disease types: Type 2 Diabetes and Cardiovascular Disease (Myocardial Infarction), with two types of content: clinical references and real patient discussions from public forums. --- Step 1: Data Collection Data was collected from two places: trusted medical sites and Reddit. Medical pages were scraped with an HTML parser that parsed only the main article content. Reddit threads were pulled using the JSON API, with HTML parsing as a fallback. The final dataset includes 350+ clinical articles and 1,000+ forum threads, saved as raw JSON for cleaning. --- Step 2: Cleaning and Normalization Raw scraped data is noisy, so it was cleaned using a cleaning script that: - Converts everything into plain text with a consistent format - Strips page clutter like navigation, buttons, and popups - Normalizes spacing and formatting - Adds a metadata header containing the title, source type (clinical or social), and the original URL During this step, a manifest.json is also generated as a master lookup: --- Step 3: Chunking and Indexing After cleaning, the processed text files are loaded from S3, chunked into small passages, and embedded using Amazon Titan Embeddings v2. Chunk size and overlap are tuned so each chunk keeps enough context. Each chunk is then indexed into OpenSearch with its vector plus metadata like file ID, title, and URL. The OpenSearch index stores a k-NN vector field for semantic search and text fields for BM25 keyword search, enabling hybrid retrieval at runtime with low latency.
System Architecture: Custom RAG Pipeline on AWS
Our system is designed to be modular, giving us granular control over each component. The orchestration logic is centralized on an EC2 instance, which connects with AWS services for storage, search, and AI model inference. --- Orchestration Layer (Amazon EC2 + Python) - Compute Engine: An EC2 instance serves as the central system, running our primary Python application - Core Logic: The application uses LangChain as the framework to structure our RAG pipeline --- Data & Persistence Layer - Raw & Processed Data (Amazon S3): Stores scraped content, normalized .txt files, and our critical metadata file - Vector Store (Amazon OpenSearch): Dedicated cluster configured for high-performance vector search --- AI Model Layer (Amazon Bedrock) - Embedding Model: Amazon Titan Text Embedding v2 for text-to-vector transformations - LLM: Meta's Llama 3 8B Instruct model for generating the final response --- UI (Chainlit) A Chainlit front-end provides the real-time, interactive chat interface with streaming responses and clickable citations.
Answering Questions: Runtime RAG Pipeline (Online)
Here is what happens when someone asks TrustMed AI a question. This is the online RAG flow that runs for every query. It follows four steps: embed, retrieve, generate, and respond. --- 1. Query Embedding The user question is sent to Amazon Titan Text Embeddings v2 through Bedrock. This converts the text into a vector so the system can search by meaning, not just keywords. --- 2. Hybrid Retrieval The system queries OpenSearch using both the vector and the raw query text. OpenSearch runs k-NN vector search for semantic matches and BM25 keyword search for exact terms, then merges results. > [!TIP] This ensures that if a patient uses slang ("sugar crash"), the vector search catches it. If they use a specific drug name, the keyword search catches it. The retriever returns the top chunks with metadata like title, file ID, and URL. --- 3. Prompt and Generation LangChain assembles a prompt by combining the user question with the retrieved chunks. This prompt is sent to Llama 3 8B Instruct on Bedrock. The prompt constraints push the model to answer using only the provided context, reducing hallucinations. --- 4. Citations and Response The model output is post-processed to attach citations. Chunk metadata is mapped through manifest.json to resolve clean titles and canonical URLs, then a Sources section is added. The final response is streamed back to the Chainlit UI.
Evaluation: How Do We Trust the Answers?
In healthcare, accuracy is critical. A hallucinated answer is not just a small error—it can cause real harm. To test reliability, the system was evaluated using TruLens and the RAG Triad metrics. --- The Three Key Metrics: | Metric | Score | Description | |--------|-------|-------------| | Groundedness | 0.877 | Checks whether the answer is supported by the retrieved context. A high score means the model is staying close to evidence and not making up facts. | | Answer Relevance | 0.869 | Measures whether the response directly answers the user's question instead of drifting into generic or unrelated content. | | Context Relevance | 0.764 | Evaluates retrieval quality—how relevant the retrieved chunks are to the query. This score showed the main area to improve, especially because forum data can include noise like side conversations. | --- Key Insights: > [!TIP] The system produces accurate and grounded answers (high Answer Relevance and Groundedness). Retrieval quality (Context Relevance) is the main area to optimize—some retrieved passages may be tangential or incomplete. TruLens computed these scores using embedding-based semantic similarity to compare questions, retrieved context, and generated answers. Along with automated scoring, responses were also manually reviewed to confirm clinical correctness and source quality.
The UI: Chainlit for Conversational AI
We used Chainlit to build our conversational interface. Chainlit is an open-source Python library designed to quickly build chat applications. --- Why Chainlit? - Write backend logic (calling Bedrock) and front-end UI in a single Python script (app.py) - Ideal for RAG demos—it easily displays streaming LLM responses - Native support for formatted, clickable source citations - Real-time message streaming for better user experience > [!NOTE] This setup keeps answers grounded, modular, and easy to improve. Retrieval logic, prompt format, and model choice can be tuned independently.
Conclusion & Key Takeaways
TrustMed AI is not just a Q&A chatbot. It is a dual-source medical knowledge system that combines trusted clinical content with real patient language. Using Amazon Bedrock for secure model access, Llama 3 for fast and instruction-focused generation, and OpenSearch hybrid retrieval for accurate context, the system can stay grounded in evidence while still sounding clear and human. --- Technical Achievements: | Component | Technology | |-----------|------------| | Data Sources | 350+ medical articles + 1,000+ Reddit threads | | Embedding | Amazon Titan Text Embedding v2 | | Vector Store | Amazon OpenSearch with k-NN | | Search | Hybrid (Semantic k-NN + BM25 Keyword) | | LLM | Meta Llama 3 8B Instruct via Bedrock | | Orchestration | LangChain on EC2 | | Evaluation | TruLens (0.87 Answer Relevance, 0.88 Groundedness) | --- Key Lessons Learned: 1. Dual-source datasets work: Combining clinical facts with patient voice creates more empathetic responses 2. Hybrid search beats pure vector search: BM25 keyword matching catches specific medical terms that pure semantic search might miss 3. Metadata is critical: Storing source URLs with chunks enables accurate, clickable citations 4. Evaluation frameworks matter: TruLens helps quantify and improve trust in LLM outputs > [!IMPORTANT] This work highlights what matters most in a medical RAG setup: a strong ingestion pipeline, reliable retrieval, and continuous evaluation. The result is a system that can understand everyday questions, retrieve the right medical context, and respond with citations so users can verify the source. --- > [!NOTE] Project Repository: github.com/shitijkarsolia/trustmed-ai --- Team: Shitij Mathur, Advaith Venkatsubramanian, Suhas Gajula, Thanishka Bolisetty, Varad More, Vishnu Menon