Projects
RAGInformation RetrievalLLMsEvaluationBiomedical QA

TREC BioGen: Evidence-grounded RAG and LLM evaluation system

Built and evaluated sparse and dense RAG pipelines over 26.8 million PubMed documents, with Qwen-based generation, reranking, and a dashboard for evidence-linked inspection.

2025-09-01 • 1 min read
Live demoCodePaper

Overview

Build a biomedical QA system where each generated claim remains traceable to retrieved evidence and easy to inspect during evaluation. BioEvidence is the productized continuation of this work, and its Examples view is now the live interface for exploring detailed TREC runs.

System design

  • Sparse and dense retrieval over a 26.8 million document PubMed corpus
  • BM25, Pyserini/Lucene, MedCPT, TF-IDF/MMR, FAISS, and cross-encoder reranking
  • Qwen-based generation served on GPU infrastructure through vLLM and Slurm
  • Human and AI feedback loops to refine reformulation, ranking, and answer generation behavior

What was built

  • End-to-end pipeline for TREC BioGen Task-B submissions
  • Citation-linked answer generation with PubMed evidence traces
  • BioEvidence Examples view for comparing detailed runs, inspecting outputs, and debugging evidence quality

Results

  • Contributed to a published TREC 2025 system paper
  • Produced reusable diagnostics for answer accuracy, retrieval quality, evidence grounding, and citation reliability
  • Carried the TREC pipeline into BioEvidence so the system remains inspectable beyond the shared task