RAGInformation RetrievalLLMsEvaluationBiomedical QA

TREC BioGen: Evidence-grounded RAG and LLM evaluation system

Built and evaluated sparse and dense RAG pipelines over 26.8 million PubMed documents, with Qwen-based generation, reranking, and a dashboard for evidence-linked inspection.

2025-09-011 min read

Overview

Build a biomedical QA system where each generated claim remains traceable to retrieved evidence and easy to inspect during evaluation.

System design

  • Sparse and dense retrieval over a 26.8 million document PubMed corpus
  • BM25, Pyserini/Lucene, MedCPT, TF-IDF/MMR, FAISS, and cross-encoder reranking
  • Qwen-based generation served on GPU infrastructure through vLLM and Slurm
  • Human and AI feedback loops to refine reformulation, ranking, and answer generation behavior

What was built

  • End-to-end pipeline for TREC BioGen Task-B submissions
  • Citation-linked answer generation with PubMed evidence traces
  • Interactive dashboard for comparing runs, inspecting outputs, and debugging evidence quality

Results

  • Contributed to a published TREC 2025 system paper
  • Produced reusable diagnostics for answer accuracy, retrieval quality, evidence grounding, and citation reliability
  • Made system behavior easier to inspect for both manual review and shared-task iteration