LIGHT logo
Building intelligent Global Health and Humanitarian Response Technologies (BiGHT)

Project Briefs

Overview

This page summarizes the proposed course projects. Use the links below to jump to each full project brief.

Before preparing Milestone 1, read your assigned project brief carefully and align your technical design with its proof-of-concept expectations.

Project numberProject titleTeam sizeShort description
Project 1Offline Speech-To-Speech System To Support Humanitarian Triage Communication3Offline speech-to-speech system to support communication between humanitarian health staff in Yemen and migrants during staff-led triage.
Project 2Interactive Public Health Messenger3Build an interactive public health messenger that helps people understand crisis instructions, checks whether guidance was understood, and aggregates misunderstanding patterns for responders.
Project 3Geospatial Intelligence For Public Health Decisions3The main aim of the project is to provide reliable Geospatial Intelligence for Public Health decisions.
Project 4AI-Powered Disease Risk Prediction System For Zoonotic Disease OutbreaksWHO logo3The aim of this project is to build an AI-powered disease risk prediction system that integrates human, animal, and environmental data to identify and visualize areas at high risk of zoonotic disease outbreaks.
Project 5AI-Powered Medical Assistant For Health Workers In Rwanda3Build an AI-powered medical assistant for health workers in Rwanda, with a focus on adapting medical AI to support Kinyarwanda, English, and French.
Project 6AI-Powered Dengue Early-Warning Platform For Public Health Decision-Makers3Build an AI-powered dengue early-warning platform that predicts upcoming disease incidence and turns these forecasts into interpretable information that public health decision-makers can explore and use.
Project 7WHO Skin AI For Offline DermatologyWHO logo3Build an offline mobile application that can be used to reliably detect skin disease in the Global South.
Project 8Conversational AI For Frontline Health WorkersWHO logo3Build a chatbot that draws from clinical and national health guidelines in answering questions posed by frontline healthcare workers.
Project 9Bayesian Conversational System For Ebola Virus3Curate and document the lab's Ebola dataset, then use it as the empirical knowledge base for a MoBayes style modular Bayesian dialogue system for Ebola triage.
Project 10Medical Personas And Expert Routing For Clinical QA3Investigate whether specialist medical personas can improve the accuracy and consistency of large language models on clinical question-answering tasks.

The proof-of-concept expectations of your project are binding for Milestone 2.

Project 1: Offline Speech-To-Speech System To Support Humanitarian Triage Communication

Proposed team size: 3 students.

Project lead: Xavier and David.

Partner: Benita Rowe and Fabrice Lauper.

Keywords: natural language processing, conversational AI.

Short Description

Offline speech-to-speech system to support communication between humanitarian health staff in Yemen and migrants during staff-led triage.

Motivation

This project explores the development of a small, open-source, offline speech-to-speech system to support communication during a staff-led initial medical triage exchange in low-resource settings. It is motivated by a real-world use case in Yemen, where IOM health teams serve newly arrived migrants from the Horn of Africa at migrant response points, on the coast, and through mobile medical teams. Arabic-speaking medical staff may need to communicate with migrants who speak Amharic, Oromo, Somali, or Tigrinya, and where reliable internet access or interpreters may not be available.

Students will investigate and compare open-source components for automatic speech recognition, machine translation, and speech synthesis. They will assess how effectively these components can be combined to provide two-way spoken translation between Arabic and the target languages, with Oromo as the highest priority.

The project will also explore interactive and collaborative translation workflows. These may include using an LLM to discuss or refine possible translations, mechanisms for users or conversation partners to verify and correct translations, and forward and backward translation as a possible sanity check. Students may also investigate recording and multi-turn interaction workflows, for example allowing utterances to be revisited, reviewed by another person, or refined across a conversation.

The project will focus particularly on what can run fully offline on low-cost, resource-constrained hardware. The goal is not to produce a deployment-ready clinical application, but to establish what is currently technically possible, identify promising models and architectures, explore effective human-in-the-loop interaction patterns, and determine the key limitations that would need to be addressed before such a system could eventually be deployed on ordinary Android phones in Yemen or similar low-resource settings.

The staff member is responsible for the questions and for all triage and referral decisions. The system only translates the exchange: it must not independently question or route people, interpret symptoms, identify warning signs, recommend actions, or make clinical decisions.

Intended Users

Potential users include:

  • humanitarian health staff conducting initial medical triage with migrants in Yemen;
  • Arabic-speaking medical staff communicating with patients who speak Amharic, Oromo, Somali, or Tigrinya;
  • patients who need to describe symptoms, history, or concerns across a language barrier;
  • field teams working without reliable internet access or interpreters;
  • technical and humanitarian teams assessing offline translation feasibility for low-resource settings.

Possible Features

Possible features include:

  • offline speech input and recording;
  • automatic speech recognition for Arabic and one or more target languages;
  • machine translation between Arabic and Amharic, Oromo, Somali, or Tigrinya;
  • speech synthesis for translated outputs;
  • two-way spoken translation workflow;
  • forward and backward translation as a sanity check;
  • LLM-assisted discussion or refinement of possible translations when feasible;
  • mechanisms for users or conversation partners to verify, reject, or correct translations;
  • multi-turn conversation history;
  • delayed review of recorded utterances by another person;
  • lightweight interface designed to support triage communication in constrained settings.

The team should not attempt all features. The technical design should identify a core triage communication workflow, prioritize Oromo where feasible, and define a realistic proof-of-concept path.

Design Questions

Consider:

  • Which language pair should be prioritized for the proof of concept, and why?
  • What can realistically run fully offline on low-cost, resource-constrained hardware?
  • How accurate are available open-source ASR, translation, and speech synthesis components for Arabic, Oromo, Amharic, Somali, and Tigrinya?
  • How should the interface communicate uncertainty, alternative translations, and possible errors?
  • How can a patient or conversation partner verify or correct a translation without speaking the staff member's language?
  • When is backtranslation useful, and when might it create false confidence?
  • How should recordings, conversation history, and reviewed utterances be stored, deleted, or protected?
  • How can the interface make clear that the system only translates, and that all questions and triage decisions remain with the staff member?

Technical Directions

Possible technical components include:

  • comparison of open-source ASR models;
  • comparison of open-source machine translation models;
  • comparison of open-source speech synthesis options;
  • offline inference pipeline design;
  • model-size, latency, storage, and hardware-feasibility analysis;
  • Arabic-to-target-language and target-language-to-Arabic translation path;
  • LLM-assisted translation refinement where feasible, with clear limits if offline LLM use is not realistic;
  • user interface for recording, playback, translation display, correction, and conversation history;
  • scripted medical triage scenarios for evaluation;
  • bilingual or expert review of translation outputs.

The system should be designed so that the core path can run reproducibly in the project repository. The project should make clear which components run offline, which require optional connectivity, and which remain simulated or experimental.

Proof-Of-Concept Expectation

The proof of concept should demonstrate one thin vertical slice through an offline triage translation workflow.

Minimum convincing POC:

  • the user records or enters one triage-relevant utterance;
  • the system processes the utterance through an ASR, translation, and output path, or clearly documents which speech components are simulated;
  • the system supports at least one Arabic-to-target-language or target-language-to-Arabic translation path, with Oromo prioritized where feasible;
  • the translated result is displayed and, if feasible, spoken back to the user;
  • the interface provides at least one mechanism for verification or refinement, such as backtranslation, alternative translations, partner confirmation, correction, or LLM-assisted discussion;
  • the demo uses at least one scripted medical triage scenario;
  • the system clearly identifies what runs offline, what is simulated, and what requires optional connectivity.

The POC may use one prioritized language pair, a small model, sample audio, or scripted utterances. It should not be only a static UI mock-up or a generic text translator without speech, triage context, or offline-feasibility analysis.

Evaluation Ideas

Possible evaluation approaches include:

  • comparing ASR outputs against reference transcripts for scripted triage utterances;
  • comparing translation outputs against bilingual reference translations or bilingual review;
  • measuring whether backtranslation or partner verification catches obvious translation failures;
  • assessing latency, model size, storage requirements, and offline runtime feasibility;
  • testing the workflow on low-cost or resource-constrained hardware where possible;
  • evaluating whether the interface makes uncertainty, alternatives, verification, and corrections understandable;
  • analyzing privacy and data-retention risks for recordings and conversation history;
  • documenting language coverage gaps and points of failure across Arabic, Oromo, Amharic, Somali, and Tigrinya.

Final Demo Target

The final demo should show a short medical triage exchange in which the system records or receives speech, translates between Arabic and a target language, presents or speaks the translated output, and supports verification or correction. The demo should also explain what ran offline, what hardware or resource constraints were tested, which models and architectures were most promising, and what limitations must be solved before deployment on ordinary Android phones in Yemen or similar low-resource settings.

Data And Resources

Internal IOM scripts and materials cannot be shared during the semester, so scripted scenarios should be built from public medical and humanitarian materials.

Public speech datasets for initial work on speech recognition, model compatibility, and the offline pipeline:

These are mostly general read speech rather than medical conversations.

Background reading on the context:

Project 2: Interactive Public Health Messenger

Proposed team size: 3 students.

Project lead: Jeremy.

Keywords: LLM, conversational AI, retrieval-augmented generation, public health communication, understanding-aware communication, UI/UX.

Short Description

Build an interactive public health messenger that helps people understand crisis instructions, checks whether guidance was understood, and aggregates misunderstanding patterns for responders.

Motivation

During a disaster or public health crisis, authorities broadcast critical instructions about water safety, evacuation, disease prevention, or other protective actions. But people may misunderstand, reinterpret, or partially remember what they hear. And in this context, small misunderstandings may lead to serious consequences. Thus, the core challenge is not just sending information, but ensuring it is actually understood.

This project turns dissemination into an interactive loop. Instead of one-way communication, users can voluntarily engage with a low-connectivity-friendly chatbot through voice or text to ask questions, clarify doubts, or hear the latest guidance. The system should answer using vetted information, for example through retrieval-augmented generation over trusted sources, while also assessing whether the user truly understood the answer.

For example, after explaining something, the system might ask: "Just to check: What would you do in this situation?" The goal is not to test or shame the use, but rather to reveal whether the message was understood well enough to support safe action.

The most interesting part is what happens across many users. Rather than only monitoring passively, the system should aggregate patterns of misunderstanding. If users misinterpret the same message in the same way, that signals a problem with how the guidance is phrased or delivered. The system can then suggest alternative formulations or highlight risky misconceptions to responders.

Intended Users

Potential users include:

  • people affected by a disaster, outbreak, or crisis;
  • public health responders communicating urgent guidance;
  • emergency communication teams;
  • local authorities responsible for public messaging;
  • humanitarian organizations monitoring whether guidance is understood;
  • community engagement teams improving message clarity.

Possible Features

Possible features include:

  • interactive question answering grounded in trusted public health or crisis guidance;
  • text or voice interface designed for low-connectivity settings;
  • retrieval-augmented generation over vetted documents;
  • lightweight understanding checks, such as explain-back or scenario-based questions;
  • detection of partial, repeated, or dangerous misunderstandings;
  • aggregation of misunderstanding patterns across conversations;
  • responder dashboard highlighting confusing guidance and risky misconceptions;
  • suggestions for clearer alternative formulations;
  • participatory feedback where users compare phrasings or mark responses as clear or confusing;
  • privacy-aware logging and aggregation.

The team should choose a focused crisis or public health scenario and avoid building a generic chatbot without a clear understanding loop.

Design Questions

Consider:

  • How do you ask someone to demonstrate understanding without sounding like a test?
  • How do you design prompts that reveal real comprehension instead of encouraging users to repeat what they just heard?
  • What makes an answer safely grounded in trusted sources?
  • What counts as a partial or dangerous misunderstanding?
  • How should misunderstanding patterns be aggregated without exposing individual users?
  • How should a responder dashboard flag guidance that is often misunderstood?
  • How can users help refine explanations, compare phrasings, or signal which responses felt clear or confusing?
  • What should happen when trusted guidance is ambiguous, missing, or internally inconsistent?
  • What should work in low-connectivity settings?

Technical Directions

Possible technical components include:

  • retrieval-augmented generation over a small trusted document collection;
  • prompt design for public health explanations and understanding checks;
  • voice or text-based conversational interface;
  • comprehension-check response capture;
  • classification, clustering, or rule-based detection of misunderstanding patterns;
  • responder dashboard showing frequently misunderstood guidance;
  • alternative phrasing generation or ranking;
  • low-connectivity-friendly UI/UX design;
  • privacy-aware logging and aggregation of interaction signals.

The system should make the source of guidance visible and should distinguish between trusted content, generated explanation, user understanding signal, and aggregate responder insight.

Proof-Of-Concept Expectation

The proof of concept should demonstrate one complete understanding-aware communication loop.

Minimum convincing POC:

  • the user submits one public health or crisis-related question through text or voice;
  • the system retrieves at least one relevant passage from a trusted source;
  • the system generates or displays an answer grounded in that source;
  • the system asks one lightweight understanding check, such as an explain-back or scenario-based question;
  • the user response is captured as a comprehension or misunderstanding signal;
  • the system identifies at least one partial or dangerous misunderstanding pattern from one or more example interactions;
  • at least one aggregate signal is displayed, logged, or shown in a simple responder dashboard;
  • the system clearly identifies what is retrieved, what is generated, and what is inferred from the user's response.

The POC may use a small trusted document set, scripted example conversations, and a simple dashboard. It should not be only a chatbot that answers questions without checking understanding, and it should not be only a dashboard without an interactive communication loop.

Evaluation Ideas

Possible evaluation approaches include:

  • checking whether generated answers are supported by retrieved trusted passages;
  • testing understanding-check prompts on scripted or example user responses;
  • measuring whether known misconceptions are detected;
  • evaluating whether the dashboard highlights recurring misunderstandings;
  • comparing alternative phrasings for clarity;
  • assessing whether explain-back prompts reveal comprehension rather than memorization;
  • reviewing privacy risks in interaction logs and aggregate reporting;
  • qualitative review of UI/UX for users and responders.

Final Demo Target

The final demo should show a user asking for crisis or public health guidance, receiving an answer grounded in trusted information, responding to an understanding check, and the responder view updating with a meaningful comprehension or misunderstanding signal. The demo should also show how repeated misunderstanding patterns could lead responders to revise guidance or flag risky misconceptions.

Project 3: Geospatial Intelligence For Public Health Decisions

Proposed team size: 3 students.

Project lead: David.

Keywords: geospatial intelligence.

Short Description

The main aim of the project is to provide reliable Geospatial Intelligence for Public Health decisions.

Motivation

Public health decisions are usually spatial-related decisions. Public health agencies need to know which services are missing in particular areas, where vulnerable populations might exist, and what environmental or infrastructural risks to look out for in particular areas.

Due to the recent popularity of LLMs, many organizations turn to them today to assist with decision-making and planning. The issue is that LLMs are notoriously unreliable for this use case because they do not reason well across maps and other forms of visual spatial data.

The aim of this project is therefore to build a working prototype of a geospatial intelligence system that can assist with public health decision-making. Instead of relying solely on LLMs to generate answers using their internal knowledge, the system should connect them to open-source geospatial analysis tools so that public health questions can be answered with structured and verifiable evidence.

The prototype should include a user interface that allows users to submit public health questions, inspect the resulting geospatial analysis and recommendations on a map, and review the evidence used to support them. Students should also systematically evaluate the reliability of the system on representative public health decision-making tasks, including the correctness of the geospatial analysis and the quality and verifiability of the resulting recommendations.

Intended Users

Potential users include:

  • public health agencies;
  • humanitarian coordination teams;
  • responders planning service deployment;
  • analysts without deep GIS expertise;
  • decision-makers who need spatial evidence quickly;
  • organizations using LLMs for public health planning but needing more reliable geospatial reasoning.

Example Query

An example query could be:

Where is the best place to deploy a mobile clinic in Freetown within Sierra Leone to best tackle the current increase in Ebola cases?

The system should find, leverage, and organize relevant open-source geospatial data so that it can be analyzed with geospatial tools and, where appropriate, summarized with an LLM to produce a specific and actionable response.

Possible Features

Possible features include:

  • natural-language public health query input;
  • user interface for asking questions and inspecting recommendations;
  • geospatial data retrieval, loading, and organization;
  • connection to open-source geospatial analysis tools;
  • map display of data layers, analysis outputs, and recommendations;
  • structured evidence table supporting each recommendation;
  • explanation of assumptions, data sources, and spatial operations;
  • LLM-generated summary grounded in geospatial outputs;
  • uncertainty, missing-data, or low-quality-data warnings;
  • representative task set for reliability evaluation.

The team should avoid building a generic map chatbot. The central goal is to connect user questions to verifiable geospatial analysis and evidence-backed recommendations.

Design Questions

Consider:

  • What public health decision-making tasks should the system support?
  • What spatial operation is needed to answer each task?
  • What open-source data is required, and is it available for the target region?
  • What should the LLM do, and what should deterministic geospatial tools do?
  • How can users inspect the map, evidence, assumptions, and recommendation path?
  • How can the system make its evidence verifiable?
  • How should missing, stale, low-resolution, or conflicting geospatial data be communicated?
  • What makes a recommendation specific and actionable for public health planning?
  • How will the team evaluate whether the geospatial analysis is correct?

Technical Directions

Possible technical components include:

  • public health task and query design;
  • open-source geospatial data loading and preprocessing;
  • spatial joins, buffers, distance calculations, accessibility analysis, or service-area analysis;
  • map visualization and layer inspection;
  • backend orchestration between query handling, geospatial tools, and LLM summarization;
  • structured evidence generation;
  • reproducible query examples;
  • reliability evaluation against representative public health tasks;
  • checks that LLM-generated recommendations match computed geospatial evidence.

The system should clearly separate geospatial computation, structured evidence, and LLM-generated explanation. A reviewer should be able to trace a recommendation back to the data sources and spatial operations that support it.

Proof-Of-Concept Expectation

The proof of concept should demonstrate one evidence-backed geospatial public health query.

Minimum convincing POC:

  • the user submits one predefined public health decision-making query through a basic interface;
  • the system loads or retrieves at least one relevant open-source geospatial dataset;
  • the system runs at least one verifiable spatial operation;
  • the result is shown on a map or in a structured spatial output;
  • the system produces a recommendation that cites or displays the evidence used;
  • the user can inspect the map, analysis output, and supporting evidence;
  • the system clearly separates geospatial computation from LLM-generated explanation;
  • the team evaluates the correctness of the geospatial operation on at least one representative task.

The POC may use a simplified region, a small dataset, or a predefined query. It should not be only an LLM response to a map-related question, and it should not be only a map visualization without a decision-support recommendation.

Evaluation Ideas

Possible evaluation approaches include:

  • checking whether the spatial operation is appropriate for each representative task;
  • validating geospatial outputs against known examples, manual GIS analysis, or expert review;
  • inspecting whether LLM-generated summaries and recommendations match the computed evidence;
  • testing sensitivity to missing, noisy, stale, or low-resolution data;
  • measuring reproducibility of query execution;
  • reviewing whether recommendations are specific, actionable, and evidence-backed;
  • evaluating whether users can inspect the map and evidence used to support a recommendation;
  • comparing the tool-supported answer with a generic LLM answer for the same public health question.

Final Demo Target

The final demo should show a user submitting a public health question, the system loading or organizing relevant open-source geospatial data, running spatial analysis, displaying the results and recommendation on a map, and exposing the structured evidence used to support the answer. The demo should make clear why the recommendation is more reliable and verifiable than an unsupported LLM response.

Project 4: AI-Powered Disease Risk Prediction System For Zoonotic Disease Outbreaks

Proposed team size: 3 students.

Project lead: David.

WHO logo

Keywords: machine learning, disease risk prediction, geospatial intelligence, explainable AI.

Short Description

The aim of this project is to build an AI-powered disease risk prediction system that integrates human, animal, and environmental data to identify and visualize areas at high risk of zoonotic disease outbreaks.

Motivation

Emerging infectious diseases are increasingly driven by interactions between humans, animals, and the environment. However, data from these three domains are often collected in isolation, making it difficult for public health agencies to detect emerging threats early enough to respond effectively.

Existing surveillance systems also require significant technical expertise in GIS and statistical modelling, limiting their accessibility and widespread use.

This WHO project for AI4Risk aims to develop a working prototype of an AI-powered risk analysis platform that integrates heterogeneous datasets from human health, animal health, and environmental sources to predict and visualize zoonotic disease risks. Students will design a pipeline that collects and harmonizes multi-source data, engineers relevant spatial and temporal features, trains machine learning models for disease risk prediction, and generates interactive geospatial risk maps.

The system should also provide interpretable explanations of its predictions and enable users to explore the factors contributing to disease risk in different locations. Students will systematically evaluate the predictive performance, robustness, and calibration of the models using appropriate spatial and temporal validation strategies and compare them against suitable baselines.

By the end of the project, the prototype should demonstrate how AI can support early warning, hotspot detection, and evidence-based decision-making for epidemic preparedness and public health planning.

Intended Users

Potential users include:

  • public health agencies;
  • epidemic preparedness teams;
  • zoonotic disease surveillance teams;
  • environmental health analysts;
  • WHO and AI4Risk stakeholders;
  • decision-makers planning interventions or resource allocation.

Possible Features

Possible features include:

  • data ingestion from human health, animal health, and environmental sources;
  • harmonization of heterogeneous spatial and temporal datasets;
  • spatial and temporal feature engineering;
  • baseline disease risk prediction model;
  • comparison of multiple forecasting or risk-scoring approaches;
  • interactive geospatial risk maps;
  • hotspot detection;
  • interpretable explanation of risk factors;
  • exploration of factors contributing to disease risk in different locations;
  • uncertainty, robustness, or calibration views;
  • evidence export or summary for decision-makers.

The team should avoid building only a static map or only a model notebook. The central goal is an integrated risk analysis platform that connects data, prediction, explanation, and decision-support visualization.

Design Questions

Consider:

  • What is the prediction target, and what does a high-risk area mean?
  • What spatial unit, temporal unit, and forecast horizon define a prediction?
  • Which human, animal, and environmental data sources are available and compatible?
  • How will the split avoid temporal and spatial leakage?
  • What baselines are suitable for disease risk prediction?
  • How should uncertainty, robustness, and calibration be communicated to public health users?
  • What explanations are useful for understanding disease risk in different locations?
  • How should the system support early warning, hotspot detection, and evidence-based planning?
  • What limitations arise from missing, biased, sparse, or misaligned data?

Technical Directions

Possible technical components include:

  • multi-source data collection and documentation;
  • geospatial and temporal data harmonization;
  • spatial and temporal feature engineering;
  • baseline machine learning model for disease risk prediction;
  • model comparison against suitable baselines;
  • spatial and temporal validation strategy;
  • robustness and calibration evaluation;
  • explainability method for risk factors;
  • interactive geospatial risk map;
  • dashboard or interface for exploring risk, predictions, explanations, and uncertainty;
  • error analysis by geography, time, and outbreak context.

The system should make clear how data sources, features, model predictions, explanations, and map outputs are connected. A reviewer should be able to trace a displayed high-risk area back to the data and model evidence supporting it.

Proof-Of-Concept Expectation

The proof of concept should demonstrate one valid zoonotic disease risk-prediction path.

Minimum convincing POC:

  • the system loads a small multi-source dataset or a documented representative sample from human, animal, and environmental sources;
  • the team defines a clear prediction target, spatial unit, temporal unit, and forecast horizon;
  • the system engineers at least a small set of spatial and temporal features;
  • the system trains or runs at least one suitable baseline model;
  • the system generates disease risk predictions or scores;
  • the predictions are displayed on a map or spatial visualization;
  • the system provides at least one explanation signal for the prediction;
  • the evaluation uses a spatial and temporal validation strategy that is appropriate for the target and avoids leakage;
  • the demo includes at least one current result on predictive performance, robustness, or calibration.

The POC may use a limited region, simplified dataset, or reduced feature set. It should not be only a map visualization without a prediction target and baseline, and it should not report model performance without a valid spatial or temporal validation strategy.

Evaluation Ideas

Possible evaluation approaches include:

  • comparing against a simple spatial, temporal, seasonal, or persistence baseline;
  • using temporal validation, spatial validation, or combined spatial-temporal backtesting;
  • measuring predictive performance with justified metrics;
  • evaluating calibration of predicted risks;
  • testing robustness to missing, noisy, sparse, or misaligned data;
  • evaluating performance across regions, time periods, and outbreak contexts;
  • analyzing false positives and false negatives;
  • testing whether explanations align with known risk factors or plausible domain assumptions;
  • reviewing whether the map and dashboard support evidence-based decision-making.

Final Demo Target

The final demo should show multi-source data ingestion, harmonization, feature engineering, disease risk prediction, hotspot visualization, explanation of selected high-risk areas, and evaluation results. It should demonstrate how the prototype could support early warning, hotspot detection, and evidence-based decision-making for epidemic preparedness and public health planning, while clearly communicating limitations around data quality, robustness, calibration, and deployment.

Project 5: AI-Powered Medical Assistant For Health Workers In Rwanda

Proposed team size: 3 students.

Project lead: Xavier.

Keywords: machine learning, multilingual AI, foundation models, LLM, global health, NLP.

Short Description

Build an AI-powered medical assistant for health workers in Rwanda, with a focus on adapting medical AI to support Kinyarwanda, English, and French.

Motivation

Health workers, triage nurses, and clinicians in Rwanda regularly navigate clinical protocols across Kinyarwanda, English, French, and mixed-language medical terminology. Code-switching between a local language and English clinical terms is standard, yet general-purpose clinical models are rarely evaluated in this low-resource setting.

The core deliverable is a text-based, multi-turn conversational interface where health workers submit clinical queries in Kinyarwanda, English, or French to a specialized medical LLM. The MVP focuses on a text-based medical chatbot supporting Kinyarwanda, with speech interaction (ASR/TTS) and retrieval-augmented generation (RAG) over Rwandan clinical guidelines serving as optional stretch goals.

Candidate corpus: https://huggingface.co/datasets/EPFLiGHT/fully-open-meditron

Reference: https://arxiv.org/abs/2605.16215

Intended Users

Potential users include:

  • Health workers, clinicians, and community triage nurses in Rwanda consulting protocols or seeking rapid guidance in local and mixed languages.
  • Supervisors and researchers evaluating multilingual adaptation, cross-lingual transfer, and clinical safety tools for low-resource languages.

Possible Features

Possible features include:

  • multi-turn conversational interface for clinical question answering in Kinyarwanda, English, and French;
  • handling of medical code-switching within Kinyarwanda text;
  • preservation of standardized English clinical terminology in generated answers;
  • clinical safety guardrails and refusal behavior for emergency triage or unsupported diagnoses;
  • uncertainty cues and explicit disclaimers in the interface;
  • datasets and evaluation examples in Kinyarwanda and French;
  • lightweight interface suitable for low-bandwidth clinic settings;
  • speech-to-text and text-to-speech interaction for spoken clinical queries (stretch goal);
  • retrieval-augmented generation over Rwandan clinical guidelines and national health protocols (stretch goal).

The team should define which language paths are core for the final system and which are stretch goals.

Design Questions

Consider:

  • How does direct fine-tuning of a clinical LLM compare against a translation-mediated pipeline built on NLLB-200 or Gemma-4 in terms of medical accuracy, translation fidelity, latency, and operational cost?
  • How can the system preserve critical clinical terminology such as drug names, dosages, and physiological terms when generating or translating Kinyarwanda?
  • What datasets are available for Kinyarwanda and French medical adaptation, and what has to be created or translated?
  • What mechanisms best detect out-of-scope, emergency, or high-risk clinical queries to trigger refusal and escalation?
  • How should uncertainty and confidence be surfaced to health workers to avoid over-reliance on generated answers?
  • How will medical accuracy and language accuracy be evaluated separately?
  • If speech is attempted, what are the latency and accuracy trade-offs of streaming versus batch ASR and TTS for low-resource languages in low-bandwidth clinics?

Technical Directions

Possible technical components include:

  • parameter-efficient fine-tuning of an open medical foundation model such as EPFLiGHT/Apertus-8B-MeditronFO using LoRA or QLoRA;
  • instruction tuning on curated or translated parallel Kinyarwanda clinical question-answering data;
  • a translation-mediated pipeline that translates input into English, prompts the medical LLM, and translates the answer back into Kinyarwanda, using a translation model such as NLLB-200 or Gemma-4;
  • explicit constraints to keep technical medical vocabulary intact across translation steps;
  • dataset curation for Kinyarwanda and French medical examples;
  • confidence scoring, uncertainty estimation, and structured refusal logic;
  • conversation state management for multi-turn interaction;
  • evaluation of medical correctness and language quality;
  • error analysis on mixed-language inputs and medical terminology;
  • lightweight ASR such as a fine-tuned Whisper model and TTS for Kinyarwanda speech (stretch goal).

The system should make clear which language paths are fully implemented, which are partial, and which are simulated. It should also distinguish model adaptation work from any optional retrieval or speech component.

Proof-Of-Concept Expectation

The proof of concept should demonstrate one complete multilingual medical-assistant path in text.

Minimum convincing POC:

  • the user submits a medical question in Kinyarwanda through a command-line or minimal web interface;
  • the question passes through the language, model, and interface pipeline;
  • the system displays a medical assistant response and supports at least minimal multi-turn context;
  • the team demonstrates both baseline approaches, a translation-mediated pipeline and an initial fine-tuned medical model checkpoint;
  • the team reports an initial qualitative and quantitative comparison of the two approaches on a representative subset of clinical questions;
  • the system refuses at least one high-risk emergency query and shows clinical disclaimers;
  • the team documents a data collection, curation, and validation plan for Kinyarwanda clinical evaluation data;
  • the team clearly identifies which languages and components are fully working, partial, or simulated.

The POC may start with one primary language path. The adaptation work itself should be visible in the repository, so that the answers can be traced back to a documented translation or fine-tuning step.

Evaluation Ideas

Possible evaluation approaches include:

  • testing a small set of clinician-reviewed medical questions across Kinyarwanda, English, and French for factual correctness and hallucination rate;
  • evaluating medical correctness of answers using clinician or expert review where possible;
  • measuring language quality, fluency, and meaning preservation in Kinyarwanda with human ratings or reference-based metrics such as BLEU, chrF, or COMET;
  • benchmarking correct retention versus mistranslation of essential clinical terms such as medication names, units, and diagnostic terms;
  • comparing direct fine-tuning against the translation-mediated pipeline built on NLLB-200 or Gemma-4 on accuracy, inference latency, memory footprint, and compute requirements;
  • testing refusal behavior and graceful degradation on out-of-scope, emergency, or adversarial medical prompts;
  • measuring latency for the text path, and for the voice path if attempted;
  • error analysis on mixed-language inputs, medical terminology, and speech transcripts.

Final Demo Target

The final demo should show a health worker holding a multi-turn conversation with the medical assistant in Kinyarwanda through the chatbot interface. The demo should also show evidence of model adaptation, separate evaluation of language and clinical quality, visible safety and limitation cues, and, if attempted as a stretch goal, working speech-to-text and text-to-speech interaction.

Project 6: AI-Powered Dengue Early-Warning Platform For Public Health Decision-Makers

Proposed team size: 3 students.

Project lead: Fabrice.

Keywords: machine learning, time series forecasting, public health, data visualization, full-stack development, explainable AI.

Short Description

Build an AI-powered dengue early-warning platform that predicts upcoming disease incidence and turns these forecasts into interpretable information that public health decision-makers can explore and use.

Motivation

Dengue outbreaks can place substantial pressure on health systems, and anticipating increases in cases can help public health agencies prepare resources and interventions.

Students will build a complete disease forecasting and decision-support platform around the DrivenData DengAI: Predicting Disease Spread challenge. The competition provides a well-defined machine-learning forecasting task and benchmark, while students will develop rigorous temporal validation, compare forecasting approaches, and integrate the resulting models into a complete decision-support system.

Beyond the competition, students will turn their model into a usable application: a backend serving forecasts and an interactive dashboard where users can explore historical incidence, environmental conditions, predicted cases, uncertainty, and explanations of the factors influencing predictions.

The final prototype should demonstrate the complete pipeline from reproducible data processing and ML experimentation to an interpretable forecasting service that could support public-health decision-making.

Challenge link: https://www.drivendata.org/competitions/44/dengai-predicting-disease-spread/

Intended Users

Potential users include:

  • public health decision-makers;
  • epidemiological surveillance teams;
  • outbreak preparedness teams;
  • analysts comparing dengue risk across time and locations;
  • health agencies planning resources or interventions.

Possible Features

Possible features include:

  • reproducible DengAI data ingestion and preprocessing;
  • comparison between baseline forecasting models and your models;
  • API endpoint serving forecasts;
  • interactive dashboard for historical incidence and forecasts;
  • visualization of environmental conditions;
  • uncertainty visualization;
  • explanation of factors influencing predictions;
  • forecast export or summary for decision-makers.

Design Questions

Consider:

  • What forecast horizon is most useful for a public health decision-maker?
  • Which baseline is meaningful for dengue incidence forecasting?
  • How should uncertainty be represented to non-technical users?
  • What environmental or seasonal factors appear to influence predictions?
  • How should the system distinguish historical observations from forecasts?
  • What action could a decision-maker take based on the dashboard?
  • What concrete decision could a public-health decision-maker make differently because of the forecast, and what information or uncertainty would they need before acting?

Data note: Beware, pay attention to the DengAI data-use and competition terms and follow them. Competition data should not be committed to the course repository, the repository should instead document how authorized users obtain the data (and the rest of the code can still assume it was downloaded in the right place).

Technical Directions

Possible technical components include:

  • DrivenData DengAI dataset processing;
  • time series feature engineering;
  • baseline forecasting models;
  • machine learning models for incidence prediction;
  • temporal cross-validation or backtesting;
  • API forecast service;
  • interactive dashboard;
  • visualization of historical cases, predicted cases, uncertainty, and drivers.

Proof-Of-Concept Expectation

The proof of concept should demonstrate one valid dengue forecasting and decision-support path.

Minimum convincing POC:

  • the system loads the DengAI data or a documented representative subset;
  • the team defines a clear forecast target, city, time unit, and forecast horizon;
  • the system uses a temporal split that avoids future leakage;
  • the system trains or runs at least one baseline forecasting model;
  • the system generates dengue incidence forecasts for a held-out period;
  • the forecasts are displayed in a simple dashboard or visualization alongside historical incidence;
  • the system shows at least one uncertainty, error, or explanation signal useful to a public health user.

The POC may use one city, one baseline model, and a simple dashboard. It should not be only a competition notebook or only a static visualization without a reproducible forecasting path.

Evaluation Ideas

Possible evaluation approaches include:

  • comparing against a seasonal or persistence baseline;
  • using temporal backtesting;
  • reporting the DrivenData metric and at least one interpretable error summary;
  • analyzing errors by season, city, or outbreak period;
  • evaluating calibration or uncertainty quality if uncertainty is provided;
  • checking whether explanations align with known temporal or environmental patterns;
  • testing whether the dashboard makes forecast limitations visible.

Final Demo Target

The final demo should show the full path from data processing to forecast generation and dashboard exploration. A public health decision-maker should be able to inspect historical dengue incidence, predicted cases, uncertainty or error information, and explanations of important factors influencing the forecast.

Project 7: WHO Skin AI For Offline Dermatology

Proposed team size: 3 students.

Project lead: David.

WHO logo

Keywords: machine learning, skin disease, dermatology.

Short Description

Build an offline mobile application that can be used to reliably detect skin disease in the Global South.

Motivation

Skin diseases are common in many low-resource settings, but specialist dermatology expertise is often unavailable or difficult to access. Connectivity constraints can also make cloud-based AI tools unsuitable for frontline clinical use.

This project will focus on the development and evaluation of a fully open-source, AI-based skin disease detection tool designed for use in low-resource and connectivity-constrained healthcare settings. Students will identify and curate suitable open-source dermatology image datasets, train and benchmark one or more computer vision models for the recognition of both selected common and neglected tropical rare skin diseases, and integrate the resulting model into a lightweight application that can operate entirely offline, without requiring access to cloud services or external APIs.

Particular attention will be given to model efficiency, robustness across different skin tones and populations, and suitability for deployment on resource-constrained devices such as smartphones or local edge-computing hardware. The resulting system will be independently evaluated using the clinical evaluation data collected during the first validation phase of the existing tool, allowing its performance to be compared with the current solution under realistic clinical conditions.

The overarching goal is to establish a transparent and reproducible foundation for an openly available skin disease detection system whose source code, trained models, and, where licensing and ethical requirements permit, supporting training and evaluation resources can be shared with the wider research and global health community.

Intended Users

Potential users include:

  • frontline health workers in low-resource settings;
  • clinicians without immediate access to dermatology specialists;
  • mobile health teams working in rural or connectivity-constrained areas;
  • global health researchers evaluating open-source clinical AI tools;
  • public health programs addressing common and neglected tropical skin diseases.

Possible Features

Possible features include:

  • curated dermatology image dataset pipeline using open-source data;
  • model training for selected common and neglected tropical rare skin diseases;
  • image capture or upload flow for skin lesion classification;
  • fully offline inference without cloud services or external APIs;
  • lightweight mobile or mobile-like interface;
  • model confidence and uncertainty display;
  • explanations or guidance about limitations and appropriate use;
  • benchmarking across skin tones, populations, or disease groups;
  • exportable evaluation reports or model cards.

The team should avoid building a diagnostic black box. The prototype should make clear what the model can and cannot support, and how it should be interpreted by a clinical or public health user.

Design Questions

Consider:

  • Which skin diseases are in scope for the first version, and why?
  • Which open-source dermatology datasets are suitable, licensed, and ethically usable?
  • How should the system handle uncertain, low-quality, or out-of-distribution images?
  • How can the interface communicate that the tool supports decision-making rather than replacing clinical judgment?
  • What device, latency, memory, and offline constraints matter for deployment?
  • How will performance be assessed across skin tones, populations, and disease groups?
  • What information should be shared openly with the wider research and global health community?

Technical Directions

Possible technical components include:

  • discovery and curation of open-source dermatology image datasets;
  • dataset documentation, licensing review, and provenance tracking;
  • computer vision model training or fine-tuning;
  • model compression, quantization, or lightweight architecture selection;
  • offline inference pipeline for smartphones or edge devices;
  • mobile application, mobile web prototype, or local-device demo;
  • evaluation using clinical validation data from the existing tool's first validation phase;
  • fairness and robustness analysis across skin tones and populations;
  • model card and reproducibility documentation.

The resulting system should be designed so that source code, trained models, and supporting training or evaluation resources can be shared where licensing and ethical requirements permit.

Proof-Of-Concept Expectation

The proof of concept should demonstrate one valid offline skin-disease detection path from image input to model output and user-facing interpretation.

Minimum convincing POC:

  • the team identifies and documents at least one suitable open-source dermatology dataset;
  • the system trains, fine-tunes, or runs a computer vision model for a clearly scoped set of skin diseases;
  • the model can run inference locally without cloud services or external APIs;
  • the prototype accepts a representative image input and returns a classification or triage-style output;
  • the interface communicates confidence, uncertainty, or limitations;
  • the team reports at least one evaluation result using a held-out dataset or validation subset;
  • the repository includes reproducible instructions for dataset preparation, model execution, and demo use.

The POC may focus on a limited disease set, a small dataset, and a simple mobile or local-device interface. It should not be only a training notebook without an offline user-facing inference path.

Evaluation Ideas

Possible evaluation approaches include:

  • comparing model performance against a simple baseline or the current existing solution;
  • evaluating on clinical validation data collected during the first validation phase of the existing tool;
  • reporting performance by disease class, skin tone, population group, or image quality;
  • measuring latency, memory use, and offline readiness on a constrained device or simulated target environment;
  • testing robustness to image quality variation, lighting, framing, and camera differences;
  • documenting failure modes and unsafe use cases;
  • reviewing licensing, ethical constraints, and reproducibility of released resources.

Final Demo Target

The final demo should show a fully offline skin-disease detection workflow. A user should be able to provide a representative skin image, run local inference, view the model output with confidence or limitations, and see evidence that the system has been evaluated against realistic clinical data or a documented validation subset.

Project 8: Conversational AI For Frontline Health Workers

Proposed team size: 3 students.

Project lead: David.

WHO logo

Keywords: natural language processing, RAG, conversational AI.

Short Description

Build a chatbot that draws from clinical and national health guidelines in answering questions posed by frontline healthcare workers.

Motivation

Frontline health workers often need rapid access to trustworthy clinical guidance while working under time pressure, limited supervision, and variable connectivity. National clinical guidelines and approved health guidance may be available, but they can be difficult to search, interpret, or apply during routine care.

This project will focus on building a conversational chatbot interface through which frontline health workers can ask questions and receive responses grounded in the relevant national clinical guidelines and other approved health guidance applicable within the country. The languages of interaction will be English, Khmer, Lao, and Vietnamese.

The main technical focus of the project is on building a reliable RAG system using mmore over country-specific clinical guidance: https://github.com/EPFLiGHT/mmore

Students will identify and collect relevant national clinical guidelines and approved health guidance, and transform these documents into a structured and searchable corpus suitable for multilingual retrieval, including document parsing, cleaning, chunking, and metadata extraction. Students will investigate and compare approaches for multilingual retrieval, reranking, and grounded response generation, with particular attention to ensuring that answers are supported by the appropriate source documents.

The resulting system should be systematically evaluated for retrieval quality, correctness and grounding of generated answers, hallucination or unsupported claims, and performance across the supported languages.

Intended Users

Potential users include:

  • frontline health workers in primary care or community health settings;
  • nurses, midwives, and clinical officers seeking quick guidance;
  • supervisors supporting health workers across multiple facilities;
  • health programs that maintain national clinical guidelines;
  • implementation teams evaluating multilingual digital health tools.

Possible Features

Possible features include:

  • ingestion and indexing of national clinical guidelines and approved health guidance;
  • retrieval-augmented generation over guideline documents;
  • multilingual question answering in English, Khmer, Lao, and Vietnamese;
  • source citations or excerpts for each answer;
  • answer confidence, uncertainty, or escalation guidance;
  • conversation history for follow-up questions;
  • guardrails for out-of-scope, unsafe, or emergency questions;
  • lightweight interface for mobile or low-resource settings;
  • evaluation dashboard for answer quality and retrieval performance.

The team should avoid building a general medical chatbot. The system should be explicitly grounded in approved guidance and should make clear when a question cannot be answered from the available documents.

Design Questions

Consider:

  • Which guideline documents are in scope, and how should they be chunked or indexed?
  • How should the system handle questions that require local policy, clinical judgment, or urgent escalation?
  • How can answers remain faithful to the retrieved guidance instead of relying on model memory?
  • What does a useful citation or evidence excerpt look like for a busy frontline health worker?
  • How should multilingual interaction work across English, Khmer, Lao, and Vietnamese?
  • What should happen when retrieval fails or retrieved passages disagree?
  • How can the interface communicate limitations without making the tool unusable?

Technical Directions

Possible technical components include:

  • document ingestion pipeline for clinical and national health guidelines;
  • text cleaning, chunking, metadata extraction, and provenance tracking;
  • retrieval system using embeddings or keyword search;
  • RAG pipeline with source-grounded answer generation;
  • multilingual translation, cross-lingual retrieval, or multilingual embeddings;
  • prompt design and clinical-safety guardrails;
  • answer citation and source display;
  • evaluation dataset of representative frontline-worker questions;
  • automated and manual evaluation of retrieval quality, faithfulness, and answer usefulness.

The prototype should be reproducible from the project repository and should not depend on undocumented document collections, hidden prompts, or unavailable APIs unless those constraints are explicitly documented.

Proof-Of-Concept Expectation

The proof of concept should demonstrate one valid source-grounded question-answering path for frontline health workers.

Minimum convincing POC:

  • the team ingests a documented set of approved clinical or national health guidance;
  • the system retrieves relevant passages for representative health-worker questions;
  • the chatbot generates answers grounded in retrieved sources;
  • answers include citations, source snippets, or clear references back to the guidance;
  • the interface supports at least one multilingual interaction path relevant to English, Khmer, Lao, or Vietnamese;
  • the system handles at least one out-of-scope or unsafe question by refusing, escalating, or asking for clarification;
  • the team reports retrieval and answer-quality results on a small evaluation set.

The POC may use a limited guideline corpus, a small set of diseases or workflows, and a simple chat interface. It should not be only a generic chatbot prompt without retrieval, citations, and evaluation.

Evaluation Ideas

Possible evaluation approaches include:

  • measuring retrieval precision or recall on a set of guideline-grounded questions;
  • checking whether answers are faithful to cited passages;
  • comparing RAG answers with answers generated without retrieval;
  • evaluating multilingual questions for semantic preservation and answer usefulness;
  • testing refusal or escalation behavior for unsafe, emergency, or out-of-scope questions;
  • collecting expert or staff review of answer clarity, safety, and usefulness;
  • analyzing common failure modes such as missing guidance, wrong citations, hallucinated recommendations, or overconfident answers.

Final Demo Target

The final demo should show a frontline health worker asking clinical or guideline questions through a chat interface. The system should retrieve relevant guidance, answer in a supported language, show the source evidence behind the answer, and demonstrate how it handles uncertainty, missing information, or unsafe requests.

Project 9: Bayesian Conversational System For Ebola Virus

Proposed team size: 3 students.

Project lead: Yusuf.

Keywords: Bayesian inference, Clinical Decision Support, Conversational AI, Data Curation, Explainable AI, LLMs, Natural Language Processing.

Short Description

Curate and document the lab's Ebola dataset, then use it as the empirical knowledge base for a MoBayes style modular Bayesian dialogue system for Ebola triage.

Motivation

Clinical conversational AI is risky when language generation and clinical reasoning are mixed together in an opaque way. In high-stakes Ebola triage, a system should be able to explain what evidence it has, what it still does not know, and why it asks the next question.

This project has two objectives. The first is data work: students will investigate the Ebola dataset available in the lab, reconstruct what each variable and label actually means, and produce a cleaned, well-documented dataset with a proper data card.

The second objective is to implement the MoBayes framework (Kesmen et al., arXiv:2604.20022) on top of this dataset, and make a user interface for it. MoBayes separates language from reasoning: the LLM acts only as a sensor and verbalizer, parsing patient conversation into structured observations, while a Bayesian module maintains an explicit posterior over conditions, selects the next question by expected information gain, and decides when to stop or defer using calibrated thresholds.

Students will evaluate the resulting system in terms of predictive performance, calibration, dialogue efficiency, and the accuracy with which the LLM extracts structured observations from patient conversations.

Reference: https://arxiv.org/abs/2604.20022

Intended Users

Potential users include:

  • clinicians or triage workers evaluating possible Ebola cases;
  • outbreak response teams using structured symptom and exposure information;
  • researchers studying interpretable clinical dialogue systems;
  • data stewards documenting clinical or outbreak datasets;
  • public health teams comparing transparent reasoning systems with generic LLM chatbots.

Possible Features

Possible features include:

  • data audit and cleaning workflow for the lab Ebola dataset;
  • data dictionary and data card documenting variables, labels, provenance, and limitations;
  • structured observation schema for symptoms, exposures, timing, and risk factors;
  • LLM-based parsing of patient dialogue into structured observations;
  • Bayesian inference module maintaining posterior probabilities over conditions or triage states;
  • expected-information-gain question selection;
  • calibrated stopping or deferral thresholds;
  • user interface for patient conversation and reasoning inspection;
  • natural-language verbalization of the current reasoning state;
  • explanation view showing observations, posterior updates, and next-question rationale.

The team should avoid building a generic Ebola chatbot. The core of the project is the explicit separation between language processing and Bayesian clinical reasoning.

Design Questions

Consider:

  • What does each variable and label in the lab Ebola dataset actually mean?
  • Which variables can be used as observations in a patient dialogue?
  • What target conditions, triage states should the Bayesian module represent?
  • How should the system handle missing, uncertain, contradictory, or ambiguous patient answers?
  • What question should be asked next, and how is expected information gain computed?
  • When should the system stop, provide a recommendation, ask for more information, or defer to a clinician?
  • What interface makes the separation between conversation, extracted observations, and Bayesian reasoning visible?

Technical Directions

Possible technical components include:

  • dataset audit, cleaning, and documentation;
  • data dictionary construction and data-card writing;
  • structured observation extraction from dialogue using an LLM;
  • modular Bayesian inference engine;
  • expected-information-gain calculation for question selection;
  • threshold-based stopping, deferral, or escalation policy;
  • conversational interface that separates user dialogue from reasoning state;
  • evaluation against scripted patient cases or held-out dataset records.

The system should make the Bayesian state inspectable. A reviewer should be able to see which observations changed the posterior and why the next question was selected.

Proof-Of-Concept Expectation

The proof of concept should demonstrate one valid path from curated Ebola data to modular Bayesian dialogue.

Minimum convincing POC:

  • the team produces a cleaned subset of the Ebola dataset with a clear data dictionary;
  • the repository includes a data card describing provenance, variables, labels, missingness, and limitations;
  • the Bayesian module maintains an explicit posterior over a clearly defined set of conditions or triage states;
  • the system accepts at least one patient-style conversational input and converts it into structured observations;
  • the system updates the posterior after each observation;
  • the system selects at least one next question using an information-gain or uncertainty-reduction criterion;
  • the system can decide to continue, stop, or defer using documented thresholds;
  • the demo includes a user interface that explains the reasoning state in a way a technical reviewer can inspect.

Evaluation Ideas

Possible evaluation approaches include:

  • checking data documentation quality with a data-card rubric;
  • validating variable meanings and labels with staff or domain experts where possible;
  • measuring predictive performance and calibration;
  • measuring dialogue efficiency, such as number of questions before stopping or deferral;
  • measuring whether the selected next questions reduce uncertainty;
  • testing robustness to missing, ambiguous, or contradictory patient answers;
  • evaluating LLM observation extraction accuracy against manually labeled dialogue snippets;
  • comparing explanations from the modular system with those from a generic clinical chatbot;
  • analyzing calibration of stopping or deferral thresholds.

Final Demo Target

The final demo should show a short Ebola triage conversation in which the system parses patient answers into structured observations, updates an explicit Bayesian posterior, chooses the next question using information gain, and explains why it continues, stops, or defers. The demo should also show the cleaned dataset documentation, data card, predictive performance, calibration, dialogue efficiency, and LLM extraction accuracy evidence that support the reasoning module.

Project 10: Medical Personas And Expert Routing For Clinical QA

Proposed team size: 3 students.

Project lead: Jeremy.

Keywords: LLM, Clinical Q&A, Medical Personas, Prompt Engineering, Medical AI.

Short Description

Investigate whether specialist medical personas can improve the accuracy and consistency of large language models on clinical question-answering tasks.

Motivation

Large language models are increasingly being used for medical question answering, but their performance can vary significantly depending on the type and complexity of the clinical question. One possible way to improve their performance is to prompt them to adopt the perspective of a relevant medical specialist, such as an infectious-disease physician, radiologist, epidemiologist, or another clinical expert.

However, it is not yet clear which specialist personas are most effective for different categories of medical questions, or whether specialization consistently improves model reasoning and accuracy. A persona that helps with one specialty may not help another. Some personas may even make the model more confident while reducing correctness.

The aim of this project is to systematically evaluate how different medical personas affect performance across clinical QA benchmarks and to identify which personas improve or degrade particular types of questions. Students will identify and gather suitable clinical QA datasets, then curate and transform them into a benchmark organized by question type, medical specialty, and complexity where possible.

Based on these findings, students will develop an expert-routing system that predicts the most appropriate persona, or combination of personas, for each question. The system should be compared against a generic physician prompt and multi-persona ensemble approaches to determine whether dynamic specialist routing can improve and standardize clinical QA performance without requiring additional model training.

Students will also develop an interactive interface where users can submit clinical questions, inspect the selected specialist persona or personas, compare alternative routing strategies, and explore the resulting answers and evaluation results.

Intended Users

Potential users include:

  • clinicians and clinical educators studying how LLM prompts affect medical answers;
  • medical AI researchers evaluating specialist prompting strategies;
  • benchmark designers organizing clinical QA tasks by specialty and complexity;
  • developers building safer clinical decision-support interfaces;
  • reviewers who need to inspect why a system selected a particular expert persona.

Possible Features

Possible features include:

  • curated benchmark of clinical QA questions with specialty, question type, and complexity labels;
  • prompt library for generic physician and specialist medical personas;
  • evaluation pipeline comparing persona prompts across clinical QA datasets;
  • expert router that selects one or more personas for a given question;
  • multi-persona ensemble baseline for comparison;
  • answer generation interface showing the selected persona and final answer;
  • evaluation dashboard showing accuracy, consistency, calibration, and failure patterns;
  • side-by-side comparison of generic, routed, and ensemble strategies;
  • analysis tools for identifying which personas improve or degrade specific question categories.

The team should avoid building only a generic medical chatbot. The core of the project is the systematic evaluation and routing of medical personas for clinical question answering.

Design Questions

Consider:

  • Which clinical QA datasets are suitable for evaluating persona-based prompting?
  • How should questions be labeled by specialty, type, difficulty, or complexity?
  • Which specialist personas should be tested, and how should their prompts be written?
  • Does a specialist persona improve factual accuracy, reasoning consistency, calibration, or only style?
  • When does specialization degrade performance or introduce overconfidence?
  • Should the router select one persona, multiple personas, or an ensemble strategy?
  • What features of a question should influence expert routing?
  • How should the interface make the selected persona and routing rationale inspectable?
  • How should the system communicate uncertainty and avoid presenting generated answers as medical advice?

Technical Directions

Possible technical components include:

  • dataset collection and curation for clinical QA benchmarks;
  • specialty, question-type, and complexity annotation workflow;
  • prompt engineering for generic physician and specialist personas;
  • automated benchmark runner for evaluating multiple prompts and models;
  • metric calculation for accuracy, consistency, calibration, and answer stability;
  • expert-routing model based on rules, embeddings, classifiers, or LLM-based routing;
  • multi-persona ensemble baseline and aggregation strategy;
  • interactive web interface for question submission, routing inspection, and result comparison;
  • dashboard for per-specialty and per-question-type performance analysis.

The system should clearly separate the question, selected persona, generated answer, evaluation evidence, and routing rationale.

Proof-Of-Concept Expectation

The proof of concept should demonstrate one complete persona-evaluation and routing workflow.

Minimum convincing POC:

  • the team curates a small clinical QA benchmark with documented question labels;
  • at least one generic physician prompt and several specialist persona prompts are implemented;
  • the system evaluates persona prompts on a subset of the benchmark;
  • the team reports where specific personas improve or degrade performance;
  • an expert router selects a persona or persona set for a new clinical question;
  • the selected routing strategy is compared with a generic physician prompt and at least one ensemble baseline;
  • the demo interface shows the submitted question, selected persona, generated answer, and comparison results;
  • the repository includes clear documentation of datasets, prompts, metrics, and limitations.

The POC may use a small benchmark subset and a limited set of medical specialties. It should not be only a prompt collection, and it should expose the routing logic from reviewers.

Evaluation Ideas

Possible evaluation approaches include:

  • comparing routed personas against generic physician prompts on accuracy;
  • measuring consistency across repeated runs or prompt variants;
  • analyzing performance by specialty, question type, and complexity;
  • testing whether routing improves difficult or specialty-specific questions;
  • checking for overconfidence, unsupported claims, or unsafe medical advice;
  • comparing single-persona routing with multi-persona ensemble approaches;
  • evaluating the router's specialty selection against labeled questions;
  • reviewing whether the interface makes routing decisions and limitations understandable;
  • documenting failure cases where persona prompting worsens performance.

Final Demo Target

The final demo should show a user submitting a clinical question, the router selecting an appropriate specialist persona or persona set, and the system comparing the routed answer with a generic physician prompt and an ensemble baseline. The demo should also show benchmark-level evidence about which personas helped or hurt different categories of clinical questions, along with clear limitations and safeguards around medical use.