PHI Redaction System Using LLMs.

Brown University & RI Hospital

Developing an Automated Redaction Pipeline with Local LLMs

Role

Research Engineer

Duration

May 2024 - Present

Team

MedDB Group @ Brown CS

Neurosurgery @ RI Hospital

Disciplines

LLMs

Prompt Engineering

Fine Tuning

Medical Database

Research Overview

How can we automate and improve the efficiency of redacting Protected Health Information (PHI) in medical datasets?

In collaboration with the MedDB Group at Brown University’s Database Systems Lab and Lifespan Hospital, I am working on an automated system designed to redact Protected Health Information (PHI) from medical datasets. Using large language models (LLMs), this system focuses on removing PHI from unstructured clinical notes, enhancing both accuracy and context sensitivity.

Role

As an undergraduate research engineer with the MedDB Lab, I worked on developing and testing a redaction pipeline using the OpenAI API and the Llama 3.1 8B model. I mainly used Python, Ollama, DSPy, and Harvard Medical School’s Clinical Note Dataset to optimize the pipeline and prepared it for real patient data integration from Lifespan RI Hospital.

Current Challenges & Practices

Why Develop an Automated Redaction Pipeline?

Strict adherence to ethical and legal requirements, such as HIPAA, is critical when handling sensitive patient information. Under HIPAA, 18 identifiers are classified as PHI, and failing to anonymize these can result in significant legal and ethical issues. Traditional methods—such as manual redaction, rule-based systems, and conventional NLP models—struggle with context-dependent PHI, leading to potential oversights. The LLM-based pipeline addresses these limitations, offering a more sophisticated approach by integrating local LLMs into the redaction process.

While findings and research pipeline implementation details will be shared in an upcoming publication, the main approach centers around training and optimizing LLM performance using prompt engineering frameworks like DSPy to enhance redaction accuracy.

Feel free to reach out if you’re curious about more details or updates!