The NLP Working Group: Improving Clinical Data
February 17, 2020 · Matt Brock
On Monday, February 10, NCQA held its first formal gathering of a specially selected group of technology professionals. The Natural Language Processing (NLP) Working Group will be convening throughout the Spring of 2020 to help NCQA explore how NLP may be suitable for use in quality measurement, and reporting. Ultimately, the goal is to weigh whether NCQA can evaluate NLP solutions in the fair, thorough and rigorous manner our customers expect.
NCQA invited a broad and diverse list of NLP suppliers from across the country to apply for the NLP Working Group. We’re pleased to announce its final composition: Apixio, UPMC, Wave Health Technologies LLC. This select cohort showed us not only an explanation of methodological rigor, but also a practical demonstration of how each uses NLP with its clients.
What is NLP?
For those not deeply entrenched in the technology, natural language processing refers to the process by which computer algorithms sift through unstructured, every-day-language to translate and organize it into computer-friendly data.
A good example would be if your doctor’s computer identified key words from her freeform encounter notes and converted them into data points. That’s new data to make diagnoses, new data to see trends and patterns more clearly. We know that most clinical data (50-80%) in electronic clinical data systems (ECDS) is unstructured. So, NLP has the potential to increase the efficiency, accuracy, and integrability of clinical data. That makes it very valuable, especially for organizations participating in pay-for-performance care delivery.
But How Can I be Sure it Works?
Here’s the issue, and the reason for the working group. We have concerns that some NLP implementations can yield potentially flawed results. That flawed data could then make its way into pivotal data streams like the ones used for HEDIS® reporting.
In our monthly conversations with the Working Group, NCQA wants to explore ways to avoid flawed data. The entire health care sector needs data that is accurate and actionable. So, the working group will look at how we can validate the accuracy of an NLP output as part of an electronic clinical data system (ECDS). We expect the working group’s participants will show us how their NLP software converts unstructured clinical data into structured data. We’ll also look at how they ensure quality for their customers and discuss what their customers expect from NLP solutions. All of this will inform our development of a standard for validation of NLP data.
Lessons learned with the Working Group will be shared in parallel monthly meetings with an Independent Advisory Panel (IAP). The IAP consists of NLP experts—academicians, researchers, industry thought leaders—who will provide an impartial and independent perspective on the validation model NCQA expects to develop. The advisory panel’s impartiality and familiarity with industry best practices will ensure NCQA’s NLP validation model provides a rigorous assessment of NLP technologies.
Then What? From Learning to Launch
With the help of some of the industry’s leading minds on the use of NLP in healthcare, NCQA will evaluate our ability to develop a comprehensive approach to validate the integrity of NLP solutions. We want to help organizations that use data to discern which technologies are most appropriate, particularly when used in conjunction with HEDIS® reporting.
NCQA has built a 30-year history of providing our third-party, independent and trusted assessment of healthcare quality. That assessment relies on validated data for use in these evaluations. There is tremendous potential for the use of NLP. But, like always, we rely on others to help us.
The Working Group is a pivotal part of this process, to assuring NCQA’s validation model can be trusted, and to improving health care quality. So, as we embark on this investigative journey, we’ll start with a simple expression of gratitude. Thank you, Apixio. Thank you, UPMC. Thank You, Wave Health Technologies LLC. We couldn’t do it without you.
