Measure Up: Hold Ourselves to a Higher Standard

June 23, 2017 · Rick Moore

More than a year ago, I wrote a blog “Trust is Critical When Change is Needed.”  To summarize, I concluded that for the push toward Value-Based Payments (VBP) to work, clinicians must trust that their performance rates are accurately reported. In addition, the public (patients) must also trust that their care is of high quality and value. We now have two developments that indicate flaws in the foundational programs designed to protect patients as the health care system shifts from a fee-for-service payment model to one that pays based on value.

First, some background.

In 2011, the Centers for Medicare and Medicaid Services (CMS) established the Meaningful Use (MU) Program through the Health information Technology (HITECH) Act which was part of the American Recovery and Reinvestment Act (ARRA) signed by President Obama in 2009. As of April, the government paid more than $35 billion dollars in the Medicaid/Medicare EHR incentive funds to more than 500,000 eligible professionals. To earn the incentives, clinicians had to use Office of the National Coordinator (ONC)-certified health information technology (HIT) and attest to several performance metrics, including reporting of clinical quality measures.

Now, those developments.

In May, the government accused eClinicalWorks of fraudulently representing its ONC-certified software (Electronic Health Record/EHR) to its customers which posed serious safety risks to patients. eClinicalWorks admitted no wrongdoing, but settled. They paid $155 million dollars in fines and agreed to, at no charge, transition any customer off the eClinicalWorks platform to any certified platform of the customer’s choosing.

Rick Moore, NCQA CIO

Then in June, the Office of the Inspector General (OIG) estimated that the government erroneously paid more than $700 million dollars in EHR incentives. They did it by either double paying clinicians (both Medicaid and Medicare incentives) and by paying clinicians who inaccurately and/or fraudulently self-attested without necessary evidence to prove they earned the incentive.

All this as the government sets new legislation (The Medicare Access and CHIP Reauthorization Act of 2015 – MACRA) into action. MACRA attempts to take the best elements of the MU program (and others) to form the new CMS Quality Payment Program (QPP). The QPP program requires clinicians to take on even greater financial risks with managing the care they deliver. The government pays them upon their performance indicators. This system, already proven to be less than accurate, poses potential for inaccurate payment as indicated in the OIG report.

So, what have we learned from the past?

How will we move forward to regain the trust of clinicians and the patients they serve? How can we assure all of them that performance rates are accurate?

Based on the past decade of progress toward a more digital healthcare enterprise, I’d say we’ve learned the following things that should make us pause, reflect, and course correct:

1. Self-attestation may minimize reporting burden, but it is not a policy upon which CMS should rely for performance-based payments.
2. The current minimalist set of certification test cases for quality reporting is not sufficiently rigorous to accurately generate quality performance rates upon which to base payment.
3. Validation of a system in a test lab is satisfactory, but the reported data from those systems require additional validation/audit.

NCQA does not have all the answers, but we believe we can be part of the solution. With more than 25 years of experience, NCQA is deeply familiar with complexities in the extraction, translation, and loading of clinical (and other) data that produce the country’s most widely-used nationally comparable performance measurement system—the Healthcare Effectiveness Data and Information Set (HEDIS©). Building on our vast experience with validating HEDIS® rates, we developed the NCQA eMeasure testing program.

NCQA Measures Up

Since 2011, we worked with ONC to earn approval of the NCQA eMeasure testing program as an equivalent process/methodology to the ONC-provided and MITRE-developed Project Cypress test tool. We support the Project Cypress toolset as a satisfactory approach for measure developers to assess their measure logic. Still, we are focused on testing measures in ways that more effectively validate the various complexities of a measure. For example, NCQA’s algorithm validation methodology requires a system to accurately sift through ~1500 cases per measure. The Project Cypress tool sifts through ~100 cases.  We announced here in February that ONC tested the NCQA eMeasure test process as equivalent to the ONC Project Cypress toolset. Now, we can tell you the good news that ONC officially approved the NCQA eMeasure test process as an alternative testing method in the ONC Health IT Certification Program.

It’s a start–a building block to shore up the foundation of how we measure and report performance rates. NCQA’s expertise in measurement and evaluation will serve this effort well. To that end, NCQA is pursuing ONC approval as an Authorized Test Lab (ATL).

In addition, we strongly encourage CMS to implement a data validation audit process. As part of our annual HEDIS reporting process, each data submission must undergo an independent, third party audit to be considered valid. Our experience tells us that it is satisfactory to test the system producing the rates, but not sufficient to be considered a valid rate unless the HEDIS audit occurred.

We believe that we offer the industry a much higher standard of validation with our robust testing/audit approach. We are eager to assist the entire industry as we all hold ourselves to a higher standard of measure validation.