In 2022, Volkswagen recalled over 43,000 vehicles after a software update designed to improve the entertainment system had the unintended consequence of causing the rearview camera to go black, an obvious safety hazard. That same year, Audi recalled thousands of vehicles because new gas tank sensors were transmitting incorrect information to the fuel gauge, causing drivers to run out of gas even though their indicator showed fuel in the tank.
These are just two of a myriad of examples you can find across industries where seemingly minor changes to one area of an interconnected electronic system significantly impacted another.
When it comes to electronic health records (EHR), every system upgrade or update – of which there are many – has the potential to disrupt a hospital or health system’s existing EHR system and its functionality. The stakes are incredibly high, as even the slightest system break caused by an upgrade can profoundly impact everything from patient safety to revenue cycle performance. So, it is imperative to have measures in place to detect issues associated with an upgrade before they go “live” throughout an EHR system.
In industries where complex systems must work seamlessly, unexpected incompatibilities can lead to costly and time-consuming fixes and recalls. In healthcare, hospitals and health systems have an easier path thanks to regression testing. This efficient and cost-effective solution identifies potential issues related to an EHR upgrade before they disrupt clinicians or impact patient care.
What Is Regression Testing?
Regression testing confirms that recent upgrades, updates, or changes to an EHR system have not created new defects in its existing functionality. It is not about testing every single update or feature – far from it, as we’ll explore in the next section. Instead, the goal of regression testing is to verify that what worked yesterday in the EHR system still works the same way today, even after changes have been made.
What Regression Testing Isn’t
Regression testing isn’t meant to individually test every single new feature or change included in a vendor update. For instance, a system update might include hundreds, if not thousands, of changes or release notes. Instead of testing each change in isolation, regression testing takes a strategic approach, concentrating on areas where changes are most likely to impact functionality. This ensures that the system’s core operations remain reliable and efficient while maintaining a thorough and efficient testing process.
Attempting to test every change in a major system update would overwhelm an IT team and be highly inefficient—they wouldn’t have the capacity to focus on anything else. The good news is that while a quarterly update might include thousands of changes, only a small number are likely to impact the core functionality of an EHR system.
Going back to our car analogy, if you change your windshield wipers, the addition is extremely unlikely to affect the brakes, so you can still drive with confidence, knowing that your car will stop when you want it to. It’s the same thing with regression testing. Suppose an EHR update changes the font color from green to red or a dropdown menu to a radio button list on a particular screen. These alterations are extremely unlikely to impact your essential workflows, so they don’t need to be added to your test scripts. The same goes for other cosmetic and peripheral changes that are similarly benign. Especially since adding too many changes to your scripts can make them too big, which complicates pinpointing the source of a system break when the need arises.
Regression testing focuses on system changes that have the greatest potential impact, such as those affecting critical areas like patient safety, operational efficiency, and revenue cycle outcomes. This targeted approach prioritizes evaluating and maintaining the reliability of the most important aspects of the system.
Configurable to a Fault?
EHR systems are purposely customizable. Vendors like Epic and Cerner, among others, design their systems with basic functionality, providing healthcare organizations with tremendous leeway to make their own manual adjustments to accommodate specific clinical workflows, data entry, terminology, and more. While this flexibility is certainly a plus, the fact that the systems are so configurable can potentially lead to issues when changes are made.
For example, if a clinician wants information presented in a different order on a separate screen, it could lead to unintended consequences in another area of the system, effectively causing a break. The error could have serious ramifications if it is not caught in time. A significant issue for health systems that test manually is that the outcomes of system changes are accommodated for in the testing “on the fly,” which puts a tremendous onus on the analyst to validate the outcomes. When you consider all the changes being made within a single hospital, it’s understandable how some could go undocumented in the test scripts, or conversely, how too many are added to the test scripts, leaving them bloated and compromised. The potential for inefficiencies and errors associated with a manual system underscores the need for structured and disciplined regression testing.
Testing With Precision
A static bed of test scripts provides the ballast for successful regression testing. A consistent, streamlined set of scripts provides a clear expectation of how an EHR system should perform when a regression test executes, and if it doesn’t, allows testers to pinpoint the source of the break. Less structured testing, with test scripts consistently updated with non-essential changes, leaves testers with many more potential causes to investigate. These larger-than-necessary scripts introduce several levels of complexity in diagnosing whether a failure is a system issue, a script problem, or something introduced by the vendor.
The value of structured, automated regression testing, like the kind performed by SureTest, is that with an exact set of test scripts, containing fixed inputs and outputs, we can rightfully expect an EHR system to perform in a very specific way. If it does not, we only have a few possibilities to consider as to what might be wrong. When anomalies occur, we can explore key system functionality to identify potential problems. As we smoke test the core conditions of an EHR system and markers point to potential flames, we can then investigate on a deeper level. The entire process is made exponentially easier and more efficient thanks to the static bed of test scripts and clearly defined expectations that SureTest provides.
For Every Action, There’s A Reaction
Every update or modification to your EHR system has the potential to upset existing workflows.
Regression testing helps ensure that for every action – whether it’s a quarterly update or a minor tweak – there is an expected reaction, and that changes made in one area haven’t caused inadvertent consequences elsewhere, particularly in critical areas that impact patient safety.
For that reason, healthcare organizations should view a structured regression testing protocol not as a luxury, but as a necessity.
Just like in the automotive industry, where every new feature should be tested to make sure it doesn’t upend a vehicle’s sensitive ecosystem and inspire a recall, regression testing serves as something of a safety net for IT teams and their healthcare organizations, making sure every action ultimately leads to the right, and appropriate reaction.
SureTest’s regression testing approach functions as a quality control system for your EHR, much like the rigorous checks in the automotive industry that prevent minor design flaws from escalating into costly recalls. It mitigates the risk of disruptions, protecting the integrity of your system, the well-being of your patients, and the seamless operation of your organization’s workflows.
Watch this webinar to learn about the benefits Novant Health has realized by leveraging SureManage™, SureTest’s fully managed test automation solution, to eliminate the burden of manual integration and regression testing.