Correlation is Not Causation: Electrical Analysis of St. Jude Implant Shows Normal Pacing

St. Jude Merlin Error Indicators Are Not Evidence of Malfunction

Battle of the bands: Here's what we listened to while
writing this summary.

Here's an abbreviated technical analysis of some claims by Muddy Waters and St. Jude regarding pacemaker/defibrillator security. We will show you why correlation is not causation in the sense that a scary-looking screen is not a reliable indicator of a clinically relevant security problem. We did this analysis based on our experience over the last ten years analyzing pacemaker and defibrillator security and our experience building cardiac arrhythmia simulators for humanitarian pacemaker reuse. Read more at our ancient research website. Or see our index of previous blog posts on medical device security. This is a fun extracurricular activity for our team at the University of Michigan and Virta Labs, and we may post more thoughts before we return to our regular lives baking hearth breads and helping hospitals with cybersecurity risks.

The Muddy Waters report of August 25 showed a screenshot which they say shows an “apparent malfunction.” They also say that red error marks “are also indicators that the device is malfunctioning.” We were curious about these claims and decided to see if we could produce the same onscreen displays without causing any malfunction. This summary shows the screenshot is correlated with normal pacing and sensing, suggesting that the Muddy Waters report misinterprets clinical relevance of the screenshot.

Figure 1: Our experiment shows that a Merlin programmer screenshot from p. 17 of the Muddy Waters report is not supportive evidence of a successful attack. The top photo shows our reproduction of the Merlin programmer screen photo, but without causing changes to the pacing pulses. Our end-to-end oscilloscope measurements (bottom photo) show that pacing pulses continue normally despite the three benign alerts that are expected when not connected to cardiac tissue. 

Hypothesis: The Merlin programmer screen photo on page 17 of the Muddy Waters report is not supportive evidence of appearing “to have caused the device to pace at a rapid rate.”

Approach: Produce the same on-screen screen output, and externally measure electrical signals to test safety and effectiveness of pacing and sensing.

Result: We reliably produced the same screen output while the implant continued to pace normally.

Material: St. Jude Medical Fortify Assura ICD, Merlin programmer (software version 22.0.1 rev1)

Clinical validation: Verified by Dr. Thomas Crawford, a cardiologist and a clinical electrophysiologist at the University of Michigan Health System's Frankel Cardiovascular Center.

Commentary:

To verify pacing, we configured the device to emit 40 bpm pacing pulses at 2.5 V, then connected a clipped lead (~20 cm) to the V (IS-1 Bi) sense/pace port, connected an oscilloscope to the clipped lead with 50 Ω probes, and visually confirmed that the device was emitting 40 pulses per minute (Figure 1 bottom). To verify sensing, we used a signal generator to produce a 0.5 Hz square wave (consisting of 2 events, a rising then a falling edge, for a total of 1 event per second or 60 pbm) at 2 mV which we fed into the sense/pace port via the same lead; the programmer recognized a 60 bpm beat as expected. We tested other square-wave frequencies between 0.5 Hz and 2 Hz to verify that the sensing worked as expected.

To reproduce the markers that the Muddy Waters report highlights as indicators of a successful attack, we introduced benign electrical noise on the sense/pace port via the clipped lead by connecting the lead to a separately grounded oscilloscope (i.e., not grounded to the “can” of the device, which typically acts as ground). This noise was sufficient to trigger the “VS2” markers on the programmer screen, indicating that the device sensed a “ventricular beat.” While sampling the 40 bpm pacing output as described above, we reproduced the count of three alerts visible in the Muddy Waters report’s screen photo: two alerts from high impedance on two leads (since those were not connected to cardiac tissue), and one indicating “ventricular noise reversion.” The pacing and sensing continued to function normally. ■

The team from the University of Michigan and Virta Labs is continuing to investigate the contrasting claims by Muddy Waters and St. Jude Medical. To receive notifications of updates, follow the Archimedes Center for Medical Device security @ARC_MedSec and @DrKevinFu on Twitter. Virta Labs also plans to issue a separate white paper.

Study on St. Jude medical device security deemed “inconclusive” by University of Michigan researchers

A recent report that alleged security flaws in St. Jude Medical’s pacemakers and other life-saving medical devices has major flaws of its own. That’s according to a team of University of Michigan researchers who say they’ve reproduced the experiments that led to the allegations, and come to strikingly different conclusions.

The U-M team is composed of several leading medical device security researchers and a cardiologist from the U-M Health System's Frankel Cardiovascular Center. “Hyperbolic” and “sloppy” are words they use to describe the unorthodox report, which was released last week by short-selling investment research firm Muddy Waters Capital and medical device security firm MedSec, Ltd.

The U-M team reproduced the error messages the report cites as evidence of a successful “crash attack” into a home-monitored implantable cardiac defibrillator. But they showed that the messages are actually the same set of errors you’d get if you didn’t have the device properly plugged in.

When it’s implanted, a defibrillator’s electrodes are connected to heart tissue via wires that are woven through blood vessels, explains Kevin Fu, associate professor of computer science and engineering at U-M and director of the Archimedes Center for Medical Device Security. Fu is also co-founder of medical device security startup Virta Labs.

Through these wires, implantable defibrillators can perform sensing operations and also send shocks if necessary.

“When these wires are disconnected, the device generates a series of error messages: two indicate high impedance, and a third indicates that the pacemaker is interfering with itself,” said Denis Foo Kune, former U-M postdoctoral researcher and co-founder of Virta Labs.

On page 17 of the Muddy Waters report, a screenshot cites these very error messages as proof of a security breach.

“But really the pacemaker is acting correctly,” Fu said. “To the armchair engineer it may look startling, but to a clinician it just means you didn’t plug it in. In layman’s terms, it’s like claiming that hackers took over your computer, but then later discovering that you simply forgot to plug in your keyboard.”

Added Foo Kune, “While there still could be security problems, the screenshot is anything but supportive of the claim. When researchers with limited medical training go public with unvetted claims, it’s easy to jump to conclusions.”

Ethicists and other researchers have criticized MedSec’s technique of teaming with a short-seller to publicize its preliminary findings—and benefit financially, no less.

Short-selling is an investment practice that essentially involves betting that a particular stock will decline in value. If it does, then the investment firm profits. In this case, MedSec made a deal with Muddy Waters to receive a share of those profits. St. Jude’s stock fell sharply over the weekend.

“It was the irresponsible thing to do. Think about whether you believe everything a used car dealer claims when deciding whether to buy,” said Wenyuan Xu, a visiting professor of electrical engineering and computer science at U-M and an expert in automotive and medical device security. She recently hacked into Tesla’s autopilot system to demonstrate its vulnerabilities.

To conduct the experiments, the U-M team used a new and properly functioning model of the same defibrillator that the Muddy Waters study used—the Fortify Assura VR. In several additional instances, they found that the device operated properly.

Even while the U-M research team finds fault with the Muddy Waters report, they don’t mean to suggest that these medical devices—or any medical devices for that matter—are necessarily secure. They stress the importance of establishing security workflows early on in the design process of medical devices.

“While medical device manufacturers must improve the security of their products, claiming the sky is falling is counterproductive,” Fu said. “Healthcare cybersecurity is about safety and risk management and patients who are prescribed a medical device are far safer with the device than without it.”

Thomas Crawford, an assistant professor of medicine and a clinical electrophysiologist at U-M, agrees. Crawford implants and follows patients with pacemakers and implantable defibrillators.

“Given the significant benefits from home monitoring, patients should continue to engage in it via St. Jude Medical Merlin, and other companies’ respective proprietary home monitoring systems, before independent research can substantiate the claims made by MedSec and their financial partner Muddy Waters Capital, LLC,” Crawford said.

Crawford adds that home monitoring has been shown to reduce a variety of adverse events, with some studies even showing reduction in overall mortality over periodic checks of devices in the doctor’s office. The devices can send actionable alerts to a central monitoring service, which then is forwarded to the physician, so that it can be dealt with immediately if necessary. Alerts include low battery status, potential malfunction of the device, or changes in heart rhythm, which may require treatment.

The Archimedes Center for Medical Device Security offers a Medical Security 101 training in Orlando Jan. 15-17, 2017. Details will be forthcoming online. In the meantime, for more information, email archimedes@umich.edu.