More than anything over the last few days I have seen some of the so called industry professionals I have to consider my peers, wax lyrical about how the NHS should have done better. I want to take a minute to explore that statement and offer my own experience and opinion. For the record, if parts of this article offend you, tough.
Over the last 10 years I have had the pleasure of working with a large number of different NHS trusts in the context of a supplier and partner. During this time, I have made a number of observations that make me proud and sad in equal measure.
Firstly, during my entire time, I have not met a single IT or Security Professional in the NHS that does not put patient care or clinical excellence first. They truly understand the impact of a bad decision far more than the average equivalent in the private sector. This is a good thing, but can lead to a higher than normal level of risk aversion. This risk aversion however, has led to some of the most advanced, and amazing network architectures I have ever seen. I have personally worked in network architecture for many years and worked on some very interesting networks with very specific needs so I know what “good” looks like. I actually remember walking into my first NHS trust to do an independent security architecture review and genuinely being blown away by the equipment they had deployed, the overall architecture and way it was managed. Unlike some people have been saying, your typical trust has better connectivity than your average enterprise.
Now, I am not saying every trust is like this as they are all run independently and some invest more than others, but in my personal experience it’s the norm to be on the latest and greatest kit.
Where the infrastructure tends to fall down is on the endpoints. Why? Simple. Because when you spend a million pounds on an MRI scanner and it comes with a windows XP machine, then XP stops being supported and the equipment manufacturer wants an astronomical amount of money for an upgrade, what do you do? Who is really at fault? The NHS or the huge multi-global equipment manufacturer?
So, a good trust does what it does and proceeds to put in numerous countermeasures to isolate and ring-fence that machine as best it can while still allowing patient care and clinical excellence to be its priority.
So let’s look at those countermeasure options:
- Patch it – Nope, can’t, doesn’t exit
- Isolate it in a dedicated LAN – Sure, but I have 30 doctors in 6 clinical departments on 14 campuses that need to access its data and none of them understand technology enough to deal with any form of air gap, so yeah, if by isolate you mean limit, I’m all in!
- Firewall it, see 2.
- Prevent malware execution – (a) The manufacture said if we install anything on the box it will crash. (b) we tried, it crashed, (c) We tried, it didn’t crash
So let’s look at 4c for a minute. According to the cyber security vendor world they all have a product that would have stopped WannaCry and any other unknown malware dead in its tracks. Sounds great. First problem…. These tools hook the kernel and or low level drivers and end up interfering with any number of things they haven’t been tested with, say for instance like a very specific piece of MRI scanner software, of which there are only 34 copies installed in the UK. Second problem, let’s assume that it works with everything and really is the answer…. A typical NHS trust has 10’s of thousands of users / endpoints. For instance, we regularly run phishing campaigns for NHS trusts and it’s not unusual to be given 20,000 email targets in a single campaign.
These silver bullet anti-malware execution solutions are expensive and licensed per seat so the cost of deploying it into an NHS trust can be more than the available annual budget for that trust to maintain what they have! It’s a simple economics equation.
My challenge to those vendors that all have “the solution to malware” do the right thing and donate it to the NHS, after all you will probably need them one day and it would be ironic that if on the day you did, the systems were offline with a ransomware attack.
In fact, 4c is the main underlying problem. I have seen trusts with entire Cisco infrastructures that are 100% TrustSEC complaint end to end, and all they would need to enable some of the most advanced packet level manipulation in existence is a simple license. But that cost of that license is so prohibitive, they just can’t switch it on? How crazy is that? Come on Cisco, your big and bad enough to let that one go!
Next up I want to tackle the N3. The N3 is the backbone that connects every trust to every other trust and provides carrier internet connectivity as well. It also provides the secure email for the NHS and many other core network services. There is a common conception within the trusts that the “N3 protects us from bad things” and to a certain extent it does. But the flip side of that statement is that it has nearly 1mn endpoints on it! That’s a small internet!!! I mean seriously, 1mn endpoints that all trust each other? Are you crazy? with that sort of scale trust is not possible.
The irony is most trusts actually have their own firewalls, but they trust the N3 and each other so they are essentially fire-windows. This is why WannaCrypt and Conficker before it tore through the N3 taking trust after trust out along the way. Even trusts that are more restrictive than others and trust no one, still have to trust common shared infrastructure that allows infections to easily jump from trust to trust. I actually spoke to a few of my contacts in the NHS on Friday the 12th of May and they said they had “disconnected form the N3 for safety” as they had not yet been infected but knew if they stuck around they would be. That statement in itself says all that needs to be said about the N3. It should not be a trusted entity. There is a solid argument for the devolution of trust inside that boundary and that message to be hammered home to the individual trusts by NHS Digital.
So could the NHS have done more? Yes, of course in hindsight, but at the time those trusts made decisions based on risk, the risk to life, not that corporate risk assessment you make and feel all warm and fuzzy about. Actual loss of life. So before you start to rip into the NHS or try and push your silver bullet down their throat, spare a minute to think about it.
This has been a RANT sponsored by the “Security is a vocation not popularity contest” Party.
About Jay Abbott
Jay Abbott is the Managing Director of Falanx Cyber Defence and Executive Director of Falanx Group Limited. Jay’s background and experience is in the “design it, build it & break it” space, where he has spent most of his career engineering technical solutions to business problems, and the rest reverse engineering technology solutions to ensure that they are secure. Jay has held senior positions with organisations including PricewaterhouseCoopers LLP, Electronic Arts and Barclays Bank to name but a few.
He is a celebrated key-note speaker and is regularly quoted in the media on the subject of Cyber Security. Over the past 20 years within the industry,