Insights Into Security & Breach Response Planning

You hear that Mr. Anderson? That’s the sound of inevitability…

— Agent Smith, The Matrix

I’m reminded of a story of a breach I worked on a few years ago with one of my clients. This client had just suffered a breach of one of their healthcare related web servers that housed HIPAA related information. Sounds like a really bad day, right?

Well, it gets worse. This particular client was a large company (and still is), and there were multiple internal parties that were interested in this data breach. It turned out that we were one of two teams brought in to conduct independent breach investigations… and this is where it got interesting.

Our competitor released their investigation report first, and their conclusion was that they had to assume the bad guys had accessed patient health information (PHI) because there was nothing on the server to prevent the attackers from doing so.

We, on the other hand, drew the opposite conclusion based on the evidence in front of us.  Specifically, we saw clear evidence the attacker was interested in taking over the web server because it was a legitimate server with a legitimate domain, which gave it credibility in the eyes of spam filters.

That’s all the attackers were interested in and needed, and with it they were able to send out spam e-mails from this server during the time of the incident. There was no evidence whatsoever that they even took notice of what else was running on the server, or that it stored PHI.

This should be a lesson for anyone running an eCommerce site. For years I’ve heard merchants tell me that they have a limited customer base, or they sell “socks” or “spatulas” and hackers wouldn’t be interested in any of that. The fact is that attackers may simply want to abuse your hosting account or server to:

Send spam e-mail (that helps them sell their wares)

Get visitors to download their malware

Redirect visitors to their own sites

By having a live website, you may be a target whether you know it or not, so being proactive is key.

Getting back to the story, most companies won’t face a situation where two professional services firms are drawing opposite conclusions. (I say that because it’s rare to want to hire two companies to provide the same service at the same time.) But you can still derive an extremely valuable lesson, which is:

Do your due diligence before you hire an incident response firm. Each firm has very different methodologies, strengths, and sets of experiences.

What does it mean to do your due diligence? Start by asking each firm about their experience in your scenario. Have them describe their step by step methodology. You’re looking for that warm and fuzzy feeling that they know exactly what they’re doing and they’ll be able to advise you.

The firm I used to work for was tops at investigating outside intrusions and scoping the breach and eradicating the attackers. But we wouldn’t touch incidents of fraud where you have to analyze large amounts of log data coming from multiple systems and follow the breadcrumbs. That’s a totally different skillset, toolset, and set of experiences.

Luckily, my client had a breach response plan, but many eCommerce businesses don’t have incident response plans or teams. If you’re not practicing the actions you’ll be expected to take during an incident, you’ll be awkward and unrehearsed when it comes time for the real thing.

Another valuable lesson is having a process in place and training your team (no matter how large or small) so they understand the process. In other words, you play how you practice, and if you don’t practice, your play reflects it.

Without practice, this is what can happen: During the same breach I was describing earlier, my client asked an IT person to perform a manual backup on the server after the breach was discovered. But their backup process was generally automated, so running manual backups was rare.

And it was probably late at night, as these things typically only happen late at night. Anyway, this IT person accidentally ran the backup of the C drive onto the D drive, overwriting half of the data contained on the D drive. Which made the investigation that much harder on all of us.

You should consider documenting processes that are likely to come up during an incident. Examples could include running backups, taking a forensic image of a drive or memory, making emergency changes to a firewall, and providing emergency access to an outside firm in order for them to access your systems.

Working with experienced practitioners, including managed hosting services that understand best-practices, is key if you don’t want a breach to go from a bad situation to a worse one.

With eCommerce platforms like Magento, your site could be breached through Magento’s software, through an extension, through some susceptible coding in your site, or any number of other vectors. Having a good hosting partner that knows what to do and that you can reach instantly, a solid breach plan, and experts on call is always best-practice.