Equifax Breach – Early lessons learned and six point action plan

In this post, we’ll discuss a few early lessons learned from the Equifax breach announced yesterday.  We’ll also recommend a six point plan to avoid becoming “the next Equifax” based on what we know today about the breach. Rendition is in no way involved with the breach assessment for Equifax and we have no inside knowledge.  However, we will discuss the publicly available information so organizations can take action to avoid a similar breach.

Note: In the coming days and weeks, you’ll likely be inundated with vendor pitches claiming they can stop you from becoming “the next Equifax.” Be wary, be very wary.  If it sounds too good to be true, it probably is. In information security, there are no silver bullets. But that’s okay – werewolves probably aren’t part of your threat model anyway…

At Rendition Infosec, we endorse the SANS Institute six step Incident Response (IR) process.  For those not familiar with the process, the steps are:

1. Preparation
2. Identification
3. Containment
4. Eradication
5. Recovery
6. Lessons Learned

This conveniently spells PICERL.  A handy mnemonic to remember this is “Patched Infrastructure Could’ve Easily Reduced Losses.”  This is great because it’s simple to remember AND true.

For this post, we’re going to focus on the preparation and identification phases since those are what we know the most about so far.

Web Application Vulnerabilities

It is clear that some controls were lacking in the Equifax environment.  While we don’t know much about their internal network, we do know some details about problems in their web applications.  This post shows that the Equifax site was vulnerable to trivial XSS.  The XSS was disclosed in March, 2016 but was still actively exploitable yesterday. This probably wasn’t the source of the breach.  However, this points to three probable issues in the Equifax network:

1. Lack of proper web application penetration tests (preparation)
2. Absent or misconfigured web application firewall (preparation/identification)
3. Poor handling of externally reported vulnerabilities (preparation/identification)

Stack traces / error messages printed

You can also see from this screenshot (via of Twitter user @notdan) that there are unhandled exceptions in some Equifax web pages.

Not only are these conditions potentially exploitable, but printing error messages and stack traces provide attackers a roadmap for exploitation. Best practices demand these be removed.  This issue points to three probable issues:

1. Lack of proper web application penetration tests (preparation)
2. Absent or misconfigured web application firewall (preparation/identification)
3. Lack of monitoring to detect unhandled exceptions that result in 500 series HTTP response codes (identification)

Incident Communication / Public Relations

Rendition Infosec always advises our clients to perform tabletop exercises for incident response scenarios. Part of these scenarios involve contacting Public Relations (PR) to help them understand how to work with the IR team.  It seems clear that if PR was involved in the breach notification, they failed miserably.

First, you should not register a new domain to communicate a breach notification.  Users have no good way to determine whether the new domain you just registered is legitimate or not. Further, this opens opportunities for opportunistic attackers to register lookalike domains and exploit your victims them again.  Twitter user @notdan demonstrated this brilliantly with a Twitter poll.  The poll highlights the confusion about which domain should actually be trusted.  These results are especially concerning because most of Dan’s following is infosec and IT related.

After registering the domain, corporate communications forgot to let reputation monitors know about the new domain.  When never before seen domains become popular quickly, they tend to be phishing and/or malware distribution domains. Reputation monitors know this.  When they observed the explosive growth in popularity for the new never before seen domain, it was listed as potentially malicious.  OpenDNS listed the domain as a phishing domain initially, but overrode their algorithms quickly so their users could access the domain again.

During the initial hours after the breach was announced, users who tried to call Equifax reported waiting on hold for an hour or more, only to be told that the call center didn’t have the list of impacted people.  Others, including Brian Krebs, reported that the server wasn’t responding to requests at various points.

There are a few probable issues highlighted in this portion of the IR:

1. The public relations staff failed to communicate with call center employees about the incident (preparation / recovery)
2. Equifax staff registered a new domain, resulting in victims receiving phishing domain warnings when they tried to register for monitoring (preparation / recovery)
3. The choice to register a new domain exposed victims to further victimization through lookalike domains (preparation / recovery)
4. Despite prior planning, the servers at the new domain were unable to handle the load of users visiting the site after the breach was made public (preparation / recovery)

Six point plan 

There’s more that can be said about the Equifax breach now, but this is getting long already. Certainly we’ll learn more as time progresses.  If your board of directors is asking you today “how do we make sure we’re not the next Equifax,” Rendition has a six point plan to help you make that happen.

1. Have a qualified third party conduct a web application penetration test and then take action on the results.  It’s clear that one or both of these didn’t happen with Equifax.
2. Review your vulnerability notification process and incident response plans.  A third party reported an XSS vulnerability to Equifax. They didn’t act on it for more than a year. Something broke down somewhere.  Do you have a security@domain email alias? Who monitors it? Review your policies and verify they’re being actioned.
3. Perform incident response tabletop exercises with your staff.  A tabletop exercise is only as good as the scenarios and injects your moderator brings to the table (pun definitely intended).  If you only drill on softball IR scenarios, that’s all you’ll be ready to handle in real life.  Use challenging tabletop exercises to find (and fix) gaps in your response plan before an incident.
4. Once you have a strong IR team, bring in non-infosec staff to work mock incidents with your team.  Public relations, IT, legal, corporate communications, etc. will all play a role in a real incident response. It only makes sense that they have some exposure to the process before a real incident occurs.  Rendition doesn’t recommend you bring these non-infosec staff into the mix immediately.  Get your own team working like a well oiled machine before involving others outside your team.
5. Make sure your network monitoring is actually capable of finding attacks.  Weekly (or even daily) log review by systems administrators isn’t enough anymore (honestly, it never was). You need a dedicated monitoring team that understands what they are looking for.  The Equifax breach was discovered July 29, 2017. But the investigation points to mid-May as the time of the initial breach.  This puts the attacker dwell time in the network (time between initial breach and discovery) at about 75 days.  While this is better than many breaches we’ve worked at Rendition (and probably better than the industry average per the Verizon DBIR), it is still 75 days longer than the board wants an attacker in the network.  If you don’t have a SIEM, get one. If you have a SIEM, but you don’t have a dedicated monitoring team, change that now.  Many organizations benefit from using a monitoring service rather than staffing, training, and maintaining their own monitoring team.  But however you do it, get monitoring in locked down.
6. Have a plan for incident response surge staff.  All but the largest organizations will need assistance from a qualified third party during a large incident.  Even if your staff are qualified to perform a forensic investigation (most aren’t), there are only so many hours in the day.  Operations don’t stand still because of an incident (at least they shouldn’t).  Negotiate an incident response retainer with an outside firm before the breach and you can always say “we know who we’re calling.”  This lets you negotiate rates without the emotion of a breach and expectations are already set.  Good incident response firms retained in this manner will look at your logging posture and make recommendations before a breach occurs.  This removes a lot of the “I wish we’d configured PowerShell block logging” (or similar) statements during the response.  It’s not clear whether Equifax already had a retained firm before the breach, but they are definitely using a third party firm for the investigation.

Parting thoughts

We hope that you’ve found this useful.  Earlier, we noted that you should be wary of anyone trying to capitalize on the Equifax breach by trying to sell you a magic bullet.  But nothing we’ve suggested is a silver bullet or a magic pill.  We’ve just recommended some of the same services that Rendition delivers to our customers each and every day.  Of course the selection of services specifically relates to the topic at hand (the Equifax breach).  If you need assistance with any security services or just want the peace of mind that comes with an industry leading firm at your side, contact Renditon – we stand ready to help.