Last Thursday, the City Of Atlanta suffered outages from a ransomware attack. During the press conference (recorded here), city officials indicated that they were invested in cyber security. They noted that they were working with state and federal law enforcement to resolve the incident and had even been in contact with the Secret Service. Officials noted that this type of attack (and outage) were happening to many organizations. Officials attempted to convey that despite adopting cyber security best practices, the City of Atlanta was victimized. This prompts the question “Was the City of Atlanta following cyber security best practices?”
Though little is known about the internals of the city’s cyber security posture, we quickly learned that the city had exposed remote desktop protocol (RDP) to the Internet with no multi-factor authentication*. This is extremely important because if attackers get a valid username and password combination, they can directly access your information systems if no multi-factor authentication is in place.
*Full disclosure: We’re a little biased on the need for multi-factor authentication, Rendition Infosec installs and monitors multi-factor authentication solutions, click here to learn more.
Leaving RDP open to the Internet is bad, but leaving SMB (windows file sharing, or Server Message Block) open to the Internet is much worse. Most readers probably remember the WannaCry ransomware campaign that shut down services at the UK’s National Health Service and elsewhere in May 2017. These attacks were powered by the leaked NSA (allegedly) exploit EternalBlue. In June, the same leaked exploit was used with the NotPetya attacks to target Ukrainian businesses (though impacts were felt worldwide). The EternalBlue exploit targets the SMB service on unpatched computers.
Twitter user “Huy” noted that after the announcement of the ransomware attack that a number of computers apparently owned by the City of Atlanta had SMB exposed to the Internet. This is obviously a departure from best practices. However, it should be noted that just because SMB or RDP is exposed to the Internet, this doesn’t necessarily mean that these machines were compromised by attackers. It is indicative of a poor overall cybersecurity posture.
Rendition Infosec’s Scan Data
When EternalBlue was released into the wild in April 2017, Rendition Infosec realized that this exploit was going to be a big deal. The vulnerability (MS17-010) was patched in March, 2017. But we know that many organizations don’t patch for 30-60 days or more. Further, those who are exposing services like SMB to the Internet are already displaying substandard cybersecurity hygiene, making it less likely that they’ll be patching in anything approaching a timely manner.
To understand patterns of exploitation, Rendition began scanning for machines that were exposing SMB to the Internet. For each machine we discovered that was exposing this service, we sent a special “ping” command to communicate with the DoublePulsar malware that is temporarily installed when a computer is compromised with the EternalBlue exploit. When the DoublePulsar malware is present, the ping command returns a special response. Using this response, we can conclusively determine which machines have been compromised. With DoublePulsar running on a compromised machine, any attacker can upload malware to the machine and execute it with system privileges (no further exploitation or authentication is required).
Because of ethical and legal concerns, Rendition did not attempt to interact any further with machines that were discovered to be running DoublePulsar. However, we can say that those machines were compromised for some reason by some third party. In most incident response cases we’ve worked at Rendition involving EternalBlue and DoublePulsar, these tools are just used to install end-stage malware (such as ransomware) on the compromised machines.
It’s also important to note that the DoublePulsar malware disappears when a machine is rebooted. This means that our scan data is very likely incomplete. Any positive scan results should be considered an absolute indication of compromise. However, the absence of scan results does not necessarily indicate that no compromise occurred (it just means we didn’t see it). A more thorough investigation (typically referred to as threat hunting) should be conducted if you have any suspicion of compromise.
Rendition performed a number of scans in late April and the first few days of May. Of course we notified our clients of any of their IP’s and hostnames discovered in the scan data. However, the magnitude of the scan data prohibited reaching out to everyone compromised. Our largest scan indicated that more than 148k machines were compromised – the raw number of results simply precluded us from contacting each victim individually. We were discussing with legal counsel how to best communicate vulnerability data when WannaCry hit in mid May 2017. After WannaCry, we decided to shelve the idea of reaching out to victims, figuring most had also been hit with WannaCry (and we’re not ambulance chasers). However, given the events in Atlanta, we’re now seeing that this data is relevant even now and we’re dusting off our data set.
The City of Atlanta Had Five Systems Compromised in April 2017
Our scan data indicates that the City Of Atlanta had the following five systems fully compromised in April 2017:
Note that DNS names were resolved at the time of scans and rely on correct DNS PTR records to be supplied by atltantaga.gov’s DNS server. Of these systems, the webmail10.atlantaga.gov server is probably the most concerning since an attacker could conceivably download all email (given the superuser privileges obtained with EternalBlue).
This scan data conclusively shows that the City of Atlanta was not patching its Internet facing hosts more than a month after *critical* patches were released my Microsoft. Microsoft released patches on March 14, 2017. Our scan data shows these hosts being vulnerable (and compromised by unknown attackers) on dates spanning from April 23, 2017 to May 1, 2017. After doing some searching for statements from the City or Atlanta, we can’t find any indication that they were aware of this compromise at all. We reached out to let them know that they were previously compromised before this was posted publicly.
It’s further worth noting that our scan data only includes machines that had SMB exposed to the Internet. We know that public facing machines tend to receive patches (especially those labeled as critical with publicly available exploits) before the rest of the network. There are an unknown number of machines in the City of Atlanta’s internal network that were likely also unpatched at the time we performed our scans. In our experience, it is generally safe to infer overall cyber security hygiene from a representative sample like what we have. Even if we don’t infer overall hygiene, the fact remains that the City of Atlanta couldn’t be bothered to patch at least five Internet facing servers for more than a month in April 2017 – even when there was a publicly available exploit.
Was my organization compromised?
Like we mentioned earlier, we simply didn’t have the resources to contact everyone compromised. Fun fact: as of last night, I still wasn’t able to easily find a way to report a cyber security compromise on the City of Atlanta website – and that’s after they suffered an attack that drew national media coverage. Imagine repeating this notification exercise across tens of thousands of organizations…
We won’t be making our full data set available (our lawyer said no). However, as a public service, Rendition will query our dataset for your domain names and IP addresses. If you want to know if your systems showed up in our scans, please fill out the form below and one of our analysts will be in contact with you. Please don’t use Gmail, Yahoo, etc. addresses. We’re not giving you someone else’s scan data, so we need to be able to validate that you own the domain or control the IP addresses in question.
Obtain your scan results