SeHing the Stage
1.1 Forensic Evidence
When I enlisted as an I.T. foot soldier at SFSU, it was like being airlifted to a hot landing zone. Bullets were flying everywhere. The university's network (a collection of subnets in a class B address range) didn't have a firewall to speak of, not even a NAT device. Thousands of machines were just sitting out in the open with public IP addresses, listening for connections. In so many words, we were free game for every script kiddy and bot-herder on the planet.
The college that hired me managed roughly 500 desktop machines and a rack of servers. At the time, these computers were being held down by a lone sys-tem administrator and a contingent of student assistants. To be honest, the best that this guy could hope to do was focus on the visible problems and pray that the less conspicuous problems didn't creep up and bite him in the
3
Chapter 1 / Selling the Stage
backside. The caveat of this mindset is that it tends to allow the smaller fires to grow into larger fires, until the fires unite into one big firestorm. But, then again, who doesn't like a good train wreck?
It was in this chaotic environment that I ended up on the receiving end of attacks that used rootkit technology. A couple of weeks into the job, a coworker and I found the remnants of an intrusion on a computer that had been configured to share files. The evidence was stashed in the System Volume Information directory. This is one of those proprietary spots that Windows wants you to blissfully ignore. According to Microsoft's online docu-mentation, the System Volume Information folder is "a hidden system folder that the System Restore tool uses to store its information and restore points."! The official documentation also states that "you might need to gain access to this folder for troubleshooting purposes." Normally, only the operat-ing system has permissions to this folder and many system administrators simply dismiss it (making it the perfect place to stash hack tools).
The following series of batch file snippets is a replay of the actions that attackers took once they had a foothold. My guess is they left this script behind so they could access it quickly without having to send files across the WAN link. The attackers began by changing the permissions on the System Volume Information folder. In particular, they changed things so that every-one had full access. They also created a backup folder where they could store files and nested this folder within the System Volume directory to conceal it.
@echo off
xcacls "c: \System Volume Information" IG EVERYONE: F IV mkdir "c:\System Volume Information\catalog\{GUID}\backup"
attrib.exe +h +s +r "c:\System Volume Information"
attrib.exe +h +s +r "c:\System Volume Information\catalog"
attrib.exe +h +s +r "c:\System Volume Information\catalog\{GUID}"
attrib.exe +h +s +r "c:\System Volume Information\catalog\{GUID}\backup"
caclsENG "c:\System Volume Information" IT IG system:f Administrators:R caclsENG "c:\System Volume Information\catalog" IT IG system:f caclsENG "c:\System Volume Information\catalog\{GUID}" IT IG system:f caclsENG "c:\System Volume Information\catalog\{GUID}\backup" IT IG system:f The calcsENG. exe program doesn't exist on the standard Windows install.
It's a special tool that the attackers brought with them. They also brought their own copy of touch. exe, which was a Windows port of the standard UNIX program.
1 Microsoft Corporation, "How to gain access to the System Volume Information folder,"
Knowledge Base Article 309531, May 7, 2007.
4
I
Port IChapter 1 / Selling the Stage
>
Nole: For the sake of brevity, I have used the string "GUID"to represent the global unique identifier
"F7S0E6C3-38EE-ll Dl-8SES-OOC04FC29SEE."
To help cover their tracks, they changed the timestamp on the System Volume Information directory structure so that it matched that of the Recycle Bin, and then further modified the permissions on the System Volume Infor-mation directory to lock down everything but the backup folder. The tools that they used probably ran under the System account (which means that they had compromised the server completely). Notice how they placed their backup folder at least two levels down from the folder that has DENY access permissions. This was, no doubt, a move to hide their presence on the com-promised machine.
touch -g "c: \RECYCLER" "c: \System Volume Infonnation"
touch -g "c: \RECYCLER" "c: \System Volume Infonnation\catalog"
touch -g "c: \RECYCLER" "c: \System Volume Infonnation\catalog\{GUID}"
touch -g "c: \RECYCLER" "c: \System Volume Infonnation\catalog\{GUID}\backup"
xcacls "c: \System Volume Infonnation\catalog\{GUID}\backup" IG EVERYONE: F IY
xcacls "c:\System Volume Infonnation\catalog\{GUID}" IG SYSTEM:F IY
xcacls "c: \System Volume Infonnation\catalog" ID EVERYONE IY
xcacls "c: \System Volume Infonnation" IG SYSTEM: F IY
After setting up a working folder, they changed their focus to the System32 folder, where they installed several files (see Table 1-1). One of these files was a remote access program named qttask. exe.
cd\
c:
cd %systemroot%
cd system32 qttask.exe Ii net start LdmSvc
Under normal circumstances, the qttask. exe executable would be Apple's QuickTime player, a standard program on many desktop installations. A forensic analysis of this executable on a test machine proved otherwise (we'll discuss forensics and anti-forensics later on in the book). In our case, qttask. exe was a modified FiP server that, among other things, provided a remote shell. The banner displayed by the FiP server announced that the attack was the work of "Team WzM." I have no idea what WzM stands for, perhaps "Wort zum Montag." The attack originated on an
IRe
port from the IP address 195.157.35.1, a network managed by Dircon.net, which is head-quartered in London.Port I
I
5Chapter 1 / Selling the Stage
Table 1-1
File name Desmptlon
qttask.exe FTP-based command and control server pWdumpS.exe Dumps password hashes from the SAM database2
lyae.cmm ASCII bannerfile
pci. acx ASCII text, configuration parameters wci.acx ASCII text, filter sellings of some sort icp.nls,icw.nls Language support files
libeay32.dll,ssleay32.dll DLLs used by OpenSSL svcon. crt PKI certificate used by DLLs3
svcon.key ASCII text, registry key entry used during installation
Once the ITP server was installed, the batch file launched the server. The qttask. exe executable ran as a service named LdmSvc (the display name was
"Logical Disk Management Service"). In addition to allowing the rootkit to survive a reboot, running as a service was also an attempt to escape detec-tion. A harried system administrator might glance at the list of running services and (particularly on a dedicated file server) decide that the Logical Disk Management Service was just some special "value-added" OEM program.
The attackers made removal difficult for us by configuring several key ser-vices, like RPC and the event logging service, to be dependent upon the LdmSvc service. They did this by editing service entries in the registry (see HKLM\SYSTEM\CurrentControlSet\Services). Some of the service registry keys possess a REG_MUL TI_SZ value named DependOnService that fulfills this purpose. Any attempt to stop LdmSvc would be stymied because the OS would protest (i.e., display a pop-up window), reporting to the user that core services would also cease to function. We ended up having to manually edit the registry to remove the dependency entries, delete the LdmSvc sub-key, and then reboot the machine to start with a clean slate.
On a compromised machine, we'd sometimes see entries that looked like:
C:\>reg query HKLM\SYSTEM\CurrentControlSet\Services\RpcSs HKEY_lOCAl_MACHINE\SYSTEM\CurrentControlSet\Services\RpcSs
DisplayName REG_SZ @oleres.dIl,-se10
Group REG_SZ CCM Infrastructure
ImagePath REG_EXPAND_SZ svchost.exe -k rpcss
2 http://passwords.openwall.netlmicrosoft-windows-nt-2000-xp-2003-vista 3 http://www.openssl.org/
6
I
Port IChapter 1 I Selling the Stage
Description REG_SZ @oleres.dll,-Sall
ObjectName REG_SZ NT AUTHORITY\NetworkService
ErrorControl REG_!HlRD axl
Start REG_!HlRD ax2
Type REG_!HlRD ax2a
DependOnService REG_MJL TI_SZ DcomLaunch\LdmSvc
ServiceSidType REG_!HlRD axl
Note how the DependOnService field has been set to include LdmSvc, the faux logical disk management service.
Like many attackers, after they had established an outpost, they went about securing the machine so that other attackers wouldn't be able to get in. For example, they shut off the default hidden shares.
net share Idelete C$ Iy net share Idelete D$ Iy
REM skipping E$ to Y$ for brevity net share Idelete Z$ Iy
net share Idelete $RPC net share Idelete $NT net share Idelete $RA SERVER net share Idelete $SQL SERVER net share Idelete ADMIN$ Iy net share Idelete IPC$ Iy net share Idelete lwc$ Iy net share Idelete printS
reg add "HKLM\SYSTEM\CurrentControlSet\Services\LanManServer\Parameters"
Iv AutoShareServer It REG_!HlRD Id a If
reg add "HKLM\SYSTEM\CurrentControlSet\Services\LanManServer\Parameters"
Iv AutoShareWks It REG_!HlRD Id a If
Years earlier, the college's original IT director had decided that all of the machines (servers, desktops, and laptops) should all have the same password for the local system administrator account. I assume this decision was insti-tuted so that we wouldn't have to remember that many passwords, or be tempted to write them down. However, once the attackers ran pwdump5, giv-ing them a text file containgiv-ing the file server's LM and NTLM hashes, it was the beginning of the end. No doubt, they brute forced the LM hashes offline with a tool like John the Ripperi and then had free reign to every machine under our supervision (including the domain controllers). Game over, they sank our battleship.
In the wake of this initial discovery, it became evident that Hacker Defender had found its way onto several of our mission-critical systems and the intrud-ers were gleefully watching us thrash about in panic. To further amuse
4 http://www.openwall.com/john/
Part I
I
7Chapter 1 / Setting the Stage
themselves, they surreptitiously installed Microsoft's Software Update Ser-vices (SUS) on our web server and then adjusted the domain's group policy to point domain members to the rogue SUS server.
Just in case you're wondering, Microsoft's SUS product was released as a way to help administrators provide updates to their machines by acting as a LAN-based distribution point. This is particularly effective on networks that have a slow WAN link. While gigabit bandwidth is fairly common in American universities, there are still local area networks (e.g., Kazakhstan) where dial-up to the outside is as good as it gets. In slow-link cases, the idea is to download updates to a set of one or more web servers on the LAN, and then have local machines access updates without having to get on the Internet.
Ostensibly this saves bandwidth because the updates only need to be down-loaded from the Internet once.
While this sounds great on paper, and the MCSE exams would have you believe that it's the greatest thing since sliced bread, SUS servers can become a single point of failure and a truly devious weapon if compromised.
The intruders used their faux SUS server to install a remote administration suite called DameWare on our besieged desktop machines (which dutifully installed the .msi files as if they were a legitimate update). Yes, you heard right. Our update server was patching our machines with tools that gave the attackers a better foothold on the network. The ensuing cleanup took the better part of a year. I can't count the number of machines that we rebuilt from scratch. When a machine was slow to respond, or had locked out a user, the first thing we did was to look for DameWare.