Infosanity's Blog

I’ve recently been involved in a couple of discussions for different ways for identifying malware. One of the possibilities that has been brought up a couple of times is fuzzy hashing, intended to locate files based on similarities to known files. I must admit that I don’t fully understand the maths and logic behind creating fuzzy hash signatures or comparing them. If you’re curious Dustin Hurlbut has released a paper on the subject, Hurlbut’s abstract does a better job of explaining the general idea behind fuzzy hashing.

Fuzzy hashing allows the discovery of potentially incriminating documents that may not be located using traditional hashing methods. The use of the fuzzy hash is much like the fuzzy logic search; it is looking for documents that are similar but not exactly the same, called homologous files. Homologous files have identical strings of binary data; however they are not exact duplicates. An example would be two identical word processor documents, with a new paragraph added in the middle of one. To locate homologous files, they must be hashed traditionally in segments to identify the strings of identical data.

I have previously experimented with a tool called ssdeep, which implements the theory behind fuzzy hashing. To use ssdeep to find files similar to known malicious files you can run ssdeep against the known samples to generate a signature hash, then run ssdeep against the files you are searching, comparing with the previously generated sample.

One scenarios I’ve used ssdeep for in the past is to try and group malware samples collected by malware honeypot systems based on functionality. In my attempts I haven’t found this to be a promising line of research, as different malware can typically have the same and similar functionality most of the samples showed a high level of comparison whether actually related or not.

Another scenario that I had developed was running ssdeep against a clean WinXP install with a malicious binary. In the tests I had run I haven’t found this to be a useful process, given the disk capacity available to modern systems running ssdeep against a large HDD can be a time consuming process. It can also generate a good number of false positives when run against the OS.

After recently reading Leon van der Eijk’s post on malware carving I have been mulling a method for combining techniques to improve fuzzy hashing’s ability to identify malicious files, while reducing the number of false positives and workload required for an investigator. The theory was that, while any unexpected files on a system are not desirable, if they aren’t running in memory then they are less threatening than those that are active.

To test the theory I infected an XP SP2 victim with a sample of Blaster that had been harvested by my Dionaea honeypot and dumped the RAM following Leon’s methodology. Once the image was dissected by foremost I ran ssdeep against extracted resources. Ssdeep successfully identified the malicious files with a 100% comparison to the maliciuos sample. So far so good.

With my previous experience with ssdeep I ran a control test, repeating the procedure against the dumped memory of a completely clean install. Unsurprisingly the comparison did not find a similar 100% match, however it did falsely flag several files and artifacts with a 90%+ comparison so there is still a significant risk of false positives.

From the process I have learnt a fair deal (reading and understanding Leon’s methodolgy was no comparison to putting it into practice) but don’t intend to utilise the methods and techniques attempted in real-world scenarios any time soon. Similar, and likely faster, results can be achieved by following Leon’s process completely and running the files carved by Foremost against an anti-virus scan.

Being able to test scenarios similar to this was the main reason for me to build up the my test and development lab which I have described previously. In particular, if I had run the investigation on physical hardware I would likely not have rebuilt the environment for the control test with a clean system, losing the additional data for comparison, virtualisation snap shots made re-running the scenario trivial.

–Andrew Waite

P.S. Big thanks to Leon for writing up the memory capture and carving process used as a foundation for testing this scenario.

I recently read Christian Wojner’s excellent paper on Mass Malware Analysis and it re-ignited my desire to build an automated environment to improve and speed up my current malware analysis capabilities. The paper details a step by step for duplicating Wojner’s environment, but I as I don’t have any spare equipment I’ve been looking for alternative routes.

Fortunately the paper also explains the theory, thought process and design of the system so that the reader can modify to suit their own requirements. To achieve this I’ve been trying replace the Xubuntu and Virtual Box host with my existing  ESXi environment detailed in previous posts.

With a bit of Googling the vSphere CLI became the obvious choice to replace the control component for the infected machine in the automated malware environment. vmware-cmd.pl provides the functionality to both stop/start virtual guests and to revert the guest to previous snapshots, exactly what is needed for the malware analysis environment. The commands to be utilised would be (– is a double dash):

vmware-cmd.pl –server <ESXi Host> –username <user> –password <pass> /path/to/guest.vmx getstate

vmware-cmd.pl –server <ESXi Host> –username <user> –password <pass> /path/to/guest.vmx start

vmware-cmd.pl –server <ESXi Host> –username <user> –password <pass> /path/to/guest.vmx stop

vmware-cmd.pl –server <ESXi Host> –username <user> –password <pass> /path/to/guest.vmx revertsnapshot

This should have been enough to adapt Wojner’s control scripts to use ESXi instead of Virtual box, but it appears that for the first time I’ve encountered a crippled feature not available in the VMware’s free offering. Running the stop/start/revert commands results in the below exception:

Fault:
SOAP Fault:
———–
Fault string: fault.RestrictedVersion.summary
Fault detail: RestrictedVersionFault

So that’s that, unless I happen to win the lottery (which I don’t play) or someone is able and willing to provide a full ESX license to a struggling researcher (which I don’t expect to happen) I’m back to looking for a replacement Wojner’s VirtualBox control process. On with the next…

– Andrew Waite

The previous post on lab machines seemed to generate a high level of interest so I thought it may be worthwhile to expand further and share my lab’s network setup.

My recent exploration of the Vyatta platform’s capabilities has provided a simple method for segregating and connecting lab networks without requiring a hardware router. I currently split my network into three seperate subnets.

• 10.0.0.0/24       – Physical network for none virtual machines
• 192.168.1.0/24 – Primary lab subnet
• 192.168.2.0/24 – Secondary lab subnet

The primary lab subnet contains the majority of my victims. Helpfully the machines configured as part of Metasploit Unleashed and most of the machines released by Heorot.net (De-ICE level1 and Hackerdemia) both use the 192.168.1.0/24 subnet. To fit I’ve configured my custom machines to match. While providing additional targets, the custom machines in the primary lab double as my malware analysis environment (with the Vyatta appliance powered off to provide isolation).

My secondary lab subnet currently only contains the single publically available level 2 De-ICE machine. In the future I’m intending to expand the usage of the secondary lab by dual-homing one or more of the of the lab machines and demoing pivot and techniques to use one a compromised machine to attack otherwise inaccessible targets.

With the machines and environment detailed above and in the previous post I’ve managed to develop a highly versatile lab environment for both tool/exploit development and training/practice. Not bad for a total outlay of under £200 plus some time and effort.

– Andrew Waite

Since working through and reviewing Wilhelm’s ‘Professional Penetration Testing’ I’ve been trying to build up and improve my personal lab environment, still running ESXi and still running on my HP Proliant ML110 . Having just about got all of my target machines in place I thought this would be a good place to list the machines in my lab, and to share the sources for others looking for a test environment themselves.

ESXi Inventory listing

Off the back of the Professional Penetration Testing book I include the machines created and maintained on Heorot.net;

• The De-ICE LiveCDs – Example target machines, goal is to gain root access.
• Hackerdemia – “The Hackerdemia Project is a LiveCD that provides both an instructional platform (in the form of a wiki) and an attack target to practice newly acquired skills.”
• pWnOS – Target machine created by a member of Heorot.net forums, Bond00.

The recent release of Metasploit Unleashed has provided a new excellent source of information for anyone looking to learn the ins and outs of the Metasploit framework. The material provides a guide for setting up two targets used throughout the courseware:

• An XP machine from NISTs FDCC project, with instructions for downgrading the security and running SQL Express
• A Ubuntu 7.04 machine running Samba

From my own experimentation I also run:

• Two XP machines (SP1 & SP2) – mainly used for malware analysis
• A Debian 4.0 victim -  for working with Linux exploits and shellcode
• BackTrack 4 – as an attack platform
• LiveCD – Used for running additional liveCDs in the lab that aren’t permanent residence, often Samurai or Helix (before it went commercial)

For most testing I will run only a handful of the above machines at any one time, just whatever is necessary for a particular scenario. However I am able to run all the above at the same time to test scanning and information gathering tools, nmap, Nessus, etc.

If you’re looking to develop information security skills and get hands on experience using the relevant tools and techniques I’d fully suggest reading through the links above. The amount and quality of freely available information is outstanding, and as my kit proves it doesn’t take great hardware to take advantage.

– Andrew Waite

<Update>If you’re running a Mac take a look at phenotyne’s post for getting similar environments working under Apple hardware</Update>

I’ve known about Vyatta for a while, but whilst the premise has always seemed appealing I’ve not had a reason to dig deeper. Vyatta propose to be ‘The open source alternative to Cisco’, which appeals as a nice fit into a low-cost training and development lab so tonight I decided to take a closer look.

I started by downloading Vyatta’s prebuilt VMware image, which can be downloaded here along with a Xen image and an ISO file for physical install. The VMware image is designed for workstation applications, but a quick run through my new friend in VMware Converter I quickly had the image transfered across to my ESXi based environment and booting without issue.

Vyatta provide a wealth of information in the documentation section (which requires registration, although it did not require the usual ‘activation’ email so dummy values may be enough). I haven’t had a chance to delve fully into the documentation and functionality but starting out has so far been simple enough: Logging onto the Vyatta device at the command-line requires the default user credentials of vyatta/vyatta. Once logged in you can start the configuration by entering ‘configure’

Once in configuration mode setting up interfaces is simple enough:

vyatta@vyatta# set interfaces ethernet eth0 description “WAN”

vyatta@vyatta# set interfaces ethernet eth0 address 192.168.1.254/24

vyatta@vyatta# save

vyatta@vyatta# commit

vyatta@vyatta# exit

Configuring different parts of the system are similarly simple, and with a bit of experience theVyatta systems seems intuitive enough and from basic testing performance is more that adequate, at least for my requirements. The time I’ve spent getting to grips with a new system has paid of so far, and for the time being I have a nice new addition to my lab environment. I’m hoping this system can provide some seperation between between between target/test systems and provide additional realism t my lab.

– Andrew Waite

For anyone that has had to migrate machines to a virtual environment VMware’s Converter will likely be nothing new. It allows a straight forward way to migrate an existing server (both physical and most common virtual environments) to VMware’s Infrastracture, Server or Workstation product suites.

Whilst this is hugely useful in a real-world environment for p2v or v2v migration strategies it doesn’t have too much use in a lab environment as you would typically build your environment and servers once and then test away. But I’ve recently found another use, with a few simple clicks I can now easily transfer a virtual server/servers from my ESXi lab environment to my laptop to continue working away from the office, and without the need for maintaining parallel victim machines within each of my virtual environments.

The transfer process does take some time, image below shows the start of the transfer of a 20GB machine from my laptop to ESXi server of local 100Mbps network. However don’t be too put off initially, original estimated run time is nearly four hours, when in actuality it completed in a little over one. Good for fire and forget transfers whilst you make dinner.

VMware converter

Some people I’ve discussed the tool with have anecdotal stories of having issues and failures with VMware Converter, I haven’t encountered any problems with my usage but your mileage may vary depending on scenario. At the very least is should be simpler than my previous method utilising DD.

– Andrew Waite

I was recently asked by Don over at EH-Net if I would be interested in reviewing a new book by Thomas Wilhelm of Heorot.net: ‘Professional Penetration Testing: Creating and operating a formal hacking lab’. Naturally I jumped at the opportunity.

I don’t want to discuss the book in too much detail here, as you can read the full review at Ethical Hacker here, but the book is a great addition to my home library. Don also worked his magic to convince the publisher to release a chapter from the book free of charge, chapter four covers the initial setup and configuration of hack lab environment, and can be downloaded from the review.

Hope the review is of use to someone out there, thanks to Thomas for writing the book in the first place and to Don for hooking me up with the review.

– Andrew Waite

As part of an upcoming project I’ve been playing with some screen capture and editing software. As I’ve never been one for for the graphical/fluffy side of IT it’s a new area for me, and I was shocked with how simple it can be.

For screen capture I used the free CamStudio application, at first try it seems small, lightweight and most importantly simple and intuitive to use.

Finding decent editing software for free was difficult, @usedtire suggested Cinelerra for Linux. From the site it looks to be an impressive application, but I’ll admit I found no easy way to get this running under Debian/Ubuntu and ended up in dependency hell, so I installed Windows Movie Maker thanks to the links/instructions I found here.

Whilst experimenting with my new found tools I’ve created the somewhat obligitory Metasploit tutorials:

• Creating a binary meterpreter payload
• Setting up payload/multi/handler to accept payloads

– Andrew Waite

<update-20091129>
Very grateful to Timmedin for pointing me in the direction of his recent work with the same issue. In usual form, Tim has even packaged up a powershell script to automate the workaround. Check his fix here, much cleaner and slicker than my own. If your still curious, read on for the backstory.
</update>

Since rebuilding part of my toolkit I’ve had issues connecting to my ESXi host server. I had originally thought this was a result of an upgrade from ESXi 3.5 to ESXi 4.0, and the resultant change from VMWare infrastructure client to the new vSphere client. After several hours and days fighting down a blind alley I found a forum post that highlighted Windows 7 as the culprit.

Further reading indicated that this is a widespread issue with no real solution. Best workaround appears to be to run the client within a sandbox via Microsoft’s Virtual XP environment for Windows 7.

After a couple of false starts the install process was fairly straightforward, found here. Simply select hardware architecture (32/64-bit), install a patch, then finally the Virtual XP image. Everything beyond this works as expected, a virtual XP machine. Once in the virtual environment install the vSphere client as normal to regain access your VMWare environment.

vSphere via virtualXP

Knowing my preferences, observant readers may be wondering why not achieve the same results using a VMWare guest with the vSphere client installed. VMWare Server is already installed on my machine, and was one of my initial thoughts. However, Virtual XP and VMWare utilise virtualisation for different results. The Virtual XP client has several intergration features (can be disabled if prefered) that allow simple, native access of resources on the host machine (files, directories, peripherals etc) from within the guest. This makes working with either, and between, host and guess seamless. Obviously such intergration would be unsuitable for a lab environment as you want/need isolation, seperation and protection from the guest machines so VMWare still has it’s place. As usual, using the right tool for the right job is essential.

At this point I’m back in my lab, and the R&D rolls on, but the experience has led me to look more indepth and Virtual PC. I have started building a BackTrack4 guest with Virtual PC to run within my standard machine for everyday use. Having access to a Linux environment as simply as a double-click as per normal applications will hopefully be a nice addition to my usual working practice.

<UPDATE> BT4 works fine, but the X GUI fails to start. Guess I’ll need to polish up on my commandline kung fu </UPDATE>

– Andrew Waite

Zero Wine is:

an open source (GPL v2) research project to dynamically analyze the behavior of malware. Zero wine just runs the malware using WINE in a safe virtual sandbox (in an isolated environment) collecting information about the APIs called by the program.

The output generated by wine (using the debug environment variable WINEDEBUG) are the API calls used by the malware (and the values used by it, of course). With this information, analyzing malware’s behavior turns out to be very easy.

Install was fairly simple as ZeroWine is distributed as a Qemu virtual image. Qemu, is downloaded here, and ZeroWine here.

To start the ZeroWine image I use the command (change filepaths to suit your install):

>qemu.exe c:\zerowine_vm\zerowine.img -no-kqemu -L . -redir tcp:8000::8000

Once running you can access the service by pointing a browser to localhost/8000 (the ‘-redir tcp:8000::8000′ parameter redirects the ZeroWine image’s port to your local system). This provides a simple web interface to upload and analyse your malware sample:

For a test run I uploaded the most recent sample collected by my Nepenthes honeypot, MD5 hash 3c9563dacd9afe8f2dbbe86d5d0d4c5e. The report generated shows the results of ZeroWines analysis, example below:The first section shows the behavioural analysis of the malware, this should be the most useful aspect of the ZeroWine framework. However as the ZeroWine page itself states, the output is ‘very long and, as so, hard to understand‘ and is unable to distinguish between system calls made by the malware and the underlying analysis framework. As a result I personally find the information provided by the report less useful than it could be.

There are definitely better sources for generating automated analysis of malware samples, for example VirusTotal or CWSandbox. However, depending on how the malware sample was obtained legal or business requirements may prevent you from releasing the sample to a third party, and not all provided services can provide the immediate response of a local system; meaning ZeroWine can still be a valid and useful tool in your arsenal.

Taking the concept forward, Jim Clausing recently released an excellent paper on setting up an automated malware environment with open source tools. I haven’t had a chance to try out any of Jim’s suggestions, but have read the paper and listened to the related podcast and the recommendations are definitely on my todo list to improve my malware analysis toolkit.

– Andrew Waite