It’s a while since I’ve found time to add a new tool to my malware environment, so when a ISC post highlighted a new update to Cuckoo sandbox it served as a good reminder that I hadn’t got around to trying Cuckoo, something that has now changed. For those that don’t know, from it’s own site:
[...] Cuckoo Sandbox is a malware analysis system.
Its goal is to provide you a way to automatically analyze files and collect comprehensive results describing and outlining what such files do while executed inside an isolated environment.
It’s mostly used to analyze Windows executables, DLL files, PDF documents, Office documents, PHP scripts, Python scripts, Internet URLs and almost anything else you can imagine.
Considering Cuckoo is the combined product of several tools, mostly focused around VirtualBox, I found install and setup was largely trouble free, mostly thanks to the detailed installation instructions from the tools online documentation. I only encountered a couple of snags.
[2011-12-29 17:21:56,470] [Core.Init] INFO: Started.
[2011-12-29 17:21:56,686] [VirtualMachine.Check] INFO: Your VirtualBox version is: “4.1.2_Ubuntu”, good!
[2011-12-29 17:21:56,688] [Core.Init] INFO: Populating virtual machines pool…
[2011-12-29 17:21:56,703] [VirtualMachine] ERROR: Virtual machine “cuckoo1″ not found: 0x80bb0001 (Could not find a registered machine named ‘cuckoo1′)
[2011-12-29 17:21:56,704] [VirtualMachine.Infos] ERROR: No virtual machine handle.
[2011-12-29 17:21:56,705] [Core.Init] CRITICAL: None of the virtual machines are available. Please review the errors.
The online documentation specifies creating a dedicated user for the cuckoo process. Sound advice, but if you create your virtual guest machines under a different user (like I did, under a standard user account), then the cuckoo process cannot interact with the virtualbox guests. Either changing ownership of cuckoo, or specifically creating the guest VMs as the cuckoo user will solve the issue.
Last problem encountered was Cuckoo’s database, which if it doesn’t exist when the process will create a blank database. Which (obviously, in hindsight) will fail if the running user doesn’t have permissions to write to Cuckoo’s base directory.
With problems out of the way, Cuckoo runs quite nicely, with three main parts. the cuckoo.py script does the bulk of the heavy lifting and needs to be running before doing anything else. If all is well it should run through some initialisation and wait for further instructions:
/opt/cuckoo $ ./cuckoo.py
____ _ _ ____| | _ ___ ___
/ ___) | | |/ ___) |_/ ) _ \ / _ \
( (___| |_| ( (___| _ ( |_| | |_| |
\____)____/ \____)_| \_)___/ \___/ v0.3.1
Copyright (C) 2010-2011
[2011-12-29 20:27:17,120] [Core.Init] INFO: Started.
[2011-12-29 20:27:17,719] [VirtualMachine.Check] INFO: Your VirtualBox version is: “4.1.2_Ubuntu”, good!
[2011-12-29 20:27:17,720] [Core.Init] INFO: Populating virtual machines pool…
[2011-12-29 20:27:17,779] [VirtualMachine.Infos] INFO: Virtual machine “cuckoo1″ information:
[2011-12-29 20:27:17,780] [VirtualMachine.Infos] INFO: \_| Name: cuckoo1
[2011-12-29 20:27:17,781] [VirtualMachine.Infos] INFO: | ID: 9a9dddd8-f7d6-40ea-aed3-9a0dc0f30e79
[2011-12-29 20:27:17,782] [VirtualMachine.Infos] INFO: | CPU Count: 1 Core/s
[2011-12-29 20:27:17,783] [VirtualMachine.Infos] INFO: | Memory Size: 512 MB
[2011-12-29 20:27:17,783] [VirtualMachine.Infos] INFO: | VRAM Size: 16 MB
[2011-12-29 20:27:17,784] [VirtualMachine.Infos] INFO: | State: Saved
[2011-12-29 20:27:17,785] [VirtualMachine.Infos] INFO: | Current Snapshot: “cuckoo1_base”
[2011-12-29 20:27:17,785] [VirtualMachine.Infos] INFO: | MAC Address: 08:00:27:BD:9C:4F
[2011-12-29 20:27:17,786] [Core.Init] INFO: 1 virtual machine/s added to pool.
The submit.py script is one of the ways for getting cuckoo to analysis files:
python submit.py –help
Usage: submit.py [options] filepath
-h, –help show this help message and exit
-t TIMEOUT, –timeout=TIMEOUT Specify analysis execution time limit
-p PACKAGE, –package=PACKAGE Specify custom analysis package name
-r PRIORITY, –priority=PRIORITY Specify an analysis priority expressed in integer
-c CUSTOM, –custom=CUSTOM Specify any custom value to be passed to postprocessing
-d, –download Specify if the target is an URL to be downloaded
-u, –url Specify if the target is an URL to be analyzed
-m MACHINE, –machine=MACHINE Specify a virtual machine you want to specifically use for this analysis
Most of the options above are self-explanatory, just make sure to select the relevant analysis package depending on what you’re working with; possibilities are listed here.
Finally, web.py provides a web interface for reviewing the results of all analysis performed by cuckoo, bound to localhost:8080.
I’d like to thank the team that developed and continue to develop the cuckoo sandbox. I look forward to getting more automated results going forward and hopefully getting to a point where I’m able to add back to the project; until then I’d recommend getting your hands dirty, from my initial experiments I doubt you’ll be disappointed. But if you won’t take my word for it, watch Cuckoo in action analysing Zeus here.
– Andrew Waite
Like most techies I get the job of fixing and maintaining relatives’ PCs. As part of this after fixing whatever is broken I have some common clean-up and install routines that I go through to both help the system run faster and to extend the period before I’m called back, and I’ve used AVG free as part of this for many years to keep costs down for my users.
During a recent job I came across a new (I’m assuming, hadn’t noticed it before) feature of AVG free, the PC Analyzer component. Being the curious sort I hit the go button, scan ran for around 5 minutes and I was presented with this:
Ouch, I was surprised with the number of errors as this is a machine I keep a regular eye on, and in some cases use myself (it’s the missus’). Time to panic? Let’s see:
- Registry errors: Errors affect system stability: (125)
That doesn’t sound good, checking the ‘Details…’ link presented me with a long list Registry keys, which to a standard end-user would result in turning on BofH’s Dummy Mode. In reality, it found a lot of keys to set the ‘open with’ right-click function depending on file extension. ‘Affect system stability’? Not so much, and I find the links useful enough that I’ve previously researched how to add my own…
- Junk Files: These files take up disk space: (599)
Again checking the details, long list of randomly named files. In the temporary folder. All ~600 took a total of less the 300MB, and the machine has more the 200GB free. Something to correct come next house cleaning session, but not really a problem.
- Fragmentation: Reduces disk access speed
In fairness to the tool, it did come back clean and we know that fragmentation can be an issue. But that’s why every machine I’ve ever used has come with a defrag utility, as standard, for free. (OK, my BBC Micro B didn’t, but then it also had a cassette deck rather than a hard disk).
- Broken Shortcuts: Reduces explorer browsing speed(42)
Ok, so I forget a folder of shortcuts to junk that came pre-installed with the system. I’d deleted the junk, forgot the shortcuts. Thanks for the reminder, fixed.
Plenty of ‘problems’ highlighted, time to run out and drop £25 for an annual subscription to the clean-up tool? Nope, ignoring the fact that many of these issues are system settings that actually aid the end user, the remaining issues won’t have any negative impact that the end-user will notice.
In my own opinion, AVG is taking a leaf out of the fake AV scams and scaring non-techies into parting with their hard earned coin in a bid to keep the computer running and bank details away from the scary hackers that the nice lady on the news keeps taking about. Presenting a list of meaningless (to most) information and saying it’s bad is exactly the tactic I encountered with cold call scammers earlier in the year.
As a final side note, I’ve lost two of my ‘users’ this year to AVG simply because when the AVG free license I’d installed expired, they couldn’t find a link to download the latest free version, only MANY links to the paid version. As my users are nice people (latest ‘victim’ was my grandfather), they decided themselves that it was better for them to pay the small fee than have to call me and interrupt my life.
Can anyone recommend a free AV suite that doesn’t con the unwitting into unnecessary purchases to perform a cleanup that could be performed manually with around 5 minutes and half a clue? AVG Free is a great tool, and for free I shouldn’t really complain, but when the sales tactics change to make money selling things people don’t need, to those that don’t know any better?
<update>Further investigation has shown that data has been restored, but the tree structure isn’t perfect. Use at own risk</update>
Anyone who’s taken Offensive Security training should be familiar with KeepNote (similar to Leo, for those that took early versions of the courses). If you’re not familiar with KeepNote it does exactly what you’d expect from the name, provide a handy way to keep and organise information. And it does a good job of this, until….
Traceback (most recent call last): File "/usr/local/lib/python2.6/dist-packages/keepnote/gui/__init__.py", line 469, in open_notebook version = notebooklib.get_notebook_version(filename) File "/usr/local/lib/python2.6/dist-packages/keepnote/notebook/__init__.py", line 248, in get_notebook_version raise NoteBookError(_("Notebook preference data is corrupt"), e) NoteBookError: SyntaxError('junk after document element: line 11, column 0',) Notebook preference data is corrupt root@bt:~/pwbv3/labnotes#
After much searching I found several posts discussing similar issues but following the same resolutions did not resolve by problems. With this I resorted to my fallback plan, create new notebook and begin to repopulate with my content (each node is stored as a plain text file, so I was looking at lots of cut and paste). Then a thunderbolt hit me.
I copied each branch of the original note tree to the new tree, and hay-presto! functioning notebook retained and disaster averted.
cp -r ~/old-notebook/branch/ ~/new-notebook/.
Of course a better solution is just to hit the ‘File>Backup Notebook’ option occasionally.
Towards the end of last year I spent a few hours trialling SSH tunnels, I knew how the process worked but hadn’t had much cause to use it in anger; so my lab got some use instead, and a post was written covering the basics; SSH port forwarding 101.
Since I now know how to quickly and successfully implement a tunnel, it turns out that I previously had plenty of cause to use tunnels in the past, I just didn’t know SSH tunnels were the right tool for the job. A couple of recent conversations has made me realise others don’t always know the flexibility of tunnels either so I wanted to try and describe a common scenario to highlight the usefulness of tunnels.
Above is a fairly common setup. You’ve got an internal resource (for example an intranet wiki for documentation), this is in turn protected by a firewall that only allows access from trusted location. Under normal circumstances all staff can access the resource without problems, and any malicious sources (human or automated) can’t access the service.
This works well, until someone needs access and they aren’t at one of the trusted locations (we’re assuming this is an unusual problem and remote access solutions aren’t in place). In a lot of environments SSH is a ‘trusted’ system management solution and is world accessible (and hopefully secured well enough to keep the barbarians from the door, but that’s for different posts).
SSH tunnels (but you guessed that). Tunnel the server’s HTTP (or whatever) service back to your local system, and then connect locally. Using the syntax I discussed previously, from a ‘nix shell you can use this command:
ssh -L 8000:127.0.0.1:80 ssh-server.domain.com
This makes an SSH connection to the server (ssh-server.domain.com), tunnelling the local HTTP service running on port 80 (127.0.0.1:80) and binds it to your machines TCP 8000 port. Now you can connect to the service by typing 127.0.0.1:8000 into a browser, thus traversing the firewall source IP restrictions.
If you’re living in a Windows world, then the PuTTY equivalent configuration will be:
Next time you’re sat in the coffee shop on a Sunday morning, and the boss rings with an ‘emergency'; are you sure that you can’t access the resources you need from where you are? If you can, that coffee (and extra slice of cake) just became expense-able ;)
For those that don’t already know, John the Ripper is:
a fast password cracker, currently available for many flavors of Unix, Windows, DOS, BeOS, and OpenVMS. Its primary purpose is to detect weak Unix passwords. Besides several crypt(3) password hash types most commonly found on various Unix systems, supported out of the box are Windows LM hashes, plus many more with contributed patches.
John is commonly available for most ‘nix systems under their package management systems. However, I’d strongly recommend compiling from source to ensure John takes full advantage of more powerful hardware capabilities than the ‘generic’ build. Compiling from source is straightforward, and I’m yet to encounter any difficulties. The additional time spent with manual compilation definitely pays for itself in the long run, read the INSTALL file for instructions.
To get a feel for the available performance john includes ‘–test’ functionality. My system (iMac i7 w/8GB ram), manually compiled with system specific functions achieved:
enchmarking: Traditional DES [128/128 BS SSE2-16]… DONE
Many salts: 3370K c/s real, 3370K c/s virtual
Only one salt: 2840K c/s real, 2840K c/s virtual
Benchmarking: BSDI DES (x725) [128/128 BS SSE2-16]… DONE
Many salts: 108990 c/s real, 110080 c/s virtual
Only one salt: 107264 c/s real, 107264 c/s virtual
Using the same system and john compiled with a ‘generic’ system I achieved less than 50% performance:
Benchmarking: Traditional DES [64/64 BS]… DONE
Many salts: 1613K c/s real, 1613K c/s virtual
Only one salt: 1476K c/s real, 1487K c/s virtual
Benchmarking: BSDI DES (x725) [64/64 BS]… DONE
Many salts: 52800 c/s real, 52377 c/s virtual
Only one salt: 51961 c/s real, 51961 c/s virtual
Unfortunately, john doesn’t currently take advantage of multi-core systems and only uses a single core (see below). There are methods for sharing the processing load between multiple instances of john, or even across john instances running on multiple systems, but that’s a topic for another day.
Before you can crack any passwords, you first need to ‘acquire’ the hashes to be cracked. There are many options for this including the pwdump family (I’d recommend fgdump), the Cain and Abel tools and many others. For testing purposes I extracted the Windows LM hashes using Metasploit’ Meterpreter’s hashdump functionality:
meterpreter > hashdump
John comes packaged with a default password list, containing a little over 3000 entries, and several methods for generating hybrid guesses including appending numbers to entry or performing ‘l33t’ replacements. Rules for hybrid attacks can found (and edited) in the john.conf file. Even without tweaking, john is still effective at cracking hashes as the below shows.
Wintermute:john-testing awaite$ time /Applications/john-1.7.6-jumbo-12/run/john 192-168-1-127_Andrew.dump
Loaded 2 password hashes with no different salts (LM DES [128/128 BS SSE2-16])
guesses: 2 time: 0:00:09:45 (3) c/s: 19803K trying: INFODTP – INFOSS9
Wintermute:john-testing awaite$ time /Applications/john-1.7.6-jumbo-12/run/john 192-168-1-127_Barney.dump
Loaded 2 password hashes with no different salts (LM DES [128/128 BS SSE2-16])
guesses: 2 time: 0:00:00:01 (3) c/s: 9547K trying: DINOSH8 – DINOVYS
Wintermute:john-testing awaite$ time /Applications/john-1.7.6-jumbo-12/run/john 192-168-1-127_Charlie.dump
Loaded 1 password hash (LM DES [128/128 BS SSE2-16])
guesses: 1 time: 0:00:28:38 (3) c/s: 20126K trying: CH4LB5# – CH4LCGN
Cracked hashes are stored in the john.pot file, preventing john from wasting CPU cycles cracking a hash that has already been cracked. Previously cracked passwords can be found using –show:
Wintermute:run awaite$ ./john –show /Users/awaite/Documents/john-testing/192-168-1-127.dump
5 password hashes cracked, 12 left
John has a lot of additional functionality; go install, tweak and crack (only with permission, obviously).
I’ve tried messing around with SSH port forwarding in the past, but always struggled to get my head around what I was trying to connect to where, and ultimately didn’t result in anything useful. This time around I’ve put in some dedicated time to get to the bottom forwarding ports within SSH tunnels. And I’m glad I did, my with only a handful of connections the possibilities are making my head spin.
To help my head get around problems I encountered I found a number of resources helpful.
Example Scenario: A server (10.0.0.100) on an internal LAN has web services that need to be accessed, but has no remote access through the firewall. Luckily, there is remote access to an SSH server (10.0.0.200) sat on the same LAN.
ssh -L 8000:10.0.0.100:80 ssh-server.somedomain.tld
This connects to the machine at ssh-server.somedomain.tld and, once authenticated, forwards 10.0.0.100:80 to port 8000 on the local machine. Now accessing the remote services is as simple as pointing your browser to http://127.0.0.1:8000.
The same functionality can be achieved using configuration files. For example, edit ~/.ssh/config:
LocalForward 127.0.0.1:8000 10.0.0.100:80
To establish a connection with the configuration file in place simply run; ssh tunnel.
The port forward will remain active for as long as the SSH session is connected. If you don’t need to interact with the SSH session in addition to the forwarded port passing ssh the -fN flags will cause the session to be backgrounded once authentication can be established.
If you haven’t already, I suggest you investigate the possibilities within your own environment; and if an evil grin doesn’t spread across your face then you still don’t fully get it ;)
Development of new features for Dionaea has been fairly impressive of late, and I’ve been lax in keeping up to date. When Markus asked if I’d tested the graph utility that he created and wrote about here, it served as a kick to stop putting off some of the jobs I’ve got on the growing to-do list.
I won’t go into too much detail about running the script as Markus has already done a better job than I could. However I will point out that if you run your Dionaea installation on Debian stable, then your out of luck; the standard packages for sqlite are too old to take the script. Best advice is to copy your logsql.sqlite database to a Ubuntu machine and work from there (oh, and in case you didn’t guess from the script name, make sure you’ve actually installed gnuplot…).
A powerful machine is recommended, the only Ubuntu system I had to hand whilst testing was my AA1 netbook, which took 85 minutes to crunch through the script and my database.
I have immediately found the graphs produced useful as they’ve highlighted a couple of obvious spikes (see below) in activity that I would have (and did) miss if solely relying on log files and databases. This really shows the power and importance of visualising security and log information.
If you’re interested the output for the InfoSanity’s installation is now online here. I’m looking to expand the statistics from the InfoSanity honeypot environment that are publicly available, this makes a nice start. As always, big thanks to Markus and carnivore.it team for the effort.
– Andrew Waite
<UPDATE>Live download mirror: carnivore.it</UPDATE>
Mercury Live DVD was initially (I believe) announced in a post to the Nepenthes Mailing list. It is a remastered Ubuntu distribution with pre-installed honeypot applications and malware analysis tools created by John Moore. From the ReadMe:
This live DVD is a remastered version of Ubuntu 10.0 Beta LTS x86_32. It was designed due to my being disappointed with another reverse engineering malware live CD that was released recently. I have decided to call my creation MERCURY, which is an acronym for Malware Enumeration, Capture, and Reverse Engineering.
The Mercury live DVD contains tools used for digital forensics, data recovery, network monitoring, and spoofing. It should primarily be used as a honeypot or network monitoring platform as well as a laboratory and teaching aid. There are three honeypots installed – honeyd, nepenthes, and dionaea. Four, if you include netcat.
The majority of the additional applications reside in /opt:
- Dionaea (0.1.0) – Dionaea is a malware collection honeypot focusing primarily on SMB emulation, covered on InfoSanity numerous times before.
- FFP – Fuzzy Fingerprinting is a util to aid SSH MitM attacks.
- Kippo (svn rev.169) – Kippo is an low-medium interaction SSH honeypot, Also covered
- mitm-ssh – Unsurprisingly, a utility for aiding man in the middle attacks against SSH connections.
- Origami & pdftools – Two frameworks for analysing malicious PDF files.
- Volatility – an excellent memory analysis toolkit
- Zerowine-vm – A malware behavior analysis platform. I’ve covered ZeroWine here before, and whilst I find it useful for initial analysis I found it a pain to setup and get running. The fact this works out of the box on Mercury is enough reason alone to keep the .iso handy.
Other tools are installed on the system as started, access from standard locations (/etc, /usr/bin, etc.). I won’t try to list them all, but some highlights include:
- Nepenthes – Dionaea’s predecessor
- Honeyd – Honeypot system, perfect for emulating multiple different systems from one platform. Covered in more depth here.
- John – John the Ripper, password cracker
- ircd-hybrid – irc server daemon, useful for analysis irc-based malware’s interaction with command and control systems.
- Snort – de-facto intrusion detection system.
- Wireshark – Packet capture and network analysis tools.
I could go on, but I’m sure you get the idea.
Setting up a honeypot, and analysing the results, has never been easier. And I’m sure the toolkit’s functionality will also be useful in other scenarios; incident response, general network administration or as a safe learning platform. So what are you waiting for?
N.B. there have been several mirror’s and downloads established, the most reliable download source I’ve used is Markus’ mirror at carnivore.it
This morning I cause myself a problem. Annoyingly it was foreseeable and avoidable, this is my excuse (not great, but I’ll stick to it). But as every problem is merely an opportunity in disguise whist I’m re-building systems I might as well document the process. The original InfoSanity guide for installing Kippo was based off of the latest stable version, but I rapidly migrated to the development SVN on learning of the MySQL logging capabilities, so this guide covers that.
As I’m using a Debian system a lot of the system pre-requisites are packaged, this aren’t all needed immediately but we might as well grab them all at once.
apt-get install subversion #for svn
apt-get install python-twisted python-mysqldb # Python and required modules
apt-get install mysql-server #
Basic Kippo setup
Grab Kippo direct from svn, at time of writing I got version 160. (latest instructions):
svn checkout http://kippo.googlecode.com/svn/trunk/ /opt/kippo-svn
Now we can start the honeypot system:
That’s it, all that is required to get the system running. To confirm you can ssh locally with ssh -p2222 firstname.lastname@example.org, unless you’ve jumped ahead and edited the config, password will be 123456.
Log into MySQL via commanline, assuming you’ve not modified the kippo.cfg database directives build the database:
create database kippo;
grant all on kippo.* to ‘kippo’@’localhost’ identified by ‘secret';
Next edit the kippo.cfg accordingly you database/user/password and uncomment the [database] configuration directives. REMEBER to uncomment ;[database] line not just the parameters, that has now caught me out twice.
Finally, build the database structure with the script that can be found in <kippo>/doc/sql/:
doc/sql/# mysql -ukippo -psecret kippo < mysql.sql
Restart your Kippo process and you should be good; re-test access to the shell and view the database tables to confirm that logs are being written to the database.
I’ve been a bit lax in writing this post; around a month ago Miguel Jacq got in contact to let me know about a couple of errors he encountered when running InfoSanity’s mimic-nepstats.py with a small data set. Basically if your log file did not include any submissions, or was for a period shorter than 24hours the script would crash out, not the biggest problem as most will be working with larger data sets but annoying non the less.
Not only did Miguel let me know about the issues, he was also gracious enough to provide a fix, the updated script can be found here. An example of the script in action is below:
cat /opt/dionaea/var/log/dionaea.log| python mimic-nepstats_v1-1.py
Statistics engine written by Andrew Waite – http://www.infosanity.co.uk
Number of submissions: 84
Number of unique samples: 39
Number of unique source IPs: 65
First sample seen: 2010-06-08 08:25:39.569003
Last sample seen: 2010-06-21 15:24:37.105594
System Uptime: 13 days, 6:58:57.536591
Average daily submissions: 6
Most recent submissions:
2010-06-21 15:24:37.105594, 18.104.22.168, emulate://, 56b8047f0f50238b62fa386ef109174e
2010-06-21 15:18:08.347568, 22.214.171.124, tftp://126.96.36.199/ssms.exe, fd28c5e1c38caa35bf5e1987e6167f4c
2010-06-21 15:17:08.391267, 188.8.131.52, tftp://184.108.40.206/ssms.exe, bb39f29fad85db12d9cf7195da0e1bfe
2010-06-21 06:29:03.565988, 220.127.116.11, tftp://18.104.22.168/ssms.exe, fd28c5e1c38caa35bf5e1987e6167f4c
2010-06-20 23:34:15.967299, 22.214.171.124, http://126.96.36.199/trying.exe, 094e2eae3644691711771699f4947536
– Andrew Waite