Working my way through the compilation instructions from Dionaea whilst building up my latest sensor I was reminded of some optional functionality that I’d always intended to implement, but never found the time. First on my list was p0f (that’s a zero).
From p0f’s homepage:
P0f v2 is a versatile passive OS fingerprinting tool. P0f can identify the operating system on:
– machines that connect to your box (SYN mode),
– machines you connect to (SYN+ACK mode),
– machine you cannot connect to (RST+ mode),
– machines whose communications you can observe.
P0f can also do many other tricks, and can detect or measure the following:
– firewall presence, NAT use (useful for policy enforcement),
– existence of a load balancer setup,
– the distance to the remote system and its uptime,
– other guy’s network hookup (DSL, OC3, avian carriers) and his ISP.
Setting p0f up on the sensor should have been straightforward;
- Install with: apt-get install p0f
- Run p0f as suggested in dionaea.conf: sudo p0f -i any -u root -Q /tmp/p0f.sock -q -l
- And edit dionaea.conf’s ihandler section to enable p0f
This mostly worked, watching the p0f output it was correctly (I’m assuming) providing stats about connecting systems. The problem was that p0f info wasn’t getting saved into Dionaea’s logsqlite database, dionaea-error.log was reporting the below error with each connection:
[04122010 13:48:44] connection connection.c:827-warning: Could not connect un:///tmp/p0f.sock:0 (Permission denied)
Which seemed odd, /tmp/p0f.sock was showing as globally readable. Re-reading the Dionaea compilation instructions I noticed a comment about p0f struggling with IPv6 so has problems it Dionaea is listening on ::, which mine was. Problem solved, I edited dionaea.conf so that Listen mode was set to “manual”, and provided the interface/IP details of my network connection. Only this didn’t solve my problem…
So, why go to the effort? Main purpose behind running honeypot systems (for me) is to get a better idea understanding of what threats are actively targeting systems in the wild. At first glance the information provided by p0f can quickly help evaluate the attacking system; what OS? what connection type? is it local?
With the limited (few hours) of data I’ve already collected heres a sample of the info you can gather:
last 5 connections:
|328||822||Windows||IPv6/IPIP||2000 SP4, XP SP1+||-1||17||0||0|
|327||821||Windows||IPv6/IPIP||2000 SP4, XP SP1+||-1||17||0||0|
|326||820||Windows||2000 SP4, XP SP1+||-1||14||0||0|
|325||819||Windows||2000 SP4, XP SP1+||-1||14||0||0|
Breakdown by OS
select count(p0f_genre) as count, p0f_genre as OS
group by p0f_genre
order by count(p0f_genre) desc;
Umm, so most systems spreading malware are (likely) infected Windows systems. No great surprise there…
|5||GPRS, T1, FreeS/WAN|
|3||PIX, SMC, sometimes wireless|
|2||sometimes DSL (2)|
|2||sometimes DSL (4)|
select count(p0f_link), p0f_link
group by p0f_link
order by count(p0f_link) desc;
Unfortunately the the information provided by p0f isn’t an exact science, and as devices and systems are constantly changing it’s only going to be as accurate as it’s latest signatures/fingerprints. But setup is fairly quick, and the information and insight provided fairly interesting. So why not give it a go?
Markus keeps adding great features and functionality to Dionaea, when I read the post introducing a new web interface carniwwwhore I couldn’t help thinking I’d got lucky timing, start of a week’s vacation and no real plan for what to do with it. I’ve struggled previously with some of my Dionaea setups, largely because my system was running Debian, whilst Dionaea was built under Ubuntu; doesn’t cause too many problems, just a bit of google-fu, headscratching and stupidity that could have been avoided.
From this background I looked through the carniwwwhore pre-reqs with dread, plenty of version requirements that weren’t upto date with my Debian setup; so it’s time to bite the bullet and build a fresh system with Ubuntu. Unlike some of my previous setups, installation/compilation worked flawless, working on the same distro as the lead dev definitely makes life easier. If you’re looking for a fresh Dionaea installation, go with Ubuntu, you won’t regret it.
(oh, and carniwwwhore? Vacation got the better of me so it’s added to the to-do list; watch this space…)
I’m sure this is basic for most of you, but I seem to keep making the same daft mistakes whilst updating Dionaea, so I’m hoping documenting the issues and corrections will work as a memory aid in the future.
Firstly, I can never remember Git’s equivalent to ‘svn update’, which is:
Next up is recompiling Dionaea from your updated source directory, this is no different to an initial install as per Markus’ excellent instructions.
I keep forgetting to update my configuration file to include any directives needed for the new shiny new functionality you’re upgrading to get access to. I find diff useful for identifying any new additions, for example
/opt/dionaea/etc/dionaea# diff dionaea.conf dionaea.conf.dist
Assuming there’s no major changes you should only see differences specific to your installation, for example your email address to receive analysis reports, or your VirusTotal api key. If there are any other differences you’ll need to add the new content.
From the experience I’ve had in the past week, if you encounter any unexpected problems after updating Dionaea, make sure your pre-requisites are also upto date. After updating Dionaea last week to gain access to the new integration with VirusTotal’s api my Dionaea sensor started to die randomly. Markus was a great help with troubleshooting (thanks again) and my problems were eventually corrected after it was noted that my libemu installation was outdated; after a quick ‘git pull’ and ‘make’ (again following Markus’ instructions).
As I said, this is probably basic for most of you out there, but as I keep making similar mistakes I plan to refer back to this list of daft issues before bugging anyone for support in future. You never know, it might allow someone else to retain an air of competence before proving otherwise 🙂
Development of new features for Dionaea has been fairly impressive of late, and I’ve been lax in keeping up to date. When Markus asked if I’d tested the graph utility that he created and wrote about here, it served as a kick to stop putting off some of the jobs I’ve got on the growing to-do list.
I won’t go into too much detail about running the script as Markus has already done a better job than I could. However I will point out that if you run your Dionaea installation on Debian stable, then your out of luck; the standard packages for sqlite are too old to take the script. Best advice is to copy your logsql.sqlite database to a Ubuntu machine and work from there (oh, and in case you didn’t guess from the script name, make sure you’ve actually installed gnuplot…).
A powerful machine is recommended, the only Ubuntu system I had to hand whilst testing was my AA1 netbook, which took 85 minutes to crunch through the script and my database.
I have immediately found the graphs produced useful as they’ve highlighted a couple of obvious spikes (see below) in activity that I would have (and did) miss if solely relying on log files and databases. This really shows the power and importance of visualising security and log information.
If you’re interested the output for the InfoSanity’s installation is now online here. I’m looking to expand the statistics from the InfoSanity honeypot environment that are publicly available, this makes a nice start. As always, big thanks to Markus and carnivore.it team for the effort.
— Andrew Waite
<UPDATE>Live download mirror: carnivore.it</UPDATE>
Mercury Live DVD was initially (I believe) announced in a post to the Nepenthes Mailing list. It is a remastered Ubuntu distribution with pre-installed honeypot applications and malware analysis tools created by John Moore. From the ReadMe:
This live DVD is a remastered version of Ubuntu 10.0 Beta LTS x86_32. It was designed due to my being disappointed with another reverse engineering malware live CD that was released recently. I have decided to call my creation MERCURY, which is an acronym for Malware Enumeration, Capture, and Reverse Engineering.
The Mercury live DVD contains tools used for digital forensics, data recovery, network monitoring, and spoofing. It should primarily be used as a honeypot or network monitoring platform as well as a laboratory and teaching aid. There are three honeypots installed – honeyd, nepenthes, and dionaea. Four, if you include netcat.
The majority of the additional applications reside in /opt:
- Dionaea (0.1.0) – Dionaea is a malware collection honeypot focusing primarily on SMB emulation, covered on InfoSanity numerous times before.
- FFP – Fuzzy Fingerprinting is a util to aid SSH MitM attacks.
- Kippo (svn rev.169) – Kippo is an low-medium interaction SSH honeypot, Also covered
- mitm-ssh – Unsurprisingly, a utility for aiding man in the middle attacks against SSH connections.
- Origami & pdftools – Two frameworks for analysing malicious PDF files.
- Volatility – an excellent memory analysis toolkit
- Zerowine-vm – A malware behavior analysis platform. I’ve covered ZeroWine here before, and whilst I find it useful for initial analysis I found it a pain to setup and get running. The fact this works out of the box on Mercury is enough reason alone to keep the .iso handy.
Other tools are installed on the system as started, access from standard locations (/etc, /usr/bin, etc.). I won’t try to list them all, but some highlights include:
- Nepenthes – Dionaea’s predecessor
- Honeyd – Honeypot system, perfect for emulating multiple different systems from one platform. Covered in more depth here.
- John – John the Ripper, password cracker
- ircd-hybrid – irc server daemon, useful for analysis irc-based malware’s interaction with command and control systems.
- Snort – de-facto intrusion detection system.
- Wireshark – Packet capture and network analysis tools.
I could go on, but I’m sure you get the idea.
Setting up a honeypot, and analysing the results, has never been easier. And I’m sure the toolkit’s functionality will also be useful in other scenarios; incident response, general network administration or as a safe learning platform. So what are you waiting for?
N.B. there have been several mirror’s and downloads established, the most reliable download source I’ve used is Markus’ mirror at carnivore.it
I’ve been a bit lax in writing this post; around a month ago Miguel Jacq got in contact to let me know about a couple of errors he encountered when running InfoSanity’s mimic-nepstats.py with a small data set. Basically if your log file did not include any submissions, or was for a period shorter than 24hours the script would crash out, not the biggest problem as most will be working with larger data sets but annoying non the less.
Not only did Miguel let me know about the issues, he was also gracious enough to provide a fix, the updated script can be found here. An example of the script in action is below:
cat /opt/dionaea/var/log/dionaea.log| python mimic-nepstats_v1-1.py
Statistics engine written by Andrew Waite – http://www.infosanity.co.uk
Number of submissions: 84
Number of unique samples: 39
Number of unique source IPs: 65
First sample seen: 2010-06-08 08:25:39.569003
Last sample seen: 2010-06-21 15:24:37.105594
System Uptime: 13 days, 6:58:57.536591
Average daily submissions: 6
Most recent submissions:
2010-06-21 15:24:37.105594, 18.104.22.168, emulate://, 56b8047f0f50238b62fa386ef109174e
2010-06-21 15:18:08.347568, 22.214.171.124, tftp://126.96.36.199/ssms.exe, fd28c5e1c38caa35bf5e1987e6167f4c
2010-06-21 15:17:08.391267, 188.8.131.52, tftp://184.108.40.206/ssms.exe, bb39f29fad85db12d9cf7195da0e1bfe
2010-06-21 06:29:03.565988, 220.127.116.11, tftp://18.104.22.168/ssms.exe, fd28c5e1c38caa35bf5e1987e6167f4c
2010-06-20 23:34:15.967299, 22.214.171.124, http://126.96.36.199/trying.exe, 094e2eae3644691711771699f4947536
— Andrew Waite