Infosanity's Blog

I was recently asked about the network configuration I use for my honeyd sensor. I had thought I’d already written about this so initially went to find the article on honeyd configuration; but my memory was wrong and the original post only covered configuring the guest systems, not the honeyd host itself. So, as I now have a pretty(ish) network diagram showing my setup I may as well correct the earlier omission.

<DISCLAIMER: This may not be the best network design for running honeyd, this is merely how my environment is configured and it works for me as a research platform. As usual, your mileage may vary, especially if your use-case differs from my own>

As can be seen, the design has three distinct network segments:

• Publicly route-able IPs
• Internal network for honeypot hosts
• Virtual network for honeyd guest systems. These IP addresses sit on loopback interface on the host, with a static route on the firewall to pass all virtual traffic to the honeyd host.

Using a perimeter firewall with NAT/PAT capabilities allows easy switching between emulated systems and services if your public IP resources are limited; a large network of guests can be configured in advance and left static, then a quick firewall change is all that is required to expose different systems to the world.

Additionally, as much as honeypot systems are designed to be compromised and collect information of malicious attacks (or perhaps more correctly, because of this) , low-interaction systems like honeyd is designed to avoid full compromise. If something goes wrong and the host system gets fully compromised, a (sufficiently configured) perimeter firewall provides some control of outgoing traffic, limiting the attackers options for using the honeypot sensor to attack other systems.

Not much to it really; if you use an different setup and/or can suggest ways to improve the setup let me know, always looking to improve my systems where possible.

– Andrew Waite

On Friday I arrived home looking forward to a well-earned rest; unfortunately Dave Kennedy seemed to have other ideas for my weekend as he announced the alpha release of a new honeypot, Artillery.

Artillery is a combination of a honeypot, file monitoring and integrity, alerting, and brute force prevention tool. It’s extremely light weight, has multiple different methods for detecting specific attacks and eventually will also notify you of insecure nix configurations.

Installation of Artillery is currently really simple, download via svn, run the installer script, edit the config file (if necessary) and run:

$svn co http://svn.secmaniac.com/artillery artillery/

$./installer.py

$nano config

$./artillery.py

N.B. don’t make the same daft error I made initially by editing the files in the svn download. Once the installer.py script has been run, cd to /var/artillery.

Artillery goes beyond typical honeypots, as it actively blocks remote clients and protects the system it’s running on. Artillery listens on a number of common ports (configurable, look at the PORTS variable), if it receives a connection on any of the fake ports it permanently blocks the source IP address by adding a DROP rule to iptables.

From my experience Artillery gets results REALLY quickly. After getting the system online I performed a quick test from another host under my control and starting writing up this post; in the time it’s taken to write the content above Artillery has already added 8 addresses  to iptables:

Chain INPUT (policy ACCEPT)
target     prot opt source                                destination
DROP       all  --  host-31-42-163-53.pois.com.ua         anywhere
DROP       all  --  net242.187.188-2.oren.ertelecom.ru    anywhere
DROP       all  --  94-21-36-156.pool.digikabel.hu        anywhere
DROP       all  --  89.122.216.109                        anywhere
DROP       all  --  ras.beamtele.net                      anywhere
DROP       all  --  dsl5401A8C9.pool.t-online.hu          anywhere
DROP       all  --  catv-178-48-151-67.catv.broadband.hu  anywhere
DROP       all  --  176.14.205.91                         anywhere

Other functionality included in Artillery mirrors that of Tripwire, monitoring the contents of different directories (again, configurable) and generating alerts if the contents of the directories and files changes.

I really like the premise of Artillery, and Dave in his usual fashion is coding like a madman adding fixes and new functionality (new version, 0.1.1 was released 24hrs after initial announcement). I’d be wary where you set this system up to test it though due to the automatic lockout; if Artillery is on a remote system, and you connect to a dummy port from your location to test you’ve just been locked out of your own server ;)

Looking forward to seeing Artillery mature, thanks Dave.

–Andrew Waite

Very quick post to highlight a process for clearing all entries from your pass.db file. I had thought that this would take a bit of quick scripting utilising the list and remove modes from the passdb.py utility script. Turns out it’s even easier; simply pass a non-existent file to passdb.pyass.db parameter and it’ll create a fresh database file, no questions asked.

For example:

/opt/kippo/utils/passdb.py /opt/kippo/data/idontexistyet.db add testpassword

Will create a new database file with a single entry of ‘testpassword’ (this can be removed once you’ve established everything works). N.B. even with a blank pass.db, kippo will provide access to the password(s) configured in kippo.cfg.

Not sure yet if clearing the database will net any interesting results, only time will tell…..

–Andrew Waite

After a few weeks running my daily Kippo review script I’ve noticed that whilst I’m still mostly receiving several logins per day, it’s rare for a connection to actually interact with my emulated system. (For those new here, Kippo is a medium interaction honeypot emulating an SSH daemon, get started here). So I started trying to investigate what was causing the trend.

One of Kippo’s features is the password database. Basically once an intruder gains access to the shell if they try to change the password or add a different account the system adds the password to the list of allowed. This then allows connections to log into the shell with the new password. Kippo ships with a small utility script to interact with the password database:

@kippo01:/opt/kippo-svn/utils$ ./passdb.py
Usage: passdb.py <pass.db> <add|remove|list> [password]

My pass.db file contains 26 entries added by malicious ‘users’; I’m still analysing the contents in detail, but it looks like the Bad Guys(tm) are paying attention to user education 101 and using long, complex passwords.

Using the password used to log into the system, I’ve had a new (to me) way to link disparate logins. For example the query below linked connections spanning two months, originating from multiple source IP address, across three different continents (according to WHOIS records).

Source IPs for same user (based on pass)

SELECT sessions.id AS Session, sessions.ip AS Source, auth.password AS Password, auth.timestamp AS Time
FROM sessions, auth
WHERE
sessions.id = auth.session
AND auth.success = 1
AND auth.password = 'mariusbogdan';

Similarly I looked for a connection between multiple successful logins from the same source IP address. The query below provided a list of report offenders:

Successful logins from same source

SELECT COUNT(sessions.ip) AS Num, sessions.ip AS Source
FROM sessions, auth
WHERE
auth.success = 1
AND auth.session = sessions.id
GROUP BY sessions.ip
ORDER BY COUNT(sessions.ip) desc
LIMIT 25;

My summary from this is that Kippo is receiving a lower level of ‘interesting’ connections the longer the system is operational, as attackers login to check if they’ve maintained access to an ’0wned’ resource, without utilising the resource. I’m intending to clear my pass.db to remove existing access; hopefully this will return to more interesting connections and I’m also curious to see if any of my current tenants return from either the same source location(s) and/or re-using passwords (and proving me wrong with previous comment about user education).

–Andrew Waite

When I first started running Kippo almost a year ago I had no difficulty getting motivated to log into the honeypot, check for new connections and generally get a feel for what my victims visitors have been up to. As time went by, sessions started to follow familiar patterns and some days would get no hits. Slowly I’d check the logs less frequently, and when I did I’d get an ever increasing backlog to review, decreasing my motivation further.

Recently I got annoyed with myself, my system was ticking along in the background but I was gaining no benefit from it. So in a moment of madness I dusted off my bash and built a quick script to provide a daily review of activity on my system. Essentially this does two things, lists session interaction and files downloaded within the last 24hours.

I’ve had the routine running daily for around a week; for days there was minmal activity on my system, either no logins at all, or logins with immediate disconnects. Today was different, and marked the first success of the script. Delivered to my morning inbox, along with the rest of my regular quick tasks and RSS feed as an interesting session. Malicious user connects, downloads a scanner (archive contents looks like gosh), an irc bot (looks like EnergyMech derivative); and when attempts to run toolkit fail, downloads and runs three (yes, three, paranoia is strong with this one) log cleaners.

Example (snipped) output:

:~$ /opt/kippo-svn/kippo-sessions.sh
***Sessions***
---START:/opt/kippo-svn/log/tty/20110519-220029-5503.log---
www-dev:~# w
 22:00:38 up 14 days,  3:53,  1 user,  load average: 0.08, 0.02, 0.01
USER     TTY      FROM              LOGIN@   IDLE   JCPU   PCPU WHAT
root     pts/0    77.210.18.212     22:00    0.00s  0.00s  0.00s w
<SNIP>
***DOWNLOADS***
/opt/kippo-svn/dl/20110519220445_http___eduteam_home_ro_mech_gz: gzip compressed data, from Unix, last modified: Sun Oct  4 17:46:52 2009
<SNIP>

The script can be downloaded here, as usual it’s released under the Beerware License

–Andrew Waite

Running through my morning routine of catching up with email, twitter, etc. I came across this post showing Sequal7′s first hits on a Kippo installation. In addition to making amusing reading, it gave me a nudge to check back on the InfoSanity Kippo sensor. Initially I was looking to see if the same individual had stumbled across my sensor; they hadn’t at least not from the information I have available.

However, when checking if the newly changed password matched anything in my database I found a new ‘realm’ entry in the ‘input’ table, ‘ssh’. This got me curious, one of my ‘guests’ decided to hit another system whilst logged in to mine; ssh’ing to another IP, accepting the certificate and providing the password to said system (I’m assuming).

It should also be worth noting that by this point the user had already failed to notice that input hadn’t returned to their own system. After (attempting) to change my sensor’s root password (to ‘yahoo’, really) the user exited, but was caught out by Kippo’s trick of clearing the terminal and changing prompt to ‘localhost’, in total I viewed a ~20 minute terminal session of the user trying to compromise other systems, and failing in the same manner.

My assumption is that the user was running through a list of vulnerable systems identified by SSH scanners similar to the kit I wrote about earlier (it wasn’t the same gosh.tgz kit, but first glance shows similar functionality). From this I feel it’s safe to assume that the systems connected to in the logs available are those of other (probably 0wned) systems, rather than anything connected to my guest. Likewise it is probable that the source connection is also a compromised third party rather than belonging to by guest.

e.tgz :
For the curious, archive contents:

e/
e/exp_moosecox.c
e/funny.jpg
e/exp_powerglove.so
e/exp_ingom0wnar.c
e/pwnkernel.c
e/exp_cheddarbay.so
e/exp_powerglove.c
e/exp_ingom0wnar.so
e/exp_therebel.so
e/run_nonnull_exploits.sh
e/exp_paokara.so
e/exp_framework.h
e/exp_moosecox.so
e/exp_wunderbar.c
e/exp_cheddarbay.c
e/run_null_exploits.sh
e/exploit.c
e/exp_therebel.c
e/exp_vmware.so
e/exp_vmware.c
e/exp_wunderbar.so
e/exploit
e/exp_paokara.c

Whilst investigating the individual exploits and files; I came across this post, indicating ‘my’ archive is a known fire and forget post exploit kit. Here be Skiddies…

–Andrew Waite

Working my way through the compilation instructions from Dionaea whilst building up my latest sensor I was reminded of some optional functionality that I’d always intended to implement, but never found the time. First on my list was p0f (that’s a zero).

From p0f’s homepage:

P0f v2 is a versatile passive OS fingerprinting tool. P0f can identify the operating system on:

- machines that connect to your box (SYN mode),
- machines you connect to (SYN+ACK mode),
- machine you cannot connect to (RST+ mode),
- machines whose communications you can observe.

P0f can also do many other tricks, and can detect or measure the following:

- firewall presence, NAT use (useful for policy enforcement),
- existence of a load balancer setup,
- the distance to the remote system and its uptime,
- other guy’s network hookup (DSL, OC3, avian carriers) and his ISP.

Setting p0f up on the sensor should have been straightforward;

• Install with: apt-get install p0f
• Run p0f as suggested in dionaea.conf: sudo p0f -i any -u root -Q /tmp/p0f.sock -q -l
• And edit dionaea.conf’s ihandler section to enable p0f

This mostly worked, watching the p0f output it was correctly (I’m assuming) providing stats about connecting systems. The problem was that p0f info wasn’t getting saved into Dionaea’s logsqlite database, dionaea-error.log was reporting the below error with each connection:

[04122010 13:48:44] connection connection.c:827-warning: Could not connect un:///tmp/p0f.sock:0 (Permission denied)

Which seemed odd, /tmp/p0f.sock was showing as globally readable. Re-reading the Dionaea compilation instructions I noticed a comment about p0f struggling with IPv6 so has problems it Dionaea is listening on ::, which mine was. Problem solved, I edited dionaea.conf so that Listen mode was set to “manual”, and provided the interface/IP details of my network connection. Only this didn’t solve my problem…

Head thoroughly hurting I swallowed my ego and asked for assistance on the mailing list, and promptly (thanks again Ryan) got a reply that provided a functional workaround.

So, why go to the effort? Main purpose behind running honeypot systems (for me) is to get a better idea understanding of what threats are actively targeting systems in the wild. At first glance the information provided by p0f can quickly help evaluate the attacking system; what OS? what connection type? is it local?

With the limited (few hours) of data I’ve already collected heres a sample of the info you can gather:

last 5 connections:

SQL Query:

select *
from p0fs
order by
connection desc
limit 5;

Breakdown by OS

SQL Query:

select count(p0f_genre) as count, p0f_genre as OS
from p0fs
group by p0f_genre
order by count(p0f_genre) desc;

Umm, so most systems spreading malware are (likely) infected Windows systems. No great surprise there…

Connectivity Types

SQL Query:

select count(p0f_link), p0f_link
from p0fs
group by p0f_link
order by count(p0f_link) desc;

Summary

Unfortunately the the information provided by p0f isn’t an exact science, and as devices and systems are constantly changing it’s only going to be as accurate as it’s latest signatures/fingerprints. But setup is fairly quick, and the information and insight provided fairly interesting. So why not give it a go?

–Andrew Waite

–Andrew Waite

(p.s. sorry, couldn’t resist…)

Markus keeps adding great features and functionality to Dionaea, when I read the post introducing a new web interface carniwwwhore I couldn’t help thinking I’d got lucky timing, start of a week’s vacation and no real plan for what to do with it. I’ve struggled previously with some of my Dionaea setups, largely because my system was running Debian, whilst Dionaea was built under Ubuntu; doesn’t cause too many problems, just a bit of google-fu, headscratching and stupidity that could have been avoided.

From this background I looked through the carniwwwhore pre-reqs with dread, plenty of version requirements that weren’t upto date with my Debian setup; so it’s time to bite the bullet and build a fresh system with Ubuntu. Unlike some of my previous setups, installation/compilation worked flawless, working on the same distro as the lead dev definitely makes life easier. If you’re looking for a fresh Dionaea installation, go with Ubuntu, you won’t regret it.

–Andrew Waite

(oh, and carniwwwhore? Vacation got the better of me so it’s added to the to-do list; watch this space…)

As proven by the logs generated by Kippo honeypot sensors have shown, attacks against SSH services are regularly seen in the wild. Even if you follow best practices for securing the service, the malicious scans will utilise resources available to your environment; CPU, bandwidth etc. In sufficient volume legitimate operation may be impacted as the server rejects failed login attempts.

This is where utilities like Breakinguard come into their own. Basically Breakinguard monitors log files for signs of malicious activity, and once a single source has triggered enough alerts blocks all connections from the source location. Other utilities (most notably fail2ban) perform the same activities, but I’m partial to Breakinguard due to it’s small size and simple configuration (and from knowing the author ).

Installation is straightforward, and for the most part automated. Once downloaded and extracted installation is handled by the configure script. On Debian based systems this will install the pre-requisite Perl modules and transfer the utilities components to the standard locations:

• breakinguard script – /usr/local/sbin/breakinguard
• config – /etc/breakinguard.conf
• init script – /etc/init.d/breakinguard

Once installed you need to edit your configuration. The breakinguard.conf file is fairly self explanatory, I normally edit:

• $alert_email -> set to the email address that you want to receive notifications of blocked attacks. On a publicly accessible system these alerts can be high volume, you may want to use a specific email account or at least setup some auto-move rules in your email client to avoid your inbox being spammed.
• $number_of_attempts -> This specifies the number of malicious log entries need to be generated by a specific IP address before the source is blocked. Due to the timing of the Breakinguard route this isn’t always an exact science, the default of 5 does a good job of avoid false positives whilst still blocking an attack in it’s infancy.
• $log_to_watch -> selects the logfile to monitor for signs of malicious activity. /var/log/auth.log is the obvious choice on a Debian based system.
• @safe_ips -> This array allows you to specify a number of trusted networks that will not be blocked, regardless of the number of times the they trigger alerts in the logs. This is useful for insuring that you don’t get locked out of your own systems in the event your keyboard ‘breaks’. On higher end systems with hardware remote management systems (iLo, DRAC, etc.) or virtual systems that provide remote access to the ‘physical’ console I leave this list to local subnet only and use the alternative options to access the server if I do lock myself out.
• $DEBUG -> set to 1 by default, this runs the utility in test mode without actually blocking malicious sources, perfect for testing configuration before going live. Once you’re good to set set $DEBUG to 0 and wait for the attacks to start. Example testing in debug mode is below:

Breakinguard - Debug Mode

Blocking and unblocking of malicious sources is handled via iptables. Once the $number_of_attempts limit is hit Breakinguard will run the $block_command (configurable in /etc/breakinguard.conf) which by default is ‘/sbin/iptables -I INPUT -s %s -j DROP’, with %s being replaced with the attacking IP. After a configurable timeout ($block_length), the $unblock_command removes the restriction.

You can see the IP addresses currently blocked as they are listed in /var/run/breakinguard/, alternatively listing the current iptables configuration will show sources currently being blocked, for example:

Breakinguard iptables

Download Breakinguard here

–Andrew Waite