AVG & FUD?

Like most techies I get the job of fixing and maintaining relatives’ PCs. As part of this after fixing whatever is broken I have some common clean-up and install routines that I go through to both help the system run faster and to extend the period before I’m called back, and I’ve used AVG free as part of this for many years to keep costs down for my users.
During a recent job I came across a new (I’m assuming, hadn’t noticed it before) feature of AVG free, the PC Analyzer component. Being the curious sort I hit the go button, scan ran for around 5 minutes and I was presented with this:

PCAnaylzer-results
PCAnaylzer-results

Ouch, I was surprised with the number of errors as this is a machine I keep a regular eye on, and in some cases use myself (it’s the missus’). Time to panic? Let’s see:

  • Registry errors: Errors affect system stability: (125)

That doesn’t sound good, checking the ‘Details…’ link presented me with a long list Registry keys, which to a standard end-user would result in turning on BofH’s Dummy Mode. In reality, it found a lot of keys to set the ‘open with’ right-click function depending on file extension. ‘Affect system stability’? Not so much, and I find the links useful enough that I’ve previously researched how to add my own

  • Junk Files: These files take up disk space: (599)

Again checking the details, long list of randomly named files. In the temporary folder. All ~600 took a total of less the 300MB, and the machine has more the 200GB free. Something to correct come next house cleaning session, but not really a problem.

  • Fragmentation: Reduces disk access speed

In fairness to the tool, it did come back clean and we know that fragmentation can be an issue. But that’s why every machine I’ve ever used has come with a defrag utility, as standard, for free. (OK, my BBC Micro B didn’t, but then it also had a cassette deck rather than a hard disk).

  • Broken Shortcuts: Reduces explorer browsing speed(42)

Ok, so I forget a folder of shortcuts to junk that came pre-installed with the system. I’d deleted the junk, forgot the shortcuts. Thanks for the reminder, fixed.
Summary
Plenty of ‘problems’ highlighted, time to run out and drop £25 for an annual subscription to the clean-up tool? Nope, ignoring the fact that many of these issues are system settings that actually aid the end user, the remaining issues won’t have any negative impact that the end-user will notice.
In my own opinion, AVG is taking a leaf out of the fake AV scams and scaring non-techies into parting with their hard earned coin in a bid to keep the computer running and bank details away from the scary hackers that the nice lady on the news keeps taking about. Presenting a list of meaningless (to most) information and saying it’s bad is exactly the tactic I encountered with cold call scammers earlier in the year.
As a final side note, I’ve lost two of my ‘users’ this year to AVG simply because when the AVG free license I’d installed expired, they couldn’t find a link to download the latest free version, only MANY links to the paid version. As my users are nice people (latest ‘victim’ was my grandfather), they decided themselves that it was better for them to pay the small fee than have to call me and interrupt my life.
Can anyone recommend a free AV suite that doesn’t con the unwitting into unnecessary purchases to perform a cleanup that could be performed manually with around 5 minutes and half a clue? AVG Free is a great tool, and for free I shouldn’t really complain, but when the sales tactics change to make money selling things people don’t need, to those that don’t know any better?
–Andrew Waite

Recover corrupt KeepNote filestructure

<update>Further investigation has shown that data has been restored, but the tree structure isn’t perfect. Use at own risk</update>
Anyone who’s taken Offensive Security training should be familiar with KeepNote (similar to Leo, for those that took early versions of the courses). If you’re not familiar with KeepNote it does exactly what you’d expect from the name, provide a handy way to keep and organise information. And it does a good job of this, until….

Traceback (most recent call last):
  File "/usr/local/lib/python2.6/dist-packages/keepnote/gui/__init__.py", line 469, in open_notebook
    version = notebooklib.get_notebook_version(filename)
  File "/usr/local/lib/python2.6/dist-packages/keepnote/notebook/__init__.py", line 248, in get_notebook_version
    raise NoteBookError(_("Notebook preference data is corrupt"), e)
NoteBookError: SyntaxError('junk after document element: line 11, column 0',)
Notebook preference data is corrupt
root@bt:~/pwbv3/labnotes#

Aaaarrgh!
After much searching I found several posts discussing similar issues but following the same resolutions did not resolve by problems. With this I resorted to my fallback plan, create new notebook and begin to repopulate with my content (each node is stored as a plain text file, so I was looking at lots of cut and paste). Then a thunderbolt hit me.
I copied each branch of the original note tree to the new tree, and hay-presto! functioning notebook retained and disaster averted.

cp -r ~/old-notebook/branch/ ~/new-notebook/.

Of course a better solution is just to hit the ‘File>Backup Notebook’ option occasionally.
–Andrew Waite

Kippo – Clearing pass.db

Very quick post to highlight a process for clearing all entries from your pass.db file. I had thought that this would take a bit of quick scripting utilising the list and remove modes from the passdb.py utility script. Turns out it’s even easier; simply pass a non-existent file to passdb.pyass.db parameter and it’ll create a fresh database file, no questions asked.
For example:

/opt/kippo/utils/passdb.py /opt/kippo/data/idontexistyet.db add testpassword

Will create a new database file with a single entry of ‘testpassword’ (this can be removed once you’ve established everything works). N.B. even with a blank pass.db, kippo will provide access to the password(s) configured in kippo.cfg.
Not sure yet if clearing the database will net any interesting results, only time will tell…..
–Andrew Waite

Kippo – pass.db

After a few weeks running my daily Kippo review script I’ve noticed that whilst I’m still mostly receiving several logins per day, it’s rare for a connection to actually interact with my emulated system. (For those new here, Kippo is a medium interaction honeypot emulating an SSH daemon, get started here). So I started trying to investigate what was causing the trend.
One of Kippo’s features is the password database. Basically once an intruder gains access to the shell if they try to change the password or add a different account the system adds the password to the list of allowed. This then allows connections to log into the shell with the new password. Kippo ships with a small utility script to interact with the password database:

@kippo01:/opt/kippo-svn/utils$ ./passdb.py
Usage: passdb.py <pass.db> <add|remove|list> [password]

My pass.db file contains 26 entries added by malicious ‘users’; I’m still analysing the contents in detail, but it looks like the Bad Guys(tm) are paying attention to user education 101 and using long, complex passwords.
Using the password used to log into the system, I’ve had a new (to me) way to link disparate logins. For example the query below linked connections spanning two months, originating from multiple source IP address, across three different continents (according to WHOIS records).
Source IPs for same user (based on pass)

SELECT sessions.id AS Session, sessions.ip AS Source, auth.password AS Password, auth.timestamp AS Time
FROM sessions, auth
WHERE
sessions.id = auth.session
AND auth.success = 1
AND auth.password = 'mariusbogdan';

Similarly I looked for a connection between multiple successful logins from the same source IP address. The query below provided a list of report offenders:
Successful logins from same source

SELECT COUNT(sessions.ip) AS Num, sessions.ip AS Source
FROM sessions, auth
WHERE
auth.success = 1
AND auth.session = sessions.id
GROUP BY sessions.ip
ORDER BY COUNT(sessions.ip) desc
LIMIT 25;

My summary from this is that Kippo is receiving a lower level of ‘interesting’ connections the longer the system is operational, as attackers login to check if they’ve maintained access to an ‘0wned’ resource, without utilising the resource. I’m intending to clear my pass.db to remove existing access; hopefully this will return to more interesting connections and I’m also curious to see if any of my current tenants return from either the same source location(s) and/or re-using passwords (and proving me wrong with previous comment about user education).
–Andrew Waite

Reviewing Kippo Logs

When I first started running Kippo almost a year ago I had no difficulty getting motivated to log into the honeypot, check for new connections and generally get a feel for what my victims visitors have been up to. As time went by, sessions started to follow familiar patterns and some days would get no hits. Slowly I’d check the logs less frequently, and when I did I’d get an ever increasing backlog to review, decreasing my motivation further.
Recently I got annoyed with myself, my system was ticking along in the background but I was gaining no benefit from it. So in a moment of madness I dusted off my bash and built a quick script to provide a daily review of activity on my system. Essentially this does two things, lists session interaction and files downloaded within the last 24hours.
I’ve had the routine running daily for around a week; for days there was minmal activity on my system, either no logins at all, or logins with immediate disconnects. Today was different, and marked the first success of the script. Delivered to my morning inbox, along with the rest of my regular quick tasks and RSS feed as an interesting session. Malicious user connects, downloads a scanner (archive contents looks like gosh), an irc bot (looks like EnergyMech derivative); and when attempts to run toolkit fail, downloads and runs three (yes, three, paranoia is strong with this one) log cleaners.
Example (snipped) output:

:~$ /opt/kippo-svn/kippo-sessions.sh
***Sessions***
---START:/opt/kippo-svn/log/tty/20110519-220029-5503.log---
www-dev:~# w
 22:00:38 up 14 days,  3:53,  1 user,  load average: 0.08, 0.02, 0.01
USER     TTY      FROM              LOGIN@   IDLE   JCPU   PCPU WHAT
root     pts/0    77.210.18.212     22:00    0.00s  0.00s  0.00s w
<SNIP>
***DOWNLOADS***
/opt/kippo-svn/dl/20110519220445_http___eduteam_home_ro_mech_gz: gzip compressed data, from Unix, last modified: Sun Oct  4 17:46:52 2009
<SNIP>

The script can be downloaded here, as usual it’s released under the Beerware License
–Andrew Waite

Network sniffing in VMware ESXi

VMWare ESXi is perfect for a self contained lab, but as I’m used to having full access to a ‘real’ network there are a few things I miss not having control over for testing and other things. The biggest of these is a spanf port (or mirror port depending on your hardware). If you’re not familiar, the basic premise is to configure one (or more ports) to reproduce any traffic flowing through any port(s). This provides packet level access for debugging network problems, passing to an I[D/P]S, etc.
ESXi doesn’t provide this functionality, but does allow you to set a vSwitch to be ‘promiscuous’. Unfortunately this isn’t as controllable as a span/mirror port as (from the quick tests I’ve run) essentially turns the vSwitch into a vHub. Not a problem in my lab environment but you probably want to give it some serious thought before enabling in a production environment; do you really want every server on the network to be able to see all traffic on the (virtual) wire?
To make the change in ESXi you need host -> Configuration -> Networking and set the properties as shown below:

vSwitch properties
vSwitch properties

Once this change is made, any guests connected to the vSwitch can all see any of the network traffic on that switch.
For testing you can build a quick lab scenario with 3 live boot BackTrack systems. Each machine has a different role; server, client and ‘sniffer’. The sniffing machine is now able to view direct communication between the other two systems. Using wireshark’s Follow TCP stream functionality shows the conversation:
sniffed TCP stream
sniffed TCP stream

–Andrew Waite

SSH Tunnelling Example

Towards the end of last year I spent a few hours trialling SSH tunnels, I knew how the process worked but hadn’t had much cause to use it in anger; so my lab got some use instead, and a post was written covering the basics; SSH port forwarding 101.
Since I now know how to quickly and successfully implement a tunnel, it turns out that I previously had plenty of cause to use tunnels in the past, I just didn’t know SSH tunnels were the right tool for the job. A couple of recent conversations has made me realise others don’t always know the flexibility of tunnels either so I wanted to try and describe a common scenario to highlight the usefulness of tunnels.
Scenario:

Above is a fairly common setup. You’ve got an internal resource (for example an intranet wiki for documentation), this is in turn protected by a firewall that only allows access from trusted location. Under normal circumstances all staff can access the resource without problems, and any malicious sources (human or automated) can’t access the service.
This works well, until someone needs access and they aren’t at one of the trusted locations (we’re assuming this is an unusual problem and remote access solutions aren’t in place). In a lot of environments SSH is a ‘trusted’ system management solution and is world accessible (and hopefully secured well enough to keep the barbarians from the door, but that’s for different posts).
Solution?:
SSH tunnels (but you guessed that). Tunnel the server’s HTTP (or whatever) service back to your local system, and then connect locally. Using the syntax I discussed previously, from a ‘nix shell you can use this command:

ssh -L 8000:127.0.0.1:80 ssh-server.domain.com

This makes an SSH connection to the server (ssh-server.domain.com), tunnelling the local HTTP service running on port 80 (127.0.0.1:80) and binds it to your machines TCP 8000 port. Now you can connect to the service by typing 127.0.0.1:8000 into a browser, thus traversing the firewall source IP restrictions.
If you’re living in a Windows world, then the PuTTY equivalent configuration will be:

Next time you’re sat in the coffee shop on a Sunday morning, and the boss rings with an ’emergency’; are you sure that you can’t access the resources you need from where you are? If you can, that coffee (and extra slice of cake) just became expense-able 😉
–Andrew Waite

A Northern Geek's trip South

Tuesday started fine, train down the capital a chance to meet up with the London work team. So far so good, until a colleague suggested a ‘quiet’ drink after work. Ended up not being too quiet after all.
With Wednesday starting off with ‘why?….’, I found some energy and headed for Security BSides London. As I’d already reconnoitered the location on Tuesday getting to the location was a breeze, only to find the door locked. Javvad Malik to the rescue, arrived at same time and managed to call one of the organisers to let us in. After brief introductions all round I met Soraya Iggy in person for the first time, absolutely nothing like I was expecting but great in every way. After receiving goodie bag (and getting repeated grief from Iggy to change into con shirt) I enjoyed some good geek chat whilst watching the venue fill.
After the official opening of the event, I headed upstairs to track two, which started with Aaron Finnon discussing DNS tunneling techniques. I was looking forward to this talk as I’d got half of the information over a drink after Aaron gave his famous SSL talk when OWASP Leeds travelled to Newcastle. My main takeaway from the discussion was that with the use of some relatively simple tools it can be relatively simple to bypass most captive wireless portals if they aren’t sufficiently tying down egress traffic. First on my to-do list of ‘I wonder what happens if you try this in my environment?’.
Second session was David Rook and Chris Wysopal, discussing ‘Jedi Mind’ tricks for building security programs. Having watched recordings of both presents from other events I was looking forward to getting the live  experience, and neither disappointed. The presentation was great and I took a lot away for how to both discuss security issues with non-infosec people, and how to talk about the problems in business terms to get buy-in to effect real change in an organisation. I was somewhat surprised, as this started a trend of the event with my favourite presentations being non-technical in nature.
Third session was one that I’d heard a few people dismiss before the event as being a bit lame. I’d already picked it out as my preferred session for this timeslot (it was a tough call, other track was Justin Clark discuss web app attacks, but the end would have over run with the next talk I wanted to see). I’m glad I didn’t let the naysayers dissuade me, Ellen Moar and Colin McLean did a great job demonstrating just how simple it is for anyone with basic computer knowledge (script kiddie) to cut and paste their way past defensive countermeasures (AV). Content wasn’t anything groundbreaking (which is why I think some weren’t keen), but I think it’s the first time I’ve actually seen someone ‘prove’ what we all accept as gospel. Scary stuff.
Final session of the morning was Xavier Mertens discuss logging and event management. Not the most thrilling of topics I’ll admit, but it’s something that so few organisations seem to get right I was interested to find out if there were any ‘better’ ways that could improve the process. Not only are there apparently better ways, but apparently there are also free better ways, so I’m going to talk a closer look at OSSEC.
After lunch Steve Lord provided an ‘interesting’ look into different types/levels of pentester and what it means to be in the industry. The talk received a lot of laughs, but in hindsight I wish I seen a talk with more technical content. For me, bsides was for education and networking, I’ll leave comedy to the comedians.
Next talk was better, Wicked Clown, expanding on his Brucon Lightening talk showing how to break out of a restricted RDP session. This was a great presentation, and was another attack to add to my ‘what if’ to-do list. More importantly he also provided a simple fix to prevent the attack vector, considering it’s a single checkbox, and the workaround breaks how most would ‘expect’ the service to behave I’ll echo his confusion as to why Microsoft don’t have the checkbox ticked by default. Perhaps secure out of the box is too much to ask?
David Rook took to the stage again, this time alone and discussing static code analysis with Agnitio. I’d taken a look at Agnitio since David released it, but as I’m not much of a dev (see the utilities I release for proof…) haven’t been able to try it in anger. If you’re interested, the talk slides are available on the Security Ninja blog. If the tool can reach the stated end of it’s road map of being the ‘Burp Suite of static analysis’ then it should be a fantastic tool.
Next talk I saw was Manuel demo (reverse~re)engineering of DRM within Android applications. I found the talk fascinating, mostly by how quick Manuel was able to put the pieces of the puzzle together and bypass the protections put in place to do exactly what he was attempting. Whilst the presentation was good, it was one of those where you felt your comparative IQ drop as you see black magic being wielded at the keyboard before your eyes.
The event finished with ‘Security YMCA’, words cannot describe this ‘experience’ so I won’t attempt to, and leave you with this YouTube video. (WARNING: once seen, cannot be unseen). Unfortunately trying to hide at the back didn’t help in the end, so I must apologise to Ellen as I managed to say thanks for an interesting talk by subjecting her to my ‘singing’ attempts. To ensure the the guilty aren’t protected, those at the front assaulting your sensors are:

During most cons I’m usually sat in the office getting snippets from Twitter, or reading a blow by blow account as Chris John Riley posts during every session. I always wondered how he found the time to get it all done, after seeing it in person I’ve still got no clue…
Thursday provided no let-up, with Infosec Europe next on the agenda. Feeling lively I left hotel in Farringdon early just after seven, and proceed to walk to Earls Court. Yes, for those that know London this is around 5 miles as the crow flies, it’s even longer if you keep taking the wrong turn as you’re too busy admiring the sights of London (managed to cover Oxford Street, Regent Street, Piccardilly,  Buckingham Palace and Hyde Park on the way. Unsurprisingly I was slightly tired when arriving. As this was my first visit to InfoSec I was surprised by the size, but with bag for freebies in hand, I hit the stands to talk to vendors. Without a boring stand by stand account it was good to meet some people in person for the first time, and to get some hands on demos of products I hadn’t yet seen in person. Some of the marketing was in high spirits however, I got my favourite quote from a vendor who shall remain nameless (I’m nice like that) stating:
we don’t have a solution for the iPhone, as it’s a secure platform why bother?
The offering had looked promising until then; after that comment? Thanks, but I’ll pass…
I did take advantage of the Syngress stand’s discounts and filled out my to read pile. (Ninja Hacking, Seven Deadliest USB Attacks, Cybercrime and Espionage and Digital Triage Forensics). Although I didn’t take as much advantage as the gentleman in front of me in the queue, who literally bought one copy of every book on display; totalling over £450.
The trip ended with a bottle of lager sat outside the British Museum in some glorious weather (which unfortunately didn’t follow me back home). I don’t want to name names, as undoubtably I’ll forget someone, but my most common phrase this week has been ‘Good to finally be able to put a face to the twitter handle’, it really was good to meet people I’ve spoken to online for a while, and to make some new contacts as well. Looking forward to the next time we’re able to meet up.
–Andrew Waite
P.S. sorry for formatting towards the end, seems to be a strange limit with the number of paragraphs wordpress will accept per post. Will try to correct in due course.

John the Ripper 101

For those that don’t already know, John the Ripper is:

a fast password cracker, currently available for many flavors of Unix, Windows, DOS, BeOS, and OpenVMS. Its primary purpose is to detect weak Unix passwords. Besides several crypt(3) password hash types most commonly found on various Unix systems, supported out of the box are Windows LM hashes, plus many more with contributed patches.

John is commonly available for most ‘nix systems under their package management systems. However, I’d strongly recommend compiling from source to ensure John takes full advantage of more powerful hardware capabilities than the ‘generic’ build. Compiling from source is straightforward, and I’m yet to encounter any difficulties. The additional time spent with manual compilation definitely pays for itself in the long run, read the INSTALL file for instructions.
To get a feel for the available performance john includes ‘–test’ functionality. My system (iMac i7 w/8GB ram), manually compiled with system specific functions achieved:

enchmarking: Traditional DES [128/128 BS SSE2-16]… DONE
Many salts: 3370K c/s real, 3370K c/s virtual
Only one salt: 2840K c/s real, 2840K c/s virtual
Benchmarking: BSDI DES (x725) [128/128 BS SSE2-16]… DONE
Many salts: 108990 c/s real, 110080 c/s virtual
Only one salt: 107264 c/s real, 107264 c/s virtual

Using the same system and john compiled with a ‘generic’ system I achieved less than 50% performance:

Benchmarking: Traditional DES [64/64 BS]… DONE
Many salts: 1613K c/s real, 1613K c/s virtual
Only one salt: 1476K c/s real, 1487K c/s virtual
Benchmarking: BSDI DES (x725) [64/64 BS]… DONE
Many salts: 52800 c/s real, 52377 c/s virtual
Only one salt: 51961 c/s real, 51961 c/s virtual

Unfortunately, john doesn’t currently take advantage of multi-core systems and only uses a single core (see below). There are methods for sharing the processing load between multiple instances of john, or even across john instances running on multiple systems, but that’s a topic for another day.
CPU utilisation under john
Before you can crack any passwords, you first need to ‘acquire’ the hashes to be cracked. There are many options for this including the pwdump family (I’d recommend fgdump), the Cain and Abel tools and many others. For testing purposes I extracted the Windows LM hashes using Metasploit’ Meterpreter’s hashdump functionality:

meterpreter > hashdump
andrew:1011:f0c59dece4734d86959d531ee5401834:fa77122d7a1f75a91b8550e6488ba9a9:::
ASPNET:1007:62cc422eb3a006c2e9b348c9a8c62222:5e5a4a1058e775c3be4fc49775a5bac8:::
Barney:1012:44fbba209c861c56944e2df489a880e4:e463c514849b1c902f6e118fa75c77f9:::
Charlie:1013:49a1a1e421beeadcaad3b435b51404ee:ac27498221e64565152d08bdf2d34369:::
HelpAssistant:1000:551028a43e2ea865d6bce7b0f3da97ee:7bde535fb3fe1432fc7125354492aa6b:::
IUSR_FDCC-CD744D84E2:1005:c50115cc265c5db0691bfe28aa492514:6615b358c184382319cdbd59b22ab9fc:::
IWAM_FDCC-CD744D84E2:1006:628a234fbbf992c5bc5c513445f4fabd:dea6a5c2c54e7e88178fe64a6e6e4c68:::
Renamed_Admin:500:921988ba001dc8e1c41a0e2828864838:5f23a05483c0b292fdd922e6b05afa05:::
Renamed_Guest:501:aad3b435b51404eeaad3b435b51404ee:31d6cfe0d16ae931b73c59d7e0c089c0:::
SUPPORT_388945a0:1002:aad3b435b51404eeaad3b435b51404ee:005f9658b579d394f8a315506fe238de:::

John comes packaged with a default password list, containing a little over 3000 entries, and several methods for generating hybrid guesses including appending numbers to entry or performing ‘l33t’ replacements. Rules for hybrid attacks can found (and edited) in the john.conf file. Even without tweaking, john is still effective  at cracking hashes as the below shows.
Cracking ‘andrew’:

Wintermute:john-testing awaite$ time /Applications/john-1.7.6-jumbo-12/run/john 192-168-1-127_Andrew.dump
Loaded 2 password hashes with no different salts (LM DES [128/128 BS SSE2-16])
ITY              (andrew:2)
INFOSAN          (andrew:1)
guesses: 2  time: 0:00:09:45 (3)  c/s: 19803K  trying: INFODTP – INFOSS9
real 9m45.359s
user 9m45.193s
sys 0m0.125s

Cracking ‘Barney’:

Wintermute:john-testing awaite$ time /Applications/john-1.7.6-jumbo-12/run/john 192-168-1-127_Barney.dump
Loaded 2 password hashes with no different salts (LM DES [128/128 BS SSE2-16])
R                (Barney:2)
DINOSAU          (Barney:1)
guesses: 2  time: 0:00:00:01 (3)  c/s: 9547K  trying: DINOSH8 – DINOVYS
real 0m1.685s
user 0m1.642s
sys 0m0.019s

Cracking ‘Charlie’:

Wintermute:john-testing awaite$ time /Applications/john-1.7.6-jumbo-12/run/john 192-168-1-127_Charlie.dump
Loaded 1 password hash (LM DES [128/128 BS SSE2-16])
CH4LK01          (Charlie)
guesses: 1  time: 0:00:28:38 (3)  c/s: 20126K  trying: CH4LB5# – CH4LCGN
real 28m38.011s
user 28m37.645s
sys 0m0.301s

Cracked hashes are stored in the john.pot file, preventing john from wasting CPU cycles cracking a hash that has already been cracked. Previously cracked passwords can be found using –show:

Wintermute:run awaite$ ./john –show /Users/awaite/Documents/john-testing/192-168-1-127.dump
andrew:INFOSANITY:1011:fa77122d7a1f75a91b8550e6488ba9a9:::
Barney:DINOSAUR:1012:e463c514849b1c902f6e118fa75c77f9:::
Charlie:CH4LK01:1013:ac27498221e64565152d08bdf2d34369:::
5 password hashes cracked, 12 left

John has a lot of additional functionality; go install, tweak and crack (only with permission, obviously).
–Andrew Waite

Daily Paranoia

As a security guy I find my paranoia levels are slightly high than most, a little something inside me picks up on things that general users miss that indicate that something isn’t right. This morning was no exception….
After acquiring coffee, morning inbox was opened which presented the following:
Nagios Email Alerts
These are email alerts sent by the monitoring system Nagios, running within InfoSanity’ networks. The NRPE-check_users parameters have been modified from the Debian defaults to be more paranoid; triggering a warning alert with a single user logged into the server, and go critical if more than one. So; from this, someone is logged into the web server, and it isn’t me.
Feeling the onrush of panic, I log into server and chuck commands at a shell to see who is violating my system. Last, who and /var/log/auth all showed that no one had accessed the server at the time of my alerts. Everything good? Not if your paranoid, starting to smell a rootkit causing the system to lie to me.
There are a couple of anti-rootkit utilities that have served me well in the past, chkrootkit and rkhunter. Wondering which one to run? As we’ve already discovered I’m paranoid, answer was both. And both gave a clean bill of health to the system. Now I’m really getting paranoid.
Wanting to see if I’ve missed a trend, or if the issue is present with other servers in the environment I log into Nagios interface for a more detailed look, and find the answer:

Nagios webUI notifications
Nagios webUI notifications

Anyone spotted it? Yep, the service alert went critical when the check’s socket timed out (network issue), and then dropped to warning showing a single user when the connectivity returned; which was correct, I’d forgotten to log out of the console on my last VMWare connection. Stepping down from high alert…..
Morale of the story?
Don’t respond to system alerts before finishing first coffee of the morning. The events weren’t a total loss though, besides getting my heart-rate up and blood flowing, it was a good(ish) refresher for incident response (can’t beat the adrenaline rush of responding to an incident, real or imagined) and rkhunter uncovered a potential weakness in the server configuration which has since been corrected (no, I’m not telling you what).
–Andrew Waite
Oh yeah, had I just opened the email, could have avoid the whole situation:Nagios Email Details