So, external network connectivity is slow today. Again. But it seems more than just congested. I suspect rogue network traffic.
Right, on with my investigative hats (tcpdump, nmap etc.). I try to figure out what’s going on. I see ARP requests for all IPs on our subnet, lots of them. Not just the active machines, the non-existent ones too. Hmmm, that looks like virus activity. Can’t see the source IPs via the ARP requests (since they all come from the router), so I need to figure out what the traffic associated with them is.
First guess is that it’s a ping flood, since we’ve seen those before. Check for ICMP traffic – nothing unusual there, not this time. Second guess is that it’s a Windows worm using the usual ports … bingo! I see lots of hits to the existing machines on port 135. This port isn’t open on any of our systems (Linux servers, Linux workstations and Windows 98 desktops – the owners of Windows 2000 and Windows XP laptops aren’t around today, fortunately).
Monitoring the port 135 traffic gets me the IP addresses of the source machines – about 50 or so on about five separate subnets, all on the corporate wide-area network. Call HQ, tell them bad news. Sounds like they understood – although I was hoping they were already aware of the problem, but it seems not. I trust they can track down the problems. I’ve sent them my list of IP addresses, anyway.
So it seems like another Windows worm, affecting Windows 2000 and Windows XP, is probably to blame here. To be honest, one can’t really criticise the tech staff at HQ for not keeping Windows 2000 and Windows XP fully patched with security updates etc., since this is a really difficult task. One could criticise them for deploying such a potentially fragile setup in the first place, though.
Our department has Windows 98 for most of the desktops, running no public services. The few laptops which run Windows 2000 or Windows XP have personal firewalls and anti-virus software religiously updated (something which can’t be said for those on the other subnets at HQ, it would seem).
Let’s hope they can clean their systems quickly. Because until they do, the flooding is sufficient to make our low-bandwidth link almost unusable.
Updated 20:30 on 28 September: Seems like the worm in question was a Randex variant. Appears to have infected around 250 systems. And the IT Department have ‘asked everyone nicely’ to install the fix/update. That’s not really good enough – kick every infected machine off the network and make a personal visit to clean it up. Seems to have been a cock up with keeping anti-virus definitions up-to-date.
yes they are to blame – a large windows installation NEEDS patch management software. This is part of the risk with delploying a large network of any sort – you need to be able to patch quickly and easily.
Same applies to *nix, MacOS/whatever. The says of allowing months from patch release to deplayment are gone – you’re lucky if you get hours now..
Permalinkoh and I’m in LA right now trying to get an SDSL line sorted out, with some 1st line support dude …..along with sort out a die-ing Mac, a Filmaker DB, and a whole lot of other stuff as part of the offce move. Hopefully get the SDSL line sorted so people can get at their email etc.
PermalinkOf course updating AV is only part of the problem….
PermalinkYes, at least now that they’ve blocked port 135 (see other post), we’re not suffering needlessly because of others’ incompetence/problems. Which makes a change.
Permalink