Uncategorized

802.1x : RADIUS : IAS : Fiasco

We use 802.1x for our wireless security at work. The wireless controller uses Microsoft’s IAS as the RADIUS server. Recently during one of our maintenance windows, we installed a couple of critical patches and rebooted the IAS server. This was over the weekend and we didn’t check if wireless was working after the maintenance (one of the lessons learnt from his story :), put in automated monitoring so that you don’t have to worry about what services have come up or not after a maintenance window).

On Monday, our helpdesk gets swamped with calls of “wireless is not working”. We checked the controller and everything looked okay. Only error on the controller was that the RADIUS server was not responding. We checked the RADIUS server and the IAS service was running fine. But there were a ton of errors in the System event log with the following details

Access request for user XXXX\XXXXXX was discarded.
Fully-Qualified-User-Name = XXX.XXXX.NET/XXX.XXX/TECHNOLOGY/DEVELOPMENT/XXX XXXXXX
NAS-IP-Address = 192.168.128.10
NAS-Identifier = ACHIAS01IT
Called-Station-Identifier = 00-0B-85-06-0C-A0:wacker
Calling-Station-Identifier = 00-14-A4-28-4C-EE
Client-Friendly-Name = ACHIAS01IT
Client-IP-Address = 192.168.128.10
NAS-Port-Type = Wireless – IEEE 802.11
NAS-Port = 1
Proxy-Policy-Name = Use Windows authentication for all users
Authentication-Provider = Windows
Authentication-Server =
Reason-Code = 23
Reason = Unexpected error. Possible error in server or client configuration.
For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
Unexpected error. Possible error in server or client configuration.“, now that is real informational :). We scratched our heads.. Thought it might be an issue with the controller (it was on an older firmware). Upgraded the firmware and rebooted the controller. Still no go. Same error. Finally frustrated, we opened up a case with MSFT. Even the eng. from MSFT was flabbergasted. The usual “Everything looks good, it should work!!”.

Finally we resolved the issue to an expired computer certificate for the IAS server. The certificate had expired a couple of weeks ago, but looks like the authentication was cached and when the server was rebooted, it caused the IAS service to error out. Renewing the cert caused the wireless clients to start authenticating immediately.

Am looking into what other services depend on a valid cert to work properly.

Cool picture..

I think of myself as a good photographer :).. And fortunately for me, the shots do turn out good sometimes. Another thing to brag about :). Here’s a picture I took of Naveed, when we visited the Hoover Dam during the Vegas trip.

[Naveed : Cowboy!!]

DNS over VPN

We use Microsoft PPTP as our VPN solution at work. Although PPTP is not robust as a IPSEC based vpn, it is a lot easier to deploy and maintain. The biggest advantage is that most of the Microsoft OS’s (Windows 2000, XP, 98) have PPTP clients built into the OS. With IPSec, one has to deploy client software to the workstations. Recently SSL VPN‘s have made strides in the remote connectivity market and are even easier to maintain.. but that is for another post.

We had a very interesting problem with remote users trying access hosts/URLs in our network recently. We have several nodes in the network, which we publish on external and internal DNS. So users on the internal network would access the node via the internal IP address and get access to privileged areas, while public users accessing the node would be restricted to only some areas of the site. Users connecting to the office network via VPN were being directed to the external address even though their VPN was configured to use the remote gateway as the default gateway. This in theory should direct all traffic to the internal network. We scratched our heads for a while 🙂 and during the troubleshooting session discovered that the DNS of the node was resolving to the external IP address. So the workstation was using the DNS server provided by the ethernet interface rather than the PPTP interface!!!.. How do we solve this?? Well Google, to the rescue and we find this obscure article on Mircrosoft’s website, which tells you how to change the bind order of the network interfaces so that the PPTP interface is used by default. Since this is a registry change, we have to figure out a way to push this out to the workstations. But that is a story for another post :)..

Banff!!!

Picture of Banff city center. Highly recommend visitng this Ski resort. Beautiful place. More of it in a later post..

Gotta check this out..

I am still working on the Vegas trip post.. But here is something I stumbled upon. A very unique way of using the collective power of the web.

www.stumbleupon.com. This is a site, where you can tag interesting sites based on your interest, you can “stumble” on it. You can install a plugin for Firefox for it too. BTW.. Firefox 1.5 is out. If you haven’t tried it out yet.. Highly reccomend it. Fast, Small, Easy to use broswer as an alternate to MSFT’ Internet Explorer.

Vegas Trip : Day 1

What a day!! :).Highlights of the day.

Airport : Sucks!! I guess I just had a lot of expectations for the airport. It was dingy, smoky and dark. And I got the first glimpse of the “addicted” gamblers. The ones with the weary, worn out look on them. At least that is my theory :). Naveed thinks that I am crazy..

The Strip : One of the most magnificient creations of the human race!! Regardless of what one is told, I don’t think anyone is prepared for the “garishness” of the strip. It is bright, it is loud, it is large and it is addicting :).

The Hotel : I think this was one of our (or the only!!) bad decision for the trip. We decided to skimp on the money a bit and decided to stay at CirusCircus. As soon as we dropped off at the hotel, we were surprised at the lenght of the line for registration. It took me ~40 minutes to get to the counter. And then, we were offered a “smoking room”!!! because they didn’t have any non-smoking left. Me being the “take-whatever-you-get” type, just took the keys. When we went to the room, Naveed was like “Hell no!!!, we are not staying here”. He called the frontdesk and gave them a piece of his mind (him being smoke allergic and all!! 🙂 ). Surprise-Surprise-Surprise, the hotel suddenly found a non-smoking room for us. The rooms are still not something one would be excited about… I would not reccomend “CircusCircus” to anyone going to Vegas. Unless, you are there just for “gambling” :)..

Breakfast : We had breakfast at the “Bagel Store” in Circus Circus. Made plans for the day and had some bagels.. Hmm.. creamcheese. Shailah wanted to go the Spa and the rest of us decided to run some errands and get tickets for some shows. We planned on meeting up at noon.

Morning Fiasco : So, my IPod Nano decided to take a crap recently. It just wouldn’t start up. A “www.apple.com/support” error message shows up when booting it. Formatting it or restoring it, didn’t help. Hearing about Apple’slegendary support, I decided to take it to the apple store in Vegas. We had to wait for ~1.5 hours, but they swapped my Nano without any questions. Kudos to the apple support team.

Afternoon : Went to Wal Mart to get a camera for the trip.. here is one of my most memorable shots from the trip..

I know..I know!! It is crazy… Don’t ask, why we have to have the anonymous shot :).

First trip to Vegas

Am going to Vegas for the first time to celebrate a friends 30th birthday.. Am very excited and looking forward to what Vegas can offer :). Am sure that there will be a few pictures to post.

Cacti – Installing on Windows

Cacti is a great tool to graph network utilization. I discovered it during my previous job to create some utilization graphs of satelite links. I highly admire the talent of the kid maintaining this software… Cacti can be used by any organization wanting to graph network utilization. It is also flexible enough to graph other stats (disk utilization, CPU utilization.. etc.)

I used Linux for all of my previous installs of Cacti. The whole install is very well documented. My team is still not very Linux savvy and wanted to try Cacti out in a Windows environment. There is adequate documentation for installing Cacti on Windows, but I ran into several issues when following this guide. If I was starting all over again, I would rather try this documentation. It is more up-to-date and detailed. The author missed mentioning that you have to change the “DocumentRoot” value in the Apache conf file.. But that is a minor issue.

Am still having issues with the scheduler tool in Windows to run the poller every 5 minutes. The scheduled job is only running when someone is logged into the server. As soon as you log off the server, the scheduler seems to be stopping. I will post an update as soon as I fix this.