Technology

Project Uptime : Progress Report – 4

Continuing to lock down the server as part of project uptime a bit more.. I highly recommend enabling and using iptables on every Linux server. I want to restrict inbound traffic to the server to only SSH (tcp port 22) and HTTP(S) (tcp port 80/443). Here’s the process

Check the current rules on the server

[code]sudo iptables -L [/code]

Add rules to allow SSH, HTTP and HTTPS traffic and all traffic from the loopback interface

[code]sudo iptables -I INPUT -i lo -j ACCEPT
sudo iptables -A INPUT -m conntrack –ctstate RELATED,ESTABLISHED -j ACCEPT
sudo iptables -A INPUT -p tcp –dport ssh -j ACCEPT
sudo iptables -A INPUT -p tcp –dport http -j ACCEPT
sudo iptables -A INPUT -p tcp –dport https -j ACCEPT
[/code]

Drop any traffic that doesn’t match the above mentioned criteria

[code]sudo iptables -A INPUT -j DROP [/code]

save the config and create script for the rules to survive reboots by running

[code]sudo su –
iptables-save > /etc/firewall.rules[/code]

now create a simple script that will load these rules during startup. Ubuntu provides a pretty neat way to do this. You can write a simple script and place it in /etc/network/if-pre-up.d and the system will execute this before bringing up the interfaces. You can get pretty fancy with this, but here is a simple scrip that I use

[code]
samurai@samurai:/etc/network/if-pre-up.d$ cat startfirewall
#!/bin/bash

# Import iptables rules if the rules file exists

if [ -f /etc/firewall.rules ]; then
iptables-restore </etc/firewall.rules
fi

exit 0
[/code]

Now you can reboot the server and check if your firewall rules are still in effect by running

[code]sudo iptables -L [/code]

Project Uptime : Progress Report – 3

Things have been a bit hectic at work.. so didn’t get a lot of time to work on this project. Now that that the new server has been setup and the kernel updated, we get down to the mundane tasks of installing the software.

One of the first things I do, when configuring any new server is to restrict root user from logging into the server remotely. SSH is the default remote shell access method nowadays. Pls don’t tell me you are still using telnet :).

And before restricting the root user for remote access, add a new user that you want to use for regular activities, add the user to sudo group and ensure you can login and sudo to root as this user. Here are the steps I follow to do this on a Ubuntu server

Add a new user

[code]useradd xxxx [/code]

Add user to sudo group

[code]usermod -G sudo -a xxxx[/code]

Check user can sudo to gain root access

[code]sudo su – xxxx
su – [/code]

Now moving into the software installation part

Install Mysql

[code]sudo apt-get install mysql-server [/code]

you will be prompted to set the root user during this install. This is quite convenient, unlike the older installs, where you had to set the root password later on.

Install PHP

[code]sudo apt-get install php5-mysql [/code]

In addition to installing the PHP5-mysql, this will also install apache. I know, I mentioned, I would like to try out the new version of Apache. But it looks like Ubuntu, doesn’t have a package for it yet. And I am too lazy to compile from source :).

With this you have all the basic software for wordpress. Next, we will tweak this software to use less system resources.

DID YOU KNOW : Windows mobile and wildcard certs don't work together

Wildcard SSL certificates allow you to use one certificate for all sub domains (up to one level) of a host. Say I got a wildcard SSL certificate for *.kudithipudi.org, I would be able to use it to provide SSL on blah.kudithipudi.org, ssltest.kudithipudi.org, youcannotbeserious.kudithipudi.org and the clients won’t complaint about it.

For some reason though, Windows Mobile phones don’t like wildcard certs. So if you are ever scratching your head, why every other client works, but windows mobile devices don’t..stop scratching and get a regular SSL certificate for your website/application.

Apparently, this is the case with

  • Windows CE
  • Windows Mobile 5.0
  • Windows Mobile 6.0
  • Windows Mobile 7.0

Don’t you get the feeling that someone keeps using the same library and never bothered to check/fix it? And searching on MSDN or any other Microsoft resource won’t provide you this information. This is my own deduction after beating my head against the wall for more than 3 days :).

Project Uptime : Progress Report – 2

After I installed the new kernel as mentioned in this update, I was still seeing the server booting up with the older version. I tried to have the kernel install option (apt-get install linux-image-VERSION) to overwrite the grub config, but the server wouldn’t boot up after that.

So after a lot of head scratching and googling, I found the solution. Rather than have the install program automatically update the grub config file, you have to manually edit it.

So install the latest kernel using the commands as mentioned in my previous post (https://kudithipudi.org/2012/03/07/project-uptime-progress-report-1/) and when prompted about the having an existing grub menu and if you want to overwrite it, just say no. Then do the following

  • Check the new kernel image filename by running [code] ll /boot/vmlinuz* [/code]
  • Edit the grub boot menu by editing the file /boot/grub/menu.1st and add a section for the new kernel image.
  • Edit the default boot option to the new kernel image (NOTE.. the sequence starts from 0)
  • Reboot the server and enjoy the new kernel

Here’s my before and after comparisons for the server I am building

BEFORE  (this is after the line ## End Default Options ##)

[code]

title Ubuntu 11.10, kernel 3.0.0-12-virtual
root (hd0)
kernel /boot/vmlinuz-3.0.0-12-virtual root=/dev/xvda1 console=hvc0 ro quiet splash
initrd /boot/initrd.img-3.0.0-12-virtual

title Ubuntu 11.10, kernel 3.0.0-12-virtual (recovery mode)
root (hd0)
kernel /boot/vmlinuz-3.0.0-12-virtual root=/dev/xvda1 console=hvc0 ro single
initrd /boot/initrd.img-3.0.0-12-virtual

title Ubuntu 11.10, kernel 3.0.0-16-virtual
root (hd0)
kernel /boot/vmlinuz-3.0.0-16-virtual root=/dev/xvda1 console=hvc0 ro quiet splash
initrd /boot/initrd.img-3.0.0-16-virtual
title Chainload into GRUB 2
root (hd0)
kernel /boot/grub/core.img

title Ubuntu 11.10, memtest86+
root (hd0)

[/code]

AFTER

[code]

title Ubuntu 11.10, kernel 3.0.0-12-virtual
root (hd0)
kernel /boot/vmlinuz-3.0.0-12-virtual root=/dev/xvda1 console=hvc0 ro quiet splash
initrd /boot/initrd.img-3.0.0-12-virtual

title Ubuntu 11.10, kernel 3.0.0-12-virtual (recovery mode)
root (hd0)
kernel /boot/vmlinuz-3.0.0-12-virtual root=/dev/xvda1 console=hvc0 ro single
initrd /boot/initrd.img-3.0.0-12-virtual

title Ubuntu 11.10, kernel 3.0.0-16-virtual
root (hd0)
kernel /boot/vmlinuz-3.0.0-16-virtual root=/dev/xvda1 console=hvc0 ro quiet splash
initrd /boot/initrd.img-3.0.0-16-virtual
title Chainload into GRUB 2
root (hd0)
kernel /boot/grub/core.img

title Ubuntu 11.10, memtest86+
root (hd0)

[/code]

BTW.. Grub is the boot manager (and a lot more) in Linux.

HOW TO : Configure Jboss for writing web access logs

One of the capabilities of Jboss is that it can serve HTTP traffic. By default Jboss does not log any of the HTTP traffic in it’s log files. Here is a quick howto on enabling this logging. This post is specific to Jboss 4.x (ancient!!) and I will post another one soon on how do it in version 5.x and newer.

Edit the server.xml file located in $JBOSS_HOME/servers/$PROFILE/deploy/jboss-web.deployer and replace the commented out access logger section as such

FROM

[code]<!–
<Valve className="org.apache.catalina.valves.AccessLogValve"
prefix="localhost_access_log." suffix=".log"
pattern="common" directory="${jboss.server.log.dir}"
resolveHosts="false" />
–> [/code]

TO

[code]<Valve className="org.apache.catalina.valves.AccessLogValve"
prefix="localhost_access_log." suffix=".log"
pattern="common" directory="${jboss.server.log.dir}"
resolveHosts="false" /> [/code]

This will start creating a file with the format localhost_access_log.CURRENT_DATE.log in the $JBOSS_HOME/server/$PROFILE/log folder

But it isn’t fun if you just leave the default logging right :). The pattern formats of common and combined are similar to the standard apache logging options. But if you wanted to have certain content and format in the log files, you have a lot of options. Jboss community has documented all the data that is exposed through this valve at http://docs.jboss.org/jbossweb/latest/api/org/apache/catalina/valves/AccessLogValve.html

So say, I want to log the referrer header, user agent and the value of a cookie called JSESSONID and log all this data into a file called jboss_web_access_log, I setup the options as such

[code]<Valve className="org.apache.catalina.valves.AccessLogValve"
prefix="jboss_web_access_log." suffix=".log"
pattern="%h %p %l %u %t %r %s %b ‘%{Referer}i’ ‘%{User-Agent}i’ ‘%{JSESSIONID}c’"
directory="${jboss.server.log.dir}"
resolveHosts="false" /> [/code]

HOW TO : Clear unused swap memory in Linux

Inspired by a G+  post by Thomas Weeks .

swap memory is something used by the OS to essentially swap data to and forth if the main memory is not available. It is several times slower than RAM, since it uses hard disk to store the memory. And if you are constantly swapping, your system performance is going to be impacted quite a lot. You should always ensure that  your system is not swapping by adding the required RAM and/or stopping your application(s) from using so much memory. At times, because of spike in utilization, the OS might briefly use swap. And when it does, it doesn’t release the memory from swap. So from an analysis prospective, it makes it difficult to check (quickly) if your system is using swap or not. This is similar to errors on an interface in a router. Unless you clear them and monitor, you don’t know when the errors happened.

I was not aware that you could turn off swap devices while the OS is running and then enable them again. So here are the commands to do that in Linux

[code]swapoff -a[/code]

This essentially disables swap on all devices configured for swap in /etc/fstab

[code]swapon -a[/code]

This does the opposite of the first command. Enabled swap on all devices that have swap configured.

Tom put this into a nice alias by doing the following

[code]alias unswap=’sudo swapoff -a && sudo swapon -a'[/code]

Thx Tom…

HOW TO : Clear screen based on OS in python scripts

I like shiny new toys :). Even though perl is pretty powerful and more than enough for the simple tasks I get to automate from time to time, I want to start learning python and find out first hand, why the whole geek community is raving about it.

As I start to write new scripts in python, I wanted to document how I used to do some things in perl and how I implemented them in python.

One of the standard features of any script I write is to “clear” the screen before starting to send output to the console. Here is the comparison between perl and python

perl

[code]system $^O eq ‘MSWin32’ ? ‘cls’ : ‘clear’; [/code]

python

[code]

# Clear screen, based on the OS
if (os.name == ‘nt’):
os.system("cls")
else:
os.system("clear")

[/code]