Linux

Project Uptime : Progress Report – 3

Things have been a bit hectic at work.. so didn’t get a lot of time to work on this project. Now that that the new server has been setup and the kernel updated, we get down to the mundane tasks of installing the software.

One of the first things I do, when configuring any new server is to restrict root user from logging into the server remotely. SSH is the default remote shell access method nowadays. Pls don’t tell me you are still using telnet :).

And before restricting the root user for remote access, add a new user that you want to use for regular activities, add the user to sudo group and ensure you can login and sudo to root as this user. Here are the steps I follow to do this on a Ubuntu server

Add a new user

[code]useradd xxxx [/code]

Add user to sudo group

[code]usermod -G sudo -a xxxx[/code]

Check user can sudo to gain root access

[code]sudo su – xxxx
su – [/code]

Now moving into the software installation part

Install Mysql

[code]sudo apt-get install mysql-server [/code]

you will be prompted to set the root user during this install. This is quite convenient, unlike the older installs, where you had to set the root password later on.

Install PHP

[code]sudo apt-get install php5-mysql [/code]

In addition to installing the PHP5-mysql, this will also install apache. I know, I mentioned, I would like to try out the new version of Apache. But it looks like Ubuntu, doesn’t have a package for it yet. And I am too lazy to compile from source :).

With this you have all the basic software for wordpress. Next, we will tweak this software to use less system resources.

March 28, 2012 by Databases HOWTO Linux Networking security Technology Uncategorized Web 0

HOW TO : Route traffic to loopback interface in Linux

Back in 2009 (last decade!! 🙂 ), I wrote a blog post on how you can trick windows to route traffic destined to a particular IP address to a black-hole. In it, I mentioned the command to route traffic to /dev/null in Linux was [code]<code>route ADD IP_ADDRESS_OF_MAIL_SERVER MASK 255.255.255.255 127.0.0.1</code> [/code]

I ran into a need to try it today and looks like the trick doesn’t work :). So here is the right command if you want to route traffic to the loopback (or blackhole) destined to a particular IP address [code]sudo route add -host IP_ADDRESS_OF_HOST/NETWORK_MASK lo [/code]

For example if I want to black-hole traffic destined to 74.205.216.2, I would do the following [code] sudo route add -host 74.205.216.2/32 lo [/code]

March 26, 2012 by HOWTO Linux Networking Uncategorized 0

Project Uptime : Progress Report – 2

After I installed the new kernel as mentioned in this update, I was still seeing the server booting up with the older version. I tried to have the kernel install option (apt-get install linux-image-VERSION) to overwrite the grub config, but the server wouldn’t boot up after that.

So after a lot of head scratching and googling, I found the solution. Rather than have the install program automatically update the grub config file, you have to manually edit it.

So install the latest kernel using the commands as mentioned in my previous post (https://kudithipudi.org/2012/03/07/project-uptime-progress-report-1/) and when prompted about the having an existing grub menu and if you want to overwrite it, just say no. Then do the following

  • Check the new kernel image filename by running [code] ll /boot/vmlinuz* [/code]
  • Edit the grub boot menu by editing the file /boot/grub/menu.1st and add a section for the new kernel image.
  • Edit the default boot option to the new kernel image (NOTE.. the sequence starts from 0)
  • Reboot the server and enjoy the new kernel

Here’s my before and after comparisons for the server I am building

BEFORE  (this is after the line ## End Default Options ##)

[code]

title Ubuntu 11.10, kernel 3.0.0-12-virtual
root (hd0)
kernel /boot/vmlinuz-3.0.0-12-virtual root=/dev/xvda1 console=hvc0 ro quiet splash
initrd /boot/initrd.img-3.0.0-12-virtual

title Ubuntu 11.10, kernel 3.0.0-12-virtual (recovery mode)
root (hd0)
kernel /boot/vmlinuz-3.0.0-12-virtual root=/dev/xvda1 console=hvc0 ro single
initrd /boot/initrd.img-3.0.0-12-virtual

title Ubuntu 11.10, kernel 3.0.0-16-virtual
root (hd0)
kernel /boot/vmlinuz-3.0.0-16-virtual root=/dev/xvda1 console=hvc0 ro quiet splash
initrd /boot/initrd.img-3.0.0-16-virtual
title Chainload into GRUB 2
root (hd0)
kernel /boot/grub/core.img

title Ubuntu 11.10, memtest86+
root (hd0)

[/code]

AFTER

[code]

title Ubuntu 11.10, kernel 3.0.0-12-virtual
root (hd0)
kernel /boot/vmlinuz-3.0.0-12-virtual root=/dev/xvda1 console=hvc0 ro quiet splash
initrd /boot/initrd.img-3.0.0-12-virtual

title Ubuntu 11.10, kernel 3.0.0-12-virtual (recovery mode)
root (hd0)
kernel /boot/vmlinuz-3.0.0-12-virtual root=/dev/xvda1 console=hvc0 ro single
initrd /boot/initrd.img-3.0.0-12-virtual

title Ubuntu 11.10, kernel 3.0.0-16-virtual
root (hd0)
kernel /boot/vmlinuz-3.0.0-16-virtual root=/dev/xvda1 console=hvc0 ro quiet splash
initrd /boot/initrd.img-3.0.0-16-virtual
title Chainload into GRUB 2
root (hd0)
kernel /boot/grub/core.img

title Ubuntu 11.10, memtest86+
root (hd0)

[/code]

BTW.. Grub is the boot manager (and a lot more) in Linux.

March 20, 2012 by HOWTO Linux Technology Uncategorized 0

HOW TO : Increasing number of processes that can be run by a user in Linux

By default, most of the Linux distros limit the number of processes that a user can spawn. This is put in place to limit (un)intended cases when a process might just fork off processes without a limit and bring down a server.

For RHEL (and CentOS), the default is 1024 processes per user. In some cases, you do need to increase the number of processes that a particular user can spawn. For example if you are running a database or an application server, you definitely want to tweak this number because these apps tend to create a lot of threads.

As a side note, if you run into this limitation on a machine running jboss, you typically see an error with the following string in your server logs [code]java.lang.OutOfMemoryError: unable to create new native thread.[/code]

. Looking at the error, one would think it is related to memory issues :).

OK.. back to the subject at hand. Here is the process for identifying your limits and then tweaking them as required in RHEL or CentOS.

  • Check the current limits on the number of processes a user can run by executing [code]ulimit -u[/code]
  • Edit the /etc/security/limits.conf file and add the required limits. You can get all the possible options by running man limits.conf. For example, if I wanted all the users to have a soft limit of 2000 and a hard limit of 4000, my limits.conf file wold look like this [code]# Increase the number of threads per process
    *       soft    nproc   200
    *       hard    nproc   4000 [/code]
  • Edit the /etc/security/limits.d/90-nproc.conf file and update it to have the same soft limits. By default, it has 1024 as the limit. So an updated file with my new limits as in the example above would look like this [code]
    # Default limit for number of user’s processes to prevent
    # accidental fork bombs.
    # See rhbz #432903 for reasoning.

    *          soft    nproc     2000[/code]

  • Restart the server. The updated settings won’t take affect until this is done
  • Check if you have the new limits by running [code]ulimit -u[/code]

You can also check the limits of a particular user by finding a process ID being executed by that user and running [code]sudo cat /proc/PROCESS_ID/limits [/code]

March 14, 2012 by HOWTO Linux Uncategorized 2

HOW TO : Clear unused swap memory in Linux

Inspired by a G+  post by Thomas Weeks .

swap memory is something used by the OS to essentially swap data to and forth if the main memory is not available. It is several times slower than RAM, since it uses hard disk to store the memory. And if you are constantly swapping, your system performance is going to be impacted quite a lot. You should always ensure that  your system is not swapping by adding the required RAM and/or stopping your application(s) from using so much memory. At times, because of spike in utilization, the OS might briefly use swap. And when it does, it doesn’t release the memory from swap. So from an analysis prospective, it makes it difficult to check (quickly) if your system is using swap or not. This is similar to errors on an interface in a router. Unless you clear them and monitor, you don’t know when the errors happened.

I was not aware that you could turn off swap devices while the OS is running and then enable them again. So here are the commands to do that in Linux

[code]swapoff -a[/code]

This essentially disables swap on all devices configured for swap in /etc/fstab

[code]swapon -a[/code]

This does the opposite of the first command. Enabled swap on all devices that have swap configured.

Tom put this into a nice alias by doing the following

[code]alias unswap=’sudo swapoff -a && sudo swapon -a'[/code]

Thx Tom…

March 9, 2012 by HOWTO Linux Technology Uncategorized 2

Project Uptime : Progress Report – 1

Here is the first update on Project Uptime. I spun up a new server (doesn’t that sound so odd.. spun up a new server!! :)) with 512MB of RAM running Ubuntu 11.1o (Oneiric Ocelot). First order of business after spinning up the server?

  • Update to the latest and greatest patches

[code] sudo apt-get update [/code]

  • Update to the latest kernel.
    • First check the version of kernel you are running

[code] uname -r [/code]

    • Check the repository for latest version

[code] apt-cache search linux-image [/code]

    • Install latest version

[code] sudo apt-get install linux-image-LATEST-VERSION [/code]

    • Restart server

[code] sudo init 6 [/code]

March 7, 2012 by HOWTO Linux Uncategorized 2

HOW TO : Find size of directories in current directory

Quick note for self. Simple bash loop to find out the size of each directory in the existing directory. This script is useful if you are running our of disk space and want to quickly find out the offending directory.

[code]for dir in $(find ./ -maxdepth 1 -type d); do echo ${dir}; du -ch ${dir} | grep -i total; done [/code]

Breaking this down

  • The find command prints out a list of directories. You can modify it to do recursive lookups by just removing the -maxdepth option. This output is fed into the bash loop
  • du gets the size of all the files (and sub directories) in the directory and grepping it for total gives you the total size of the directory

 

March 6, 2012 by HOWTO Linux Uncategorized 3

Project : Uptime

The uptime of this blog has been really bad recently. I switched to hosting it on a Rackspace virtual server last year and went with the cheapest option. A 256MB Linux virtual server that was costing me ~$12/month. I never got around to tuning the OS, so the server was always using swap and would go down pretty much every day. Last week, I upgraded the plan and moved to a 512MB server. But the uptime hasn’t been any better. Here’s a report from Pingdom (which by the way is a great service to track the uptime and responsiveness of your website) showing the availability of the site over the last year 96%!!.. And for someone that has been working in the operations and infrastructure world, that is unacceptable :). So my new goal is to maintain at least 99.5% uptime. Here is my plan to achieve this

  1. Move to a fresh VM with the latest kernel
  2. Upgrade to the latest version of Apache. Initially, I wanted to move to nginx or lighttpd, but with the recent Apache upgrade, I hear good things about Apache working well in low memory situations.
  3. Upgrade to latest version of MySQL and tune it for memory usage
  4. Configure cloudflare to serve a static version of front page, in case the server goes down. Design the static page to point people to my other digital presences (Google+, LinkedIn, Flickr etc)

I plan to blog the progress and learnings as I implement this plan.

March 5, 2012 by HOWTO Linux Networking Technology Uncategorized Web 5