Kudithipudi.Org

December 9, 2009

HOW TO : Load/Stress test a Linux based server

Filed under: HOWTO,Linux,Networking,Technology — Vinay @ 9:03 pm

We ran into an issue at work recently, which prompted us to do some performance testing on some of our Linux servers. The requirement was to stress test the key components of the server (CPU, RAM, HDD, Network) and prove that different servers with the same configuration were capable of performing identically. Pretty simple right :) .. The challenge was to find tools that could be run to stress test each of the components. There were a lot of tools for CPU and memory (RAM) testing, but not a lot for network and hard drive (HDD) testing. After searching high and low, we found a couple of tools, that I wanted to document here for future reference.

HDD Testing :

I found a pretty interesting tool called Iozone written by William Norcott (Oracle) and Don Capps. You can get the source code and builds for major OSs at http://iozone.org . Despite installing the program using RPM, we were not able to  run the program without specifying the complete path.

There are a ton of options for the program, but the easiest method to run it was in automated mode with the output going to an Excel spreadsheet (more like a glorified CSV file :) ). Here is the command we used

/opt/iozone/bin/iozone -a -Rb output_excel_file.xls

The “-a” is to tell the program in automated mode and the “-Rb” is to tell the program to format the output in Excel format. And you can then go ahead and open the spreadsheet in Excel and create 3D graphs to check and compare the output.

Network Testing :

Most of the information out there in terms of testing the network stack of a machine is either to copy large files over a network share or via FTP. We didn’t find that was enough to really max out a Gigport since there were protocol limitations that didn’t allow us to saturate the network port. After some searching, we stumbled across a tool called “ettcp” on Sourceforge. ettcp itself is an offshoot of ttcp. ttcp (stands for test tcp) was created to test network performance between two nodes. I couldn’t find any place to download ttcp itself, but you can download ettcp at http://ettcp.sourceforge.net/.

We used a server, to act as a common receiver for all the servers we intended to do a performance test on. Here are the commands we used to run the test

RECEIVER (Common Server)
./ettcp -r -s -f M

The options are

  • “-r” for designating the machine as receiver
  • “-f M” for showing the output in Mega Bytes.

TRANSMITTER (Test Servers)
./ettcp -t -s receiver_hostname -n 10000000 -f M

the options are

  • “-t” for designating the machine as transmitter
  • “-s receiver_hostname” to define the receiver
  • “-n” to define the number of packets to send to the receiver

July 8, 2009

HOW TO : Improve performance on Samba

Filed under: HOWTO,Linux,Technology,Windows — Vinay @ 10:02 pm

Adding the following options to the [Global] section in the samba (open source software providing windows compatible file sharing for *nix operating systems) will most likely speed your throughput

socket options = TCP_NODELAY IPTOS_LOWDELAY SO_SNDBUF=8192 SO_RCVBUF=8192

The receive and send buffer seem to do the trick.

May 22, 2009

{awk} – One Liners

Filed under: Linux,Technology — Vinay @ 6:37 pm

I ran across a cached Google entry for a good collection of awk one liners by Eric Pemant. Since I thought the original site was down, I decided to host it on my domain at http://kudithipudi.org/misc/awk_one_liners.txt. But I did some digging and looks like Eric has his own site now and hosts a lot more resources related to awk at http://www.pement.org/awk.htm. Thx for the great collection Eric..

HOW TO : Find which interface a particular IP address is configured on

Filed under: HOWTO,Linux,Programming,Technology — Vinay @ 3:28 pm

There are a ton of scripts to find how many IP addresses  are configured on a system, but I could not find one, whic would show me which particular network interface an IP address was configured on. Here is a one liner, that will give you this information in Linux

/sbin/ifconfig | grep -B1 10.10.10.10 | awk '{if (NR==1) print $1}'

The same script can be changes a bit to support other operating systems too. Essentially, I am doing a grep (search) of the output of ifconfig, which shows all the network information on the system for a particular IP. At the same time, I am using the -B1 option, which will show the line above the matching line. Finally, I am piping this to awk and printing the first row in the first column.

May 21, 2009

HOW TO : Pass environment variables when using sudo

Filed under: Databases,HOWTO,Linux,Technology — Vinay @ 12:27 am

Say you need to sudo as a particular user and run a command and at the same time you need to pass an environmental variable to the command, you can do it by passing the command in doublequotes.

For example, I want to start Oracle while I am logged in as another user (vinay), I can start the database using dbstart by issues

"sudo su - oracle -c "dbstart /$ORACLE_HOME"

$ORACLE_HOME is an environmental variable listed under user oracle’s environment.

Needless to say, you need to ensure that you have sudo configured to allow your userID to su to oracle.

May 15, 2009

HOW TO : Block outbound e-mails in Postfix

Filed under: HOWTO,Linux — Vinay @ 7:49 pm

I ran into a challenge at work, where we had to allow e-mail delivery for certain domains, but block all other domains. But at the same time, we had to ensure that the clients sending e-mails did not get a delivery error. We were using Postfix as the MTA running on Redhat Linux. Here’s how I resolved it

  • Edit the main.cf file (the default location is in /etc/postfix) and add “transport_maps = hash:/etc/postfix/transport” (without the quotes) to the file.
  • Create a file named “transport” in /etc/postfix, if it doesn’t exist
  • Add the following at the end of the transport file

DOMAIN1 :
DOMAIN2 :
* discard:

  • Run “postmap /etc/transport” to create a hash of the transport file
  • Run “service postfix restart” to restart the postfix service

This configuraiton will ensure that all e-mails address to DOMAIN1 and DOMAIN2 are delivered normally, but the rest of the e-mails are silently discarded.

Note : Ensure that you follow the syntax for where to place the : verbatim.

May 6, 2009

HOW TO : More examples of grep

Filed under: HOWTO,Linux,Technology — Vinay @ 8:14 am

For my reference.. good article about command line usage of grep . http://arstechnica.com/open-source/news/2009/05/command-line-made-easy-five-simple-recipes-for-grep.ars . Older how to articles on my site related to this topic are here and here.

March 26, 2009

HOW TO : Force expire sudo security permissions..

Filed under: HOWTO,Linux — Vinay @ 3:11 am

Ever run into a situation when you thought you had sudo rights on a machine and tried to issue the sudo command and upon finding that you don’t have them..get your name added to the sudoers list by begging the sysadmin.. and then frusrated when sudo keeps throwing an error that you are not part of the sudoers list? Hmm.. that is a long sentance :) ..

To expire any cached security permissions, so that sudo is forced to check the sudoers files, issue the following command

sudo -k

January 30, 2009

HOW TO : View HTML pages in Linux (command line)

Filed under: HOWTO,Linux — Vinay @ 9:07 pm

If you are stuck in a terminal on a Linux workstation and need to view a html file.. you can use the following command

links NAME_OF_HTML_FILE

Links displays the HTML code in the page by default..If you want to just view the rendered HTML, press “\” and you can toggle between HTML and Text views.

P.S : You need to have links installed to use it :) .. But most of the new distributions have it installed by default.

January 22, 2009

HOW TO : Delete duplicate lines in Linux/Unix

Filed under: HOWTO,Linux,Technology — Vinay @ 2:22 am

If you have a large text file and want to quickly delete any duplicate lines, you can use the following option of sort in *nix..

sort -u filename > newfilename

If you want to get even funkier :) , like only checking duplicates on a particular column, feel free to check out this link http://www.gnu.org/software/coreutils/manual/html_node/sort-invocation.html#sort-invocation.

« Newer Posts — Older Posts »

Powered by WordPress