Kudithipudi.Org

March 6, 2012

HOW TO : Find size of directories in current directory

Filed under: HOWTO,Linux — Vinay @ 10:02 am

Quick note for self. Simple bash loop to find out the size of each directory in the existing directory. This script is useful if you are running our of disk space and want to quickly find out the offending directory.

for dir in $(find ./ -maxdepth 1 -type d); do echo ${dir}; du -ch ${dir} | grep -i total; done

Breaking this down

  • The find command prints out a list of directories. You can modify it to do recursive lookups by just removing the -maxdepth option. This output is fed into the bash loop
  • du gets the size of all the files (and sub directories) in the directory and grepping it for total gives you the total size of the directory

 

March 5, 2012

Project : Uptime

Filed under: HOWTO,Linux,Networking,Technology,Web — Vinay @ 5:13 am

The uptime of this blog has been really bad recently. I switched to hosting it on a Rackspace virtual server last year and went with the cheapest option. A 256MB Linux virtual server that was costing me ~$12/month. I never got around to tuning the OS, so the server was always using swap and would go down pretty much every day. Last week, I upgraded the plan and moved to a 512MB server. But the uptime hasn’t been any better. Here’s a report from Pingdom (which by the way is a great service to track the uptime and responsiveness of your website) showing the availability of the site over the last year 96%!!.. And for someone that has been working in the operations and infrastructure world, that is unacceptable :) . So my new goal is to maintain at least 99.5% uptime. Here is my plan to achieve this

  1. Move to a fresh VM with the latest kernel
  2. Upgrade to the latest version of Apache. Initially, I wanted to move to nginx or lighttpd, but with the recent Apache upgrade, I hear good things about Apache working well in low memory situations.
  3. Upgrade to latest version of MySQL and tune it for memory usage
  4. Configure cloudflare to serve a static version of front page, in case the server goes down. Design the static page to point people to my other digital presences (Google+, LinkedIn, Flickr etc)

I plan to blog the progress and learnings as I implement this plan.

March 4, 2012

HOW TO : Search and Replace text in a file with Perl

Filed under: HOWTO,Technology — Vinay @ 2:10 am

There are tons of sites (and tons of different ways to do this) about this information.. But wanted to note this down for my personal records. If you ever wanted to search for and replace certain text in a file, you can do it with perl with this quick one liner

perl  -p -i -e 's/ORIGINAL_STRING/NEW_STRING/g' FILE_NAME

February 29, 2012

HOW TO : Sort Apache Web Logs for hits by Unique IP Addresses

Filed under: HOWTO,Linux,Technology,Web — Vinay @ 2:35 am

 

Say you want to find out how many hits you are getting t0 a specific page from a particular source IP, you can use this quick collection of Linux tools to get this data

grep -i "URL_TO_CHECK" PATH_TO_APACHE_ACCESS_LOG | cut -d' ' -f 1 -| sort |uniq -c | sort -rn > ~/ip_report.txt

You are using

  • grep to filter the string of the page you want the report on
  • cut to get the IP address from the log file
  • sort and uniq to sort the unique IP addresses
  • and finally sort -rn to sort the data in descending order

Example :

grep -i "GET /" /opt/apache/logs/access_log | cut -d' ' -f 1 -| sort |uniq -c | sort -rn > ~/ip_report.txt

gets you the report of hits to the index page.

February 19, 2012

Photography : Putting tips to good use

Filed under: HOWTO,Photography — Vinay @ 1:25 pm

I ran across a video of Peter Hurley, as a guest post on Scott Kelby’s blog, in which he discusses the importance of positioning the jaw lines of your subjects when taking portraits. Sri stopped by to check on Virat yesterday along with his family and I thought I should try out this tip. The effect was amazing.

BEFORE (as I would normally take portraits)

AFTER (Shebang!!) :)

February 7, 2012

HOW TO : Find list of files used by a process in Linux

Filed under: HOWTO,Linux,Technology — Vinay @ 10:01 am

Quick howto on finding the list of files being accessed by a process in Linux. I needed to find this for troubleshooting an issue where a particular process was using an abnormally high percentage of CPU. I wanted to find out what this particular process was doing and accessing.

  1. Find the process ID (pid) of the process you want to analyze by running 
     ps -ef | grep NAME_OF_PROCESS
  2. Find the files the process is accessing at a given time by running 
    sudo ls -l /proc/PROCESS_ID/fd

For example, if I wanted to find the list of files being accessed by mysql, the process would look as such

 ps -ef | grep mysqld

which would show the output as

samurai@samurai:~$ ps -ef | grep mysqld
mysql     3304     1  0 Feb04 ?        00:00:23 /usr/sbin/mysqld
samurai  23389 23374  0 14:57 pts/0    00:00:00 grep --color=auto mysqld

I can then find the list of files being used by mysql by running

 sudo ls -l /proc/3304/fd

which would give me


lrwx------ 1 root root 64 Feb  7 15:00 0 -> /dev/null
lrwx------ 1 root root 64 Feb  7 15:00 1 -> /var/log/mysql/error.log
lrwx------ 1 root root 64 Feb  7 15:00 10 -> socket:[4958]
lrwx------ 1 root root 64 Feb  7 15:00 11 -> /tmp/ibdu9WRh (deleted)
lrwx------ 1 root root 64 Feb  7 15:00 12 -> socket:[4959]
lrwx------ 1 root root 64 Feb  7 15:00 14 -> /var/lib/mysql/blog/wp_term_relatio                        nships.MYI
lrwx------ 1 root root 64 Feb  7 15:00 15 -> /var/lib/mysql/blog/wp_postmeta.MYI
lrwx------ 1 root root 64 Feb  7 15:00 17 -> /var/lib/mysql/blog/wp_term_relatio                        nships.MYD
lrwx------ 1 root root 64 Feb  7 15:00 18 -> /var/lib/mysql/blog/wp_term_taxonom                        y.MYI
lrwx------ 1 root root 64 Feb  7 15:00 2 -> /var/log/mysql/error.log
lrwx------ 1 root root 64 Feb  7 15:00 20 -> /var/lib/mysql/blog/wp_postmeta.MYD
lrwx------ 1 root root 64 Feb  7 15:00 21 -> /var/lib/mysql/blog/wp_term_taxonom                        y.MYD
lrwx------ 1 root root 64 Feb  7 15:00 22 -> /var/lib/mysql/blog/wp_terms.MYI
lrwx------ 1 root root 64 Feb  7 15:00 23 -> /var/lib/mysql/blog/wp_terms.MYD
lrwx------ 1 root root 64 Feb  7 15:00 3 -> /var/lib/mysql/ibdata1
lrwx------ 1 root root 64 Feb  7 15:00 4 -> /tmp/ibvANyz7 (deleted)
lrwx------ 1 root root 64 Feb  7 15:00 5 -> /tmp/ibonS0mU (deleted)
lrwx------ 1 root root 64 Feb  7 15:00 6 -> /tmp/ibcKctaH (deleted)
lrwx------ 1 root root 64 Feb  7 15:00 7 -> /tmp/ibB5DS5t (deleted)
lrwx------ 1 root root 64 Feb  7 15:00 8 -> /var/lib/mysql/ib_logfile0
lrwx------ 1 root root 64 Feb  7 15:00 9 -> /var/lib/mysql/ib_logfile1

December 12, 2011

HOW TO : Modify iptables rules

Filed under: HOWTO,Linux,Networking,security — Vinay @ 3:17 pm

Quick how to for my personal records. iptables is an open source firewall (and it does a lot more) included with most linux distributions.

Steps to add new rule to existing configuration

  • Check the list of rules and their corresponding sequence

sudo iptables -vL --line-numbers

  • Add the new rule at the required location/sequence

 sudo iptables -I INPUT LINE_NUMBER RULE

Example :

iptables -I INPUT 8 -s X.X.X.X/24 -p tcp -m state --state NEW -m tcp --dport 3128 -j ACCEPT

  • Save the configuration

 sudo serivce iptables save

Thx to Sijis for helping with the commands.

November 28, 2011

HOW TO : Fix Jboss startup script for CentOS

Filed under: HOWTO,Linux,Technology — Vinay @ 11:50 pm

Quick note for myself on fixing the default startup script provided by Jboss to work on CentOS. Thx to Shankar to finding the solution.

The default startup script (/$JBOSS_HOME/bin/jboss_init_redhat.sh) that Jboss provides does not work properly in CentOS. The start option works fine, but when you try to stop Jboss, it gives you a “No JBossas is currently running” message and quits.

Here’s a quick way to fix it. Edit the jboss_init_redhat.sh file and replace

JBOSSSCRIPT=$(echo $JBOSSSH | awk '{print $1}' | sed 's/\//\\\//g')

with

JBOSSSCRIPT=$(echo $JBOSSSH | awk '{print $1}')

November 13, 2011

HOW TO : Move your life into the cloud

Filed under: HOWTO,Management,Technology,Web — Vinay @ 10:41 pm

Nope… I am not too late to get on the “cloud” bandwagon :) . I started writing this post in Dec 2009 and here’s a screenshot of my drafts to prove it 

And I have finally decided that it is time to complete the post and publish it.

I change laptops every 6 months or so and a lot of my friends ask me how I manage to swap them so quickly and yet stay productive. I am sure a lot of you can relate to this. It usually take a month or so to get your workstation to a “state” that you feel comfortable with and are productive. Here are the tricks/tools I use to make the switching of a laptop/desktop to be a no-brainer activity. And I utilize the “cloud” heavily for this.

I adhere to a couple of simple rules to make sure I can be productive anywhere, even in situations, where I don’t have my workstation with me.

  • Everything I produce should be searchable
  • Everything I produce should be available on the web
  • Everything I produce should be easy to share

With these principles in mind, here are the services I use..

PHOTOS :

  • SERVICE : I use flickr to store all my pictures. I have taken ~40 thousand pictures since 2003 and everyone of them is online at http://www.flickr.com/photos/kudithipudi. I wish, flickr was around when I was a kid, so that I had a place to store all the pictures from my childhood instead of rotting away in some old cardboard box. I like Flickr for it’s simplicity and ease of use. There are other sites that offer a lot more features, but the features offered by Flickr are are more than enough for me.
  • COST : $24.95/year to upload/store unlimited number of pictures
  • OTHER CHOICES : There are plenty of photo storing/sharing sites. Some of the popular ones are picasa, photobucket, facebook

ON-LINE STORAGE :

  • SERVICE : I use dropbox to store any digital content I create. This overlaps a bit with the service I use to store documents I create. Dropbox is a service that allows you to synchronize files between different computers you have the agent installed on and at the same time stores them online for you. They offer 2GB of free space by default and you can earn more space by referring people to the service. (note : the links to dropbox are my referral links. If you sign up for the service, I get 250Mb of free space. If you don’t want to use the referral links, you can sign up for the service directly at www.dropbox.com).You would think 2GB is not a lot of space. But once you remove the music, movies and photos, you really don’t need a lot of space :) . For example, I haven’t crossed 1.8 GB, even though I have an electronic record of all my important files all the way from 2006. All I do, when I switch to a new laptop is install the dropbox agent and voila all my files are downloaded and synced with the latest copies.
  • COST : free. If you need more space, dropbox offers it for a cost.
  • OTHER CHOICES : There’s plenty of competition for dropbox, but I don’t think anyone of them have come close to making the sharing/storage work as seamless as dropbox. Some of the popular ones are box.net, SugarSync,wuala, Amazon Cloud Drive

DOCUMENTS :

  • SERVICE : I use Google Docs to create and store documents, spreadsheets and presentations. Since it’s inception in 2006 as a simple online editor and spreadsheets service, Google Docs has come a long way. There are few things you cannot do in Google Docs, that you can do in a full fledged productivity suite like Microsoft Office. Plus it gives you the capability to collaborate with other people when creating documents.
  • COST : free.
  • OTHER CHOICES : The only other service that comes close to Google Docs is Zoho Suites. Microsoft has a competing product, Office Live, but I think they are confused on how to market it because it will eat into their most profitable franchise (Microsoft Office)

EMAIL :

  • SERVICE : I use Gmail for my email. Although there is a standalone version,  I use it as part of the services provided by Google Apps for my domain (kudithipudi.org). It offers free spam protection, 7GB of space and super fast search. What else can one ask for? :)
  • COST : free
  • OTHER CHOICES : There are several free email hosting providers. Some of the popular ones are hotmail, yahoo, aol

ONLINE PRESENCE :

  • SERVICE : I strongly believe that all of us have to manage our online presence. And I don’t mean just for the folks that work in technology, but everyone that uses the Internet. And that is pretty much most of the people living on planet earth :) . There are several ways to do this (and I think that is for a another blog post), but the simplest way is to ensure you have a place where you can broadcast your presence. I use this blog as a way to document my thoughts, share ideas and in general manage  my on-line presence. I host this blog on a virtual server that I lease from Rackspace.
  • COST : $11/month
  • OTHER CHOICES : I would not recommend what I am doing for most people. There are several free platforms that you can host your blog on. I just do it this way, because I like to tinker with technology. Some of the popular blogging platforms are tumblr, blogspot, wordpress, squarespace.

October 31, 2011

HOW TO : Check web services using curl

Filed under: HOWTO,Linux,Networking,security,Web — Vinay @ 3:36 pm

Quick note for myself to check web services using curl ([L/U]nix utility to play with http(s) traffic)

 curl https://URL_TO_TEST --insecure --trace-ascii debug.txt

Comments on options :
–insecure is used if you are testing web services served over SSL using self signed certs
–trace-ascii dumps all traffic between the client (curl in this case) and the server in human readable format

« Newer Posts — Older Posts »

Powered by WordPress