admin

HOW TO : Sort Apache Web Logs for hits by Unique IP Addresses

 

Say you want to find out how many hits you are getting t0 a specific page from a particular source IP, you can use this quick collection of Linux tools to get this data

[code]grep -i "URL_TO_CHECK" PATH_TO_APACHE_ACCESS_LOG | cut -d’ ‘ -f 1 -| sort |uniq -c | sort -rn > ~/ip_report.txt[/code]

You are using

  • grep to filter the string of the page you want the report on
  • cut to get the IP address from the log file
  • sort and uniq to sort the unique IP addresses
  • and finally sort -rn to sort the data in descending order

Example :

[code]grep -i "GET /" /opt/apache/logs/access_log | cut -d’ ‘ -f 1 -| sort |uniq -c | sort -rn > ~/ip_report.txt[/code]

gets you the report of hits to the index page.

February 29, 2012 by HOWTO Linux Technology Uncategorized Web 2

HOW TO : Find list of files used by a process in Linux

Quick howto on finding the list of files being accessed by a process in Linux. I needed to find this for troubleshooting an issue where a particular process was using an abnormally high percentage of CPU. I wanted to find out what this particular process was doing and accessing.

  1. Find the process ID (pid) of the process you want to analyze by running[code] ps -ef | grep NAME_OF_PROCESS [/code]
  2. Find the files the process is accessing at a given time by running[code]sudo ls -l /proc/PROCESS_ID/fd [/code]

For example, if I wanted to find the list of files being accessed by mysql, the process would look as such

[code] ps -ef | grep mysqld [/code]

which would show the output as

[code]samurai@samurai:~$ ps -ef | grep mysqld
mysql     3304     1  0 Feb04 ?        00:00:23 /usr/sbin/mysqld
samurai  23389 23374  0 14:57 pts/0    00:00:00 grep –color=auto mysqld
[/code]

I can then find the list of files being used by mysql by running

[code] sudo ls -l /proc/3304/fd [/code]

which would give me

[code]

lrwx—— 1 root root 64 Feb  7 15:00 0 -> /dev/null
lrwx—— 1 root root 64 Feb  7 15:00 1 -> /var/log/mysql/error.log
lrwx—— 1 root root 64 Feb  7 15:00 10 -> socket:[4958]
lrwx—— 1 root root 64 Feb  7 15:00 11 -> /tmp/ibdu9WRh (deleted)
lrwx—— 1 root root 64 Feb  7 15:00 12 -> socket:[4959]
lrwx—— 1 root root 64 Feb  7 15:00 14 -> /var/lib/mysql/blog/wp_term_relatio                        nships.MYI
lrwx—— 1 root root 64 Feb  7 15:00 15 -> /var/lib/mysql/blog/wp_postmeta.MYI
lrwx—— 1 root root 64 Feb  7 15:00 17 -> /var/lib/mysql/blog/wp_term_relatio                        nships.MYD
lrwx—— 1 root root 64 Feb  7 15:00 18 -> /var/lib/mysql/blog/wp_term_taxonom                        y.MYI
lrwx—— 1 root root 64 Feb  7 15:00 2 -> /var/log/mysql/error.log
lrwx—— 1 root root 64 Feb  7 15:00 20 -> /var/lib/mysql/blog/wp_postmeta.MYD
lrwx—— 1 root root 64 Feb  7 15:00 21 -> /var/lib/mysql/blog/wp_term_taxonom                        y.MYD
lrwx—— 1 root root 64 Feb  7 15:00 22 -> /var/lib/mysql/blog/wp_terms.MYI
lrwx—— 1 root root 64 Feb  7 15:00 23 -> /var/lib/mysql/blog/wp_terms.MYD
lrwx—— 1 root root 64 Feb  7 15:00 3 -> /var/lib/mysql/ibdata1
lrwx—— 1 root root 64 Feb  7 15:00 4 -> /tmp/ibvANyz7 (deleted)
lrwx—— 1 root root 64 Feb  7 15:00 5 -> /tmp/ibonS0mU (deleted)
lrwx—— 1 root root 64 Feb  7 15:00 6 -> /tmp/ibcKctaH (deleted)
lrwx—— 1 root root 64 Feb  7 15:00 7 -> /tmp/ibB5DS5t (deleted)
lrwx—— 1 root root 64 Feb  7 15:00 8 -> /var/lib/mysql/ib_logfile0
lrwx—— 1 root root 64 Feb  7 15:00 9 -> /var/lib/mysql/ib_logfile1
[/code]

February 7, 2012 by HOWTO Linux Technology Uncategorized 1

Overheard : Comment about trust and security

Very thought provoking comment on trust and security by Mark Boyle, the Moneyless Man, on a recent episode of PRI‘s To the best of our knowledge program (I personally transcribed this.. so pls overlook any minor typos 🙂 )

What money has become is.. a substitute for trust. It has now become our primary source of security in the world and what I am trying to do personally is to find my primary source of security in the friendships I have and in my local community and my relationship with earth. Because most countries, such as Argentina and Indonesia and currently Zimbabwe have experienced this hyperinflation and you can have a million dollars in the bank. One day with devaluation, it can almost be worthless. No matter how badly I behave, my friend’s don’t devalue me that quickly. And I think real security comes in our relationships, whether to it be with our planet or whether with our local community. I think what we all can do is build a bit more diversity in how we meet our needs and to not be so reliant on cash.

You can get the full interview at http://feedproxy.google.com/~r/TTBOOK/~3/X009WjbiqB0/tbk120205a.mp3. Segment with Mark starts at ~42 min.

February 5, 2012 by OverHeard Uncategorized 0

Overheard : Comment on Work

I was standing in line to get into a plane yesterday and heard this comment made by a gentleman to his friend

You know.. funny thing about work, it has to get done!!

The guys were discussing about how their wives don’t understand the pressures of work :).

February 2, 2012 by Fun OverHeard Uncategorized 0

Protesting SOPA and PIPA

Unless you are living under a rock or outside the US :).. you probably heard about the crazy legislation that the US congress and senates are proposing to help protect content creators (AKA Hollywood) from privacy. While I personally don’t have any issues with giving protection to content creators, it should not be at the cost of freedom for the rest of the world. Go to http://americancensorship.org/ to get more information about why this proposed legislation are bad.

Today (1/18/2012) has been designated as “Protest SOPA/PIPA day” by the technology world. I believe in the old adage, put your money where your mouth is :).. so I checked on the top 25 US sites (according to Alexa) to see how many of them are supporting this protest in a visible manner. Only 4 out of the 25 sites, put visible content on their websites regarding the protest. I think Google’s message was the most effective, where they did not reduce the functionality of the website, but provided a lot of visibility to the protest. I know which companies I am going to support/use moving forward :). I was very happy to see that three of the sites that I use on a regular basis (google, amazon and wikipedia) are supporting this protest. Here are screenshots of the protest from the the  4 sites that are in the top 25 visited sites in the US

Google.com

Amazon.com

Wikipedia.Org

WordPress.com

Screenshots of some other sites that I visit on a regular basis and are supporting the protest

Boingboing.net

Wired.com

Arstechnica.com

Reddit.com

DuckDuckGo.com

January 18, 2012 by Politics Technology Uncategorized Web 0

G+ or Blog

I started using Google Plus from last November and I should say that, even though I am a big proponent of keeping control over your digital avatar, it has been very easy to make (give) quick updates on Google plus than on this blog. Plus my friends and family don’t have to specially come to this site to get updates. They get the G+ updates as part of their regular email and/or when they log into their G+ stream. It is less work on everyones part.

That is one of the reasons, I believe G+ will be one of the first real contenders to Facebook. Even though Facebook boasts of more than 800 million users, it is still a “seperate” site that folks have to log into unlike Google plus, which is fast becoming part of the regular Google experience. Esp with the tweaks that Google made last week with incorporating G+ data into the search results, the line between  a Google search and using Google Plus gets blurrier.

So the question (for me) is not if it is Facebook or G+.. but if it is the blog or G+..

 

January 15, 2012 by Rantings Technology Uncategorized 3

Virat Kudithipudi

New year… new addition to the family :).

Virat Kudithipudi, was born on 1/9/2012 at 22:22.

We went through a lot at the end of the pregnancy. First Jhanvi got shingles and then I contracted chickenpox from her. At one point, I didn’t even know if I would be able to be at his delivery. But thankfully everything worked out and I was cured by the time, Virat decided to arrive on planet earth. When the pediatrician checked him the day after his birth and announced that he was perfectly “healthy”, I chocked up. I understand you better now mom :)..

BTW.. if you didn’t get chickenpox as a kid.. run to the drugstore and get vaccinated for it. You don’t want to go through what I went 🙂

Welcome to the world kiddo..

Virat Kudithipudi

Collection of pictures of the first few days..

http://www.flickr.com/photos/kudithipudi/collections/72157628830321025/

January 15, 2012 by Family Uncategorized 5

Goodbye 2011 and Welcome 2012

 

Happy New Year!!

Wishing all of you and your families a healthy and loving 2012. If there is one thing I learnt in 2011, it is that if you are healthy rest of the things in life will follow.

I haven’t posted any updates on my 2011 resolutions in the last few months. So here is a final recap

  • Lose the gut
    • I really didn’t accomplish the goal of losing the gut. But I gained some good habits (eating breakfast, eating consistently throughout the day, keeping track of my weight)
    • I have collected some interesting data as part of tracking my weight through the year. Plan to do some analysis on it later on.
    • Final chart of my weight in 2011
  • Increase traffic to Kudithipudi.org
    • I wrote a bit more in 2011 than in 2010, but not as much as needed.
    • Snapshot of 2011 traffic compared to 2010. As you can see, the traffic has increased nearly 80%.. 
  • CISSP Certification
    • Didn’t attempt this at all.
    • Wanted to do this in the last quarter, but things got a bit hectic :).
  • Take a Vacation
    • Successfully went on a vacation to India.

Overall, 50% of the goals were accomplished. As they say, the glass is half full :). Not bad.

And yes, I am going back to my old habit of not making resolutions :).

January 2, 2012 by Resolutions Uncategorized 0

HOW TO : Modify iptables rules

Quick how to for my personal records. iptables is an open source firewall (and it does a lot more) included with most linux distributions.

Steps to add new rule to existing configuration

  • Check the list of rules and their corresponding sequence

[code]sudo iptables -vL –line-numbers [/code]

  • Add the new rule at the required location/sequence

[code] sudo iptables -I INPUT LINE_NUMBER RULE [/code]

Example :

[code]iptables -I INPUT 8 -s X.X.X.X/24 -p tcp -m state –state NEW -m tcp –dport 3128 -j ACCEPT[/code]

  • Save the configuration

[code] sudo serivce iptables save [/code]

Thx to Sijis for helping with the commands.

December 12, 2011 by HOWTO Linux Networking security Uncategorized 0