For my notes.. if you ever want to check the IO capability of a disk (local or network) on a linux machine, use the following command
[bash] dd if=/dev/zero of=test.file bs=4M count=1000 [/bash]
The above command make a copy of the output from /dev/zero to a file called test.file (you can locate the file on the disk you want to measure) with a block size of 4M for a total file size of 4000Mb.
Bold forecast :).. But in 5 to 10 years, I predict the majority of a company’s infrastructure will be hosted in a “cloud”. If you recall (circa 2000..), most of the companies were hosting “anti-spam” services in house. If anyone suggested that we can outsource that service, you would get a “are-you-crazy” look :). And now, you will get the same look if anyone suggests they run the anti-spam service in house. I believe the same is going to happen for infrastructure. You might still be running some components in house, but it will get smaller and smaller. Companies will be forced to focus on their core competency rather than try to maintain an army of engineers to perform tasks that someone else might be a lot better at.
Speaking of being visionary, apparently Netflix operates most of their infrastructure in the cloud. If Netflix can operate in the cloud, a majority of us can too :). Here are some links regd their lessons from moving to a cloud.
http://blip.tv/file/4252897 (Video of Netflix Director of Engineering explaining their move to the cloud)
https://docs.google.com/viewer?a=v&pid=sites&srcid=ZGVmYXVsdGRvbWFpbnxwcmFjdGljYWxjbG91ZGNvbXB1dGluZ3xneDo2NDc2ODVjY2ExY2Y1Zjcz&pli=1 (Write up by a Netflix engineer about the move to the cloud from a storage and DB prospective)
This picture (credited to Uptime Software) shows how you can classify outages so that you know which ones to tackle first. A very good way to take stock of things and show progress over time.
There are several ways to run commands repetively in Linux.. (cron, while loop etc). Here is a simple trick. Say you want to check the size of a directory using “du” every 5 seconds, you can use the following command
[bash]watch -n 5 du -ch[/bash]
Check out all the options available in watch by running “man watch” :).
For my notes.. If you ever wanted to check the status of a bonded interface configure in Linux (esp RHEL), you can check the status by running the following command
[root@serverxyz bin]# cat /proc/net/bonding/bond0
i.e. assuming the name of your bond interface is bond0.
Output from the command
Ethernet Channel Bonding Driver: v3.4.0 (October 7, 2008) Bonding Mode: fault-tolerance (active-backup) Primary Slave: eth3 (primary_reselect always) Currently Active Slave: eth3 MII Status: up MII Polling Interval (ms): 100 Up Delay (ms): 0 Down Delay (ms): 0 Slave Interface: eth3 MII Status: up Link Failure Count: 0 Permanent HW addr: 00:10:18:6e:b8:1a Slave Interface: eth0 MII Status: up Link Failure Count: 0 Permanent HW addr: 00:21:5e:11:34:32
The configuration files involved are
/etc/sysconfig/network-scripts/ifcfg-bond0 (Bond Interface)
DEVICE=bond0 IPADDR=10.10.40.26 NETMASK=255.255.255.0 ONBOOT=yes BOOTPROTO=none USERCTL=no GATEWAY=10.10.40.1 NETWORK=10.10.40.0 BROADCAST=10.10.40.255 TYPE=Ethernet
/etc/sysconfig/network-scripts/ifcfg-eth3 (Primary Interface)
DEVICE=eth3 BOOTPROTO=none ONBOOT=yes HWADDR=00:10:18:6e:b8:1a MASTER=bond0 SLAVE=yes TYPE=Ethernet USERCTL=no
/etc/sysconfig/network-scripts/ifcfg-eth0 (Secondary Interface)
DEVICE=eth0 HWADDR=00:21:5e:11:34:32 USERCTL=no ONBOOT=yes MASTER=bond0 SLAVE=yes BOOTPROTO=none TYPE=Ethernet
We ran into an interesting issue at work recently. Documenting the solution for my records..
BACKGROUND : We had a table in one of our databases that served as a “hopping” point for some jobs. Data was inserted into this table and at jobs get kicked off at periodic intervals to “process” the data and delete it.
CURRENT METHOD : Launch multiple jobs to process the data and delete the rows as soon as the data is processed. This is causing locks on the table because there are multiple delete operations occurring at the same time. Which in turn means that the jobs cannot complete processing the data causing the table to grow in size.
PROPOSED METHOD : Add a new column to the table called “PROCESSED_STATE” and modify the “processing” jobs to set a flag “Y” in this column as soon as the data is processed. Create a new job that will be launched periodically, which checks the PROCESSED_STATE column and if the flag is set to “Y”, deletes the row.
Morale of the story.. 🙂 .. Multiple deletes on a table are bad. Better way is to have multiple updates and one delete.
Your site goes down 🙂
And traffic to the site drops!!
Things have been a bit crazy at work recently, so I didn’t get a chance to fix the site as soon as it went down (due to an error I still haven’t figured out). And as a result, the traffic to the site dropped.
I finally took the chance to move the site to a dedicated server running on the RackSpace Cloud services. Am putting together a post on how I handled this migration and will publish it soon.
The bad news is that I have lost traffic to the site that I have built over a period of time.. the good news is that I am the master of my own house (website) at last :).
I don’t have a Facebook or twitter account and that surprises a lot of my friends since I am such a geek :). And the reason I keep (kept) giving was that I want to be in control of my destiny. In this case, destiny being content. While Facebook and Twitter provide you with a easy way to connect with friends/relatives/stalkers etc, I believe it gives a lot of leeway on control over the content for the companies running these applications. I have all the means and ways to communicate with my friends and advertise what I need to world. How I do that? That is a blog post that I have been “drafting” for the last couple of months :)..Hope to publish it soon. And looks like the wider audience is finally waking up to it.
Check out this article on ReadWriteWeb regd how the tech leaders are calling for a boycott of Facebook and advocating for an open social networking protocol (http://www.readwriteweb.com/archives/more_web_industry_leaders_quit_facebook_call_for_o.php).
Another article on the same website, speaks about a study by the Advanced Institute of Science and Technology in Korea, which shows that Twitter is really not a social networking site, but more of a medium to broadcast your content (http://www.readwriteweb.com/archives/study_twitter_isnt_very_social.php). Doesn’t really support the argument I made earlier that Twitter is not going to make it..but it certainly supports the notion that once the hype is gone the influence of twitter as a medium will decrease.
I am sure a lot of people shop on express.com , but I probably get the credits for being the first blogger to post that express.com has not been responding to DNS queries since ~7:00 PM CST (4/26). Looks like Qwest is hosting DNS for Express. The name servers (most probably global load balancers) are not responding to DNS requests.
Here’s what I get, when I queried for www.express.com
I feel for the poor ops team scrambling around to bring up the service :). Another reason, you want diversity in your DNS hosting.
Instructions for configuring the mime type mappings in Jboss. Mime types essentially tells the application processing the content (typically a browser), what the content is. More information here (http://en.wikipedia.org/wiki/Internet_media_type).
