admin

Project PaaS : Day 2 on Google App Engine

It looks like I was able to accomplish writing the application that I wanted to on the App Engine in 2 days!!  at least in it’s basic form.  After some help from Google, I updated the application I created yesterday (http://samurai-apps.appspot.com/) to display the User Agent string being sent by the client.

The code has been updated to github at https://github.com/kudithipudi/google-app-engine/

Lessons from day 2?

  • Python doesn’t like tabs :). Always use spaces to ident. I was using Notepad++ as the editor and it automatically puts tabs when you hit enter. Why? Looks like that is the best practice according to this style guide (http://www.python.org/dev/peps/pep-0008/)
  • The “Logs” console in the SDK toolkit should be your best friend. It let’s you know if there is any error in your code and what line it believes the error is at.

Next, I will try to pretty it up a bit.

Isn’t it amazing that I was able to create a simple app in a matter of 2 days and host it on an “infinitely” scalable  platform without even taking our my credit card.

Project PaaS : Day 1 on Google App Engine

Following up on my public resolution for his month..I started playing with Google App engine. I think a lot has been written about what it is and how it works, but in a nutshell, think of it as an environment to deploy your applications and not have to worry about underlying system capacity. It provides support for Java, Python and more recently Google’s own Go programming languages.

I chose Python, since I have been meaning to dabble in it for a while now. So without further adieu, here is a link to my first application on Google App Engine

http://samurai-apps.appspot.com/

And obviously it has to be hello world :).

How did I get here?

  1. As any good programmer would do, I first tried to find a good place to store my source code. I chose Github, since it seems to be the goto place for hackers (in the good sense 🙂 ) in recent times. I opened a free account on it and created a repository called google-app-engine at https://github.com/kudithipudi/google-app-engine/ .
  2. Following instructions listed here https://developers.google.com/appengine/docs/python/gettingstartedpython27/ and created the helloworld script.
  3. Enabled App Engine on my account by validating myself. Had to use my mobile phone to do the validation.
  4. Created an app called samurai-apps in the Google App Engine control panel
  5. Deployed the helloworld script to the Google Apple engine using the deploy function in the SDK tool. (note: make sure that the name of the app you create in the SDK is the same as the one you created in the app engine control panel. Or you will get an error stating “This application does not exist (app_id=u’xxx’).” where  xxx is the name of the app in the SDK tool)

Pretty simple eh.. 🙂

Now the challenge is to program something more useful than print hellworld :).

 

2012 May Project

Continuing on my project/month theme.. here is what I want to accomplish in May 2012

  • Understand Google App Engine and Windows Azure Platforms
  • Write a simple application and deploy it both the platforms
  • The application that I am envisioning will display the “user agent” string of the client trying to access the application. I know there are tons of sites that already do this.. but I think this is an useful tool to have in your bag of tricks :). It is simple enough that I think I can program it in a month.

Why am I doing this? I understand the IaaS area pretty well, but am not well versed in the PaaS arena. Hoping this adventure will teach me some new things. And yes, I do plan on documenting my journey :).

Wish me luck 🙂

HOW TO : Configure Jboss to not show backend server name when proxying https (ssl) traffic

Phew.. that was a long title :).  Was running into an issue with the setup shown in the picture below

When we try to access the web site using https, the html content being served back was showing the app server name as the reference, rather than the web site.

So in this example, let’s say the web address was kudithipudi.org and the app server was app-server-kudithipudi, the HTML content was showing https://app-server-kudithipudi:8080 as the source.

Here’s how, we fixed it.

Edit the server.xml file found in $JBOSS_HOME/server/$JBOSS_PROFILE/deploy/jboss-web.deployer and update the HTTPS connector to use the web address (kudithipudi.org) as the proxyName.

BEFORE

[code]<Connector port="8443" protocol="HTTP/1.1" SSLEnabled="true"
maxThreads="250" scheme="https" secure="true"
clientAuth="false" sslProtocol="TLS"
keystoreFile="/opt/jboss/jboss-as/server/kudithipudi/conf/ssl/kudithipudi.keystore"
keystorePass="xxxxxx" />
[/code]

AFTER

[code]<Connector port="8443" protocol="HTTP/1.1" SSLEnabled="true"
maxThreads="250" scheme="https" secure="true"
clientAuth="false" sslProtocol="TLS"
proxyName="kudithipudi.org" proxyPort="443"
keystoreFile="/opt/jboss/jboss-as/server/kudithipudi/conf/ssl/kudithipudi.keystore"
keystorePass="xxxxxx" />

[/code]

HOW TO : grep options to display before and after lines of matching content

For my own notes.. if you are using grep to parse through the contents of a file and want to see the preceding or proceeding content than the line that matched your query, you can use the following options

preceding content [code]grep -B NUMBER_OF_LINES_TO_DISPLAY query filename[/code]

for example, if I was searching for kudithipudi in a file names access.log and want to see 2 lines prior to the match, I would use [code]grep -B 2 kudithipudi access.log[/code]

proceeding content[code]grep -A NUMBER_OF_LINES_TO_DISPLAY query filename[/code]

for example, if I was searching for kudithipudi in a file names access.log and want to see 2 lines after the match, I would use [code]grep -A 2 kudithipudi access.log[/code]

preceding and proceeding content[code]grep -C NUMBER_OF_LINES_TO_DISPLAY query filename[/code]

for example, if I was searching for kudithipudi in a file names access.log and want to see 2 lines before and after the match, I would use [code]grep -C 2 kudithipudi access.log[/code]

HOW TO : Use templates in puppet to pass hostnames

puppet, is a configuration management framework that can be used to perform several different things to validate/configure your infrastructure. We have been using puppet for sometime at my work and have just started moving into some of the advanced uses of the tool.

One of the features offered by puppet is the capability to use templates to configure different servers.

For example, say you want to configure an application on server ABCD, XYZ and 123. And the configuration file for all these servers is the same, other than the hostname of the server. The configuration file has to reside in /opt/application/config.conf . The config.xml file looks like this

[code]

db.name=blah
db.user=blahblah
db.hostname=XYZ
log.level=ERROR
log.location=/var/log/application

[/code]

Here is how you can do it in puppet.

Define a module which uses a template and then configure the template to put the host specific entry in the template. Let’s name our module test_config

  • Create the module
    • cd $PUPPET_HOME/modules
    • mkdir test_config/{files,manifests,templates}
  • Create the template
    • cd templates
    • vi config.conf.template and add the following to the file[code]db.name=blah
      db.user=blahblah
      db.hostname=<%= fqdn %>
      log.level=ERROR
      log.location=/var/log/application [/code]
      • note : see how I replaced the hostname XYZ, which was specific to one server with <%= fqdn %>. This is one of the “facts” provided by puppet. you can get a list of all the facts by running facter on any of the puppet clients.
  • Configure the module to use the template. In this case, we want the module to place the file config.conf in /opt/application
    • cd manifests
    • vi init.pp and add the following to the file[code]class test_config {
      file { "/opt/application/config.conf":
      ensure => present,
      owner => appuser,
      group => appuser,
      mode => 755,
      content => template("test_config/config.conf.template"),
      }
      }[/code]
      • note : There are several other options you can use for the class file.. I just gave an example of some of the common ones. Like setting the owner, group and the rights.
  • Finally configure the clients to use the module. In the individual node config files, include the module you just created. Here is how the config for node ABCD would look like[code]node ABCD {
    include test_config
    }[/code]

The next time the puppet client runs on host ABCD, it would create the file /opt/application/config.conf with the right hostname in the config file.

HOW TO : Configure Jboss to append log files instead of overwriting them

If you use the default logging options for Jboss, it has a nasty habit of overwriting log files on a restart. So, if you were in the middle of troubleshooting an issue and had to restart Jboss, you will end up loosing all the historic data. You can change this default behavior by changing one option in the log4j config file

  • Edit the $JBOSS_HOME/server/$JBOSS_PROFILE/conf/jboss-log4j.xml and replace [code]<param name="Append" value="false"/>[/code]

    with [code]<param name="Append" value="true"/>[/code]

  • You don’t even have to restart Jboss for this new setting to take place, since Jboss reads the log4j config every 60 seconds and updates the logging parameters accordingly.

Project Uptime : Recap

Final post on Project Uptime. Before I go into the details of how well I fared against the original goals, here is a screenshot of the uptime of the site for the last two weeks.. Hope I can keep that number for the rest of this year :). Recap of the original goals and their status

GOAL : Move to a fresh VM with the latest kernel

  • STATUS : This was successfully completed. Details of the install are here and here.

GOAL : Upgrade to the latest version of Apache.

  • STATUS : Although I didn’t install 2.4 version of Apache as planned, I was able to get similar (or better) performance by installing varnish as a caching engine. Details of thee apache/varnish install are here.

GOAL : Upgrade to latest version of MySQL and tune it for memory usage

  • STATUS : This was probably the easiest part of the project. Details of the install are here. I didn’t necessarily tweak it for low memory usage though.

GOAL : Configure cloudflare to serve a static version of front page, in case the server goes down. Design the static page to point people to my other digital presences (Google+, LinkedIn, Flickr etc)

  • STATUS : Cloudflar doesn’t have the capability to direct static pages if the origin server is down. At least not for the free tier. I also didn’t design a simple page to host my digital presence. Will try to find a good template for it down the road.

All in all.. not bad 🙂