HOW TO : Configure Jboss to use hugepages in RHEL/CentOS

Most of us worry about paging to disk (swap), but if you are running a transaction intensive application the paging that happens in RAM also starts to impact the application performance. This happens due to the size of the “block” that is used to store data in memory. Hugepages allows you to store the data in bigger blocks, hence reducing the need to page while interacting with the data.

Here is how you can enable hugepages and configure jboss (actually any Java app) to use hugepages on a RHEL/CentoOS system.

OS CONFIGURATION

1. Check if your system is capable of supporting hugepages by running[code]grep HUGETLB /boot/config-`uname -r`[/code]

   If you see the response as below, you should be good[code]CONFIG_HUGETLBFS=y
   CONFIG_HUGETLB_PAGE=y
   [/code]

• Next check if huge pages are already being used by running[code]cat /proc/sys/vm/nr_hugepages [/code]

1. If the response is anything other than 0, that means hugepages have already been configured.

• Find the block size for hugepages by running[code]cat /proc/meminfo | grep -i hugepagesize [/code]
• Calculate the amount of memory you want to dedicate to hugepages. (note: memory allocated to hugepages cannot be used by other processes in the system, unless they are configured to use it)

1. For example, I want to dedicate 3GB of RAM for hugepages. So the number of hugepages would be[code](3*1024*1024)/2048[/code]

• Configure the number of hugepages on the system by editing the /etc/sysctl.conf and adding the option[code]vm.nr_hugepages = 1536[/code]

  (note: I put in 1536 since that was the value I got from the above example)

• Restart the server and check if hugepages has been enabled by running[code]cat /proc/meminfo | grep -i huge [/code]

1. You should see something like this[code]AnonHugePages:    839680 kB
   HugePages_Total:    1500
   HugePages_Free:     1500
   HugePages_Rsvd:        0
   HugePages_Surp:        0
   Hugepagesize:       2048 kB
   [/code]

JBOSS CONFIGURATION

1. At this point your system is configured with hugepages and any application that is configured to use them can leverage them.  In this example, we want to configure Jboss to utilize these hugepages
2. Add the groupid of the user that Jboss is running under to the /etc/sysctl.conf file. In my case, the jboss user group had a GID of 505, so I added this line to /etc/sysctl.conf[code]vm.hugetlb_shm_group = 505 [/code]
3. Next allocate the memory to the user by editing /etc/security/limits.conf and allocating the memory. Again, in my case, I added the following to /etc/security/limits.conf[code]# Allocate memory for Jboss user to take advantage of hugepages
   jboss   soft    memlock 1500
   jboss   hard    memlock 1500
   [/code]
4. Finally add the following to the Jboss startup parameters. I edited the $JBOSS_HOME/bin/run.sh file. (note: the startup file can be different based on your config) with the option[code] -XX:+UseLargePages[/code]
5. Restart Jboss and you are good to go

note : A lot articles that I read online say that hugepages are effective when you are allocating large amounts of RAM to the application. The use case of just using 3GB above was just that.. a use case.

While I cannot personally vouch for it, a lot of users have noted that they saw >2 fold increase in performance.