Translation(s): none


Linux "Huge page tables" (HugeTlbPage) is available in Debian since DebianLenny (actually, since 2.6.23).

Currently, there is no standard way to enable HugeTLBfs, mainly because the FHS has no provision for such kind of virtual file system, see 572733 . (Fedora mounts it in /dev/hugepages/, so don't be surprised if you find some example on the web that use this location)

Read the documentation for more information about hugetlbpage.

Enabling HugeTlbPage

  1. Create a group for users of hugepages, and retrieve it's GID (is this example, 2021) then add yourself to the group.
    Note: this should not be needed for libvirt (see /etc/libvirt/qemu.conf)

    % groupadd my-hugetlbfs
    
    % getent group my-hugetlbfs
    my-hugetlbfs:x:2021:
    
    % adduser franklin my-hugetlbfs
    Adding user `franklin' to group `my-hugetlbfs' ...
    Adding user franklin to group my-hugetlbfs
    Done.
  2. Edit /etc/sysctl.conf and add this text to specify the number of page you want to reserve (the HugePages are 4MiB on x86_32, 2MiB on x86_64 and x86_32 PAE)

    # Allocate 256*2MiB for HugePageTables (YMMV)
    vm.nr_hugepages = 256
    
    # Members of group my-hugetlbfs(2021) are allowed to use 
    vm.hugetlb_shm_group = 2021
  3. Create a mount point for the file system

    % mkdir /hugepages
  4. Add this line in /etc/fstab (The mode of 1770 allows anyone in the group to create files but not unlink or rename each other's files.1)

    hugetlbfs /hugepages hugetlbfs mode=1770,gid=2021 0 0
  5. Reboot 2

limits.conf

You should configure the amount of memory a user can lock, so an application can't crash your operating system by locking all the memory. (keep in mind that any page can be locked in RAM, not just huge pages!).

== Multiple huge page size support ===

See:

Getting informations

Well, typically you can retrieve the current available/used page from /proc/meminfo:

(read Documentation/vm/hugetlbpage.txt for more information about it)

Standard Debian Kernel have HUGETLB enabled:

Various runtime settings (see documentation)

Other settings

Some application (Java...?) may need to configure the locked memory limit (ulimit -l and/or memlock in /etc/security/limits.conf).

Some hints here on ibm.com.

Tools

Hugepage enabled applications

An application can allocate/use HugeTlbPage through two different means:

  1. mmap system call require a mounted hugetlbfs, with appropriate permissions.

  2. Shared memory segment (shmat/shmget system calls or mmap with MAP_HUGETLB), must be member of a group, configured in /proc/sys/vm/hugetlb_shm_group.

    Application hugetlbfs shared memory
    QEMU/KVM No Yes
    MySQM Yes No
    Java No Yes

MySQL

Linux HugeTLBfs: Improve MySQL Database Application Performance
http://www.cyberciti.biz/tips/linux-hugetlbfs-and-mysql-performance.html (note tested)

Java (Sun, OpenJDK)

Sun and OpenJDK can use large pages.

Basically, it seems that one have to launch java with -XX:+UseLargePages. The maximum size of the Java heap (-Xmx) should fit in your reserved Huge pages ; same for ulimit -l and/or memlock in /etc/security/limits.conf.

See:

Possible errors:

PosgreSQL

Humm doesn't seems so... but see that hack: hugetlblib above.

Virtualisation

Some consideration about hugepages an virtualisation.

  1. Before enabling hugepages in a virtual machine, you should make sure the that your virtualization tool can handle it.
  2. Whether a virtualization tools supports hugepages for it's client or for itself are probably two different aspects.

KVM

Xen

It is unclear which version of Xen supports huge pages, and how it is used.

Documentation

See also


  1. Credits: Russell Coker (1)

  2. This is the most reliable method of allocating huge pages as memory has not yet become fragmented. Yes, you finally found an operation that actually recommend a reboot in Linux ;) (2)