4834
Comment:
|
6108
Add Java (plus minor formatingà
|
Deletions are marked like this. | Additions are marked like this. |
Line 7: | Line 7: |
<<TableOfContents(2)>> | <<TableOfContents(3)>> |
Line 75: | Line 75: |
== Tools == * libhugetlbfs is a library which provides easy access to huge pages of memory. The library also comes with several userspace tools to help with huge page usability, environment setup, and control. ~-<<BR>>http://libhugetlbfs.ozlabs.org/ <<BR>> DebianBug:533708 - ITP: libhugetlbfs -- Initial package request -~ |
|
Line 83: | Line 85: |
Java ; No ; Yes | |
Line 88: | Line 91: |
* Get a performance boost by backing your KVM guest with hugetlbfs ~-<<BR>>[[http://www.linux-kvm.com/content/get-performance-boost-backing-your-kvm-guest-hugetlbfs]] (not tested)-~ | |
Line 93: | Line 96: |
=== Java (Sun, OpenJDK) === Sun and OpenJDK can use large pages. * DebianPkg:sun-java6-bin - Sun Java(TM) Runtime Environment (JRE) 6 (architecture dependent files) * DebianPkg:openjdk-6-jre-headless - OpenJDK Java runtime, using Hotspot JIT (headless) http://java.sun.com/javase/technologies/hotspot/largememory.jsp __Possible errors:__ * Insufficient {{{/proc/sys/kernel/shmmax}}} ? * Not in group defined in {{{/proc/sys/vm/hugetlb_shm_group}}} {{{#!text % java -XX:+UseLargePages Java HotSpot(TM) Server VM warning: Failed to reserve shared memory (errno = 1). Java HotSpot(TM) Server VM warning: Failed to reserve shared memory (errno = 22). }}} |
|
Line 101: | Line 122: |
* Get a performance boost by backing your KVM guest with hugetlbfs ~-<<BR>>[[http://www.linux-kvm.com/content/get-performance-boost-backing-your-kvm-guest-hugetlbfs]] (note tested)-~ | * Article about Huge pages, on LWN.net ~-<<BR>>[[http://lwn.net/Articles/374424/|Introduction]] - [[http://lwn.net/Articles/375096/|Interfaces]] - [[http://lwn.net/Articles/376606/|Administration]] - [[http://lwn.net/Articles/378641/|Benchmarking]]-~ |
Translation(s): none
Linux "Huge page tables" (HugeTlbPage) is available in Debian since DebianLenny (actually, since 2.6.23).
Contents
Currently, there is no standard way to enable HugeTLBfs, mainly because the FHS has no provision for such kind of virtual file system, see 572733 . (Fedora mounts it in /dev/hugepages/, so don't be surprised if you find some example on the web that use this location)
Read the documentation for more information about hugetlbpage.
Enabling HugeTlbPage
Create a group for users of hugepages, and retrieve it's GID (is this example, 2021) then add yourself to the group.
Note: this should not be needed for libvirt (see /etc/libvirt/qemu.conf)% groupadd my-hugetlbfs % getent group my-hugetlbfs my-hugetlbfs:x:2021: % adduser franklin my-hugetlbfs Adding user `franklin' to group `my-hugetlbfs' ... Adding user franklin to group my-hugetlbfs Done.
Edit /etc/sysctl.conf and add this text to specify the number of page you want to reserve (the HugePages are 4MiB on x86_32, 2MiB on x86_64 and x86_32 PAE)
# Allocate 256*2MiB for HugePageTables (YMMV) vm.nr_hugepages = 256 # Members of group my-hugetlbfs(2021) are allowed to use vm.hugetlb_shm_group = 2021
Create a mount point for the file system
% mkdir /hugepages
Add this line in /etc/fstab (The mode of 1770 allows anyone in the group to create files but not unlink or rename each other's files.1)
hugetlbfs /hugepages hugetlbfs mode=1770,gid=2021 0 0
Reboot 2
Getting informations
Well, typically you can retrieve the current available/used page from /proc/meminfo:
% grep Huge /proc/meminfo HugePages_Total: 256 HugePages_Free: 256 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 4096 kB
(read Documentation/vm/hugetlbpage.txt for more information about it)
Standard Debian Kernel have HUGETLB enabled:
% grep HUGETLB /boot/config-$(uname -r) CONFIG_HUGETLBFS=y CONFIG_HUGETLB_PAGE=y
Various runtime settings (see documentation)
% grep -R "" /sys/kernel/mm/hugepages/ /proc/sys/vm/*huge* /sys/kernel/mm/hugepages/hugepages-4096kB/nr_hugepages:256 /sys/kernel/mm/hugepages/hugepages-4096kB/nr_overcommit_hugepages:0 /sys/kernel/mm/hugepages/hugepages-4096kB/free_hugepages:256 /sys/kernel/mm/hugepages/hugepages-4096kB/resv_hugepages:0 /sys/kernel/mm/hugepages/hugepages-4096kB/surplus_hugepages:0 /proc/sys/vm/hugepages_treat_as_movable:0 /proc/sys/vm/hugetlb_shm_group:0 /proc/sys/vm/nr_hugepages:256 /proc/sys/vm/nr_overcommit_hugepages:0
Tools
libhugetlbfs is a library which provides easy access to huge pages of memory. The library also comes with several userspace tools to help with huge page usability, environment setup, and control.
http://libhugetlbfs.ozlabs.org/
533708 - ITP: libhugetlbfs -- Initial package request
Hugepage enabled applications
An application can allocate/use HugeTlbPage through two different means:
mmap system call require a mounted hugetlbfs, with appropriate permissions.
Shared memory segment (shmat/shmget system calls or mmap with MAP_HUGETLB), must be member of a group, configured in /proc/sys/vm/hugetlb_shm_group.
KVM
(TODO), see:
http://fedoraproject.org/wiki/Features/KVM_Huge_Page_Backed_Memory
Get a performance boost by backing your KVM guest with hugetlbfs
http://www.linux-kvm.com/content/get-performance-boost-backing-your-kvm-guest-hugetlbfs (not tested)
MySQL
(TODO), see:
Enabling Large Page Support
http://dev.mysql.com/doc/refman/5.1/en/large-page-support.html
Linux HugeTLBfs: Improve MySQL Database Application Performance
http://www.cyberciti.biz/tips/linux-hugetlbfs-and-mysql-performance.html (note tested)
Java (Sun, OpenJDK)
Sun and OpenJDK can use large pages.
sun-java6-bin - Sun Java(TM) Runtime Environment (JRE) 6 (architecture dependent files)
openjdk-6-jre-headless - OpenJDK Java runtime, using Hotspot JIT (headless)
http://java.sun.com/javase/technologies/hotspot/largememory.jsp
Possible errors:
Insufficient /proc/sys/kernel/shmmax ?
Not in group defined in /proc/sys/vm/hugetlb_shm_group
% java -XX:+UseLargePages Java HotSpot(TM) Server VM warning: Failed to reserve shared memory (errno = 1). Java HotSpot(TM) Server VM warning: Failed to reserve shared memory (errno = 22).
Documentation
Documentation http://git.kernel.org/....../Documentation/vm/hugetlbpage.txt
alloc_hugepages(2), free_hugepages(2) - allocate or free huge pages
See also
Article about Huge pages, on LWN.net
Introduction - Interfaces - Administration - Benchmarking
Credits: Russell Coker (1)
This is the most reliable method of allocating huge pages as memory has not yet become fragmented. Yes, you finally found an operation that actually recommend a reboot in Linux (2)