Differences between revisions 1 and 46 (spanning 45 versions)
Revision 1 as of 2011-11-23 15:03:49
Size: 3322
Editor: OsamuAoki
Comment:
Revision 46 as of 2013-01-21 16:10:27
Size: 9118
Editor: OsamuAoki
Comment: /tmp are normally wiped ... I see no point warning
Deletions are marked like this. Additions are marked like this.
Line 5: Line 5:
This describe SDD optimization with system having encrypted root and swap. This page is about optimal set up of SSD. This page should be keep it clean enough for beginners to get the most basic idea.
Line 7: Line 7:
## If your page gets really long, uncomment this Table of Contents
Line 10: Line 9:
== SSD Optimization on the powerful laptop PC == == Basics ==
Line 12: Line 11:
For a PC with 256GB SSD with system having encrypted root and swap, system is mounted as:  * Use a recent Linux kernel. (3.2 or newer)
 * Have enough RAM to not need any swap space under normal workloads while maintaining most of the variable data in a persistent RAMDISK that gets synced to disk periodically.
 * Set up a swap partition on a hdd (or SSD), just in case you run out of RAM and to be able to suspend to disk.
 * Use the "noatime" (or "relatime") mount option in /etc/fstab, to disable (or reduce) disk writes during each disk read access.
 * Use filesytems in the ext4 format.
 * Optionally, use the btrfs format (not yet stable). It supports additional mount options like "ssd" in /etc/fstab, that enables SSD optimized disk space allocation.
Line 14: Line 18:
== File system and partition alignment ==

File system and partition alignment to 4096 Byte page size is one of the most important optimization aspect. Here are good links for this subject.

 * [[http://en.wikipedia.org/wiki/Advanced_format|Advanced Format]]
 * [[http://www.linux-mag.com/id/8397/|Aligning SSD Partitions (Linux magazine)]]
 * [[http://tytso.livejournal.com/2009/02/20/|Aligning filesystems to an SSD’s erase block size] by Ted T'so]] (kernel FS developer)
 * http://blog.nuclex-games.com/2009/12/aligning-an-ssd-on-linux/
 * http://lifehacker.com/5837769/make-sure-your-partitions-are-correctly-aligned-for-optimal-solid-state-drive-performance

== Mounting SSD with noatime and discard options etc. ==

The performance of SSDs can also be influenced by these mounting options:

 * Maybe enable the "discard" filesystem options for automatic/online TRIM. However this is not strictly necessary if your SSD has enough overprovisioning (spare space) or you leave (unpartitioned) free space on the SSD (http://www.spinics.net/lists/raid/msg40866.html). Enabling online-trim in fstab may just slow down some SSDs significantly (https://patrick-nagel.net/blog/archives/337).
  * Set "discard" mount option in /etc/fstab for the ext4 filesystem, swap partition, Btrfs, etc. See mount(8).
  * Set optional "noatime" mount option in /etc/fstab.
  * Set "issue_discard" option in /etc/lvm/lvm.conf for LVM. See lvm.conf(5).
  * Set "discard" option in /etc/crypttab for dm-crypt.
 
'''Note that using discard with on-disk-cryptogrpahy (like dm-crypt) also has drawbacks with respect to security/cryptography!''' See crypttab(5).

dm-crypt's /etc/crypttab:
Line 15: Line 42:
$ mount
/dev/mapper/crypto-root on / type ext4 (rw,noatime,discard,commit=600,errors=remount-ro,commit=600)
tmpfs on /lib/init/rw type tmpfs (rw,nosuid,size=5242880,mode=755,size=5242880,mode=755)
tmpfs on /run type tmpfs (rw,noexec,nosuid,size=10%,mode=755,size=10%,mode=755)
tmpfs on /tmp type tmpfs (rw,noexec,nosuid,nodev,noatime,mode=1777,size=20%,mode=1777,mode=1777)
proc on /proc type proc (rw,noexec,nosuid,nodev)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
udev on /dev type tmpfs (rw,mode=0755)
tmpfs on /run/shm type tmpfs (rw,nosuid,nodev,size=20%,mode=1777,size=20%,mode=1777)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=620,gid=5,mode=620)
/dev/sda1 on /boot type ext4 (rw,noatime,discard,commit=600,commit=600)
tmpfs on /var/log type tmpfs (rw,noexec,nosuid,nodev,noatime,mode=1777)
tmpfs on /home/username/.cache type tmpfs (rw,noexec,nosuid,nodev,noatime,uid=1000,gid=1000,mode=700)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
fusectl on /sys/fs/fuse/connections type fusectl (rw)
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,noexec,nosuid,nodev)
#<target name> <source device> <key file> <options>
var UUID=01234567-89ab-cdef-0123-456789abcdef none luks,discard
Line 32: Line 45:
 You'll also need to update your initramfs: `update-initramfs -u -k all`
Line 33: Line 47:
This is done with the foloowings.  * Optionally, set up an offline-trim cronjob that runs {{{time fstrim -v }}} (or mdtrim) on the ssd mountpoints periodically. Until software raid (md device layer) has trim support, you could use something like mdtrim (https://github.com/Cyberax/mdtrim/).
Line 35: Line 49:
/etc/fstab: More:
 * http://siduction.org/index.php?module=news&func=display&sid=78
 * http://forums.debian.net/viewtopic.php?f=16&t=76921
 * https://wiki.archlinux.org/index.php/SSD
 * http://wiki.ubuntuusers.de/SSD

== Reduction of SSD write frequency via RAMDISK ==

Use of RAMDISK can stop constantly changing files from hitting on the SSD. (It may hit SSD via swap.) RAMDISK configuration may be performed via

 * Set RAMTMP, RAMRUN and RAMLOCK to "yes" (in /etc/default/rcS or /etc/default/tmpfs since wheezy)
 * /etc/fstab: line such as "tmpfs /tmp noatime,nosuid 0 0"

Enabling RAMTMP may cause some (broken) applications to run out of temporary write disk space. Setting TMPDIR environment variable for those programs pointing to a writable disk space should fix their situation.

Please note files in /tmp are wiped out upon reboot unless /etc/default/rcS is set to something other than TMPTIME=0 (if not set, 0 is the default value).

== Persistent RAMDISK ==

As an advanced option, you may consider to use persistent RAMDISK (dedicated read/write RAM buffer that gets synced periodically and on startup/shutdown) with anything-sync-daemon or goanysync set up:

 * /home (synced to work-data-fs raid only once a day?), you only risk settings the true work in /home/*/work-data is on a dedicated raid
 * /home/*/work-data/volatile (synced more frequently, once per hour?)
 * /home/*/Downloads (synced to bulk-data-fs once a day?)
 * /var completely if supported (syncing once a day? avoids spin-ups and allows to save /var also to SSD), at least set this up for
  * /var/log if suported
  * /var/cache/apt/archives
     Configure apt to delete package files after installing, to minimize the data to sync.

Options to having logs copied into RAM: http://www.debian-administration.org/articles/661, http://www.tremende.com/ramlog, https://github.com/graysky2/anything-sync-daemon (if it supports this), or https://github.com/wor/goanysync
 
If /home is not on a persistent ramdisk, use profile-sync-daemon to have the browser database and cache copied into RAM during uptime (http://ubuntuforums.org/showthread.php?t=1921800 https://github.com/graysky2/profile-sync-daemon)
 * /home/*/<browser-cache-and-profiles> (synced to root-fs or home-fs)

Further improvement: Patch anything-sync-daemon or goanysync to use a (copy-on-write) union filesystem mount (e.g. http://aufs.sourceforge.net) to keep changes in RAM and only save to SSD on unmount/shutdown (aubrsync), instead of copying all data to RAM and having to sync it all back.

Alternative to persistent ramdisk:
 
 * Make system only flush data to the disk every 10 minutes or more:
 /!\ Attention: Increasing the flushing interval from the default 5 seconds (maybe even until proper shutdown) leaves your data much more vulnerable in case of lock-ups or power failures, and seems to be a global setting.
  * Kernel settings like the "dirty_buffer_ratio" etc. may only be available as non-path/mount specific global settings.
  * Mount option "commit=600" in /etc/fstab. See mount(8).
  * Or better, use pm-utils ([[http://bugs.debian.org/659260|Debian BTS #659260]]), tlp, or laptop-mode-tools (also optimizes read buffers) to configure the laptop-mode even under AC operation.

== Low-Latency IO-Scheduler ==

The default scheduler queues data to minimize seeks on HDDs, which is not necessary for SSDs. Thus, use the deadline scheduler that just ensures bulk transactions won't slow down small transactions: Install sysfsutils and
  echo "/sys/block/sdX/queue/scheduler = deadline" > /etc/sysfs.conf
(adjust sdX to match your SSD) reboot or
  echo deadline > /sys/block/sdX/queue/scheduler

== /etc/fstab example ==

Here is an example of /etc/fstab
Line 44: Line 112:
proc /proc proc defaults 0 0
Line 47: Line 114:
/dev/mapper/crypto-root / ext4 discard,noatime,commit=600,errors=remount-ro 0 1 /dev/mapper/goofy-root / ext4 discard,noatime,commit=600,errors=remount-ro 0 1
Line 49: Line 116:
UUID=709abe4b-81c1-96cb-8ac1-dace3159a1b7 /boot ext4 discard,noatime,commit=600,defaults 0 2 UUID=709cbe4a-80c1-46cb-8bb1-dbce3059d1f7 /boot ext4 discard,noatime,commit=600,defaults 0 2
Line 51: Line 118:
/dev/mapper/crypto-swap_1 none swap sw,discard 0 0 /dev/mapper/goofy-swap none swap sw,discard 0 0
/dev/mapper/goofy-chroot /srv/chroot btrfs ssd,discard,noatime 0 2
Line 53: Line 121:
### SSD OPTIMIZATION: /home/osamu/.cache on tmpfs
tmpfs /tmp tmpfs rw,noexec,nodev,nosuid,noatime,mode=1777 0 0
tmpfs /usr/tmp tmpfs rw,noexec,nodev,nosuid,noatime,mode=1777 0 0
tmpfs /var/log tmpfs rw,noexec,nodev,nosuid,noatime,mode=1777 0 0
tmpfs /home/osamu/.cache tmpfs rw,noexec,nodev,nosuid,noatime,uid=1000,gid=1000,mode=700 0 0
Line 60: Line 123:
/etc/pm/config.d/SET_JOURNAL_COMMIT_TIME_AC
== /etc/lvm/lvm.conf example ==
Line 62: Line 127:
JOURNAL_COMMIT_TIME_AC=600 ...
# This section allows you to configure which block devices should
# be used by the LVM system.
devices {
...
    # Issue discards to a logical volumes's underlying physical volume(s) when
    # the logical volume is no longer using the physical volumes' space (e.g.
    # lvremove, lvreduce, etc). Discards inform the storage that a region is
    # no longer in use. Storage that supports discards advertise the protocol
    # specific way discards should be issued by the kernel (TRIM, UNMAP, or
    # WRITE SAME with UNMAP bit set). Not all storage will support or benefit
    # from discards but SSDs and thinly provisioned LUNs generally do. If set
    # to 1, discards will only be issued if both the storage and kernel provide
    # support.
    # 1 enables; 0 disables.
    #issue_discards = 0
    issue_discards = 1
}
...
Line 64: Line 147:
Line 70: Line 154:
 *
Line 72: Line 155:

== Larger system with SSD ==

See ___FIXME___

Translation(s): none


This page is about optimal set up of SSD. This page should be keep it clean enough for beginners to get the most basic idea.

Basics

  • Use a recent Linux kernel. (3.2 or newer)
  • Have enough RAM to not need any swap space under normal workloads while maintaining most of the variable data in a persistent RAMDISK that gets synced to disk periodically.
  • Set up a swap partition on a hdd (or SSD), just in case you run out of RAM and to be able to suspend to disk.
  • Use the "noatime" (or "relatime") mount option in /etc/fstab, to disable (or reduce) disk writes during each disk read access.
  • Use filesytems in the ext4 format.
  • Optionally, use the btrfs format (not yet stable). It supports additional mount options like "ssd" in /etc/fstab, that enables SSD optimized disk space allocation.

File system and partition alignment

File system and partition alignment to 4096 Byte page size is one of the most important optimization aspect. Here are good links for this subject.

Mounting SSD with noatime and discard options etc.

The performance of SSDs can also be influenced by these mounting options:

  • Maybe enable the "discard" filesystem options for automatic/online TRIM. However this is not strictly necessary if your SSD has enough overprovisioning (spare space) or you leave (unpartitioned) free space on the SSD (http://www.spinics.net/lists/raid/msg40866.html). Enabling online-trim in fstab may just slow down some SSDs significantly (https://patrick-nagel.net/blog/archives/337).

    • Set "discard" mount option in /etc/fstab for the ext4 filesystem, swap partition, Btrfs, etc. See mount(8).
    • Set optional "noatime" mount option in /etc/fstab.
    • Set "issue_discard" option in /etc/lvm/lvm.conf for LVM. See lvm.conf(5).
    • Set "discard" option in /etc/crypttab for dm-crypt.

Note that using discard with on-disk-cryptogrpahy (like dm-crypt) also has drawbacks with respect to security/cryptography! See crypttab(5).

dm-crypt's /etc/crypttab:

#<target name>    <source device>            <key file>  <options>
var  UUID=01234567-89ab-cdef-0123-456789abcdef  none  luks,discard
  • You'll also need to update your initramfs: update-initramfs -u -k all

  • Optionally, set up an offline-trim cronjob that runs time fstrim -v  (or mdtrim) on the ssd mountpoints periodically. Until software raid (md device layer) has trim support, you could use something like mdtrim (https://github.com/Cyberax/mdtrim/).

More:

Reduction of SSD write frequency via RAMDISK

Use of RAMDISK can stop constantly changing files from hitting on the SSD. (It may hit SSD via swap.) RAMDISK configuration may be performed via

  • Set RAMTMP, RAMRUN and RAMLOCK to "yes" (in /etc/default/rcS or /etc/default/tmpfs since wheezy)
  • /etc/fstab: line such as "tmpfs /tmp noatime,nosuid 0 0"

Enabling RAMTMP may cause some (broken) applications to run out of temporary write disk space. Setting TMPDIR environment variable for those programs pointing to a writable disk space should fix their situation.

Please note files in /tmp are wiped out upon reboot unless /etc/default/rcS is set to something other than TMPTIME=0 (if not set, 0 is the default value).

Persistent RAMDISK

As an advanced option, you may consider to use persistent RAMDISK (dedicated read/write RAM buffer that gets synced periodically and on startup/shutdown) with anything-sync-daemon or goanysync set up:

  • /home (synced to work-data-fs raid only once a day?), you only risk settings the true work in /home/*/work-data is on a dedicated raid
  • /home/*/work-data/volatile (synced more frequently, once per hour?)
  • /home/*/Downloads (synced to bulk-data-fs once a day?)
  • /var completely if supported (syncing once a day? avoids spin-ups and allows to save /var also to SSD), at least set this up for
    • /var/log if suported
    • /var/cache/apt/archives
      • Configure apt to delete package files after installing, to minimize the data to sync.

Options to having logs copied into RAM: http://www.debian-administration.org/articles/661, http://www.tremende.com/ramlog, https://github.com/graysky2/anything-sync-daemon (if it supports this), or https://github.com/wor/goanysync

If /home is not on a persistent ramdisk, use profile-sync-daemon to have the browser database and cache copied into RAM during uptime (http://ubuntuforums.org/showthread.php?t=1921800 https://github.com/graysky2/profile-sync-daemon)

  • /home/*/<browser-cache-and-profiles> (synced to root-fs or home-fs)

Further improvement: Patch anything-sync-daemon or goanysync to use a (copy-on-write) union filesystem mount (e.g. http://aufs.sourceforge.net) to keep changes in RAM and only save to SSD on unmount/shutdown (aubrsync), instead of copying all data to RAM and having to sync it all back.

Alternative to persistent ramdisk:

  • Make system only flush data to the disk every 10 minutes or more:

    /!\ Attention: Increasing the flushing interval from the default 5 seconds (maybe even until proper shutdown) leaves your data much more vulnerable in case of lock-ups or power failures, and seems to be a global setting.

    • Kernel settings like the "dirty_buffer_ratio" etc. may only be available as non-path/mount specific global settings.
    • Mount option "commit=600" in /etc/fstab. See mount(8).
    • Or better, use pm-utils (Debian BTS #659260), tlp, or laptop-mode-tools (also optimizes read buffers) to configure the laptop-mode even under AC operation.

Low-Latency IO-Scheduler

The default scheduler queues data to minimize seeks on HDDs, which is not necessary for SSDs. Thus, use the deadline scheduler that just ensures bulk transactions won't slow down small transactions: Install sysfsutils and

  • echo "/sys/block/sdX/queue/scheduler = deadline" > /etc/sysfs.conf

(adjust sdX to match your SSD) reboot or

  • echo deadline > /sys/block/sdX/queue/scheduler

/etc/fstab example

Here is an example of /etc/fstab

# /etc/fstab: static file system information.
#
# Use 'vol_id --uuid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
### SSD: discard,noatime
### match battery operation default for commit JOURNAL_COMMIT_TIME_AC in Add files in /etc/pm/config.d/*
/dev/mapper/goofy-root /               ext4    discard,noatime,commit=600,errors=remount-ro 0       1
# /boot was on /dev/sda1 during installation
UUID=709cbe4a-80c1-46cb-8bb1-dbce3059d1f7 /boot           ext4    discard,noatime,commit=600,defaults        0       2
### SSD: discard
/dev/mapper/goofy-swap none            swap    sw,discard              0       0
/dev/mapper/goofy-chroot /srv/chroot         btrfs    ssd,discard,noatime 0       2
/dev/scd0       /media/cdrom0   udf,iso9660 user,noauto     0       0

/etc/lvm/lvm.conf example

...
# This section allows you to configure which block devices should
# be used by the LVM system.
devices {
...
    # Issue discards to a logical volumes's underlying physical volume(s) when
    # the logical volume is no longer using the physical volumes' space (e.g.
    # lvremove, lvreduce, etc).  Discards inform the storage that a region is
    # no longer in use.  Storage that supports discards advertise the protocol
    # specific way discards should be issued by the kernel (TRIM, UNMAP, or
    # WRITE SAME with UNMAP bit set).  Not all storage will support or benefit
    # from discards but SSDs and thinly provisioned LUNs generally do.  If set
    # to 1, discards will only be issued if both the storage and kernel provide
    # support.
    # 1 enables; 0 disables.
    #issue_discards = 0
    issue_discards = 1
}
...

Smaller system with SSD

See

Larger system with SSD

See _FIXME_