Diff for "InitramfsDebug"

Differences between revisions 1 and 26 (spanning 25 versions)

initramfs-tools is used in Debian to build the initramfs that contains the early userspace and the needed kernel modules to be able to mount any possible root.

Boot

Boot with

  rootdelay=9

or such to shorten the time of waiting for the root device to come up. Also it adds time to scsi device to settle before calling mdadm or lvm thus excluding potential races. Alternatively you may have success with

  scsi_mod.scan=sync

To fix wrong fs recognition work around boot with

  rootfstype=<rootfstype>

Rescue shell (also known as initramfs shell)

Read man initramfs-tools to learn about the break=something kernel parameter (where valid arguments for something are: top, modules, premount, mount, mountroot, bottom, init), which starts a debug shell. You can try, for example, break=premount. You can edit /boot/grub/grub.cfg to add this to the end of the kernel line, or you can do it interactively from the grub boot menu: "e" to edit, and "b" to boot once you've edited the kernel line.

Ok, now you landed in the initramfs debug shell. This is a surprisingly complete environment provided by "busybox". You can use vi, for example, but not everything is fully the same as bash. For example, sleep only accepts integer arguments. You may find more complete commands in /bin (for example, /bin/sleep does accept decimal arguments, but you will need to use /bin/sleep in your scripts).

The following steps should help to narrow down the problem.

Check root device existence
```
 ls -l /dev/[hs]da*
```
If above does not exist checkout for eventual driver conflict or other information
```
 dmesg
```
Check root boot argument
- It needs to be the same than above root device
```
 cat /proc/cmdline
```
Check correct fs recognition
```
 # for releases >= squeeze:
 blkid -o value -s TYPE /dev/sda1 
 # for releases <= lenny:
 /lib/udev/vol_id -t /dev/sda1
 # compare with:
 fstype /dev/sda1
```
- If the output of aboves command differ, your partition was not cleanly formated. For example mkfs.ext2 does not clear all sectors when formating a partition.
Check if the ata/ide/scsi driver is loaded
```
 cat /proc/modules
```
saving output from an initramfs script into a logfile you can review later
- Add "debug" bootparam and find the logfile in /dev/.initramfs/initramfs.debug. (You'll need initramfs-tools 0.92m at least for that feature. Also see further down in this document on how to save other data, or save data across reboots.

You can exit the shell with exit or ^D and the boot should resume.

initramfs content

It can happen that due to a bug in the initramfs creation a newer initramfs lacks important libraries or utilities. Easiest way is to check contents with lsinitramfs

 lsinitramfs /boot/initrd.img-2.6.32-5-686 | less

Old way is to compare the file list of working contra non-working initramfs.

 cd $TMPDIR
 mkdir initramfs{,-old}
 cd initramfs
 gunzip -c -9 /boot/initrd.img-2.6.32-5-686 \
  | cpio -i -d -H newc --no-absolute-filenames
 find > ../initramfs.content
 cd ../initramfs-old
 gunzip -c -9 /boot/initrd.img-2.6.32-5-686.bak \
  | cpio -i -d -H newc --no-absolute-filenames
 find > ../initramfs-old.content
 cd ..
 diff -u initramfs-old.content initramfs.content

If the intiramfs creation of both those images went well there should be no diff. If you are comparing initramfs for different kernels you should see a difference in the modules aka other dir and newer drivers in /lib/modules/$(uname -r)

Saving debug information

If you specify debug as a kernel option during boot, initramfs-tools (>= 0.92m) writes shell script traces to /dev/.initramfs/initramfs.debug, which will be available after the system has booted.

You can write other data to files in that directory to be able to access them later, when the system has booted. Since /dev is a tmpfs however, they won't persist across a reboot.

If you need to get debug data off the system and it can mount the root filesystem, you can do something like

mount -o remount,rw /root
dmesg > /root/root/initramfs.dmesg
mount -o remount,ro /root
^D

If that does not work, use a USB stick: plug it in, observe the kernel output, mkdir /mnt, mount the device node there, and write to the flash drive. Remember to umount before pulling out the stick.

CategoryKernel

-  ⇤ ← Revision 1 as of 2007-06-25 23:12:09 → 
  Size: 728
  Editor: ?maximilianattems
  Comment: initial commit
+   ← Revision 26 as of 2011-02-23 15:20:42 → ⇥
  Size: 4758
  Editor: ?maximilianattems
  Comment:
-Deletions are marked like this.
+Additions are marked like this.
 Line 1:
-[[wiki:initramfs-tools]] is used in Debian to build the initramfs that contains the early userspace and the needed kernel modules to be able to mount any possible root.
+#language en
~-[[DebianWiki/EditorGuide#translation|Translation(s)]]: none-~
----
[[initramfs-tools]] is used in Debian to build the [[initramfs]] that contains the early userspace and the needed kernel modules to be able to mount any possible root.
-Line 3:
+Line 6:
-== Debug Howto ==
+== Boot ==
-Line 5:
+Line 8:
- * Check root device existance
+Boot with 
{{{
  rootdelay=9
}}}
or such to shorten the time of waiting for the root device to come up. Also it adds time to scsi device to settle before calling mdadm or lvm thus excluding potential races. Alternatively you may have success with
{{{
  scsi_mod.scan=sync
}}}
-Line 7:
+Line 17:
- {{{ ls -l /dev/[hs]da* }}}
+To fix wrong fs recognition work around boot with
{{{
  rootfstype=<rootfstype>
}}}


== Rescue shell (also known as initramfs shell) ==

Read {{{man initramfs-tools}}} to learn about the {{{break=something}}} kernel parameter (where valid arguments for {{{something}}} are: {{{top}}}, {{{modules}}}, {{{premount}}}, {{{mount}}}, {{{mountroot}}}, {{{bottom}}}, {{{init}}}), which starts a debug shell. You can try, for example, {{{break=premount}}}. You can edit {{{/boot/grub/grub.cfg}}} to add this to the end of the kernel line, or you can do it interactively from the grub boot menu: "e" to edit, and "b" to boot once you've edited the kernel line.

Ok, now you landed in the initramfs debug shell. This is a surprisingly complete environment provided by "busybox". You can use {{{vi}}}, for example, but not everything is fully the same as bash. For example, {{{sleep}}} only accepts integer arguments. You may find more complete commands in {{{/bin}}} (for example, {{{/bin/sleep}}} does accept decimal arguments, but you will need to use {{{/bin/sleep}}} in your scripts).

The following steps should help to narrow down the problem.

 * Check root device existence

 {{{
 ls -l /dev/[hs]da*
 }}}
 If above does not exist checkout for eventual driver conflict or other information
 {{{
 dmesg
 }}}
-Line 11:
+Line 44:
-  It needs to be the same than aboves root device
+  It needs to be the same than above root device
-Line 13:
+Line 46:
- {{{ cat /proc/cmdline }}}
+ {{{
 cat /proc/cmdline
 }}}
-Line 17:
+Line 52:
-  {{{ 
      /lib/udev/vol_id -t /dev/sda1
      fstype /dev/sda1
  }}}
+ {{{ 
 # for releases >= squeeze:
 blkid -o value -s TYPE /dev/sda1 
 # for releases <= lenny:
 /lib/udev/vol_id -t /dev/sda1
 # compare with:
 fstype /dev/sda1
 }}}
-Line 26:
+Line 65:
-   {{{ cat /proc/modules }}}
+ {{{
 cat /proc/modules
 }}}

 * saving output from an initramfs script into a logfile you can review later
  
  Add "debug" bootparam and find the logfile in /dev/.initramfs/initramfs.debug. (You'll need initramfs-tools 0.92m at least for that feature.

  Also see further down in this document on how to save other data, or save data across reboots.

You can exit the shell with {{{exit}}} or {{{^D}}} and the boot should resume.

== initramfs content ==

It can happen that due to a bug in the initramfs creation a newer initramfs lacks important libraries or utilities. Easiest way is to check contents with lsinitramfs
 {{{
 lsinitramfs /boot/initrd.img-2.6.32-5-686 | less
 }}}

Old way is to compare the file list of working contra non-working initramfs.

 {{{
 cd $TMPDIR
 mkdir initramfs{,-old}
 cd initramfs
 gunzip -c -9 /boot/initrd.img-2.6.32-5-686 \
  | cpio -i -d -H newc --no-absolute-filenames
 find > ../initramfs.content
 cd ../initramfs-old
 gunzip -c -9 /boot/initrd.img-2.6.32-5-686.bak \
  | cpio -i -d -H newc --no-absolute-filenames
 find > ../initramfs-old.content
 cd ..
 diff -u initramfs-old.content initramfs.content
 }}}
 If the intiramfs creation of both those images went well there should be no diff. If you are comparing initramfs for different kernels you should see a difference in the modules aka other dir and newer drivers in /lib/modules/$(uname -r)

== Saving debug information ==

If you specify {{{debug}}} as a kernel option during boot, initramfs-tools (>= 0.92m) writes shell script traces to {{{/dev/.initramfs/initramfs.debug}}}, which will be available after the system has booted.

You can write other data to files in that directory to be able to access them later, when the system has booted. Since {{{/dev}}} is a {{{tmpfs}}} however, they won't persist across a reboot.

If you need to get debug data off the system and it can mount the root filesystem, you can do something like

{{{
mount -o remount,rw /root
dmesg > /root/root/initramfs.dmesg
mount -o remount,ro /root
^D
}}}

If that does not work, use a USB stick: plug it in, observe the kernel output, {{{mkdir /mnt}}}, {{{mount}}} the device node there, and write to the flash drive. Remember to {{{umount}}} before pulling out the stick.

----
CategoryKernel