6475
Comment: add wikipedia description
|
← Revision 17 as of 2023-02-19 19:24:15 ⇥
6490
link to FileSystem
|
Deletions are marked like this. | Additions are marked like this. |
Line 1: | Line 1: |
. '''[[WikiPedia:Lustre_(file_system)|Lustre]]''' is a type of parallel distributed file system, generally used for large-scale cluster computing. | . '''[[WikiPedia:Lustre_(file_system)|Lustre]]''' is a type of parallel distributed [[FileSystem|file system]], generally used for large-scale cluster computing. |
Lustre is a type of parallel distributed file system, generally used for large-scale cluster computing.
This pages should help you to get a running lustre cluster based on debian systems.
Contents
You can either build your own Lustre Debian packages from GIT or download these packages from
http://pkg-lustre.alioth.debian.org/backports/lustre-2.2.0-squeeze/
and
http://pkg-lustre.alioth.debian.org/backports/ldiskfsprogs-1.42.3/
If you download all packages incl. the kernel package you can continue with chapter Installation of Lustre 2.2 in Debian Squeeze.
If you download only the Lustre packages without the kernel and kernel module package, e.g. because you want to compile your own custom kernel, please continue with chapter Building Lustre kernel.
Building Lustre 2.2 for Debian Squeeze
Building Lustre packages from GIT
Change to a any Lustre building directory and clone the actual GIT tree of Debian Lustre project:
git clone http://anonscm.debian.org/git/pkg-lustre/lustre.git
Then you need the original source tarball of Lustre.
wget http://pkg-lustre.alioth.debian.org/backports/lustre-2.2.0-squeeze/lustre_2.2.0.orig.tar.gz
Finally you can build your packages like this:
cd lustre dpkg-buildpackage
(If you don't want to sign your packages, you can add the parameters -us -uc to dpkg-buildpackage)
Now you'll find your packages in your Lustre building directory.
Building Lustre 2.2 kernel for Debian Squeeze
Install the linux-patch-lustre_2.2.0-5_all.deb package:
sudo dpkg -i linux-patch-lustre_2.2.0-5_all.deb
The Lustre patches for Linux kernel 2.6.32 will be placed in /usr/src/kernel-patches/lustre.
To go on please now install the kernel source package from Debian Squeeze:
apt-get source linux-2.6 cd linux-2.6-2.6.32 /usr/src/kernel-patches/lustre/scripts/apply
All the Lustre patches are being applied to the kernel sources. Now you can create your kernel. First prepare your sources:
fakeroot make-kpkg --config=menuconfig --revision deblust.1.0 --append-to-version "-lustre-2.2" configure
Please choose a meaningful revision name. Save your configuration as .config file and then initiate the building process:
fakeroot make-kpkg --initrd --revision deblust.1.0 --append-to-version "-lustre-2.2" kernel_image kernel_headers kernel_source
Depending on your system this can take a long time.
With the help of the script /usr/src/kernel-patches/lustre/scripts/unpatch you can remove the Lustre kernel patches from the sources.
Building Lustre 2.2 modules
Install the Lustre kernel image and Lustre source package:
sudo dpkg -i linux-image-2.6.32-lustre-2.2_deblust.1.0_amd64.deb sudo dpkg -i linux-headers-2.6.32-lustre-2.2_deblust.1.0_amd64.deb sudo dpkg -i linux-source-2.6.32-lustre-2.2_deblust.1.0_all.deb sudo dpkg -i lustre-source_2.2.0-5_all.deb
Please reboot into the new Lustre kernel. Use the module-assistant to start the module building process:
sudo m-a auto-install lustre
Then the module kernel package should be there and the modules installed.
Installation of Lustre 2.2 in Debian Squeeze
For a minimal test cluster at least 4 machines are recommended:
- Lustre test client
- Lustre MGS/MDT (Management Server/Metadata Target)
- Lustre OST1 (Object Storage Target)
- Lustre OST2
The MGS stores configuration information for all the Lustre file systems in a cluster and provides this information to other Lustre components, whereas the MDT stores namespace metadata (filenames, directories, access permissions, file layout). The MGS/MDT is also an entry point. The OST stores the data.
Preparing the systems
Install on all machines:
sudo dpkg -i linux-image-2.6.32-lustre-2.2_deblust.1.0_amd64.deb sudo dpkg -i lustre-modules-2.6.32-lustre-2.2_2.2.0-5_amd64.deb sudo dpkg -i lustre-utils_2.2.0-3_amd64.deb sudo dpkg -i ldiskfsprogs_1.42.3-1_amd64.deb
Now you are ready to boot the kernel an all machines. Then load the kernel modules with:
modprobe lnet modprobe lustre modprobe ldiskfs
If you're running ypbind or something else on your system which uses port 988 you should make sure that the ptctl module is loaded very early in the boot process (as lustre needs port 988 for communications), See the ?KnowledgeBase for details.
MGS/MDT
Create a new and free partition (as large as possible), e.g. /dev/sda4 and format it as MGS/MDT:
mkfs.lustre --fsname=temp --mgs --mdt /dev/sda4
temp is the name of our filesystem. This can be changed of course. The formatting process could take a long time.
Mount the new partition:
mount -t lustre /dev/sda4 /srv/mdt
You can choose another mountpoint of course.
OST
Create a new and free partition (as large as possible), e.g. /dev/sda4 and format it as MGS/MDT:
mkfs.lustre --ost --fsname=temp --mgsnode=192.168.80.100@tcp0 /dev/sda4
temp is the name of our filesystem. Please note that we have to set the IP adress or Hostname of the Lustre MGS node (here: 192.168.80.100).
Mount the new partition:
mount -t lustre /dev/sda4 /srv/ost1
You can choose another mountpoint of course.
Repeat for OST2.
Client
Go to the client machine and mount your new Lustre file system:
mount -t lustre 192.168.80.100@tcp0:/temp /mnt/lustre
Verify that the file system started and is working:
lfs df -h
Go to /mnt/lustre (or whatever your mount point is) and run a first basic test:
dd if=/dev/zero of=/lustre/zero.dat bs=4M count=20
Please check also your kernel logs for any error messages.
Using Quota
The binaries and drivers for lustre in debian are compiled with quota enabled. So if you like to use quota this is quite simple:
lfs quotacheck -ug $path/to/mounted/lustre/fs lfs setquota -u $user 1000 2000 10000 20000 $path/to/mounted/lustre/fs lfs quota -u $user $path/to/mounted/lustre/fs
upstream specific information
CategorySoftware CategorySystemAdministration CategoryStorage