Differences between revisions 20 and 21
Revision 20 as of 2007-09-16 01:02:34
Size: 2665
Editor: OndrejCertik
Comment:
Revision 21 as of 2007-09-16 01:06:50
Size: 2003
Editor: OndrejCertik
Comment:
Deletions are marked like this. Additions are marked like this.
Line 32: Line 32:
If anyone has any explanation, why `dakol` is 20x faster than the other computers, please write it here. Is it because of the position of the disk partition, or that it's larger? `pc232` is full, but the other computers have around the same disk usage. Note: `/var` and `/tmp` is on the same partition on all computers. Note 1: XFS filesystem was created using mkfs.xfs, ext3 filesystem was created with mkfs.ext3. No options added.
Line 34: Line 34:
Update: I also tried on `august` to copy the `base.cow` to a larger 233GB partition, 0.2% full, and then copy on this partition, and it took 6s. So a little improvement, but that is not the reason, why it is so terribly slow, compared to `dakol`. Note 2: On `dakol` with XFS filesystem, cp takes 0.5s, `sudo sync` takes 3s (!), rm takes 0.5s, and another `sudo sync` takes 2.8s.
However, with ext3 filesystem, the `sudo sync` is negligible.
Line 36: Line 37:
The only explanation I can think of is that `xfs` filesystem is 64bit, and thus is simply 20x faster on amd64 debian, compared to i386 debian...
Line 38: Line 38:
Update: I think I know: on `dakol`, cp takes 0.5s, `sudo sync` takes 3s (!), rm takes 0.5s, and another `sudo sync` takes 2.8s. So it's only 2x faster than `august`. And that could be explained by the 64bits. However, in practice, since the kernel is caching the files, it's 20x faster.

Solution: So I know where the problem is. I just used an empty 10GB partition on pc232.
Conclusion: The ext3 filesystem is truly superior to XFS for the `cowbuilder` purpose. Just use it and you'll be fine.

Note: please don't edit the page, I found the solution, polishing this page at this very moment.

Benchmarks of COW directory copies and removal in [:cowbuilder: cowbuilder].

Details

The numbers in the copy column were generated using:

time sudo cp -al /var/cache/pbuilder/base.cow /tmp/new

and in the remove column:

time sudo rm -rf /tmp/new

Everything is in seconds. XFS filesystems:

computer

copy

remove

computer details

pc232

16.1

18.2

Intel Core 2 Duo, ST3250820AS /dev/sda1 (10GB, xfs, 80% full), sid, i386

pc232

9.0

10.0

Intel Core 2 Duo, ST3250820AS /dev/sda2 (10GB, xfs, 2.4% full), sid, i386

august

9.3

10.9

Intel Core 2 Quad, ST3320620AS /dev/sda1 (20GB, xfs, 20% full), sid, i386

august

5.8

9.5

Intel Core 2 Quad, ST3320620AS /dev/sda3 (233GB, xfs, 0.2% full), sid, i386

dakol

0.48

0.5

AMD Athlon 64 3800+, ST3160827AS /dev/sda7 (135GB, xfs, 32% full), sid, amd64

fuji

13.9

16.4

Intel Core Duo, WDC WD1600BEVS-2 /dev/sda1 (14GB, xfs, 33% full), sid, i386

Ext3 filesystems:

computer

copy

remove

computer details

pc232

0.2

0.14

Intel Core 2 Duo, ST3250820AS /dev/sda2 (10GB, ext3, 2.4% full), sid, i386

august

0.22

0.13

Intel Core 2 Quad, ST3320620AS /dev/sda5 (42GB, ext3, 0.9% full), sid, i386

dakol

0.22

0.15

AMD Athlon 64 3800+, ST3160827AS /dev/sda5 (2GB, ext3, 14.5% full), sid, amd64

Note 1: XFS filesystem was created using mkfs.xfs, ext3 filesystem was created with mkfs.ext3. No options added.

Note 2: On dakol with XFS filesystem, cp takes 0.5s, sudo sync takes 3s (!), rm takes 0.5s, and another sudo sync takes 2.8s. However, with ext3 filesystem, the sudo sync is negligible.

Conclusion: The ext3 filesystem is truly superior to XFS for the cowbuilder purpose. Just use it and you'll be fine.