196 replies to this topic

[dvboxer]

Stupid me. "echo 1> file" obviously redirects stderr to file, sorry. But setting it to 4 stops freezing here, too. That is without journal. I will put it back on and try again.

With journal a value of 4 gives freezers again, but 1 still stops them.

[MiLo]

The results with slice_idle=1 are very promising indeed. In particular when combined with the new ext4 format on the 3.2 kernel, I can run several recordings and bash away at the disk, and the write() calls rarely exceed 10ms, while with the default slice_idle=8 the numbers are worse. Basically, the queue favors "reading" too much in the default settings, but we're not running a desktop, so we want to have little write latencies.

I've made a patch for the kernel that changes the default to 1.

[dvboxer]

Actually I still get freezers even with slice_idle=1 if it is the first recording after mounting. Subsequent recordings are fine which is why I didn't notice it before (I hadn't remounted after changing slice_idle).

[domi-sat]

hallo,
ich hoffe ich darf hier noch in deutsch schreiben.
habe auch aussetzer, vorallem am anfang.

aber als hinweis:
heute hat et-view eine mail rausgeschickt:

Sehr geehrte Xtrend-Anwender,

bitte updaten Sie Ihr OpenPLi-Image ab Morgen und testen erneut, ob Sie weiterhin die genannten Frezzer-Probleme mit ext4 Dateisystemen feststellen koennen:

woher haben die diese informationen bezüglich openpli?
dieses forum ist doch openpli.

[Frogman]

Jeder kann diese Informationen einsehen, d.h. alles, was am OpenPLi gemacht wird, denn es ist ein OpenSource-Projekt. Hier siehst Du es.

Und das, was in der email angesprochen wurde, bezieht sich auf diesen commit von MiLo. Damit wird eine Einstellung des Kernels verändert, die zu einer besseren Schreibperformance gerade mit ext4 führt.

@MiLo
Danke auch an dieser Stelle dafür

Edited by Steffen, 18 February 2012 - 14:57.

[MiLo]

(Deutch ist kein problem, kann ich ja lesen, aber schreiben nicht so gut).

For best results, if possible, re-format the harddisk in the box. The 3.2 kernel supports large block allocation, which increases performance for large files to make it faster than any other filesystem I've tried on the ET. The downside is that any file will take at least 256k (instead of 4k) of disk space on the disk. But it reduces allocation overhead and fragmentation. Existing disks (even if already ext4) cannot be converted to this format.

[dvboxer]

@MiLo How do you explain that only the first few seconds of writing to the disk seem to be a problem? I now dd 100Mb of zeros to the disk at booting time and no freezers anymore. Isn't there any other scheduler parameter that can be tweaked to avoid this problem?

[dvboxer]

Look at this:

root@vuduo:~# umount /media/hdd

root@vuduo:~# mount /media/hdd

root@vuduo:~# dd if=/dev/zero of=/media/hdd/blanks2 bs=1024k count=100

100+0 records in

100+0 records out

104857600 bytes (100.0MB) copied, 18.168953 seconds, 5.5MB/s

root@vuduo:~# dd if=/dev/zero of=/media/hdd/blanks2 bs=1024k count=100

100+0 records in

100+0 records out

104857600 bytes (100.0MB) copied, 1.872618 seconds, 53.4MB/s

Edited by dvboxer, 18 February 2012 - 16:41.

[MiLo]

root@et9x00:/media# umount /media/hdd/

root@et9x00:/media# mount /dev/sda1 /media/hdd

root@et9x00:/media# dd if=/dev/zero of=/media/hdd/blanks2 bs=1024k count=100

100+0 records in

100+0 records out

104857600 bytes (100.0MB) copied, 1.864044 seconds, 53.6MB/s

root@et9x00:/media#

root@et9x00:/media# umount /media/hdd/

root@et9x00:/media# mount /dev/sda1 /media/hdd

root@et9x00:/media# dd if=/dev/zero of=/media/hdd/blanks2 bs=1024k count=100

100+0 records in

100+0 records out

104857600 bytes (100.0MB) copied, 1.507302 seconds, 66.3MB/s

root@et9x00:/media#

[dvboxer]

Hm what's the difference then? The kernel?

uname -a
Linux vuduo 3.1.1 #1 SMP Thu Feb 9 06:22:38 CET 2012 mips GNU/Linux

On my box slice_idle=1 is really not that important. When mounting as ext2 I get 66 Mbyte/s with the first dd even with slice_idle=8. That's why I doubt it's the scheduler. Shouldn't it have an impact on ext2 writing performance?

OTOH: I tried every sensible ext4 parameter I found on "man 8 mount". None seems to have a considerable effect.

[MiLo]

Your first dd has to create the file, the second just overwrites it. But the difference should be only small, like on my box.

Also, 100MB fits in RAM so one should actually expect rediculously fast results...

[MiLo]

Just for clarity, slice_idle has NO impact on "raw" performance whatsoever, that's normal. It DOES influence things when multiple threads are using it, which is the case while recording. It will probably LOWER performance in the form of throughput (which is what's being aimed at the most because it's easy to do), even in that case. But it reduces "peak" latencies, by allowing more disk seeks to happen. This will make the total throughput less, but it prevents "starving" the recording threads.

[MiLo]

.. When mounting as ext2 I get 66 Mbyte/s with the first dd even with slice_idle=8. That's why I doubt it's the scheduler. Shouldn't it have an impact on ext2 writing performance?

I doubt your "first dd is slow" has anything to do at all with the recording issues. Try running fsck, maybe something's wrong with the disk. Best is to just format it, as I wrote earlier.

[dvboxer]

I have experimented a lot with it, e.g. removing the file before issuing the second dd and dd-ing from files on a mounted usb stick instead of /dev/zero (using a different file on the second run). Just to exclude those possibilities. To no avail, it has always been the same pattern: the first write about 5Mbyte/s the second about 10 times as fast.

[dvboxer]

I doubt your "first dd is slow" has anything to do at all with the recording issues. Try running fsck, maybe something's wrong with the disk. Best is to just format it, as I wrote earlier.

I had checked the hdd before experimenting, it was clean. Also in my home forum aaf there are a lot of users having the same problem. And the problem first arose after updating the kernel to the 3 series.

[Stefan_Salewski]

I can confirm that the first dd command after reboot is much slower. I have formated the partition with ext4 and mounted ext4. This is a Vu-Duo box, with latest Pli software update and SAMSUNG HD204UI disk. I did echo "test" > test.txt to ensure that the disk is not in standby. (I have only minimal Ruckler/Hänger/Glitches.)

root@vus:/media# echo "test" > test.txt
root@vus:/media# ls
hdd net sda1 sda2 sda3 sda4 test.txt
root@vus:/media# dd if=/dev/zero of=/media/hdd/blanks2 bs=1024k count=100
100+0 records in
100+0 records out
104857600 bytes (100.0MB) copied, 4.445641 seconds, 22.5MB/s
root@vus:/media# dd if=/dev/zero of=/media/hdd/blanks2 bs=1024k count=100
100+0 records in
100+0 records out
104857600 bytes (100.0MB) copied, 1.890970 seconds, 52.9MB/s
root@vus:/media# rm hdd/blanks2
root@vus:/media# dd if=/dev/zero of=/media/hdd/blanks2 bs=1024k count=100
100+0 records in
100+0 records out
104857600 bytes (100.0MB) copied, 1.706991 seconds, 58.6MB/s
root@vus:/media#
root@vus:/media# tune2fs -l /dev/sda4
tune2fs 1.42 (29-Nov-2011)
Filesystem volume name: <none>
Last mounted on: /media/hdd
Filesystem UUID: e4708530-2fb0-449b-b887-10368af19717
Filesystem magic number: 0xEF53
Filesystem revision #: 1 (dynamic)
Filesystem features: has_journal ext_attr resize_inode dir_index filetype needs_recovery extent flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize
Filesystem flags: signed_directory_hash
Default mount options: (none)
Filesystem state: clean
Errors behavior: Continue
Filesystem OS type: Linux
Inode count: 100999168
Block count: 403968022
Reserved block count: 20198401
Free blocks: 320526603
Free inodes: 100998724
First block: 0
Block size: 4096
Fragment size: 4096
Reserved GDT blocks: 927
Blocks per group: 32768
Fragments per group: 32768
Inodes per group: 8192
Inode blocks per group: 512
Flex block group size: 16
Filesystem created: Wed Feb 1 15:16:03 2012
Last mount time: Thu Jan 1 01:00:04 1970
Last write time: Thu Jan 1 01:00:04 1970
Mount count: 34
Maximum mount count: 29
Last checked: Wed Feb 1 15:16:03 2012
Check interval: 15552000 (6 months)
Next check after: Mon Jul 30 16:16:03 2012
Lifetime writes: 523 GB
Reserved blocks uid: 0 (user root)
Reserved blocks gid: 0 (group root)
First inode: 11
Inode size: 256
Required extra isize: 28
Desired extra isize: 28
Journal inode: 8
Default directory hash: half_md4
Directory Hash Seed: 729ae560-aaa9-4794-9c11-8fd8d99ea7cb
Journal backup: inode blocks
root@vus:/media# cat /etc/fstab
rootfs / auto defaults 1 1
proc /proc proc defaults 0 0
devpts /dev/pts devpts mode=0620,gid=5 0 0
usbfs /proc/bus/usb usbfs defaults 0 0
tmpfs /var/volatile tmpfs defaults 0 0
/dev/sda4 /media/hdd ext4 defaults 0 0
/dev/sda1 none swap sw 0 0
root@vus:/media#
I hope the echo "1" > /sys/block/sda/queue/iosched/slice_idle will help, and I will reformat the disk for large blocks when kernel 3.2 is available for Vu-Duo.

Best regards,

Stefan Salewski

[MiLo]

You realize you did not write that "test.txt" to the harddisk

Anyway, your results aren't as dramatic as dvboxer's, but I have no explanation for the effect. Your second test shows that it isn't related to the block allocation.

[MiLo]

I hope the echo "1" > /sys/block/sda/queue/iosched/slice_idle will help...

No need to echo, it's already the default starting today.

[dvboxer]

Anyway, your results aren't as dramatic as dvboxer's, but I have no explanation for the effect. Your second test shows that it isn't related to the block allocation.

I got my first dd speeded up to 10 Mbyte/second by using data=writeback,noatime,barrier=0,delalloc and the default slice_idle=8. But that's it. It seems vu+ duo is particularly problematic. Are there any relevant kernel config switches different for that machine?

[MiLo]

If you disable journalling altogether (umount, "tune2fs -O ^has_journal /dev/sda1" and mount) does the first dd delay disappear? (not intended as a fix, but merely an investigation)