ET9000:Hänger in HD Aufnahmen
Re: ET9000:Hänger in HD Aufnahmen #61
Posted 5 February 2012 - 16:58
thank you very much for your efforts in order to find the bottleneck of the ext4 driver module. Would like to test that option with nodelalloc, but I reformated my HDD to ext2 already. But I'm sure someone will do pick up and will conduct a long-run test. I'm anxious on what the result might be.
Again, thanks a million for your hard work! - You guys are THE best!
PS: After testing many HD recordings with ext2 and ext2 module, no freezer appeared at all.
Re: ET9000:Hänger in HD Aufnahmen #62
Posted 5 February 2012 - 17:44
would you like to set the nodealloc parm als default in your image in future?
Edited by anudanan, 5 February 2012 - 17:45.
Receiver:2 x Uno4k SE (PLI 7.3 rel), 1 x ET9200 (PLI 4.0), NAS: 2 x QNAP 410, TV: LG 65C8llla, LG 47LB570V, LG 42LM615S, Sound: Yamaha RX-v663, Teufel System 5 THX
Re: ET9000:Hänger in HD Aufnahmen #63
Re: ET9000:Hänger in HD Aufnahmen #64
Posted 5 February 2012 - 18:13
would you like to set the nodealloc parm als default in your image in future?
It reduces performance considerably, so i'd prefer to find a way to solve it in user-space now that I have a good candidate for what is going on in the kernel. I was thinking along the lines of "fallocate" for example.
Re: ET9000:Hänger in HD Aufnahmen #65
Posted 5 February 2012 - 19:06
http://www.h-online....xt4-740787.html
http://article.gmane...x.kernel/811654
http://article.gmane...x.kernel/811700
http://article.gmane...x.kernel/811699
http://www.linux-mag...Ext-3-und-Ext-4 (article in german)
Re: ET9000:Hänger in HD Aufnahmen #66
Posted 5 February 2012 - 19:29
Re: ET9000:Hänger in HD Aufnahmen #67
Re: ET9000:Hänger in HD Aufnahmen #68
Posted 5 February 2012 - 20:10
Re: ET9000:Hänger in HD Aufnahmen #69
Posted 5 February 2012 - 20:39
In the past I´ve choosen ext2 because I heared about some negativ aspects with journal of ext3. Is it right that a file delete stalls all other write because the delete is an atomic operation on ext3 journaling filesystem. So a delete of a movie file will produce some freeze on the recordings? I´ve never tested that but I seems logical for me. Is that the reason for the background delete parameter in your Image?
Do you know if ext4 solves that delete stalling problem. Is there also the need for background deleting parameter?
Do you think the ext4 performance with nodealloc parameter is to low for the box? Is it lower that the ext2 performance in the past? If not I think the ext4 performance with nodealloc would be ok for the boxes.
Receiver:2 x Uno4k SE (PLI 7.3 rel), 1 x ET9200 (PLI 4.0), NAS: 2 x QNAP 410, TV: LG 65C8llla, LG 47LB570V, LG 42LM615S, Sound: Yamaha RX-v663, Teufel System 5 THX
Re: ET9000:Hänger in HD Aufnahmen #70
Posted 6 February 2012 - 08:39
ext4 does not completely solve the stalling-on-delete problem, but improves on delete performance by a large factor. The stall is caused by heavy disk IO, not the journal. This is only the case for files that were created AFTER the ext4 conversion, deleting files that were written when the filesystem was ext3 or ext2 (deleting files on ext3 and ext2 is equally slow) is still as slow.
Re: ET9000:Hänger in HD Aufnahmen #71
Posted 8 February 2012 - 16:18
Thanks a lot for your investigation.
I have tested with disabled journaling, writeback and I have much less "freezers".
I would like to test with default mount parameters and then activate "nodelalloc".
I would like to test it correclty:
Please, what are the default mount parameters (e.g. to set with tune2fs) in PLi 2.1 with ext4?
has_journal
journal_data_ordered
Edited by Qu@rk, 8 February 2012 - 16:19.
Re: ET9000:Hänger in HD Aufnahmen #72
Posted 8 February 2012 - 18:54
Are these the correct default parameters?
fsck
reboot
Then I set "nodelalloc" using tune2fs: tune2fs -o nodelalloc /dev/sda1
reboot
fsck
tune2fs -l /dev/sda1 lists the correct mount options.
But the mount command does not show "nodelalloc" as a flag for /dev/sda1
How can I check whether "nodelalloc" has been activated?
Re: ET9000:Hänger in HD Aufnahmen #73
Posted 8 February 2012 - 18:56
/dev/sda1 /media/hdd ext4 defaults,nodelalloc,data=writeback 0 0
To test, just unmount and mount the disk with "mount /dev/sda1" (without options). Then type "mount" and you should see the options in effect.
Re: ET9000:Hänger in HD Aufnahmen #74
Posted 8 February 2012 - 19:07
UUID=37408d8b-d8ba-4442-b158-88b5bbf25a22 /media/hdd ext4 defaults,nodelalloc,data=writeback 0 0
umount /dev/sda1
mount /dev/sda1
mount
rootfs on / type rootfs (rw)
ubi0:rootfs on / type ubifs (rw,relatime)
proc on /proc type proc (rw,relatime)
sysfs on /sys type sysfs (rw,relatime)
tmpfs on /dev type tmpfs (rw,relatime,size=64k,mode=755)
devpts on /dev/pts type devpts (rw,relatime,mode=600)
usbfs on /proc/bus/usb type usbfs (rw,relatime)
tmpfs on /var/volatile type tmpfs (rw,relatime)
/dev/sda1 on /media/hdd type ext4 (rw,relatime,barrier=1,data=writeback)
I can see, that the options were used because of the change from "ordered" to "writeback".
But no "nodelalloc".
What is PLi default "writeback" or "ordered"?
Journal is activated by default?
dmesg | grep EXT4
EXT4-fs (sda1): mounted filesystem with writeback data mode. Opts: nodelalloc,data=writeback
But mount does not show "nodelalloc".
Edited by Qu@rk, 8 February 2012 - 19:12.
Re: ET9000:Hänger in HD Aufnahmen #75
Posted 8 February 2012 - 20:57
For optimum better-than-ext2 performance, turn of the journaling. You can leave delalloc enabled in that case. The price you pay is that after power failure, you will need to run fsck to fix the system. That may take several minutes to complete. I've tested this, pulled the power plug while 4 HD recordings and a DD transfer were running. Booted fine, mounted okay, fsck took about 2 minutes and fixed some things.
For better brownout stability while preventing overruns, disable delalloc. You can use data=writeback and/ore commit=30 and/or barrier=0 if you want to squeeze out more performance, but this will cost you some stability, and you will lose more data and risk data corruption again.
I'm still not really convinced what's best. I don't care about losing recently written data on powerout, who cares about that 5 seconds of video when the box is down for at least a minute. But it should be able to survive power loss without corrupting the rest of the filesystem... Any takers? (btw, btrfs didn't run well on the ET, the system crashed after a few minutes of recording).
Re: ET9000:Hänger in HD Aufnahmen #76
Posted 8 February 2012 - 23:06
Receiver:2 x Uno4k SE (PLI 7.3 rel), 1 x ET9200 (PLI 4.0), NAS: 2 x QNAP 410, TV: LG 65C8llla, LG 47LB570V, LG 42LM615S, Sound: Yamaha RX-v663, Teufel System 5 THX
Re: ET9000:Hänger in HD Aufnahmen #77
Posted 9 February 2012 - 07:27
anudanan, very good idea!
During my tests, I have installed the packages e2fsprog and e2fsprog-tune2fs (to get blkid and tune2fs). I think e2fsprog-tune2fs would have been enough, but I did notice that to late.
If I uninstall e2fsprog-package, will I get back all the original binaries?
Or will I have to do a complete reinstall (oh my gosh) of my PLi 2.1 to get back the original PLi 2.1 state?
Edited by Qu@rk, 9 February 2012 - 07:27.
Re: ET9000:Hänger in HD Aufnahmen #78
Posted 9 February 2012 - 08:15
Instead of e2fsprogs, only install e2fsprogs-tune2fs, that contains the binary you need. The e2fsprogs package draws in quite a lot of unneeded dependencies.
I'll change the image to include tune2fs by default on the systems with a newer kernel. Makes it easier for the harddisk menu to use it.
Re: ET9000:Hänger in HD Aufnahmen #79
Re: ET9000:Hänger in HD Aufnahmen #80
Posted 9 February 2012 - 14:56
I have downloaded busybox from feed and reinstalled it.
busybox mount still shows no "nodelalloc" flag.
I have disabled journaling (tune2fs -O ^has_journal /dev/sda1), did an fsck and reboot.
I don't know what's wrong.
Does your mount show the "nodelalloc" flag?
My settings:
/etc/fstab
rootfs / auto defaults 1 1 proc /proc proc defaults 0 0 devpts /dev/pts devpts mode=0620,gid=5 0 0 usbfs /proc/bus/usb usbfs defaults 0 0 tmpfs /var/volatile tmpfs defaults 0 0 UUID=37408d8b-d8ba-4442-b158-88b5bbf25a22 /media/hdd ext4 defaults,nodelalloc,data=writeback 0 0
mount (or busybox mount)
rootfs on / type rootfs (rw) ubi0:rootfs on / type ubifs (rw,relatime) proc on /proc type proc (rw,relatime) sysfs on /sys type sysfs (rw,relatime) tmpfs on /dev type tmpfs (rw,relatime,size=64k,mode=755) devpts on /dev/pts type devpts (rw,relatime,mode=600) /dev/sda1 on /media/hdd type ext4 (rw,relatime,barrier=1) usbfs on /proc/bus/usb type usbfs (rw,relatime) tmpfs on /var/volatile type tmpfs (rw,relatime)
tune2fs -l /dev/sda1
tune2fs 1.41.14 (22-Dec-2010) Filesystem volume name: <none> Last mounted on: /media/hdd Filesystem UUID: 37408d8b-d8ba-4442-b158-88b5bbf25a22 Filesystem magic number: 0xEF53 Filesystem revision #: 1 (dynamic) Filesystem features: ext_attr resize_inode dir_index filetype extent flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize Filesystem flags: signed_directory_hash Default mount options: journal_data_writeback nodelalloc Filesystem state: not clean Errors behavior: Continue Filesystem OS type: Linux Inode count: 1907712 Block count: 488361943 Reserved block count: 0 Free blocks: 131638341 Free inodes: 1905775 First block: 0 Block size: 4096 Fragment size: 4096 Reserved GDT blocks: 907 Blocks per group: 32768 Fragments per group: 32768 Inodes per group: 128 Inode blocks per group: 8 Flex block group size: 16 Filesystem created: Fri Aug 5 17:11:48 2011 Last mount time: Thu Jan 1 01:00:03 1970 Last write time: Thu Feb 9 14:47:48 2012 Mount count: 1 Maximum mount count: 30 Last checked: Thu Feb 9 14:34:53 2012 Check interval: 15552000 (6 months) Next check after: Tue Aug 7 15:34:53 2012 Lifetime writes: 3136 GB Reserved blocks uid: 0 (user root) Reserved blocks gid: 0 (group root) First inode: 11 Inode size: 256 Required extra isize: 28 Desired extra isize: 28 Default directory hash: half_md4 Directory Hash Seed: 89cd731e-bce3-4f9a-ba08-2be6509aad34 Journal backup: inode blocks
dmesg | grep EXT4
EXT4-fs (sda1): mounted filesystem without journal. Opts: nodelalloc,data=writeback
Edited by Qu@rk, 9 February 2012 - 14:57.
17 user(s) are reading this topic
0 members, 17 guests, 0 anonymous users