196 replies to this topic

[MiLo]

if you do a " ls -l `which mount` " you can see which mount program is being used. You could also type "busybox mount" to run the busybox version. For what I can see in your output, the nodelalloc option is in effect.

(note: If you disable journalling, better re-enable delalloc, it actually improves performance in that case)

[Qu@rk]

Thanks again.

I have used the busybox mount (see my post). The "nodelalloc" flag will not be displayed with busybox mount.

So, I am back to what I had before. I have disabled journaling and the rest is PLi default (delalloc).

You have written that for max. performance we should disable journaling and enable nodelalloc flag.
That's why I would have liked to test nodelalloc.

With disabled journaling the writeback flag has no effect because data=writeback has only an effect with enabled journaling.

So: With ext4 just disable journaling e.g. using tune2fs and you have at least (or better) the performance of ext2 (without freezers at recordings).

So we don't need a /etc/fstab entry, just disable journaling via tune2fs?

[Qu@rk]

Another question (from me and some forum members of the et forum):

blkid of e2fsprogs showed the UUID and the real filesystem type of the partition.

The blkid from busybox shows the UUID only.

On PC-Linux there are different commands (df, vol_id etc.), e.g. the df -T command, but busybox df does not support this option.

Is there another way to determine the real filesystem type of a partition (ext2, ext3, ext4)? Not the used kernel module but the real filesystem type?

[Qu@rk]

Btw, I did another PLi update today. My busybox was updated from r41 to r42 and now mount shows "nodelalloc" correctly.

But I will not start testing again. I will take MiLo's word, disable journaling, enable delalloc.
If I have freezers again, I will report here.

Edited by Qu@rk, 10 February 2012 - 11:30.

[MiLo]

Is there another way to determine the real filesystem type of a partition (ext2, ext3, ext4)? Not the used kernel module but the real filesystem type?

"tune2fs -l" shows what's actually going on. There isn't really a ext2/3/4 format, the combination of features/flags determines what's there. And to further complicate things, you can turn things on and off on the fly...

The system type can be derived from the flags, the "extends" and "uninit_bg" flags are exclusive for ext4. Journalling indicates ext3, ext2 is the same as ext3 but without journalling. An ext3 disk can be mounted by an ext2 driver (which is the only way to disable journalling). A "true" ext4 disk can only be mounted by the ext4 driver, because the ext2/3 drivers don't understand extents. An ext4 disk without extents can be mounted by an ext2 driver.

[Qu@rk]

Great explanation, thanks a lot. Have a nice weekend.

[hulster]

A totally different guess: I am still experiencing glitches/freezer running verfied EXT2. My disk is 500 GB, so no issues with sector size. Joining the discussion in the vendor forum I was always wondering, that there is no real distinct behavior. Even EXT4 seems to be an significant factor, it seems not be the only one.
So what about if in the 3.x the thread handling or process priorisation is different. Or maybe just the priorisation of the recording processes by new PLi build.
For me it looks currently that everything ressource grabbing could have a negative affect to recordings. Some stuff more due to higher resource needs some other less.
So if the priority for recordings running in background has changed to lower for some reason, this could explain some of the behavior and especially not every proposal/chenge works the same for everybody.
Just an idea - which MiLo may wipe out easily.

Edited by hulster, 15 February 2012 - 20:50.

[Qu@rk]

@hulster
Did you install the ext2-kernel-module and did you create an entry in /etc/fstab for mounting your hdd?

If you use ext2 with automount (no change in /etc/fstab), your hdd will be mounted with the ext4-kernel-module, which is default.

All users who have tested the ext2 filesystem reported freezers when they were using the default ext4-kernel-module.

This ext2 workaround will only have a positive effect, if you use the ext2-kernel-module.

[hulster]

@hulster
Did you install the ext2-kernel-module and did you create an entry in /etc/fstab for mounting your hdd?

If you use ext2 with automount (no change in /etc/fstab), your hdd will be mounted with the ext4-kernel-module, which is default.

All users who have tested the ext2 filesystem reported freezers when they were using the default ext4-kernel-module.

This ext2 workaround will only have a positive effect, if you use the ext2-kernel-module.

Thx - but known. That is why I mentioned ext2 VERIFIED. "mount" output clearly tells ext2. And yes kernel module installed and fstab entried made.

[Hump]

HDDs formated with ext2 and run by the ext2 driver module works completely freezerfree. That is proven and confirmed.

[MiLo]

HDDs formated with ext2 and run by the ext2 driver module works completely freezerfree. That is proven and confirmed.

So we should all just ignore it when hulster contradicts that?

[Hump]

HDDs formated with ext2 and run by the ext2 driver module works completely freezerfree. That is proven and confirmed.

So we should all just ignore it when hulster contradicts that?

Yes. Think he's got a different problem. There is a monster-thread at the Xtrend Support Forum http://www.et-view-s...n-Aufnahmen-... where users have tested several weeks with journal off and different mount options with ext4. Journaling off reduced the freezers significantly, but there were still some mini-freezer every now an than. There is still some testing going on with ext3 run by the ext3 driver module, which looks also promising. But it is not fully confirmed, that it's running fully free of freezers.

Only ext2 run by the ext2 driver module ist confirmed free of freezers by all users who have switched to it. hulster is the only one that has reported problems with ext2/ext2 driver module so far.

Edited by Hump, 16 February 2012 - 16:22.

[Qu@rk]

@MiLo
We have at least a dozen of confirmations of users who had freezers with ext4 and tested ext2-kernel-module with ext2 filesystem or ext3-kernel-module with ext3 filesystem and all do not have any freezers any more.

There is definitely a problem with the ext4-kernel-module if journaling is activated. Some ext4 users (like me) reported success with deactivated journaling some still had freezers.

So you should not ignore the very successful workaround using ext2 (or ext3) with ext2/ext3-kernel-modules.

Using ext2 with ext2-kernel-module is definitely the best workaround at present for the freezers problem.

Edited by Qu@rk, 17 February 2012 - 08:57.

[MiLo]

I don't see it that way. It does not work like a democracy. If there are a thousand reports that some setting "fixes" the issue, and only one that it does not, then my conclusion is that it does NOT fix the issue.

If the ext4 driver alone is the issue, then mounting with the ext3 driver should work just as well, because we've been using that for ages now. But it doesn't. The root cause of the problem is in the kernel somewhere. It is triggered by a combination of factors, the journal is a major contributor, but also delayed allocation and probably the IO scheduler play important roles too. It's not limited to ext filesystems either, the btrfs filesystem performed even worse.

[hulster]

I don't see it that way. It does not work like a democracy. If there are a thousand reports that some setting "fixes" the issue, and only one that it does not, then my conclusion is that it does NOT fix the issue.

If the ext4 driver alone is the issue, then mounting with the ext3 driver should work just as well, because we've been using that for ages now. But it doesn't. The root cause of the problem is in the kernel somewhere. It is triggered by a combination of factors, the journal is a major contributor, but also delayed allocation and probably the IO scheduler play important roles too. It's not limited to ext filesystems either, the btrfs filesystem performed even worse.

Hi MiLo,

Thx for not ignoring me. Would be great to get a comment just regarding my guess about the priority of processes.

In general, especially for other users: Indeed EXT2 makes it MUCH better and I didn't said that there is no other issue. That there is MAYBE ANOTHER issue I tried to point out.
So my intention was to state EXT2 helps a lot, but MAYBE not for everything/everyone.
Just additionally - I have not too much plugins installed - especially no EMC. Skin HDGlass16.

[MiLo]

I'm currently looking into the IO scheduler. One of the most promising ones is:

/sys/block/sda/queue/iosched/slice_idle

(look it up on Google)
This defaults to 8, which is way to high in my opinion. After setting it to 1 I haven't been able to create overruns, while at 8 that is pretty easy to do (journalling is enabled, delalloc enabled, using ext4).

[anudanan]

Milo, many thanks for your look into the kernel scheduler. The scheduling prio may cause the reason für some delays? I´m very interessting on your results of your test with different prios and your study of the scheduler inside the kernel

Edited by anudanan, 17 February 2012 - 20:25.

[dvboxer]

I'm currently looking into the IO scheduler. One of the most promising ones is:

/sys/block/sda/queue/iosched/slice_idle

(look it up on Google)
This defaults to 8, which is way to high in my opinion. After setting it to 1 I haven't been able to create overruns, while at 8 that is pretty easy to do (journalling is enabled, delalloc enabled, using ext4).

How did you do that? I tried "echo 1> /sys/block/sda/queue/iosched/slice_idle" but a subsequent cat on that file gave 0.

[dvboxer]

How did you do that? I tried "echo 1> /sys/block/sda/queue/iosched/slice_idle" but a subsequent cat on that file gave 0.

Anyway, the problem stays with slice_idle set to 0. The only thing for me that works is mounting the drive as ext2. Disabling the journal, nodelalloc,data=writeback, all those have no effect. I always have two freezers at the beginning of a recording from ARD HD (720p).

Edited by dvboxer, 18 February 2012 - 09:37.

[dvboxer]

How did you do that? I tried "echo 1> /sys/block/sda/queue/iosched/slice_idle" but a subsequent cat on that file gave 0.

Stupid me. "echo 1> file" obviously redirects stderr to file, sorry. But setting it to 4 stops freezing here, too. That is without journal. I will put it back on and try again.