Jump to content


Photo

.del files?


  • Please log in to reply
72 replies to this topic

Re: .del files? #21 WanWizard

  • Forum Moderator
    PLi® Core member
  • 48,786 posts

+834
Excellent

Posted 30 January 2018 - 11:48

Yes, you can just delete them.

 

What it does is it renames a file to ".del", then in a loop it slowly removes a part from it, so it makes it shorter, until it is all gone. If I read it correctly, it should erase by 20MB/s.

 

It would not be a bad idea to check when Enigma starts to see if there are ".del" files in the Trashcan, and of so, start the file eraser again.


Currently in use: VU+Duo 4K (2xFBC S2), Amiko Viper T2C (T2), SAB Alpha Triple HD (S2+T2), Zgemma H3.T2C (T/C), Zgemma H6 (fallback), VU+Zero (fallback)

Many answers to your question can be found in our new and improved wiki.

note: I do not provide support via PM !

 


Re: .del files? #22 sjlouis

  • Senior Member
  • 1,422 posts

+3
Neutral

Posted 30 January 2018 - 11:57

I don't understand why it needs time to erase files. Perhaps it's not the same way with Linux but with Windows, the name of the file and its address are erased from the allocation table, then its place on the disk is free.

 

Yes it would be a good idea to check the trashcan when Enigma restarts :) .


Vu+ Ultimo 4K - OpenPli 6.2

(Xtrend ET9200 - OpenPli 4.0 - Oscam)


Re: .del files? #23 WanWizard

  • Forum Moderator
    PLi® Core member
  • 48,786 posts

+834
Excellent

Posted 30 January 2018 - 13:25

FAT does that yes, in NTFS it works different, but similar.

 

ext3 and ext4 (without extents) use an indirect block mapping scheme, where it tracks each block used. This makes deleting large files very slow. If you would just use the OS "delete" command, Enigma would be blocked (and display a spinner) for as long as is needed to delete all these blocks. And this is because Enigma is not written to be multithreaded, things don't happen in parallel.


Currently in use: VU+Duo 4K (2xFBC S2), Amiko Viper T2C (T2), SAB Alpha Triple HD (S2+T2), Zgemma H3.T2C (T/C), Zgemma H6 (fallback), VU+Zero (fallback)

Many answers to your question can be found in our new and improved wiki.

note: I do not provide support via PM !

 


Re: .del files? #24 sjlouis

  • Senior Member
  • 1,422 posts

+3
Neutral

Posted 30 January 2018 - 13:47

Ok. That's why I don't see it previously a few years ago when we have only SD files (<> HD files). There were smaller.

 

Does it explain it's so difficult to restore a deleted file with Linux while there are a lot of tools which work well with Windows?


Vu+ Ultimo 4K - OpenPli 6.2

(Xtrend ET9200 - OpenPli 4.0 - Oscam)


Re: .del files? #25 WanWizard

  • Forum Moderator
    PLi® Core member
  • 48,786 posts

+834
Excellent

Posted 30 January 2018 - 16:56

Yes.

 

A file deleted in ext3 and ext4 is gone, as there is no reference to the blocks that made up the file, and sequence of blocks. With FAT, only the directory entry is deleted, but the file allocation table isn't, so if you can recover the starting block (which is still in the directory entry if not overwritten by a new file entry) you can recover the entire file.


Currently in use: VU+Duo 4K (2xFBC S2), Amiko Viper T2C (T2), SAB Alpha Triple HD (S2+T2), Zgemma H3.T2C (T/C), Zgemma H6 (fallback), VU+Zero (fallback)

Many answers to your question can be found in our new and improved wiki.

note: I do not provide support via PM !

 


Re: .del files? #26 Pike_Bishop

  • Senior Member
  • 885 posts

+33
Good

Posted 30 January 2018 - 19:55

@Robinson,

 

 

Also, is there a command to move ALL files in the root of my HDD to HDD/movie folder?

 

you can use wildcards for that as example;

mv /media/hdd/*.* /media/hdd/movie

but in this case also the file epg.dat is moving to /media/hdd/movie and make sure that you didn't have directorys under /media/hdd with a dot in the name

as example action.thrill because otherwise they are also moved to /media/hdd/movie

 

 

regards

Biki3


Edited by Biki3, 30 January 2018 - 19:56.

Receiver: Mutant HD 2400, VU Ultimo 4k

Image: PLI 7.1


Re: .del files? #27 sjlouis

  • Senior Member
  • 1,422 posts

+3
Neutral

Posted 31 January 2018 - 11:57

Thank you WanWizard :) .


Vu+ Ultimo 4K - OpenPli 6.2

(Xtrend ET9200 - OpenPli 4.0 - Oscam)


Re: .del files? #28 Erik Slagter

  • PLi® Core member
  • 45,353 posts

+498
Excellent

Posted 31 January 2018 - 20:12

Additional note: space allocation using ext4 (recommended!) is done using "extends" which comprise large amounts of blocks. Files recorded usinge extends (and once you mounted the device as "ext4", all NEW files will have that), are removed remarkably quick.


* Wavefrontier T90 with 28E/23E/19E/13E/9E/4.8E/0.8W/5W via SCR switches 2 x 2 x 6 user bands
* Ziggo digital cable TV (FTA)
I don't read PM -> if you have something to ask or to report, do it in the forum so others can benefit. I don't take freelance jobs.
Ik lees geen PM -> als je iets te vragen of te melden hebt, doe het op het forum, zodat anderen er ook wat aan hebben.

Re: .del files? #29 sjlouis

  • Senior Member
  • 1,422 posts

+3
Neutral

Posted 1 February 2018 - 10:42

Remarkably quick ? It takes several minutes to delete one movie and it can takes 20 minutes  :o  to delete a few files.


Vu+ Ultimo 4K - OpenPli 6.2

(Xtrend ET9200 - OpenPli 4.0 - Oscam)


Re: .del files? #30 MiLo

  • PLi® Core member
  • 13,890 posts

+290
Excellent

Posted 1 February 2018 - 18:22

Additional note: space allocation using ext4 (recommended!) is done using "extends" which comprise large amounts of blocks. Files recorded usinge extends (and once you mounted the device as "ext4", all NEW files will have that), are removed remarkably quick.


Only on "native" ext4 partitions. If the disk was converted from ext3 to ext4, or if it's still ext2/3, it will not have the "extend" feature and files will be just as slow to delete.
Real musicians never die - they just decompose

Re: .del files? #31 Erik Slagter

  • PLi® Core member
  • 45,353 posts

+498
Excellent

Posted 1 February 2018 - 19:59

That's not how I remember it, the extend feature should be one as soon as you mount as ext4, but of course, only for new files extends will be used. But quite simple to test I guess. I don't have file systems from ancient converted from ext3 to ext4, but maybe someone here does?


* Wavefrontier T90 with 28E/23E/19E/13E/9E/4.8E/0.8W/5W via SCR switches 2 x 2 x 6 user bands
* Ziggo digital cable TV (FTA)
I don't read PM -> if you have something to ask or to report, do it in the forum so others can benefit. I don't take freelance jobs.
Ik lees geen PM -> als je iets te vragen of te melden hebt, doe het op het forum, zodat anderen er ook wat aan hebben.

Re: .del files? #32 sjlouis

  • Senior Member
  • 1,422 posts

+3
Neutral

Posted 2 February 2018 - 12:04

I have no old disk file but it's easy to take a "free" disk, format it in ext3, copy movies and delete them to see what you want to control ;) . I don't know how we can see these extends.


Vu+ Ultimo 4K - OpenPli 6.2

(Xtrend ET9200 - OpenPli 4.0 - Oscam)


Re: .del files? #33 MiLo

  • PLi® Core member
  • 13,890 posts

+290
Excellent

Posted 3 February 2018 - 14:47

That's not how I remember it, the extend feature should be one as soon as you mount as ext4, but of course, only for new files extends will be used. But quite simple to test I guess. I don't have file systems from ancient converted from ext3 to ext4, but maybe someone here does?

 

I do. You're right, it still gets the "extent" feature. It's the lack of bigalloc probably the slows the delete down.

 

Filesystem features:

 

Converted disk:

dir_index filetype extent sparse_super large_file uninit_bg
 

"Native" ext4 disk:

has_journal ext_attr resize_inode dir_index filetype needs_recovery extent flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize bigalloc


Real musicians never die - they just decompose

Re: .del files? #34 Erik Slagter

  • PLi® Core member
  • 45,353 posts

+498
Excellent

Posted 3 February 2018 - 16:24

Bigalloc doesn't seem to make such a big difference and, at least where I am looking, it's said to be unsafe...


* Wavefrontier T90 with 28E/23E/19E/13E/9E/4.8E/0.8W/5W via SCR switches 2 x 2 x 6 user bands
* Ziggo digital cable TV (FTA)
I don't read PM -> if you have something to ask or to report, do it in the forum so others can benefit. I don't take freelance jobs.
Ik lees geen PM -> als je iets te vragen of te melden hebt, doe het op het forum, zodat anderen er ook wat aan hebben.

Re: .del files? #35 MiLo

  • PLi® Core member
  • 13,890 posts

+290
Excellent

Posted 9 February 2018 - 13:24

It made quite a noticeable difference on the old 400MHz boxes, years ago when I introduced it. Probably with the newer "more than GHz" boxes the overhead isn't as noticable any more, I haven't done any measurements lately.

The major improvement is that it has to read less from the disk in order to write to it. Writing almost never stalls on the raw throughput, it always stalls on allocating new clusters which requires information to be read from disk, and contrary to just writing, you cannot continue until that read action returns the data you need.

Edited by MiLo, 9 February 2018 - 13:26.

Real musicians never die - they just decompose

Re: .del files? #36 anudanan

  • Senior Member
  • 1,002 posts

+14
Neutral

Posted 9 February 2018 - 14:00

I´m using an old version of EMC on my boxes (not the new version with many new code for enigma functions, it works well with openpli 4.1 und 6.1) and for deleting files the EMC uses a different linux process to do that. The advantage is, that no stalling of enigma is there. I delete my movies with EMC, and the file delete goes quit. I have never  seen any interrupts in recordings or playbacks which are runs in parallel.

 

I think deleting files in a different linux process than enigma is a good way to do that. And with that you have no .DEL files on the HDD


Edited by anudanan, 9 February 2018 - 14:03.

Receiver:2 x Uno4k SE (PLI 7.0 rel), 1 x ET9200 (PLI 4.0), NAS: 2 x QNAP 410, TV: LG 65C8llla, LG 47LB570V, LG 42LM615S, Sound: Yamaha RX-v663, Teufel System 5 THX

Re: .del files? #37 MiLo

  • PLi® Core member
  • 13,890 posts

+290
Excellent

Posted 9 February 2018 - 18:37

The reason you're not getting interrupts in recordings is that your filesystem is ext4. It's totally unrelated to EMC, to processes or to the alignment of verious planets.

The built-in delete uses a thread (light-weight process) to do the deleting. There's no advantage whatsoever in forking a process, the kernel does not treat them any different.

The ".del" files will disappear by themselves, since they're in the trashcan anyway.
Real musicians never die - they just decompose

Re: .del files? #38 anudanan

  • Senior Member
  • 1,002 posts

+14
Neutral

Posted 9 February 2018 - 18:42

thanks for information. I´ve understand in this thread enigma makes a busy waiting unlink()


Receiver:2 x Uno4k SE (PLI 7.0 rel), 1 x ET9200 (PLI 4.0), NAS: 2 x QNAP 410, TV: LG 65C8llla, LG 47LB570V, LG 42LM615S, Sound: Yamaha RX-v663, Teufel System 5 THX

Re: .del files? #39 Erik Slagter

  • PLi® Core member
  • 45,353 posts

+498
Excellent

Posted 10 February 2018 - 14:19

You might consider starting the deletion without renaming them, that way they would instantly become "invisable" (which in itself also has drawbacks).


* Wavefrontier T90 with 28E/23E/19E/13E/9E/4.8E/0.8W/5W via SCR switches 2 x 2 x 6 user bands
* Ziggo digital cable TV (FTA)
I don't read PM -> if you have something to ask or to report, do it in the forum so others can benefit. I don't take freelance jobs.
Ik lees geen PM -> als je iets te vragen of te melden hebt, doe het op het forum, zodat anderen er ook wat aan hebben.

Re: .del files? #40 WanWizard

  • Forum Moderator
    PLi® Core member
  • 48,786 posts

+834
Excellent

Posted 10 February 2018 - 14:24

That would be the case with an unlink, but that is not how it works.

 

Instead, it uses callbacks to truncate the file with a few blocks at the time, until it reaches the beginning, after which it calls unlink to delete the 0-byte file left.

 

And this is done because an unlink of a large file on ext3 takes an absolute age, producing a spinner...


Currently in use: VU+Duo 4K (2xFBC S2), Amiko Viper T2C (T2), SAB Alpha Triple HD (S2+T2), Zgemma H3.T2C (T/C), Zgemma H6 (fallback), VU+Zero (fallback)

Many answers to your question can be found in our new and improved wiki.

note: I do not provide support via PM !

 





0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users