Jump to content


Photo

.xz, the undesireable compression format


  • Please log in to reply
9 replies to this topic

#1 Huevos

  • PLi® Contributor
  • 4,767 posts

+167
Excellent

Posted 22 October 2017 - 18:01

From the author of Lzip compressing utility:

Xz has a complex format, partially specialized in the compression of executables and designed to be extended by proprietary formats. Of the four compressors tested here, xz is the only one alien to the Unix concept of "doing one thing and doing it well". It is the less appropriate for data sharing, and not appropriate at all for long-term archiving.

In general, the more complex the format, the less probable that it can be decoded in the future. But the xz format, just as its infamous predecessor lzma-alone, is specially badly designed. Xz copies almost all the defects of gzip and then adds some more, like the fragile variable-length integers. Just one bit-flip in bit 7 of any byte of one variable-length integer and the whole xz stream comes tumbling down like a house of cards. Using xz for anything other than compressing short-lived executables is not advisable.

Don't interpret me wrong. I am very grateful to Igor Pavlov for inventing/discovering LZMA, but xz is the third attempt of his followers to take advantage of the popularity of 7zip and replace gzip and bzip2 with inappropriate or badly designed formats. In particular, it is shameful that support for lzma-alone was implemented in both GNU and Linux.

 

http://www.nongnu.or..._benchmark.html


Edited by Huevos, 22 October 2017 - 18:02.


Re: .xz, the undesireable compression format #2 Huevos

  • PLi® Contributor
  • 4,767 posts

+167
Excellent

Posted 22 October 2017 - 18:09

Xz format inadequate for long-term archiving:
http://www.nongnu.or...inadequate.html



Re: .xz, the undesireable compression format #3 WanWizard

  • PLi® Core member
  • 70,897 posts

+1,833
Excellent

Posted 22 October 2017 - 18:27

Nothing new. The xz format is only used for epg data, which is short lived data, and isn't meant for archiving or storage.


Currently in use: VU+ Duo 4K (2xFBC S2), VU+ Solo 4K (1xFBC S2), uClan Usytm 4K Ultimate (S2+T2), Octagon SF8008 (S2+T2), Zgemma H9.2H (S2+T2)

Due to my bad health, I will not be very active at times and may be slow to respond. I will not read the forum or PM on a regular basis.

Many answers to your question can be found in our new and improved wiki.


Re: .xz, the undesireable compression format #4 Huevos

  • PLi® Contributor
  • 4,767 posts

+167
Excellent

Posted 22 October 2017 - 18:46

Takes a lot more processing power and much bigger RAM footprint. Receivers like the Vu+ Solo are going to crash.



Re: .xz, the undesireable compression format #5 WanWizard

  • PLi® Core member
  • 70,897 posts

+1,833
Excellent

Posted 22 October 2017 - 23:11

We'll see. If that is the case we'll have to address that. So far, there are no reports of EPG imports no longer working.


Currently in use: VU+ Duo 4K (2xFBC S2), VU+ Solo 4K (1xFBC S2), uClan Usytm 4K Ultimate (S2+T2), Octagon SF8008 (S2+T2), Zgemma H9.2H (S2+T2)

Due to my bad health, I will not be very active at times and may be slow to respond. I will not read the forum or PM on a regular basis.

Many answers to your question can be found in our new and improved wiki.


Re: .xz, the undesireable compression format #6 Erik Slagter

  • PLi® Core member
  • 46,969 posts

+542
Excellent

Posted 23 October 2017 - 19:11

Takes a lot more processing power and much bigger RAM footprint. Receivers like the Vu+ Solo are going to crash.

Compression is much much harder on the producing side than the consumer side.


* Wavefrontier T90 with 28E/23E/19E/13E via SCR switches 2 x 2 x 6 user bands
I don't read PM -> if you have something to ask or to report, do it in the forum so others can benefit. I don't take freelance jobs.
Ik lees geen PM -> als je iets te vragen of te melden hebt, doe het op het forum, zodat anderen er ook wat aan hebben.


Re: .xz, the undesireable compression format #7 doglover

  • Rytec EPG Team
  • 17,467 posts

+667
Excellent

Posted 24 October 2017 - 10:08

 

Takes a lot more processing power and much bigger RAM footprint. Receivers like the Vu+ Solo are going to crash.

Compression is much much harder on the producing side than the consumer side.

 

 

After producing now for several months the EPG in xz format, I can confirm that producing xz files takes more processing power.

But you have to see this in perspective.  Producing a gz file takes 1 sec.  Compressing the same file in xz would take 3 to 4 sec.

 

Is this a problem?  I do not think so.

 

Willy


~~Rytec Team~~
Maxytec Multibox SE OpenPli (used as mediaplayer)
Mutant HD2400 OpenPli
Vu+ Duo OpenPli (backup)

Synology NAS

Sat: 13E, 19.2E, 23.5E and 28.2E
*Pli/Rytec EPG POWERED*


Re: .xz, the undesireable compression format #8 Erik Slagter

  • PLi® Core member
  • 46,969 posts

+542
Excellent

Posted 24 October 2017 - 17:58

 

 

Takes a lot more processing power and much bigger RAM footprint. Receivers like the Vu+ Solo are going to crash.

Compression is much much harder on the producing side than the consumer side.

 

After producing now for several months the EPG in xz format, I can confirm that producing xz files takes more processing power.

But you have to see this in perspective.  Producing a gz file takes 1 sec.  Compressing the same file in xz would take 3 to 4 sec.

 

Is this a problem?  I do not think so.

You missed my point. On your side the compression may take quite a bit longer, that is something you will have to consider, but on the clients side (the receivers) the decompression will NOT take longer, so that is not relevant anyway.


* Wavefrontier T90 with 28E/23E/19E/13E via SCR switches 2 x 2 x 6 user bands
I don't read PM -> if you have something to ask or to report, do it in the forum so others can benefit. I don't take freelance jobs.
Ik lees geen PM -> als je iets te vragen of te melden hebt, doe het op het forum, zodat anderen er ook wat aan hebben.


Re: .xz, the undesireable compression format #9 MiLo

  • PLi® Core member
  • 14,055 posts

+298
Excellent

Posted 27 October 2017 - 17:22

...Just one bit-flip in bit 7 of any byte of one variable-length integer and the whole xz stream comes tumbling down like a house of cards...


For compression, that's a desirable feature. If a single bitflip does not destroy the whole archive, the compression could have been better.
Real musicians never die - they just decompose

Re: .xz, the undesireable compression format #10 Erik Slagter

  • PLi® Core member
  • 46,969 posts

+542
Excellent

Posted 28 October 2017 - 09:46

Hahaha, you're so right, +1. If you'd want the information density to become lower, I'd guess you'd need a decompressor instead of a compressor :)

 

I guess they're calculating some digest though, so you can at least know whether the file is uncompromised. That's always better than an uncompressed plain that doesn't have that protection.


* Wavefrontier T90 with 28E/23E/19E/13E via SCR switches 2 x 2 x 6 user bands
I don't read PM -> if you have something to ask or to report, do it in the forum so others can benefit. I don't take freelance jobs.
Ik lees geen PM -> als je iets te vragen of te melden hebt, doe het op het forum, zodat anderen er ook wat aan hebben.



3 user(s) are reading this topic

0 members, 3 guests, 0 anonymous users