Jump to content


Photo

EPG data distribution format


  • Please log in to reply
93 replies to this topic

Re: EPG data distribution format #61 WanWizard

  • PLi® Core member
  • 70,533 posts

+1,811
Excellent

Posted 31 July 2017 - 13:34

@Robinson,

 

At the moment, not.

 

When OpenPLi 6 is released, we will check the state of OpenPLi 4 for one last time, update what needs/can be updated, and build a final image. After that we switch the buildserver off. Due to the outdated OE version, OpenPLi 4 can not be build on our current server platform which uses a gcc compiler which is to new.

 

We're still thinking about a mechanism to update selected packages after that moment, such as the tuxbox package (satellites.xml etc), the rytec sources package, channel settings packages, etc. so that users that have/want to remain on OpenPLi 4 will still receive updates for those packages, for a limited period.


Currently in use: VU+ Duo 4K (2xFBC S2), VU+ Solo 4K (1xFBC S2), uClan Usytm 4K Ultimate (S2+T2), Octagon SF8008 (S2+T2), Zgemma H9.2H (S2+T2)

Due to my bad health, I will not be very active at times and may be slow to respond. I will not read the forum or PM on a regular basis.

Many answers to your question can be found in our new and improved wiki.


Re: EPG data distribution format #62 doglover

  • Rytec EPG Team
  • 17,391 posts

+658
Excellent

Posted 31 July 2017 - 15:24

I tried to install it on OpenPLi4.0 and got this:

root@et9x00:~# opkg install /tmp/enigma2-plugin-extensions-xmltvimport_1.0+git112+606c345-r0_mips32el.ipk
Upgrading enigma2-plugin-extensions-xmltvimport from 1.0+git110+127feaf-r0.1 to 1.0+git112+606c345-r0 on root.
enigma2-plugin-extensions-xmltvimport: unsatisfied recommendation for python-lzma
Collected errors:
* satisfy_dependencies_for: Cannot satisfy the following dependencies for enigma2-plugin-extensions-xmltvimport:
*      libc6 (>= 2.24) *
* opkg_install_cmd: Cannot install package enigma2-plugin-extensions-xmltvimport.
root@et9x00:~#
root@et9x00:~# opkg install /tmp/libc6_2.24-r0_mips32el.ipk
Not selecting libc6 2.24 as installing it would break existing dependencies.
Package libc6 (2.20-r0.0) installed in root is up to date.
root@et9x00:~#

Without any further knowledge I do not risk to go further.

 

Willy


Edited by doglover, 31 July 2017 - 15:26.

~~Rytec Team~~
Maxytec Multibox SE OpenPli (used as mediaplayer)
Mutant HD2400 OpenPli
Vu+ Duo OpenPli (backup)

Synology NAS

Sat: 13E, 19.2E, 23.5E and 28.2E
*Pli/Rytec EPG POWERED*


Re: EPG data distribution format #63 WanWizard

  • PLi® Core member
  • 70,533 posts

+1,811
Excellent

Posted 31 July 2017 - 16:37

Yes, that doesn't work, python-lzma isn't available in OpenPLi 4.


Currently in use: VU+ Duo 4K (2xFBC S2), VU+ Solo 4K (1xFBC S2), uClan Usytm 4K Ultimate (S2+T2), Octagon SF8008 (S2+T2), Zgemma H9.2H (S2+T2)

Due to my bad health, I will not be very active at times and may be slow to respond. I will not read the forum or PM on a regular basis.

Many answers to your question can be found in our new and improved wiki.


Re: EPG data distribution format #64 Abu Baniaz

  • PLi® Contributor
  • 2,499 posts

+64
Good

Posted 31 July 2017 - 17:25

The libc incompatibilities/issues are usually because of build machine. You can work around it by unzipping plugin using 7zip and then transferring it. As opposed to installing it. However that will not resolve the python lzma issue



Re: EPG data distribution format #65 MiLo

  • PLi® Core member
  • 14,055 posts

+298
Excellent

Posted 31 July 2017 - 17:56

Het hoeven geen aparte files te zijn. Alles kan ook in 1 file. Maar dan moet er een niveau (node) toegeveogd worden.
En dan vrees ik dat die xml file niet meer compatibel is met de huidige plugin.

Zal morgen eens een xml file maken met hoofdnodes en subnodes. Door het opsplitsen van de files voor ieder land word anders de hele lijst wel heel lang en onoverzichtelijk. Dat is de reden dat ik een tree view vraag.

As ik afzonderlijke files maak voor iedere hoofdnode, zijn deze gwoon compatibel met de huidige plug-in.


Denk dat 't niet veel uitmaakt. Losse files zijn denk ik wel makkelijker onderhouden. Een <description> tag in elke <sources> zou wel handig zijn voor het geven van een omschrijving.
Real musicians never die - they just decompose

Re: EPG data distribution format #66 doglover

  • Rytec EPG Team
  • 17,391 posts

+658
Excellent

Posted 31 July 2017 - 18:17

Daarin heb je gelijk.

Maar bedoel je met die description tag zoiets als dit?  Of mag dat ook <description> noemen?

<?xml version="1.0" encoding="latin-1"?>
<sources>
<desc>Belgium</desc>>
<channel name="rytec.channels.xml.gz">
    <url>http://url1/rytec.channels.xml.gz</url>
    <url>http://url2/rytec.channels.xml.gz</url>
    <url>http://url3/rytec.channels.xml.gz</url>
</channel>
<channel name="telenet.channels.xml.gz">
    <url>http://url1/telenet.channels.xml.gz</url>
    <url>http://url2/epg_data/telenet.channels.xml.gz</url>
    <url>http://url3/telenet.channels.xml.gz</url>
</channel>
<source type="gen_xmltv" channels="rytec.channels.xml.gz">
    <description>Vlaanderen - Basis</description>
     <url>http://url1/rytec_BE_VL_basic.gz</url>
     <url>http://url2/rytec_BE_VL_basic.gz</url>
     <url>http://url3/rytec_BE_VL_basic.gz</url>     
</source>
<source type="gen_xmltv" channels="rytec.channels.xml.gz">
      <description>Vlaanderen - Nederland - Combo</description>
     <url>http://url1/rytec_BE_NL_common.gz</url>
     <url>http://url2/rytec_BE_NL_common.gz</url>
     <url>http://url3/rytec_BE_NL_common.gz</url>     
</source>
<source type="gen_xmltv" channels="rytec.channels.xml.gz">
    <description>Wallonie - Telesat Base</description>
     <url>http://url1/rytec_BE_FR_basic.gz</url>
     <url>http://url2/rytec_BE_FR_basic.gz</url>
     <url>http://url3/rytec_BE_FR_basic.gz</url>     
</source>
<source type="gen_xmltv" channels="rytec.channels.xml.gz">
    <description>Français - TNT</description>
     <url>http://url1/rytec_TNT.gz</url>
     <url>http://url2/rytec_TNT.gz</url>
     <url>http://url3/rytec_TNT.gz</url>     
</source>
<source type="gen_xmltv" channels="rytec.channels.xml.gz">
    <description>Français - Commun</description>
     <url>http://url1/rytec_BE_FR_common.gz</url>
     <url>http://url2/rytec_BE_FR_common.gz</url>
     <url>http://url3/rytec_BE_FR_common.gz</url>     
</source> 
</sources>

Willy


~~Rytec Team~~
Maxytec Multibox SE OpenPli (used as mediaplayer)
Mutant HD2400 OpenPli
Vu+ Duo OpenPli (backup)

Synology NAS

Sat: 13E, 19.2E, 23.5E and 28.2E
*Pli/Rytec EPG POWERED*


Re: EPG data distribution format #67 doglover

  • Rytec EPG Team
  • 17,391 posts

+658
Excellent

Posted 31 July 2017 - 18:25

Yes, that doesn't work, python-lzma isn't available in OpenPLi 4.

 

Knew I had forgotten something in the list of things I tried:

root@et9x00:~# opkg install /tmp/python-lzma_0.0.8-r0_mips32el.ipk
Installing python-lzma (0.0.8-r0) on root.
Collected errors:
* satisfy_dependencies_for: Cannot satisfy the following dependencies for python-lzma:
*      libc6 (>= 2.24) *       liblzma5 (>= 5.2.2) *
* opkg_install_cmd: Cannot install package python-lzma.
root@et9x00:~#
root@et9x00:~# opkg install /tmp/liblzma5_5.2.2-r0_mips32el.ipk
Installing liblzma5 (5.2.2-r0) on root.
Collected errors:
* satisfy_dependencies_for: Cannot satisfy the following dependencies for liblzma5:
*      libc6 (>= 2.24) *
* opkg_install_cmd: Cannot install package liblzma5.
root@et9x00:~#

It boils down that first libc6 (>= 2.24) needs to be isntalled.  And this fails.

root@et9x00:~# opkg install /tmp/libc6_2.24-r0_mips32el.ipk
Not selecting libc6 2.24 as installing it would break existing dependencies.
Package libc6 (2.20-r0.0) installed in root is up to date.
root@et9x00:~#

All the rest just follows.

 

Willy


~~Rytec Team~~
Maxytec Multibox SE OpenPli (used as mediaplayer)
Mutant HD2400 OpenPli
Vu+ Duo OpenPli (backup)

Synology NAS

Sat: 13E, 19.2E, 23.5E and 28.2E
*Pli/Rytec EPG POWERED*


Re: EPG data distribution format #68 doglover

  • Rytec EPG Team
  • 17,391 posts

+658
Excellent

Posted 1 August 2017 - 08:48

Conclusion for OpenPLi 4.0:  It may work, but it requires some trickery.  So not to be pursued.

 

It means that everything I do should be compatible with the importer of OPenPLi4.0 for some time (at least another year)

 

Willy


~~Rytec Team~~
Maxytec Multibox SE OpenPli (used as mediaplayer)
Mutant HD2400 OpenPli
Vu+ Duo OpenPli (backup)

Synology NAS

Sat: 13E, 19.2E, 23.5E and 28.2E
*Pli/Rytec EPG POWERED*


Re: EPG data distribution format #69 littlesat

  • PLi® Core member
  • 57,176 posts

+698
Excellent

Posted 1 August 2017 - 10:04

It means that everything I do should be compatible with the importer of OPenPLi4.0 for some time (at least another year)

If you want to continue support on OpenPli4.... But should we?


WaveFrontier 28.2E | 23.5E | 19.2E | 16E | 13E | 10/9E | 7E | 5E | 1W | 4/5W | 15W


Re: EPG data distribution format #70 Dream1975

  • Senior Member
  • 1,634 posts

+14
Neutral

Posted 1 August 2017 - 10:18

I would opt for, after releasing 6 stable, the boologo (except for the brands not going to have 6) is changed to one which states that OpenPLi 6 is out and can be upgraded for free (pointing to the wiki for instructions how to upgrade) and stating 4 is eol and support has stopped. This way you don't have to put too much effort in it.


Mutant HD2400, OpenPLi nightly, 2x DVB-C & 2x DVB-S

Mutant HD51, OpenPLi nightly, 1x DVB-C & 1x DVB-S

Wavefrontier T55 (Astra 1,2,3 en HB)

Smartcards Ziggo (Irdeto) and CDS (Seca) on Oscam

 


Re: EPG data distribution format #71 doglover

  • Rytec EPG Team
  • 17,391 posts

+658
Excellent

Posted 1 August 2017 - 10:35

 

It means that everything I do should be compatible with the importer of OPenPLi4.0 for some time (at least another year)

If you want to continue support on OpenPli4.... But should we?

 

 

One more reason:  A few lines higher it was said that xz support was something for OpenPLi 6.1 RC. 

However on OpenPLi 6.0 RC all that was required to enable xz support is installing python-lzma (Tried this).  So why not include it in OpenPLi 6.0 stable as well ?

 

Willy


~~Rytec Team~~
Maxytec Multibox SE OpenPli (used as mediaplayer)
Mutant HD2400 OpenPli
Vu+ Duo OpenPli (backup)

Synology NAS

Sat: 13E, 19.2E, 23.5E and 28.2E
*Pli/Rytec EPG POWERED*


Re: EPG data distribution format #72 WanWizard

  • PLi® Core member
  • 70,533 posts

+1,811
Excellent

Posted 1 August 2017 - 12:40

The general process is:

 

- pick a moment your develop is stable

- branch that off to a release candidate

- test the release candidate in frozen state for a few days

- fix any bugs you find, and merge those fixes back into develop

- branch the release candidate to a stable, and release the product

 

This under normal circumstances takes only a few days, and requires the frozen state for proper testing, which means no new functionality must be introduced in the R/C after it is created, otherwise you have to start testing again. But since it is only a few days, it should not be a problem.

 

At the moment, we're in the situation that it is our first R/C (using this process), we're short on build capacity (we have to build the R/C during the day, and we have no capacity for a release build) so that needs to be expanded, and it's in the middle of the holiday period, so half the team is enjoying the sun somewhere.

 

This has caused the R/C to exist for much longer than is forseen in the process, so we might have to evaluate whether or not we pull features forward before the R/C is ready to be released. This feature is relatively harmless, lzma has no dependencies at all, we only have to make sure the importer itself is properly tested so it doesn't crash when you decide to switch to xz files.


Currently in use: VU+ Duo 4K (2xFBC S2), VU+ Solo 4K (1xFBC S2), uClan Usytm 4K Ultimate (S2+T2), Octagon SF8008 (S2+T2), Zgemma H9.2H (S2+T2)

Due to my bad health, I will not be very active at times and may be slow to respond. I will not read the forum or PM on a regular basis.

Many answers to your question can be found in our new and improved wiki.


Re: EPG data distribution format #73 doglover

  • Rytec EPG Team
  • 17,391 posts

+658
Excellent

Posted 1 August 2017 - 16:10

At the moment I run the 6.0 RC with python-lzma installed for a few days.  No problems so far.

 

Except the plug-in itself cannot handle channels.xml file compressed with xz.  It is not a big deal as the volume gained compared to a channels.xml gz compressed is minimal.

Reported this to Milo.

Either the xz file is faulty.  Although it is the same file as the gz file, but xz compressed (with 7zip).  Or the importer cannot handle it.

[EPGImport] Parsing channels from '/home/Fritz/epg/rytec.channels.xml.xz'
[EPGImport] failed to parse /home/Fritz/epg/rytec.channels.xml.xz Error: not well-formed (invalid token): line 1, column 0

Nothing crashes on this error.  So that is good.

 

Willy


Edited by doglover, 1 August 2017 - 16:11.

~~Rytec Team~~
Maxytec Multibox SE OpenPli (used as mediaplayer)
Mutant HD2400 OpenPli
Vu+ Duo OpenPli (backup)

Synology NAS

Sat: 13E, 19.2E, 23.5E and 28.2E
*Pli/Rytec EPG POWERED*


Re: EPG data distribution format #74 WanWizard

  • PLi® Core member
  • 70,533 posts

+1,811
Excellent

Posted 1 August 2017 - 17:05

Looking at the code, MiLo only added support for xz for the epg data files, not for the channels xml. I'll have a look.


Currently in use: VU+ Duo 4K (2xFBC S2), VU+ Solo 4K (1xFBC S2), uClan Usytm 4K Ultimate (S2+T2), Octagon SF8008 (S2+T2), Zgemma H9.2H (S2+T2)

Due to my bad health, I will not be very active at times and may be slow to respond. I will not read the forum or PM on a regular basis.

Many answers to your question can be found in our new and improved wiki.


Re: EPG data distribution format #75 WanWizard

  • PLi® Core member
  • 70,533 posts

+1,811
Excellent

Posted 1 August 2017 - 17:09

Bij deze: https://github.com/O...d3168f5908c2a18

 

Je kunt er dit bij je lokaal ook in zetten, om te testen.


Currently in use: VU+ Duo 4K (2xFBC S2), VU+ Solo 4K (1xFBC S2), uClan Usytm 4K Ultimate (S2+T2), Octagon SF8008 (S2+T2), Zgemma H9.2H (S2+T2)

Due to my bad health, I will not be very active at times and may be slow to respond. I will not read the forum or PM on a regular basis.

Many answers to your question can be found in our new and improved wiki.


Re: EPG data distribution format #76 doglover

  • Rytec EPG Team
  • 17,391 posts

+658
Excellent

Posted 2 August 2017 - 08:16

Gecompileerd, en op de ET6500 gezet.  (weet nu ook, hoe slechts één ipk te bouwen)

En perfect:

[EPGImport] nextImport, source= News Channels (xz)
[EPGImport] afterChannelDownload None
[EPGImport] Using twisted thread
[EPGImport] Parsing channels from '/home/Fritz/epg/rytec.channels.xml.xz'
[XMLTVConverter] Enumerating event information
Unknown channel:  DeutscheWelleDE.de
Unknown channel:  AlArabiya.nws
Unknown channel:  TV5MONDEMaghreb-Orient.fr
Unknown channel:  PCNEChinese.uk
Unknown channel:  SkyNwsArabia.nws
Unknown channel:  TRTInternational
[EPGImport] ### thread is ready ### Events: 13171

Ook getest met files van het net.

 

Willy


Edited by doglover, 2 August 2017 - 08:17.

~~Rytec Team~~
Maxytec Multibox SE OpenPli (used as mediaplayer)
Mutant HD2400 OpenPli
Vu+ Duo OpenPli (backup)

Synology NAS

Sat: 13E, 19.2E, 23.5E and 28.2E
*Pli/Rytec EPG POWERED*


Re: EPG data distribution format #77 WanWizard

  • PLi® Core member
  • 70,533 posts

+1,811
Excellent

Posted 2 August 2017 - 13:09

Mooi, is dat probleem ook weer uit de wereld. ;)


Currently in use: VU+ Duo 4K (2xFBC S2), VU+ Solo 4K (1xFBC S2), uClan Usytm 4K Ultimate (S2+T2), Octagon SF8008 (S2+T2), Zgemma H9.2H (S2+T2)

Due to my bad health, I will not be very active at times and may be slow to respond. I will not read the forum or PM on a regular basis.

Many answers to your question can be found in our new and improved wiki.


Re: EPG data distribution format #78 jeroenho

  • Senior Member
  • 147 posts

0
Neutral

Posted 23 August 2017 - 10:38

I was curious about my ET10k:

root@Randy:~# opkg install python-lzma

Package python-lzma (0.0.8-r0.0) installed in root is up to date.

root@Randy:~# opkg install liblzma5

Package liblzma5 (5.2.0-r0.0) installed in root is up to date.

root@Randy:~# opkg install xz

Installing xz (5.2.0-r0.0) on root.

Downloading http://downloads.pli....0_mips32el.ipk.

Configuring xz.

After this, I deselected the XML-sources and selected the .xz sources, and it's importing as I write this... That was UNexpected!...

OpenPLI 4 updated the last few days (thursday-sunday), EPGImport from https://forums.openp...-sources/page-3, #41 due to 0-files problem.

I hope this helps?

A am still in the process of figuring out what sources I need and don't need.

Later I will switch to OpenPLI6, but for now I hope this will reduce your data-usage.


Edited by jeroenho, 23 August 2017 - 10:39.

1x VU+ Ultimo 4K (1x FBC DVB-C, 1x FBC DVB-S2, 4TB HDD ST4000VN008), 1x ET10.000 (3x DVB-C, 1x DVB-S2, 4TB HDD ST4000VN000) , NFS mount naar QNAP, Astra 1,2,3, DVB-C via Ziggo (regio 5555)

Re: EPG data distribution format #79 doglover

  • Rytec EPG Team
  • 17,391 posts

+658
Excellent

Posted 23 August 2017 - 10:42

You also need to update the XMLTVimport plugin to the latest version.

 

It works now on OpenPLi 4 as well, but with the latest version from the feed.

 

Willy


~~Rytec Team~~
Maxytec Multibox SE OpenPli (used as mediaplayer)
Mutant HD2400 OpenPli
Vu+ Duo OpenPli (backup)

Synology NAS

Sat: 13E, 19.2E, 23.5E and 28.2E
*Pli/Rytec EPG POWERED*


Re: EPG data distribution format #80 jeroenho

  • Senior Member
  • 147 posts

0
Neutral

Posted 23 August 2017 - 11:00

My regular update-day is thursday. Friday and the weekend I'm free from work. This means I have plenty of time to fix any problems. ;) So I will update tomorrow and see.

Randy is my test-box. I like The Muppets, so guess the name of my production-box. Via a VPN I can safely reach the box from work when there's temporarely nothing for me to do.


1x VU+ Ultimo 4K (1x FBC DVB-C, 1x FBC DVB-S2, 4TB HDD ST4000VN008), 1x ET10.000 (3x DVB-C, 1x DVB-S2, 4TB HDD ST4000VN000) , NFS mount naar QNAP, Astra 1,2,3, DVB-C via Ziggo (regio 5555)


0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users