←  [EN] Rytec XMLTV and EPG support

Forums

»

EPGImport latest changes: explanation

Pr2's Photo Pr2 15 Sep 2018

Hi,
 
Here are a few explanation about the latest changes in EPGImport:
 
For IPTV:
 
You now need to match the service reference between the userbouquet and the channels.xml file.
So if your IPTV channel are properly added using the #SERVICE 4097:.... now in the channels.xml this MUST also be 4097!
 
 
LastUpdate.txt:
 
We have faced several times the following problem, Doglover/Rytec can no longer upload newest file to EPG mirror servers but servers still distribute obsolete EPG data files to client. Result was no EPG even if data was downloaded because data where obsolete.
 
To prevent this it was decide in accordance with Rytec/Doglover to add a control file:  LastUpdate.txt, this file must only be set with the last update date when all the files are properly uploaded to it. (date format is YYYY-mm-dd)
So if we have again a server in this bad state it will be rejected by the checkValidServer.
 
 
But people that define there own sources.xml file that point to there own server can decide not to use this check (LastUpdate.log) by adding the attribute  nocheck="1"
 
example: <source type="gen_xmltv" nocheck="1" channels="http://epg.in.ua/epg...annels.xml.gz">
 
 
Be careful:
First implementation uses LastUpdate.log but a mirror was unable to provide .log file so we need to change to LastUpdate.txt
 
 
IPTV or custom sources.xml: file custom.channels.xml
 
When defining sources.xml file it was mandatory to define again all the servers URL.
 
So for people that use IPTV and create there own sources.xml file and also channels.xml file this create a double donwload from EPG data mirror.
 
To avoid this an ease the creation of personal .channels.xml files it is now possible to just create one file:
 
custom.channels.xml
 
You include in this file all you self created service reference (the same as in your bouquets files).
Just place this file into /etc/epgimport/ folder, it will be readed and processed for every downloaded EPG data file.
 
 No more case sensitive for channel id into channels.xml file:
 
To ease maintainer life Doglove/Rytec it has been decided to have the channel id=" "  no more case sensitive.
 
So TF1.fr tf1.fr Tf1.fr will now be considered as the same channel for EPG data sources.
 
 
Pr2

Edited by Pr2, 30 January 2019 - 11:28.
Quote

Abu Baniaz's Photo Abu Baniaz 15 Sep 2018

Can the service type be omitted or treated the same whether 1, 19, 4097, 5001, 5002?

Quote

doglover's Photo doglover 16 Sep 2018

Can the service type be omitted or treated the same whether 1, 19, 4097, 5001, 5002?

 

 

Can you explain this a bit more pls.

I am not sure, I can answer this.

 

Willy

Quote

Pr2's Photo Pr2 16 Sep 2018

Can the service type be omitted or treated the same whether 1, 19, 4097, 5001, 5002?

 

I understand your question, this needs to be checked in code and tested.

Quote

Robinson's Photo Robinson 16 Sep 2018

I think it would be good to update pinned topics when the new improvements have been finalized.

Thanks. :)

Quote

Pr2's Photo Pr2 16 Sep 2018

For people concerned with fake SERVICE 1: for IPTV please test this:

 

https://github.com/p...e38ce66ba19acaf

 

I will wait for your feedback before creating the Pull Request.

 

Pr2

Quote

Pr2's Photo Pr2 16 Sep 2018

Be careful: a mirror was unable to return file with .log extension, so we decide to change the file from LastUpdate.log to LastUpdate.txt.

 

Sorry for this late change for people using there own EPG data server please update the creation of your LastUpdate.txt file accordingly or use the nocheck="1" option.

Quote

Abu Baniaz's Photo Abu Baniaz 16 Sep 2018

 

Can the service type be omitted or treated the same whether 1, 19, 4097, 5001, 5002?

 

 

Can you explain this a bit more pls.

I am not sure, I can answer this.

 

Willy

 

 

For picons, we allow all service types to work with a picon. Whether 1, 19, 4097, 5001, 5002. I was hoping the same could be done for EPG.

Quote

Abu Baniaz's Photo Abu Baniaz 16 Sep 2018

It looks like I mixed up the terminology. The point I was trying to make is below

 

https://github.com/O...icon.py#L78:L80

Quote

humaxboy's Photo humaxboy 17 Sep 2018

Guys what i am doing wrong ? still using my rytec.sources.xml and channels.xml.

Sources can be seen (import.jpg), checking Manual brings error (import2.jpg), log shows that (import3.jpg).

Attached Files

Quote

doglover's Photo doglover 18 Sep 2018

I have no idea what is going wrong.

However, it seems that you have a custom made sources file.  The one on the screenshot is definitely not the one from rytec.

So in that file you could have everything changed.  No idea what.

 

Can you show us the files you made?

 

Willy

Quote

Abu Baniaz's Photo Abu Baniaz 18 Sep 2018

That is a picture from OpenATV. The OE-A fork is broken because of the IP6 checks.

Sent from my Moto G (5S) using Forum Fiend v1.3.3.
Quote

humaxboy's Photo humaxboy 18 Sep 2018

@doglover - yes I have custom made sources file

@abu Baniaz - yes, that is OpenAtv..so it is their fault ? i just need simple yes or no...

Quote

dsayers2016's Photo dsayers2016 18 Sep 2018

Probably best posting on OpenATV forum. Issue has been opened here https://github.com/o...mport/issues/30

It's been worked on
Quote

Pr2's Photo Pr2 18 Sep 2018

The change are made are for EPGImport this one is used only by OpenPLi  from EPGImport there is a fork called XMLTVImport which is used by OE-Alliance images (check dsayers2016 provided link).

So someone port the change I made in EPGImport into XMLTVImport but this porting cause troubles, because XMLTVImport implement a legacy IPv6 download that doesn't work properly.

 

So problem is on the OE-Alliance side, not in OpenPLi one.

Quote

Rob van der Does's Photo Rob van der Does 19 Sep 2018

Correct.

And that ported change has now been reverted.

Quote

Pr2's Photo Pr2 19 Sep 2018

Yes funny solution instead a tackling down the bug in the IPv6 implementation.

Quote

Rob van der Does's Photo Rob van der Does 19 Sep 2018

Indeed :(
Quote

jenseneverest's Photo jenseneverest 20 Sep 2018

okay using latest openpli image dated 18/9/18, epgimport plugin 1.0+git162

 

Some source import okay others do not

 

the log below is for Rytec UK Sat/Virgin XMLTV

                                           UK/Ireland - Sports/Movies (xz)

[EPGImport] No settings [Errno 2] No such file or directory: '/etc/enigma2/epgimport.conf'
[EPGImport] Selected source:  ['UK/Ireland - Sports/Movies (xz)']
sourcesDone():  False None
[EPGImport] nextImport, source= UK/Ireland - Sports/Movies (xz)
[EPGImport] Downloading: http://rytecepg.dyndns.tv/~rytecepg/epg_data/rytecUK_SportMovies.xz to local path: /tmp/epgimport.xz
[EPGImport] afterDownload /tmp/epgimport.xz
[EPGImport] unlink /tmp/epgimport.xz
[EPGImport] afterChannelDownload None
[EPGImport] Using twisted thread
[XMLTVConverter] Enumerating event information
Unknown channel:  skycinemathriller.uk
Unknown channel:  skymoviespremiere.uk
Unknown channel:  skymoviescomedy.uk
Unknown channel:  skymoviesactionthriller.uk
Unknown channel:  skymoviesfamily.uk
Unknown channel:  skymoviesdrama.uk
Unknown channel:  skyscifihorror.uk
Unknown channel:  skymoviesindie.uk
Unknown channel:  skycinemahits.uk
Unknown channel:  skymoviesdisney.uk
Unknown channel:  skymoviespremiereplus1.uk
Unknown channel:  skyarts1.uk
Unknown channel:  skyatlantic.uk
Unknown channel:  skyatlanticplus1.uk
Unknown channel:  skybrits.uk
Unknown channel:  zeecinema.uk
Unknown channel:  skysports5.ie
Unknown channel:  skysports1.ie
Unknown channel:  mutv.uk
Unknown channel:  chelseatv.uk
Unknown channel:  espn.uk
Unknown channel:  skyspboxoff.uk
Unknown channel:  skyspmainev.uk
Unknown channel:  skyspcricket.uk
Unknown channel:  skysppl.uk
Unknown channel:  skyspf1.uk
Unknown channel:  skyspnews.uk
Unknown channel:  skysptheopen.uk
Unknown channel:  skyspaction.uk
Unknown channel:  eurosport1.uk
Unknown channel:  ginxesportstv.uk
Unknown channel:  eurosport2.uk
Unknown channel:  btsporteur.uk
Unknown channel:  skyspfball.uk
Unknown channel:  skysparena.uk
Unknown channel:  liverpoolfctv.uk
Unknown channel:  skyspmix.uk
[EPGImport] ### thread is ready ### Events: 0
[EPGImport] imported 0 events
[EPGImport] Save last import date and count event
[EPGImport] Run check deep standby after import
[EPGImport] #### Finished ####

Quote

Pr2's Photo Pr2 20 Sep 2018

Sorry but this log doesn't display any error.
So either:
- You don't receive the satellite that this file is intended for so nothing is loaded into memory: imported 0 events
- The references used for those channels have changed and nobody inform Doglover of the change.
- You receive those channels from another means (DVB-C for exemple) and the reference are not known by Doglover.
 
Here is the list of channels included in this package:
  <channel id="BTSportEur.uk">
  <channel id="ChelseaTV.uk">
  <channel id="ESPN.uk">
  <channel id="Eurosport1.uk">
  <channel id="Eurosport2.uk">
  <channel id="ExtremeSports.uk">
  <channel id="GINXeSportsTV.uk">
  <channel id="LiverpoolFCTV.uk">
  <channel id="MUTV.uk">
  <channel id="MovieMix.uk">
  <channel id="SkyArts1.uk">
  <channel id="SkyAtlantic.uk">
  <channel id="SkyAtlanticplus1.uk">
  <channel id="SkyBrits.uk">
  <channel id="SkyCinemaHits.uk">
  <channel id="SkyCinemaThriller.uk">
  <channel id="SkyMoviesActionThriller.uk">
  <channel id="SkyMoviesComedy.uk">
  <channel id="SkyMoviesDisney.uk">
  <channel id="SkyMoviesDrama.uk">
  <channel id="SkyMoviesFamily.uk">
  <channel id="SkyMoviesIndie.uk">
  <channel id="SkyMoviesPremiere.uk">
  <channel id="SkyMoviesPremiereplus1.uk">
  <channel id="SkySciFiHorror.uk">
  <channel id="SkySpAction.uk">
  <channel id="SkySpArena.uk">
  <channel id="SkySpBoxOff.uk">
  <channel id="SkySpCricket.uk">
  <channel id="SkySpF1.uk">
  <channel id="SkySpFball.uk">
  <channel id="SkySpMainEv.uk">
  <channel id="SkySpMix.uk">
  <channel id="SkySpNews.uk">
  <channel id="SkySpPL.uk">
  <channel id="SkySpTheOpen.uk">
  <channel id="SkySports1.ie">
  <channel id="SkySports5.ie">
  <channel id="ZeeCinema.uk">
So if you can provide the correct service reference for those channels I am sure Doglover will include them.
Edited by Pr2, 20 September 2018 - 11:42.
Quote