Jump to content


Photo

FAQ's about EPGimport (previously called XMLTV import)

FAQ XMLTV

  • This topic is locked This topic is locked
No replies to this topic

#1 doglover

  • Rytec EPG Team
  • 16,974 posts

+635
Excellent

Posted 4 May 2013 - 12:37

* The Plug-in to import EPG is called XMLTVimport in OpenPLi and EPGimport in other images.

The plug-in is essentially the same.

 

Edit: In OpenPLi the plug-in is now also called EPGimport
 

* When to download the XMLTVfiles - or perform an import.

The XMLTV files are updated during the night.  This takes a couple of hours.  Approx. around 5h00 (CET) the XMLTV files are being uploaded to the different servers.  It is quite a large amount of data, and the uploads take a while.  There is also one server which updates itself.  (It gets the data and stores it.  This update is done once each hour) So this server could be updated one hour later than the others. 

Lately there are extra updates scheduled at 7h00, 11h00 and 17h00 (all CET).  These updates load only the changed XMLTV files to the servers which have been edited or run separately.  There are in general no new files generated automatically.

 

So in general set your EPGimport not earlier than 6h30 if you want the latest EPG.

However if you do not need the afternoon schedules, it may be advantageous to import the data after 18h00.   So you have the latest corrections.

 

Certainly do not import between 4h00 and 6h30 (CET) as during this times the servers are updated.  And too much downloads during that time could hinder the update of the servers.

 

* Updates.

In order to update the sources of XMLTVimport, deinstall enigma2-plugin-extensions-xmltvimport-rytec and reinstall.  This will update the sources for updating the EPG.

Alternative run in a telnet box:

opkg update
opkg upgrade enigma2-plugin-extensions-xmltvimport-rytec

* General Info

The XMLTV system gathers its information from the internet, and delivers it to a Linux (enigma2) box via an XMLTV file.

So the info is grabbed from websites.  Sometimes these websites change and then the grabber needs adjusting.

Secondly the EPG is linked to the channels via a service ref (these 1:0:19:b70:30:46:e070000:0:0:0:).  This is the ref each channel has.

When a channel is moved by the service provider. The service ref has to be updated in the channels file.

 

When the box updates the EPG, two files are loaded.  The channels file and the XMLTV file. Normally the channels file is not updated that frequently, and it is only loaded once a day.  (And some boxes under some circumstances - which I have not been able to indentify - never load a new channels file).  But you can force a new load by restarting enigma2.

 

Therefore when you see that some channels do not have EPG, I request you restart your box and reimport the EPG.  When now the channel is still missing, pls post a request to correct the situation.  (I cannot check 2000 different channels each day).  Mostly it is a case of data missing on the websites or the service ref of the channel has been changed.  (If you know the channel has been moved by the service provider, pls. tell me so.  Also tell me the old service ref so I can remove it).

 

* EPG for IP-TV channels (with thanks to athoik)

 

In the EPGimport plug-in, do no activate the setting:  Load EPG only services in Bouquet

Leave this setting at No.  It causes strange results when activated with as symptom:  Nothing imported.

 

IP-TV channels can also have epg imported.  However you have to make your own import settings as the service references can be anything you fancy.  You will need an OpenPli image from after 20/12/2013

 

A working example:

 

In the bouquet file you have this as channel reference (here the Nostalgie radiostation in Belgium)

#SERVICE 4097:0:1:1000:1000:0:0:0:0:0:http%3a//stream10.fluoline.net%3a80/:Nostalgie
#DESCRIPTION Nostalgie
 

You have to make a channels.xml file containing:

<?xml version="1.0" encoding="latin-1"?>
<channels>
<channel id="PrivaSpice.ero">1:0:1:1000:1000:0:0:0:0:0:http%3a//example.com</channel> <!-- Private Spice -->
</channels> 

The part http%3a//example.com is really needed, but can be the same for all channels.

Also the 4097 has to be replaced by a 1 :  Not needed anymore

I called this file test.channels.xml and placed it in etc/epgimport

 

Now you have to reference this channels file in the sources.xml file.  Place in etc/epgimport a file something.sources.xml which contains:

<?xml version="1.0" encoding="latin-1"?>
<sources>

<source type="gen_xmltv" channels="test.channels.xml">
    <description>test XMLTV</description>
    <url>http://url1/rytecxmltvero.gz</url>
    <url>http://url2/rytecxmltvero.gz</url>
    <url>http://url3/rytecxmltvero.gz</url>
    <url>http://url4/rytecxmltvero.gz</url>
    <url>http://url5/rytecxmltvero.gz</url>
</source>

</sources> 

You could edit the rytec.sources.xml by adding the above, but I do not recommend it as the file will be overwritten when you update your box.  The advise is thus to make a separate something.sources.xml file.  The XMLTV importer will read both this file and the rytec.sources.xml file.

PS: with something I mean really any name you can dream up.


 

 

* At which time should I set the update.

 

It is recommended not to download before 0630 (GMT + 2).  This because the update is only ready by then.

 

* No epg is imported after the box comes out of deepstandby.

 

When using deepstandby, and autotimer, there could be a problem with the automatic update.

This is because  when the box starts, the system time is not set properly until the box is tuned to a channel.  Now if you use autotimer, this tuning to a channel could take a minute longer as usual, because of autotimer.running.  Now the XMLTV importer sets a wrong update time, because the system time is still incorrect

 

The solution is to install the systemtime plugin and select upon startup to set system time by NTP server.

 

Lately this does not seem to work anymore.  Another workaround is:

Edit /var/bin/enigma2.sh and add before the line: 

 

# hook to execute scripts always before enigma2 start

 

the next line:

ntpdate time.nist.gov > /tmp/ntdate.log
 

This also works if you prepare a /usr/bin/enigma2_pre_start.sh file and place the above command into that file.   The advantage is that this solution survives an image update. (if you use autobackup and add this file to the backup list)
 

You may have to install ntpdate.  This can be done in a telnet session:

opkg update
opkg install ntpdate

*No epg is loaded on recently added channels

 

Channels which have been added to the channels.xml file sometimes do not load epg. In order to receive the information on these channels you need the new channels file.  This file is normally loaded once a day, and the information is automatically included..

 

Now some boxes do not load this file automatically every day.  We do not know why this happens.

 

The solution (or workaround if you want to call it this way) is simple.  Restart enigma (or reboot the box) and reimport.  This forces a reload of the channels file.

 

This restart is also recommended if you suddenly experience loss of epg on channels which previously had information.   (Of course if still have no information on these channels after an enigma restart ad reimport, something else is wrong)

 

* Where is the epgimport.log

 

Open the XMLTV importer.

Press now the Menu button on your remote.

Make the choice to see the log.

Now you see the log.  This log can be saved, and you find the saved fle in /tmp/

 

* What is the purpose of epg.dat

 

The epg.dat file is commonly seen as the vehicle for XMLTV.

This is a misconception.  The epg.dat is a file from enigma, in which the epg is stored when the enigma is stopped or the receiver is shut down.  Upon restart this file is read again in memory to restore the epg.

 

* Misconceptions about epg.dat and XMLTV

 

On images without the oudeis patch, epg.dat is used to get epg into memory.  In these cases the XMLTV importer and Crossepg generate an epg.dat file.  Subsequently enigma is restarted and the new epg is read.  But this is the only use of epg.dat in this respect.  Also Crossepg uses this technique on these images. 

 

* How do I now if my image is oudeis patched? 

If you import epg via XMLTV or Crossepg and the image automatically restarts then your image is not patched.  Gemini is one of the stubborn ones.

The patched images can receive the epg data directly and do not need the use of epg.dat.  epg.dat is still used by enigma on these images to fulfil its original role (see above)

 

* Where is epg.dat stored and how to change it.

 

You can set the storage device for the epg.dat file by editing the settings file in /etc/enigma2 when enigma is stopped.

init 4  (to stop enigma)

edit now /etc/enigma2/settings – add or change


config.misc.epgcache_filename=/media/usb/epg.dat  (or another location to your choice)

init 3 (to restart enigma)

 

* How to erase all epg in your box

In the import settings make a choice to do so before import.


The second method is the manual one. I use the program DCC to do this but you also can use Putty or even a simple Telnet session in Windows.

Open a Telnet session to the IP address of your satellite receiver (in most cases Dreambox or Vu).

When you are prompted you can type behind the prompt #

init 4 (This will stop Enigma of working and if you were watching TV it will freeze.)

What I do extra is open a FTP session (in DCC you can switch easy to this) and then go to /tmp and see if there is maybe a file epg.dat, remove it and then go also to /media/hdd if you have a harddisk installed in your receiver and also check if maybe a fil epg.dat is there and remove it. Check also /media/usb
(This step is not always needed but I do it and it does not take a lot of time)

Now go back to the Telnet session and behind the prompt type #

init 3 (This will start Enigma again and after a while you will notice the screen will play again (if you were watching TV).

After this the EPG listings should be empty or only show Now/Next in the infobar, depending on how you have setup the EIT and EPG settings in the system menu.
(keep in mind if you have enabled MHW, EIT, etc then EPG listings will not be empty if you zap to those channels)

 

* Where do the double schedules come from?

 

I grab the data for 6 or 7 days. 

This info is imported in your box.  And it stays there until it is overwritten, expires or deleted.

In the course the next couple of days, the channel decided to change its schedule, the grabber will correct the schedule to the new times.   If you now import the data, sometimes the old data is not overwritten properly.  The old is not completely removed.

Result:  Double schedule.

 

The proof it is old data.  Normally only 5 days of description are imported, although 6 or 7 days are in the XMLTV fle.  This results in schedules without a description for the 6th and 7 th day.

I know this problem is especially bad on VIER and to some extent less on VTM.   A couple of days later, the description with the new schedule is added.

 

The second reason why this is happening is the fact you may have EIT and MHW still switched on.  You receive then epg from 2 different sources

 

If your are really annoyed by this, you can delete all epg in your box (see the FAQ) and reimport.

 

* Can the amount of epg be limited to only the channels I am using

 

Take a look here: http://forums.openpl...r-v100-by-acds/

It is not written by us, but could be helpfull for implementing this limitation. 

The manual method is described herefafter.

 

Yes, make your own channels file (and reference them in the sources file) in which you references only the channels you use.
Use as a template our channels file and delete everything you do not want.

This way only the channels which are referenced will be imported.

Our channels file can be found on a different websites.  Open the rytec.sources.xml file on your box (dir /etc/epgimport/)
The required urls's are found in the first part of the file, after <channel name="rytec.channels.xml.gz">
Download the file with a webbrowser, unzip the file and edit the file.  Delete only lines like these: <channel id="M6.fr">1:0:19:245E:440:1:C00000:0:0:0:</channel> <!-- M6 HD -->
Name the file anything.channels.xml. (anything can be really any name o fancy).
Store this file anywhere (except in /tmp/) on your box or a NAS drive accesible by your box.

Next create a something.sources.xml file in which you reference the newly created anything.channels.xml file and the XMLTV files.  An example is given below.

 

<?xml version="1.0" encoding="latin-1"?>
<sources>

<source type="gen_xmltv" channels="/media/hdd/epg/anything.channels.xml">
    <description>France XMLTV</description>
    <url>http://url1/rytecxmltv.gz</url>
    <url>http://url2/rytecxmltv.gz</url>
    <url>http://url3/rytecxmltv.gz</url>
    <url>http://url4/rytec/rytecxmltv.gz</url>
    <url>http://url5/rytecxmltv.gz</url>
    <url>http://url6/rytecxmltv.gz</url>>
</source>

<source type="gen_xmltv" channels="/media/hdd/epg/rytec.channels.xml">
    <description>Germany/Austria/Swiss XMLTV</description>
    <url>/media/hdd/epg/rytecxmltv.xml</url>
</source>

<source type="gen_xmltv" channels="/media/hdd/epg/rytec.channels.xml">
    <description>UK XMLTV</description>
    <url>/media/hdd/epg/rytecxmltvuk.gz</url>
</source>

</sources>
 


 
You can name this file also f.i. own.sources.xml file (The sources.xml part is needed - the own can be anything you want)
Renaming this to something else as rytec will ensure they are not overwritten when updating the box.

As you can see I use here also local xmltv files.  But these can be also from the internet as given in the rytec.sources.xml file
Files can be in xml format or gzipped files (.gz)

 

* How to capture a single service reference

 

Open the OpenWebif in your browser.

Open the channel you want the service ref from.  Double click on it.

Now press the Current button (Huidig in the attached example)

The service ref is given here:  Sref: 1_0_19_22C5_80D_2_11A0000_0_0_0  (on a yellow background in the attached image)

 

ServiceRef.png

 

 

* How to update XMLTV importer from Telnet

 

1. Connect to your box ip address with DCC or Putty
2. opkg update
3. opkg upgrade enigma2-plugin-extensions-xmltvimport-rytec enigma2-plugin-extensions-xmltvimport

If you are not sure how a package is called you can use the search function below :
( use opkg list '*xmltv*' to get the corect spelling.)

Thanks to Milo for these commandlines.

 

 

* Crossepg

 

I had to spent the umpiest time on trying to find an error caused by crossepg and/or the "repacked" files.

 

Crossepg and the repacked files (XEPGDB files) are outside my control.  I cannot change anything in it.

Do you blame me of having a hate relation with Crossepg.  Besides this Crossepg causes also other problems.  To name one not taking into account of the Namespace in the service ref causng mixed up epg.

 

Up to now, each time I tried to find a workaround for crossepg.  But I am going to stop this.  I have nicer things to do with my time as to solve problems with bug ridden software, which I cannot alter.

 

The rytec XMLTV files can be used in Crossepg, but be aware that there are issues with some channels having epg from two different channels.  This error is inherent to the Crossepg set-up and we have commented on this on several occasions.

 

You could try to edit the Crossepg configs in \usr\crossepg\providers\ by replacing:

 

rytec.channels.xml.gz

with

rytec.channels.xml-sat.gz

 

This will eliminate the cable channels which causes the problems in Crossepg with the Rytec XMLTV files.

 

* XMLTV viewer for PC  (windows)

 

A simple viewer for the XMLTV files can be found here:  http://xmltvview.bir.../epgviewer.html

 

* Versions for OE2.2

 

Crossepg

http://www.i-have-a-dreambox.com/wbb2/thread.php?threadid=174846

XMLTV import

http://www.dream-multimedia-tv.de/board/index.php?page=Thread&threadID=19447&pageNo=2

or

http://www.oozoon-board.de/wbb/index.php?page=Thread&threadID=1458

Attached Files


Edited by doglover, 13 September 2018 - 15:54.

~~Rytec Team~~
Maxytec Multibox SE OpenPli (used as mediaplayer)
Mutant HD2400 OpenPli
Vu+ Duo OpenPli (backup)

Synology NAS

Sat: 13E, 19.2E, 23.5E and 28.2E
*Pli/Rytec EPG POWERED*




Also tagged with one or more of these keywords: FAQ, XMLTV

1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users