14 replies to this topic

[alex22]

Hi,

 

I have a DM7000 with latest OpenPli (OpenPLi-beta-dm7000-20130505-305.img), pointing at Astra 19.2E.

 

What is the current way of getting the 5-day EPG for E1 boxes ?  Is there an MHW-bearing channel (the previous one, GUIDE TV, no longer exists) ?

 

If the solution is an IP-based plugin, which one ? Rytec is completely unusable on E1 boxes.

 

Thanks in advance,

 

-Akex

 

 

[doglover]

Not really.  I abonded the use of e1-boxes.  Too much problems.

The only option is MHW and EIT, and tuning to the channels.

 

Willy

Edited by doglover, 26 January 2014 - 07:31.

[WanWizard]

Which provider on 19.2? The german and benelux providers send EPG data in EIT, non encrypted.

 

Importing XMLTV data should still work, afaik that hasn't changed in the last 5 years in E1. It requires the Sqlite DB setup (use a CF card in a 7000), and you'd probably have to do some manual scripting to fetch and load the files.

[doglover]

Which provider on 19.2? The german and benelux providers send EPG data in EIT, non encrypted.

 

Importing XMLTV data should still work, afaik that hasn't changed in the last 5 years in E1. It requires the Sqlite DB setup (use a CF card in a 7000), and you'd probably have to do some manual scripting to fetch and load the files.

 

 

Problem with Rytec data.  Most sources are in utf-8 codepage, which e1 does not understand (at least the epg importer).

After all kinds of problems with special characters, most packages are now provided in utf-8.

 

Sorry.

 

Willy

[WanWizard]

Conversion from utf8 to 8859-1 is not really a problem, you could do that with a PHP oneliner. It's (amongst others) what I meant with "do some manual scripting"...

[alex22]

Which provider on 19.2? The german and benelux providers send EPG data in EIT, non encrypted.

 

Mostly CSAT; This is France :  I am not aware of any other MHW feed than the one provided by CanalSat. Now they brought it down (presumably because their own proprietary boxes no longer used it), it means that currently in France it is impossible to get any decent EPG (beyond EIT) on E1. Sad.

 

Importing XMLTV data should still work, afaik that hasn't changed in the last 5 years in E1. It requires the Sqlite DB setup (use a CF card in a 7000), and you'd probably have to do some manual scripting to fetch and load the files.

 

As a (C, sh, Tcl) developer myself, I am willing to dig. And as many-years fan of PLi, I am also willing to give you back the fixes. Can you please get me started by pointing me to any available documentation ? Architecture (the way enigma talks with external EPG sources), or specific things about the Rytec plugin (for more incremental fixes)... Thanks in advance !

 

-Alex

 

PS: I agree with you, giving up all E1 boxes due to a simple character encoding issue sounds ... suboptimal

[WanWizard]

SQLite backed EPG is part of the image, so if you have a USB stick or CF card for additional storage, you can already set that up.

 

Further instructions on how to set it up are still online at http://rytec.be/, towards the bottom, in the Enigma1 section. The "Pli Jade3 & SQlite" info should still be accurate.

 

What (probably) doesn't work anymore, are any instructions on how do download the xml files, as those are no longer online. So you need to figure out from the Enigma2 download sources which files to download, and modify the scripts to do so. And as Willy already mentioned, E2 files are in UTF-8, while E1 requires ISO-8859-1. So after download you need to convert the file's charset.

[alex22]

SQLite backed EPG is part of the image, so if you have a USB stick or CF card for additional storage, you can already set that up.

 

Further instructions on how to set it up are still online at http://rytec.be/, towards the bottom, in the Enigma1 section. The "Pli Jade3 & SQlite" info should still be accurate.

 

What (probably) doesn't work anymore, are any instructions on how do download the xml files, as those are no longer online. So you need to figure out from the Enigma2 download sources which files to download, and modify the scripts to do so. And as Willy already mentioned, E2 files are in UTF-8, while E1 requires ISO-8859-1. So after download you need to convert the file's charset.

 

Yes, I have already looked at the documentation on rytec.be, and applied the instructions there.

The EPG is in SQlite mode, as can be seen from the change date of the epg.db file. (By the way, I set it on the internal HDD, not on flash as you suggested. Any objection ?)

 

But the user interface of the plug-in is baffling:

 

 (1) The selected package is not remembered after saving: regardless of the previous selection, the first item (CD/TVV) is always selected on next invocation.

 

 (2) The blue-button "Channels" action leads to a completely cryptic interface (the one with two columns), and even after reading and re-reading the PDF documentation from rytec.be, I can't seem to understand the semantics of what it does, nor even to do anything permanent to it. Even the "Mapped" list (the only nonempty one)  is mysterious as it contains DNS-like names (like "Mezzo.be" or "117.750743.microsoft.com"). 

 

I am completely lost by this UI, and I understand that nobody will ever fix it. But I am willing to adapt on the command-line side.

 

For example, looking at the  e1_Csat.sh script, I noticed (1) a wrong "# /bin/sh" header (fixed it to #!) and (2) an outdated url (http://www.xmltvepg.be), getting redirect loops. I fixed it by looking at a working config on a friend's E2 machine (a Vu), to http://rytec.sifteam.eu.

 

As a result, I can see that the retrieval now works (I see a big  /media/hdd/rytecxmltvcsat.gz temporarily exist, and it does contain meaningful EPG summaries).

 

But clearly something goes wrong in the rest of the chain. Most likely a key piece is missing to map the channel names from that XML file to internal E1 channel idents.

I see that "db_epg" is called with an associated mapfile, which in my case only contains stuff like:

 

 2204:1:44c=I69.751903.microsoft.com

2458:1:456=I7.751945.microsoft.com

1f41:1:42e=I70.751913.microsoft.com

427e:1:446=I71.751828.microsoft.com

4281:1:446=I72.751863.microsoft.com

 

So, most likely some tiny bit is missing (though I installed everything from the openpli menu). Can you help me find which ?

 

-Alex

 PS: and yes, I'll take care of char encoding when that is the last problem

[WanWizard]

No objection to using the HDD. I never had one in all the boxes I have had, so it simply didn't enter my mind.

 

1. No clue.
2. It's not intuitive, I know. You have the list of channels in the XML (identified by DNS like names, yes), and the list of channels in your bouquets. The idea is that you select one left, then find the corresponding one right, and select it. That creates the mapping between the two.

In that snippet, the hex value on the left is the service reference (it contains channel id, orbital, pid's, etc), the DNS-like name on the right points to the channel definition in the XML.

For E2 the Rytec downloads contain a mapping file, so it's probably better to download that and convert it to the mapping format E1 wants then fiddle with the plugin.

 

If the script can do both automatically, your mapping will always be up to date, since you're using the daily updates from the Rytec team.

 

p.s. I'm trying to do this from memory, I haven't had an E1 box powered on in years!

[alex22]

No objection to using the HDD. I never had one in all the boxes I have had, so it simply didn't enter my mind.

 

1. No clue.
2. It's not intuitive, I know. You have the list of channels in the XML (identified by DNS like names, yes), and the list of channels in your bouquets. The idea is that you select one left, then find the corresponding one right, and select it. That creates the mapping between the two.

In that snippet, the hex value on the left is the service reference (it contains channel id, orbital, pid's, etc), the DNS-like name on the right points to the channel definition in the XML.

 

Thanks. This will likely make more sense once the left column is more "filled".

 

For E2 the Rytec downloads contain a mapping file, so it's probably better to download that and convert it to the mapping format E1 wants then fiddle with the plugin.

 

If the script can do both automatically, your mapping will always be up to date, since you're using the daily updates from the Rytec team.

 

Yes, I'm ready to write anything needed to bridge that gap. However I can't guess the specs ("the format E1 wants"), nor can I even reverse-engineer it since db_epg is a binary.

Would appreciate any kind of doc/source.

 

-Alex

[WanWizard]

The E1 source is here: http://sourceforge.n...code/HEAD/tree/

 

I can't really find any plugin source in there, no clue where they are. You might want to ping dAF2000, he might know it.

 

I thought you had the mapping file? So the format is known. They only thing you need to figure out from the Rytec mapping, is how the E2 service reference maps to the E1 reference in this mapping file, since E2's is a lot longer.

[alex22]

The E1 source is here: http://sourceforge.n...code/HEAD/tree/

 

I can't really find any plugin source in there, no clue where they are. You might want to ping dAF2000, he might know it.

 

I thought you had the mapping file? So the format is known. They only thing you need to figure out from the Rytec mapping, is how the E2 service reference maps to the E1 reference in this mapping file, since E2's is a lot longer.

 

OK, if all my problems amount to building the mapfile, you are right. I hope so

In the meantime I have found the (minimal, but sufficient) doc for db_epg on the openpli wiki:  http://openpli.org/wiki/dbepg

 

One last thing: do you have an url for an E2 mapfile (or larger file containing that info), that I can fetch and convert from a cron as you suggest ?

[WanWizard]

You can download rytec.channels-sat.xml.gz from the same location you downloaded the xmltv file. It contains the mapping between the ID and the service reference.

[alex22]

You can download rytec.channels-sat.xml.gz from the same location you downloaded the xmltv file. It contains the mapping between the ID and the service reference.

Thank you. First, let me just report success thanks to all your help. For the first time in years I have a one-week EPG on all French channels, with lowercase and proper accents: great !!!!

 

For the record, what needed be done for Rytec to work on E1 was:

   - fix the retrieval url to a working one http://rytec.sifteam.eu/

   - convert the retrieved XML from UTF-8 back to iso8859-1 (removing the BOM, and adapting the XML prologue). Did it in Tcl.

   - fetch once before using the mapfile-editing GUI (otherwise "unknown" is empty or unusable)

   - understand that the GUI does not "forget" settings, but presents a vector (one for each of the .cfg), where each one can independently be added/removed from the crontab

 

Now as to  rytec.channels-sat.xml.gz : it is indeed very simple to extract the reduced info useful for E1:

 

   <channel id="CANALplusCINEMA.fr">1:0:1:200B:430:1:C00000:0:0:0:</channel> <!-- CANAL+ CINEMA -->

 

becomes

 

  200b:1:430=CANALplusCINEMA.fr

 

I'll script this shortly and report

 

In summary: you rock, man !!!

 

-Alex

[WanWizard]

Would it be possible to do the conversion on the box?

 

If so, it can be done in the download script, and we've got a working EPG download again for E1.