45 replies to this topic

[shax]

First of all i would like to express gratitude for making your image open source.
Even do, I do not use your images (not that i don't think they are not excellent), source code has been valuable resource for learning.

XMLTV Import plugin itself provides lots of really unique examples.
Keeping that in mind i suppose that modifying plugin to support users epg.dat files would not be too much of a problem.
I suppose i could do it with some dirty hacks and code modifying but due to my limited knowledge i could never do it the way you would/could.

Basic idea is that user instead of generating epg on the box downloads already prepared epg.dat file and loads it.
So for example if i add

<source type="epg_dat" channels="http://www.xmltvepg.be/rytec.channels.xml.gz">
	   <description>Krkadoni Ex-Yu</description>
	   <url>http://krkadoni.com/ext.epg.dat.gz</url>
</source>

in

/etc/epgimport/rytec.sources.xml

plugin would simply first enumerate all type="epg_dat" sources from config, download them and load into cache one by one, and then continue loading other xmltv sources.

This would give users much more freedom when downloading and loading epg content, not to mention speed.

So, what do you think?

P.S. I appologize if this has been discussed before.

Regards.

[shax]

Ohh, my apologies, obviously it was too late so i didn't notice epg.dat importer
Seems like you guys are always one step ahead.

Just a suggestion, if only epg.dat sources are imported label "Importing XXX" - "X events imported" stays.
I realize you cannot know number of imported events but changing text label to: "Finished" would be much better.

Thank you,

regards.

[shax]

Talking to myself, but maybe someone will find this usefull

I modified xmltvimporter, it was quick and dirty but works.
Now plugin:

- installs automatically python dependencies if not found on system (no need to download and install them before)
- modified dependencies so it can be installed on OE 1.6
- if there already is rytec.sources.xml file it will only update it with public epg.dat providers
- supports epg.dat type providers in sources.xml
- will notify once epg.dat file is complete
- changed status bar to display info in format: "Imported XY XML events and XY dat files
- changed HDD_EPG_DAT path from static to variable, it will depend on existing devices, enumerated the way e2 looks for epg.dat
- added GEMINI_EPG_DAT - support for "gemini.epg.dat" filename

Now entire SkyIT, CPlus, Ex-Yu, etc... is loaded in a mater of seconds, instead of minutes.

I've been playing with it for few hours seems to run without problems on DreamElite OE 1.6

Changed files are only: EpgImport.py and plugin.py

You can download it HERE

Regards.

[muelleimer321]

Hm, the status bar says 0 XML events and 2 dat files

Here´s my rytec.channels.xml source

<source type="epg.dat" channels="own.channels.xml">
    <description>Rytec Sky Germany XMLTV</description>
    <url>http://www.xmltvepg.be/rytecxmltvskyde.gz</url>
</source>
<source type="epg.dat" channels="http://www.xmltvepg.be/rytec.channels.xml.gz">
    <description>Rytec SKY IT epg.dat</description>
    <url>http://www.xmltvepg.be/rytecxmltvskyit.gz</url>
</source>

[rtzhjgg0]

@shax
I installed it on OoZooN-8000 but importing of epg.dat files was unsuccessful with:

[EPGImport] Downloading: http://www.xmltvepg.be/csat/epg.dat.gz to local path: /media/hdd/epgimport.gz
[EPGImport] afterDownload /media/hdd/epgimport.gz
[EPGImport] Cannot load EPG.DAT files on unpatched enigma. Need CrossEPG patch.

As I know, OoZooN images have an own 'load patch for crossepg'. Can you please check why this happened?

Thanks

[delagroov]

I noticed that your ipk does not change the enigma2.sh file to load the xmltv setting on boot.

[doglover]

Hm, the status bar says 0 XML events and 2 dat files

Here´s my rytec.channels.xml source

<source type="epg.dat" channels="own.channels.xml">
    <description>Rytec Sky Germany XMLTV</description>
    <url>http://www.xmltvepg.be/rytecxmltvskyde.gz</url>
</source>
<source type="epg.dat" channels="http://www.xmltvepg.be/rytec.channels.xml.gz">
    <description>Rytec SKY IT epg.dat</description>
    <url>http://www.xmltvepg.be/rytecxmltvskyit.gz</url>
</source>

As I understand it, when you are importing ready made epg.dat files, you should access the epg.dat files, not the xmltv files.

for Germany the file is
<url>http://www.xmltvepg....pg.dat.gz</url>

and for italy
<url>http://www.xmltvepg....pg.dat.gz</url>

Willy

[muelleimer321]

Hm, i´ve uninstalled the Plugin and Configs, restart dreambox, install ipkg with autocreate of rytec.source.xml and reboot the dreambox. The Problem is the self. 0 XML and 1 dat file.

With the type "gen_xmltv" download it the xmls one time!

@doglover: With your tipp the plugin don´t also work

[muelleimer321]

Heres my log

action -> InfobarMenuActions mainMenu
loading mainmenu XML...
TimerEdit TimerEditList
PluginBrowser PluginBrowser
allocating new converter!
allocating new converter!
[EPGC] start caching events(1284280126)
close frontend 2
action -> OkCancelActions ok
okbuttonClick
reused converter!
reused converter!
action -> WizardActions ok
Looking for embedded skin
reused converter!
[EPGC] abort non avail schedule other reading
[EPGC] abort non avail viasat reading
[EPGC] abort non avail FreeSat nownext reading
[EPGC] abort non avail FreeSat schedule reading
[EPGC] abort non avail FreeSat schedule_other reading
[EPGC] abort non avail mhw reading
[EPGC] nownext finished(1284280133)
action -> ColorActions yellow
action -> MsgBoxActions ok
[EPGImport] oudeis patch not detected, using epg.dat instead.
[EPGImport] using C module, yay
[EPGImport] nextImport, source= Rytec SKY IT epg.dat
[EPGImport] Downloading: http://www.xmltvepg....taly/epg.dat.gz to local path: /media/hdd/epgimport.gz
ERROR reading PES (fd=54) - Value too large for defined data type
poll: unhandled POLLERR/HUP/NVAL for fd 54(8)
[EPGImport] afterDownload /media/hdd/epgimport.gz
[EPGImport] Uncompressing /media/hdd/epgimport.gz
[EPGImport] Importing /hdd/epg.dat
main thread is non-idle! display spinner!
no spinner DC!
main thread is non-idle! display spinner!
no spinner DC!
main thread is non-idle! display spinner!
no spinner DC!
main thread is non-idle! display spinner!
no spinner DC!
main thread is non-idle! display spinner!
no spinner DC!
main thread is non-idle! display spinner!
no spinner DC!
[EPGC] 48300 events read from /media/hdd/epg.dat
[EPGImport] epg.dat type provider imported, loading next
[EPGImport] imported 0 events
[EPGImport] #### Finished ####
ERROR reading section - Value too large for defined data type

poll: unhandled POLLERR/HUP/NVAL for fd 41(8)
no ancillary data in audio stream... abort radiotext pes parser
main thread is non-idle! display spinner!
no spinner DC!
[eDVBLocalTimerHandler] diff is 9
[eDVBLocalTimerHandler] diff < 120 .. use Transponder Time
[EPGC] schedule finished(1284280153)
[EPGC] stop caching events(1284280153)
[EPGC] next update in 15 min
[eDVBLocalTimerHandler] update RTC
[eDVBLocalTimerHandler] time update to 10:29:21
[eDVBLocalTimerHandler] m_time_difference is 9
[eDVBLocalTimerHandler] set Linux Time
[EPGC] start cleanloop
ERROR reading PES (fd=54) - Value too large for defined data type
poll: unhandled POLLERR/HUP/NVAL for fd 54(8)
[EPGC] stop cleanloop
[EPGC] 2366576 bytes for cache used

[doglover]

It seems that, this verson needs a bit of work.

Willy

[shax]

Ok, i definitely didn't expect THIS many replyes!
I see some things are not clear, so i'll try to explain:
- plugin can not know how many events is in epg.dat file, there for it splits informations in status bar in two parts, one is old XML events count and second is new epg.dat count. If you don't specify at least one XMLTV source event count will always be 0
- and like it's already notice, you need epg.dat sources to load epg.dat data. You can not specify gzipped xmltv in a epg.dat and expect it to work

Now, i did not get it. Where you sucessfull in installing this? Did it load epg or not? Did you try to empty your epg data before loading it with this plugin, then try to load it with it?


@rtzhjgg0
Does original plugin works?

[rtzhjgg0]

...Does original plugin works?

Yes, the original plugin works on OoZooN.

[shax]

@delagroov
I've just double checked, enigma2.sh is patched same like it was with original plugin.

@rtzhjgg0

I'll look into it.

[muelleimer321]

At First. I´ve a epg.dat at /media/hdd/ and it is not empty. 4789 events in 21 channels.

Yes, at first my path for the epg.dat source was wrong, but at my log you show the file was downloading from [EPGImport] Downloading: http://www.xmltvepg....taly/epg.dat.gz to local path: /media/hdd/epgimport.gz

The Plugin works with rytecs EPG Source. It downloads and merge it with my epg.dat.


To your Modification
When the type "epg.dat" in the source.xml is, the files from "http://www.xmltvepg....aly/epg.dat.gz" and "http://www.xmltvepg....any/epg.dat.gz" was download it to /media/hdd/.
epgimport.gz filesize: 1,2 MB.

I show the epg.dat was creating etc. it was bigger. At the end the file is 1,76MB. (temporary the files was 2,43Mb)

Here´s the log for the import at the moment

[EPGImport] oudeis patch not detected, using epg.dat instead.
[EPGImport] using C module, yay
[EPGImport] nextImport, source= OWN SKY IT epg.dat
[EPGImport] Downloading: http://www.xmltvepg....taly/epg.dat.gz to local path: /media/hdd/epgimport.gz
nr_read 233 nr_write 235
sum 468 prev_sum 463
hdd was accessed since previous check!
[IDLE] 0 600 False
[EPGImport] afterDownload /media/hdd/epgimport.gz
[EPGImport] Uncompressing /media/hdd/epgimport.gz
[EPGImport] Importing /hdd/epg.dat
[EPGC] epg file has incorrect byte order.. dont read it
[EPGImport] epg.dat type provider imported, loading next
[EPGImport] nextImport, source= OWN Sky Germany XMLTV
[EPGImport] Downloading: http://www.xmltvepg....many/epg.dat.gz to local path: /media/hdd/epgimport.gz
[EPGImport] afterDownload /media/hdd/epgimport.gz
[EPGImport] Uncompressing /media/hdd/epgimport.gz
[EPGImport] Importing /hdd/epg.dat
[EPGC] epg file has incorrect byte order.. dont read it
[EPGImport] epg.dat type provider imported, loading next
[EPGImport] imported 0 events
[EPGImport] #### Finished ####
[EPGC] start cleanloop
[EPGC] stop cleanloop
[EPGC] 619026 bytes for cache used

It looks like the download works now but
the old epg.dat was deleted and the new cannot see at the channels!


On Gemini, the File was defined at gemini.epg.dat, write to /media/hdd, save automately 2hours, cache file on

[delagroov]

@delagroov
I've just double checked, enigma2.sh is patched same like it was with original plugin.

@rtzhjgg0

I'll look into it.

I already know, what causes this.
I tried this on Pli and your ipk forces python download which is not suitable for PLi.(not OE1.6)

On Vu+ orginal it works.

[shax]

Well this is odd..
preinst checks kernel version and tries to determine if it's oe 1.5 or 1.6

kernel=`uname -r`
oe16=0

case "$kernel" in 
  *2.6.18*)
    oe16=1
    ;;
esac

if [ "$oe16" -eq 0 ]; then
....

So if it founds 2.6.18. kernel it defults to OE 1.6, else it's OE 1.5
What kernel is PLi using?

Does someone knows better way to determine OE version?

[muelleimer321]

My Gemini works with OE 1.6.

back to my Problem.

When i define the epg.file named
- gemini.epg.dat (the Plugin download it, don´t merged it) (name of /media/hdd/ was epg.dat, i can download often)
- epg.dat (the Plugin download it, the Plugin merged it) (name of /media/hdd is gemini.epg.dat, at the second download, gemini crash on the merge)

[delagroov]

Pli has kernel 2.6.18.7.3 but uses his own OE.

It uses thes packages.

python-compression Version: 2.5.1-ml4
python-zlib Version: 2.5.1-ml4

When your packages are downloaded and installed Pli enigma crashes.
So i think that PLi needs the older python packages for other applications which depend on these version.

[shax]

Well there is the problem, i supposed if kernel is 2.6.18 it's OE 1.6
I'll have to add special check for PLi

Thank you!

[MiLo]

So if it founds 2.6.18. kernel it defults to OE 1.6, else it's OE 1.5
What kernel is PLi using?

PLi uses the 2.6.18 kernel, but (currently) Python 2.5. Kernel and OE version are NOT related!

You cannot check on OE1.5 or 1.6, because there is really no such thing runtime. Check on what you need, e.g. check the Python version directly if you feel the need to, but don't make any assumptions on other versions. A 7025 image built with OE1.6 will have a 2.6.12 kernel.