15 replies to this topic

[PaphosAL]

Hi- OpenPLi (25/04/2010) on DM500S: I thought I would give rytec xmltvUK EPG a try, to see how it compared with RadioTimesUK epg. But it falls over very quickly, as you can see from the log:
------------------------------------------------------------------------------------------------------------

Using a timeOffset of 0 seconds.
Retrieving 168 hours of data for each channel
#count:1
20100707
rytecxmltvuk.gz 0% | | 0 --:-- ETA
rytecxmltvuk.gz 100% |*****************************| 51 00:00 ETA
Connecting to :80]www.xmltvepg.be[212.71.0.79]:80:80]
gunzip: Invalid magic
#cur:0
/var/bin/db_epg -t xmltv -d /tmp/epg.db -f /tmp/rytecxmltvuk -o 0 -h 168 -m /var/tuxbox/config/dbepg/e1_UK.dat -g /var/tuxbox/config/dbepg/genre.dat -e -c 1
Setting encoding to ISO-8859-1
XMLTV parse error: /tmp/rytecxmltvuk: no element found at line 1

------------------------------------------------------------------------------------------------------------

The main problem appears to be with gunzip - is this a known issue, please? Secondly, why is it insisting on retrieving 7 Days of data per channel, when I actually have it set to 1 Day only? I think this will soon fill memory and lock the box, if I do manage to find a solution for the gunzip issue!!

Comparing the e1_UK.sh and rt_uk.sh scripts, I find an anomaly:

"numOfDays")
hours=$((24*$val)) ; <this is rt_uk>

"numOfDays")
numofdays=$(($val+6))
hours=$((24*($val+6))) ; <this is e1_UK)

Why on earth is the e1_UK script adding 6 to the Number of Days in it's math?

Cheers- AL

[doglover]

where the f*** did you get the package.

I the present e1_uk.sh following instruction can be found:

hours=72
timeOffset=0
numofdays=3


OIFS=$IFS
IFS='='
while read opt val
do
case "$opt" in
"timeOffset")
timeOffset=$val ;
;;
"numOfDays")
numofdays=$(($val))
hours=$((24*($val))) ;
;;
"mapfile")
mapfile=$configdir/$val ;
;;
"genreMap")
genrefile=$configdir/$val ;
;;

"epgLocation")
epg=$val ;
datadir=`dirname $val` ;
;;
esac;


So you are using an obsolete version.

Secondly _ on a 7020 (e1) it seems that the epgdb engine is broken. It may be the case for 500S as well. So I would recommend Jade 3. Unless it is fixed by now.

Pls. reinstall!!!!

Willy

[PaphosAL]

Thanks for responding, Willy! This OpenPLi image was sent to me by a mate, to get me up and running quickly on my new DM500S. (I was previously dBox2 for 5+ years).

I'm sure it is a 25 April 2010 nightly download, but looking more closely- it might have been of Feb 10 origin. Never mind- I'll uninstall and re-install the rytec package, as you suggest, thanks.

That was a strange anomaly I found in the script code, re Number of Days, though- wasn't it? Maybe a rogue script issued by some plank who enjoys letting DM500 owners suffer, LOL!!

Cheers- AL

[PaphosAL]

Willy- I just tried a retest, but the xmltvepg.be server seems to be down.

I don't think the 500 dbepg binary is faulty, as this is working perfectly for me on Radio Times UK EPG. One thing I did notice yesterday was that when I tried to open the 51kb rytecxmltvuk.gz that had been downloaded with WinRAR, it reported it as a bad file. I thought nothing more of this, assuming that WinRAR couldn't handle .gz archives.

But this morning, I made a small games.tar.gz archive on my Ubuntu system, and ftp'd this file to my XP machine. WinRAR opened it no problem. I then renamed it to games.gz and WinRAR still accessed it, although it showed as one file, instead of seperate .cfg / .so games plugins.

This looks to me like the downloaded rytecxmltvuk.gz file is corrupt, hence the gunzip: bad magic error in the log??

Cheers- AL

[doglover]

This happens sometimes with .gz files compressed on one system and decompressing on another machine.

I tried the the above mentoined file and found no error in it.

You can see for yourself by downloading the file with wget on a linux machine (Dreambox of ubuntu)

wget http://www.xmltvepg.be/rytecxmltvuk.gz

and unzip it with:

gunzip rytecxmltvuk.gz


PS : the rytecxmltvuk.gz should be 1.9 mB

Willy

[PaphosAL]

OK, Willy- I have now tested this with the SQlite and XMLTV EPG plugins installed in today's DM500 maxvar build, and believe there is definately something broken.

I delete my 11.5mb /media/epg/epg.db (113 mapped channels- 8 Days)- and reboot the box.

A new /media/epg/epg.db of 96kb is created by PLi

./var/bin/e1_UK.sh

rytecxmltvuk.gz downloads to /media/epg and is 1.89mb

gunzip does it's work and the .gz file vanishes, replaced by:

rytecxmltvuk at 12.03mb

epg.db-journal also appears- around 500kb

After only a couple of minutes, rytecxmltvuk and epg.db-journal both vanish

All that remains is epg.db - still at the original 96kb...

It should take 7 mins approx to process, and be 11+mb!!

Can someone else please verify that I'm not imagining this?

Cheers- AL

[doglover]

The unzipped fle of 12 mb sounds OK.

pls. perform a manual import via telnet.

In telnet type in: e1_UK.sh

You should have a result like the one below - which is mine.

I am importing CDS/TVV and the Uk package ( but I limited the amount of channels) and the epg.db file is 9.90 mB.

pls report your output from the telnet session

Using a timeOffset of 0 seconds.
Retrieving 120 hours of data for each channel
Date: 20100710

rytecxmltvuk.gz 0% | | 0 --:-- ETA
rytecxmltvuk.gz 24% |****** | 473 KB 00:03 ETA
rytecxmltvuk.gz 50% |************** | 955 KB 00:01 ETA
rytecxmltvuk.gz 75% |********************* | 1436 KB 00:00 ETA
rytecxmltvuk.gz 100% |*****************************| 1896 KB 00:00 ETA
Connecting to www.xmltvepg.be[212.71.0.79]:80
/var/bin/db_epg -t xmltv -d /hdd/sqlite/epg.db -f /hdd/sqlite/rytecxmltvuk -o 0 -h 120 -m /var/tuxbox/config/dbepg/e1_UK.dat -g /var/tuxbox/config/dbepg/genre.dat -e -c 1
Setting encoding to ISO-8859-1

[PaphosAL]

Here you go, Willy. By the way, I was using Telnet in the above test this morning. But 'Retrieve' from the plugin gui menu produced identical results. Here is the log o/p from telnet method just repeated.

Again, it closes after about 2 mins, and has not applied any treatment to the small epg.db file.

-------------------------------------------------------
Using a timeOffset of 0 seconds.
Retrieving 192 hours of data for each channel
Date: 20100710
rytecxmltvuk.gz 0% | | 0 --:-- ETA
rytecxmltvuk.gz 31% |******** | 588 KB 00:02 ETA
rytecxmltvuk.gz 73% |********************* | 1394 KB 00:00 ETA
rytecxmltvuk.gz 100% |*****************************| 1896 KB 00:00 ETA
Connecting to :80]www.xmltvepg.be[109.237.208.43]:80:80]
/bin/db_epg -t xmltv -d /media/epg/epg.db -f /media/epg/rytecxmltvuk -o 0 -h 192 -m /var/tuxbox/config/dbepg/e1_UK.dat -g /var/tuxbox/config/dbepg/genre.dat -e -c 1
Setting encoding to ISO-8859-1
--------------------------------------------------------------

Cheers- AL

[doglover]

I am puzzled by this result.

Download OK.
Unzip OK
Import started normally.
No errors on import
No database locked problem.

But still no data in the epg.db SQlite store.

The only thing I can think of: Are you sure you have the channels mapped?

Normally this should not be needed. But it cannot hurt to check.
I have added my e1_UK.dat if you want to check if this is the problem.

Willy

Attached Files

•  e1_uk_2.dat   6.58KB   0 downloads

[PaphosAL]

Channels were mapped in e1_UK.dat Willy, using the 10kb one from yesterday morning's XMLTV plugin download & install.

One difference I did spot in our two respective telnet logs above was the strange-looking download url in my log, compared to yours...

However, I just retested using your map (thanks for that!)- and the download link now reads the same as yours:

[mail]root@dm500[/mail] ~ # ./var/bin/e1_UK.sh
Using a timeOffset of 0 seconds.
Retrieving 24 hours of data for each channel
Date: 20100711
Connecting to [url=http://www.xmltvepg.be[212.71.0.79]/]www.xmltvepg.be[212.71.0.79]:80/][/url]
rytecxmltvuk.gz	 100% |*****************************|  1896 KB    00:00 ETA
/bin/db_epg -t xmltv -d /media/epg/epg.db -f /media/epg/rytecxmltvuk -o 0 -h 24
-m /var/tuxbox/config/dbepg/e1_UK.dat -g /var/tuxbox/config/dbepg/genre.dat -e -
c 1
Setting encoding to ISO-8859-1
[mail]root@dm500[/mail] ~ #

But the puzzle remains; after 2 mins, the script terminates with no reported errors, and no apparent changes have been made to my epg.db file on the server mount.

I wonder, could this be Ubuntu 10 playing silly boogers? /media/epg mount point on my dm500 points to server directory epg - which is set to full share R/W permissions. Nah- that can't be the issue, thinking about it- as RT-UK has no problem creating an 11.4mb epg.db datafile automatically with cron each morning, taking around 6 mins for 113 mapped channels & 8 Day EPG).

My m8 lincsat has tested with the same e1_UK fileset on his dm600, in conjunction with RT-UK, and successfully gets 3 Day epg on his mapped e1_UK channels, and 8 Days on RT_UK. Weird!!

Cheers- AL

[doglover]

Definitely weird.

I know using a mounted drive is a problem.
I tested this out sometime ago and found out that you have to mount the drive with the options as depicted in the attached pic.

The radiotimes importer uses a different part of the dbepg engine. If it works with the radiotimes part, is no warranty that the rytec part works.

Willy

Attached Files

•  cifsmount_1.jpg   57.92KB   1 downloads

[PaphosAL]

Brilliant! Well done, Willy- you've cracked it!

It took me three trial and error attempts to correctly modify the EPG Mount settings, as for reasons explained in this recent thread, and thanks to hemi's expert advice, I'm not using any gui mount manager. Instead, the settings are made manually in /var/etc/init for my four mounts. Here is the original line - where RT works, but E1 doesn't:

/bin/mount -t cifs -o rw,nolock,intr,soft,rsize=8192,wsize=8192,username=al,password=blahblah //192.168.0.8/epg /media/epg

It was adding the linux ext parameter that threw me, as google didn't give any pointers on the correct syntax for that parameter, in an init / fstab file. This is the one that now works:

/bin/mount -t cifs -o nolock,intr,soft,linux-ext,rsize=8192,wsize=8192,username=al,password=blahblah //192.168.0.8/epg /media/epg

I deleted epg.db and rebooted; PLi created a new epg.db of 79kb. Then via telnet, I executed the e1_UK.sh script (using your mapfile and set for 24 hrs epg). Two or three minutes later, the script terminated, with epg.db now a healthy 481kb and 24 hr EPG on your mapped channels. Woooo!!!

Many thanks (again)- AL

[doglover]

Oef - as we say in dutch.

Willy

[PaphosAL]

Babelfish translates ouf in Dutch to ouf in English- so you'll need to help me out with what oef really means, Willy. (Hope it's not oaf, LOL!)

One thing I noticed with your 6/7kb e1_UK.dat file, is that lines 1 to 116 contain TSid and Sid details, but lines 117 to 290 don't. These all begin as UNKNOWN

Are these lines ignored by the engine when comiling epg.db? If I want any channels in the UNKNOWN section to be included in long EPG, do I just amend those lines with the correct TSid/Sid data?

Cheers- AL

[doglover]

oef - sigh - relief it worked.

My .dat file is a limited file. This means it contains only the channels which I more or less regurly watch.

The UNKNOWN channels are the names of the epg which are not linked to any channels yet, but are present in the rytecxmltvuk xml file.
These channels are ignored by the dbepg engine.
You can link these fairly easy with the channel button in the epg dreambox import utility. (or provide yourself the correct TSID/SID data)

Willy

[PaphosAL]

I get the idea, thanks. Now to play with rytec EPG big time!! I'm also working on plicons in the Infobar in three modified skins (Astro, Carbon, & Wise). But that's for a new thread...

Cheers- AL

so ouf in Dutch and phew! in English, LOL!