31 replies to this topic

[Tom0000]

Hi .

I replaced dm8000 (with openPLI ) with new VU+ uno4k (with openPLI).

I have big set of recordings created by old receiver . New one is able to play old recordings but not able to delete all . The problem is if national char has been used by openpli on old box during recording. And this is not for all files.

 

Sample crash log :

OpenPLi Enigma2 crash log

crashdate=śro lut 22 12:12:59 2017
compiledate=Feb 20 2017
skin=PLi-FullNightHD/skin.xml
sourcedate=2017-02-19
branch=(no branch)
rev=1297b3c
component=Enigma2

stbmodel=uno4k
stbmodel=dm8000
kernelcmdline=root=/dev/mmcblk0p4 rootfstype=ext4 rootflags=data=journal rootwait rw coherent_pool=2M vmalloc=696m bmem=688m@320m bmem=612m@2444m
nimsockets=NIM Socket 0:
imageissue=openpli 4 %h


[eDVBPESReader] Created. Opening demux
[eDVBPESReader] Created. Opening demux
[eDVBPESReader] Created. Opening demux
[eDVBPESReader] Created. Opening demux
[eDVBPESReader] Created. Opening demux
[eDVBPESReader] Created. Opening demux
[eDVBPESReader] Created. Opening demux
[eDVBPESReader] Created. Opening demux
[eDVBPESReader] Created. Opening demux
[eDVBPESReader] Created. Opening demux
[eDVBPESReader] Created. Opening demux
[eDVBPESReader] Created. Opening demux
[eDVBPESReader] Created. Opening demux
[eDVBPESReader] Created. Opening demux
[eDVBPESReader] Created. Opening demux
[eDVBPESReader] Created. Opening demux
[eDVBPESReader] Created. Opening demux
[eDVBPESReader] Created. Opening demux
[eDVBPESReader] Created. Opening demux
[eDVBPESReader] Created. Opening demux
[eDVBPESReader] Created. Opening demux
[eDVBPESReader] Created. Opening demux
Unhandled error in Deferred:
Unhandled Error
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 524, in __bootstrap
    
  File "/usr/lib/python2.7/threading.py", line 551, in __bootstrap_inner
    
  File "/usr/lib/python2.7/threading.py", line 504, in run
    
--- <exception caught here> ---
  File "/usr/lib/python2.7/site-packages/twisted/python/threadpool.py", line 191, in _worker
    
  File "/usr/lib/python2.7/site-packages/twisted/python/context.py", line 118, in callWithContext
    
  File "/usr/lib/python2.7/site-packages/twisted/python/context.py", line 81, in callWithContext
    
  File "/usr/lib/enigma2/python/Plugins/Extensions/EPGImport/EPGImport.py", line 220, in doThreadRead
    for data in self.createIterator(filename):
  File "/usr/lib/enigma2/python/Plugins/Extensions/EPGImport/EPGImport.py", line 132, in createIterator
    return getParser(self.source.parser).iterator(self.fd, self.source.channels.items)
  File "/usr/lib/enigma2/python/Plugins/Extensions/EPGImport/EPGImport.py", line 34, in getParser
    mod = relImport(module)
  File "/usr/lib/enigma2/python/Plugins/Extensions/EPGImport/EPGImport.py", line 27, in relImport
    mod = __import__('.'.join(fullname))
exceptions.ImportError: No module named epg.dat
Traceback (most recent call last):
  File "/usr/lib/enigma2/python/Components/ActionMap.py", line 46, in action
  File "/usr/lib/enigma2/python/Screens/MovieSelection.py", line 458, in do_delete
  File "/usr/lib/enigma2/python/Screens/MovieSelection.py", line 1851, in do_delete
  File "/usr/lib/enigma2/python/Screens/MovieSelection.py", line 1866, in delete
OSError: [Errno 2] No such file or directory: '/media/hdd/movie/20161212 0955 - CANAL+ FILM HD - Mi?osne antidotum.ts'

listing from the source :

SynDS213p> ls -Al 20161212*
-rwxrwxrwx    1 aaaaa    users          972 Dec 12 10:01 20161212 0955 - CANAL+ FILM HD - MiÅosne antidotum.eit
-rwxrwxrwx    1 aaaaa    users    6206560184 Dec 12 11:42 20161212 0955 - CANAL+ FILM HD - MiÅosne antidotum.ts
-rwxrwxrwx    1 aaaaa    users        74240 Dec 12 11:42 20161212 0955 - CANAL+ FILM HD - MiÅosne antidotum.ts.ap
-rwxrwxrwx    1 aaaaa    users           12 Dec 12 11:42 20161212 0955 - CANAL+ FILM HD - MiÅosne antidotum.ts.cuts
-rwxrwxrwx    1 aaaaa    users          214 Dec 12 20:12 20161212 0955 - CANAL+ FILM HD - MiÅosne antidotum.ts.meta
-rwxrwxrwx    1 aaaaa    users      4800240 Dec 12 11:42 20161212 0955 - CANAL+ FILM HD - MiÅosne antidotum.ts.sc

I can not check , but maybe enigma plays the files using name from second listing and all is OK, but tries to delete using name from crash log , and this is a source of problem?

[littlesat]

Remove epgimport plug-in

[WanWizard]

Unrelated. The error just happens while a parallel thread is importing the EPG (and it is trying to import an epg.dat file, which is expiremental and buggy at best).

 

This is more about MovieSelection not being able to handle UTF8 in filenames. Altough I wouldn't know why this all of a sudden is a problem, all OpenPLi images run the same Enigma (or was the version on your dm8000 very old?).

[Tom0000]

Thank you for a suggestion about epgimport. If I will do it , what can I use instead ?  I used crossepg years ago to the moment when it stopped working. Is it functional today and you suggest to use it ? Or something else ?

 

I have updated my dm8000 every 6 months or more often. Now it has broken main processor. The sample is dated the end of 2016 and the version of openpli could be from September 2016 .

Edited by Tom0000, 22 February 2017 - 13:32.

[Tom0000]

But I want to return to my finds :

 

the same file is played correct, only deleting is problem. Enigma uses correct name for access the .ts file for play. Why is it possible ?

 

The /movie is a resource in NAS,  managed by linux. Enigma uses it by this connection :

           mount -t cifs //192.168.0.55/DM8000_v1 /media/hdd -o user=user,pass=pass,domain=WORKGROUP,rw
The same line connected /movie in my old  dm8000.

 

Previous listing was from nas . This one is from uno4k :

root@vuuno4k:~# ls -Al  /media/hdd/movie/20161212*
-rwxr-xr-x    1 root     root           972 Dec 12 10:01 /media/hdd/movie/20161212 0955 - CANAL+ FILM HD - Mi?osne antidotum.eit
-rwxr-xr-x    1 root     root     6206560184 Dec 12 11:42 /media/hdd/movie/20161212 0955 - CANAL+ FILM HD - Mi?osne antidotum.ts
-rwxr-xr-x    1 root     root         74240 Dec 12 11:42 /media/hdd/movie/20161212 0955 - CANAL+ FILM HD - Mi?osne antidotum.ts.ap
-rwxr-xr-x    1 root     root            12 Dec 12 11:42 /media/hdd/movie/20161212 0955 - CANAL+ FILM HD - Mi?osne antidotum.ts.cuts
-rwxr-xr-x    1 root     root           214 Dec 12 20:12 /media/hdd/movie/20161212 0955 - CANAL+ FILM HD - Mi?osne antidotum.ts.meta
-rwxr-xr-x    1 root     root       4800240 Dec 12 11:42 /media/hdd/movie/20161212 0955 - CANAL+ FILM HD - Mi?osne antidotum.ts.sc

If enigma sees it like here , it should always use name with wrong char and always has a problem with accessing the file.

The last question, how can it be possible the enigma on dm8000 created a name with this char?

Edited by Tom0000, 22 February 2017 - 13:49.

[Tom0000]

Maybe the last thing. If enigma is not able to delete the file because of strange chars and fixing it is not possible, the last to do can be checking if file not exist and show message box instead of crashing the system.

[WanWizard]

The connection to the NAS, is that CIFS by any chance?

 

If the display on the NAS is correct, and on the box wrong, it might be the DM8000 had a codepage mapping in the mount command of the NAS share.

[Tom0000]

The only connection used by me was CIFS (mount -t cifs //192.168.0.55/DM8000_v1 /media/hdd -o user=user,pass=pass,domain=WORKGROUP,rw) . I used it from 5 years.

Dm8000 has not any additional codepage mapping . New device has all defined the same. The both have had polish language set.

[WanWizard]

I assume the Å character isn't the correct one either, what should have been the correct one?

 

What version of OpenPLi were you running on the DM8000? I can remember a lot of codepage issues in the past, so if the DM8000 was running an old OpenPLi version, it could well be the cause, meaning the filename was written incorrectly, so now that this issue is addressed, it can't be read back anymore because Python doesn't see it as valid utf-8.

[Tom0000]

I am not able to check the version on dm8000 because it has broken main processor.

The story of system ... I used openPLI many years ago. When openPlI team stopped dreambox service, I experimented with others , but returned to openPLI. When openPLI returned limited support for dreamboxes version 1 , I regulary updated openPLI using build in update system in the firmware. The sample from this topic has been created using version of openPLI for dm8000 which was prepared to update between September and Octover 2016.

[Tom0000]

I checked the archive for version of base firmware and I can not write if it was 3.0 or 4.0 . I have both from 2014. If only 4.0 had updates in September 2016 , it will be 4.0 , if both had updates - I am not able to point correct version.

[WanWizard]

Ok, that would mean both boxes run the same OpenPLi version, but the DM8000 has a much older kernel (and therefore kernel drivers) than the Uno 4K, so I can only think of that as a possible difference.
 
Could you try to add any of these to the mount options on the Uno, and see if they make any difference?

iocharset=utf8
iocharset=utf8,codepage=cp437
iocharset=utf8,codepage=cp850

[Erik Slagter]

Yeah, exactly. Codepages included or not included into the kernel, defaults for CIFS codepages... Oh, I hate CIFS, can I mention that? After all, it's just the Linux kernel you're dealing with and what's in the kernel or not, is determined by the manufacturer.

[Tom0000]

Thank you WanWizard ,

I used iocharset=utf8 and I am able to delete a file. Additional I see corect chars on the enigma2's  list of recordings.

I will use it  ... but will it make the same problem all time , and all new created recordings will use chars in file names which will be able to manage only with option iocharset=utf8 ?

Is any way to prevent enigma to use national chars in recording file names ?

Edited by Tom0000, 23 February 2017 - 15:07.

[Pr2]

You already have an option to limit the charset use for the recording, just enable it. Go into Recording menu and you should find it (in the bottom of 1st screen).

[WanWizard]

And if you want to do that, use the following trick:

 

mount the share twice, once at /media/old with the iocharset, and once at /media/new, without the iocharset.

 

Then use the box to move everything from "old" to "new", that would preserve the filenames and solve the problem (may take a while with a lot of files though).

[Pr2]

Why not simply rename the files (even from the STB)? Because copying the whole files just to change a char seems to me an overkill solution.

 

File - menu - rename and delete and/or replace the wrong char(s) with something valid.

[SpaceRat]

BTW, there is another aspect for CIFS mounts, besides the kernel version:

The kernel only does the core work, but the CIFS mount helper "mount.cifs" does all the dirty work around it, e.g. resolving the hostname and stuff like that.
I didn't study the sources of it, but it might as well be in charge when negotiating other capabilities between server and client.
It then finally passes the necessary parameters to the kernel cifs driver, which does exactly what the mount helper tells him.

OpenPLi 4.0 comes without a proper mount.cifs but only has the lame excuse for a mount helper from busybox.
oe-a images like OpenATV, OpenHDF, ... use the genuine mount.cifs - currently version 5.9 - from Samba.org for this.

In OpenPLi 5.0 there would be the chance to not only get rid of the ancient Samba 3.0.37 in favor of Samba 4.x, but to also use the genuine mount.cifs.
The recipe for it - version 6.x - is already inside OpenEmbedded.

[WanWizard]

So you are actually guessing, in other words another opportunity for you to do some OpenPLi bashing? 

If you have nothing useful to contribute, may I suggest you refrain from responding?

 

If you look at the original data from the TS, you can clearly see that the original file names are already incorrect. And this is because the DM8000 CIFS mounts were never utf8 compatible, so this is about damage limitation.

 

@Pr2,

 

Obviously, renaming is a lot faster. I just wanted to be complete, if the TS wanted to preserve the name, this was an option to do some "autoconversion".

[SpaceRat]

So you are actually guessing, in other words another opportunity for you to do some OpenPLi bashing? 
If you have nothing useful to contribute, may I suggest you refrain from responding?

Sure.
You can of course also accompany the nice Samba 4.x server side support in OpenPLi 5.0 with the shitty busybox-mount.cifs for the client side.
Solely your choice.

 

If you look at the original data from the TS, you can clearly see that the original file names are already incorrect. And this is because the DM8000 CIFS mounts were never utf8 compatible, so this is about damage limitation.

The file names become incorrent if client and server side disagree on the charset to use.
You already presented a fix for the damage already done, so that part of the topic is already covered.

So I usually do not care about discussions about that it got broken but only about why it broke and how to fix that for the future.
But if you don't care about solutions I can keep my thoughts to myself, no problem with that.