164 replies to this topic

[Huevos]

Attachment:

Attached Files

•  plugin.py   47.17KB   5 downloads

[littlesat]

Indeed uppercase should also not be needed... and again you never told me how you got leading zero’s as we should use them all time or never!!!! I did not see them!!! From my point of view there should be no diviation! Always leading zero’s or never. And better never as they are muda ( do not add any value). Converting to integer is muda makes no sense and again solve it at the source where the muda is made!!!!!!! (Actually the same for uppercase,, but that does not afd muda). That enigma accepted them is not an excuse that it is right. As far I know enigma ‘writes’ the lamedb and bouquets always without leading zero’s. I would go to fix it at the source by a patch remove leading zero’s as they are muda??? By arranging it is the same code is simplified....

Edited by littlesat, 23 June 2019 - 17:05.

[Huevos]

The reason I say it is because it is a valid way to write a hex value. As to where it comes from who knows. There are so many sources for settings it could come from any of them. And the purpose of this thread was to come up with a comprehensive way to match service references even when there are differences in the presentation. So we are not looking for who is at fault, but a way to ensure the code is bullet proof.

 

Lots of people use leading zeros to beatify their files so all numbers are of equal length.

Edited by Huevos, 23 June 2019 - 17:54.

[Huevos]

Also your code fails on namespace subnet differences.

[Huevos]

Python says leadiing zeros and upper/lower are both acceptable.

Python 2.7.13 (default, Jan  7 2019, 04:24:52)
[GCC 6.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
>>>
>>> print 0x0000FFFF == 0xffff
True
>>>

[littlesat]

Again... how do you get the leading zero’s?????? I know a much better work around but first I prefer to know it... and that has nothing to do e2 might accept it and if python can do it. I know it is accepted... you can put thousands of leading zero’s but even one is muda.

Edited by littlesat, 23 June 2019 - 20:34.

[Huevos]

Again... how do you get the leading zero’s?????? I know a much better work around but first I prefer to know it... and that has nothing to do e2 might accept it and if python can do it. I know it is accepted... you can put thousands of leading zero’s but even one is muda.

You still haven't replied about namespace subnet.

 

Leading zeros could come from anywhere. If you download settings the files may contain leading zeros.

 

Leading zeros are used by people to make the content of the file look well formatted when viewed in a text editor.

 

Also, leading zeros allow correct sorting when numbers are contained in strings.

Edited by Huevos, 23 June 2019 - 20:58.

[littlesat]

Then the downloaded settings are bad.... and when do proper storting they are also muda...
Again how do you get the problem? With downloaded settings?
And again to remove leading zero’s there is a cheaper method... but before I implement I need to understand the source and reason of the real need. What do you think of adding lstrip(‘0’) after upper() ? And we can also speed it up with [x.lstrip(‘0’).upper() for x in refstr.split(‘:’)[3:7]] so not all ‘elements’ are converted.
As you see a bit more expensive integer conversion is not needed at all.
Basically I hate to adapt code for deviations that should not be there...

Edited by littlesat, 23 June 2019 - 21:17.

[Huevos]

Yes, just download a few different settings packages and you will find stuff like that, lstrip() should work too

 

How will you handle namespace subnet? This is user configurable in PLi.

[littlesat]

Can you please explain a bit more about namespace. I see in our code that you can ignore it... but I don't understand why?

So with downloaded settings you have fhe issue... sounds like these downloaded settings do have some muda? Which settings do have them.... I still think that it should be cleaned at the ‘source’..... until now I cannot find those.

Edited by littlesat, 24 June 2019 - 08:11.

[littlesat]

Now I see... we have namespace default '0000' and it is a separate config for DVB-T/C/S (ATSC not included - bug?). Even with e..g. hanssat settings I see he has for some channels namespace and others not.

[littlesat]

When we always try to ignore the namespace this might do...

 

def stripref(self, ref):
    return ["%s0000" % x[1].upper().lstrip('0')[:-4] if x[0] == 3 else x[1].upper().lstrip('0') for x in enumerate(ref.split(':')[3:7])]

 

But better to create a function... something like this... that also checks if the subnet should be removed...

 

i hope this does not slow down too much... But when it is sufficiant maybe always ignore it??? (Actually it seems to be our default)

def stripref(self, ref):
        strippedref = [x.upper().lstrip('0') for x in ref.split(':')[3:7]]
        if config.usage.subnetwork_cable and strippedref[6].startswith('FFFF') or config.usage.subnetwork_terrestrial and strippedref[6].startswith('EEEE') or config.usage.subnetwork:
                strippedref = "%s0000" % strippedref[:-4]
        return strippedref

Edited by littlesat, 24 June 2019 - 08:44.

[littlesat]

BTW.. GM's code doesn't also remove leading zero's

t = '1:0:1:0001139:02af8:13e:82a9df:0:0:0:'
print ["%s0000" % x[1].upper().lstrip('0')[:-4] if x[0] == 3 else x[1].upper().lstrip('0') for x in enumerate(t.split(':')[3:7])]
sp = t.split(":")
print ":%s:%s:%s:%s:" % (sp[3].upper(),sp[4].upper(),sp[5].upper(),sp[6].upper())

['1139', '2AF8', '13E', '820000']
:0001139:02AF8:13E:82A9DF:

Edited by littlesat, 24 June 2019 - 08:50.

[doglover]

You cannot ignore the Namespace.  It is part of the identificatio of a channels.

Channels do exist with the same SID:TID:NID but a different Namespace. 

And this could be ditterent channel.

 

 

In this case it is the same channel:

 

<!-- 5.0W --><channel id="TF1.fr">1:0:19:601:6:20FA:DDE0000:0:0:0:</channel><!-- TF 1 HD -->

<!-- DVB-T --><channel id="TF1.fr">1:0:19:601:6:20FA:EEEE0000:0:0:0:</channel><!-- TF1 HD -->

 

One is located on 5W and the other is a DVB-T channel

 

But I have seen (and had to resolve it) were it is different channel.  (Although I cannot find one right now)

(Crossepg does not take into account the Namespace, and so you end up there with the wrong EPG)

 

Willy

[littlesat]

This is indeed confusing...

 

DEE is different from EEEE.... It is not about 'satposition' part of the namespace but the last 4 characters... In short this is 'just' about the four zero's behind it.

 

DDE0000

EEEE0000

 

When they are (forced to) zero's they are ignored...

 

I was also not (anymore) aware of the fact we had this by default 0000... and I did also not found the 'disclaimer' about this yet.

 

Sounds like we also can improve (again) the picon renderer.

That is also the reason why I removed the last 4 characters and replaced them by '0000'

 

"%s0000" % x[1].upper().lstrip('0')[:-4]

 

Regarding the Hanssat settings for example I'm planning to help help Hans with a script that cleaned up his setting.

Edited by littlesat, 24 June 2019 - 09:51.

[Huevos]

Littlesat, all you need to do is trim the last 4 figures from the hex. So just completely ignore the subnet.

 

My code suggestion is so much simpler than what you are now suggesting and runs just as fast. And not prone to errors/bugs that could be caused by different presentations.

 

Also deals with namespace subnet.

def sRefToNum(sref):
	s, t, o, n = sref.toString().split(':')[3:7]
	return int(s, 16) << 48 | int(t, 16) << 32 | int(o, 16) << 16 | int(n, 16) >> 16

Willy please test.

Attached Files

•  plugin.py   45.08KB   1 downloads

Edited by Huevos, 24 June 2019 - 10:17.

[Huevos]

@Littlesat, namespace subnet should always be off by default. i.e.ignored. The only real use for it is cases of dual illumination where you want to stop one transponder overwriting another. You only switch it on to scan individual transponders and then you disable it again afterwards. Used any other way it will cause big problems. So always "XXXX0000" except in special cases.

[littlesat]

Thanks... it is also mega fast (~2 times) I did not predict that as I was afraid that convert to integer was slower...  and it also can be made faster (~4 times) with a check that we have indeed 4 items (to avoid a GSOD)

 

import time
ref = '1:0:1:0001139:02af8:13e:82a9df:0:0:0:'
 
start_time = time.time()
print ["%s0000" % x[1].upper().lstrip('0')[:-4] if x[0] == 3 else x[1].upper().lstrip('0') for x in enumerate(ref.split(':')[3:7])]
print "%.6f" % (time.time() - start_time)
 
start_time = time.time()
s, t, o, n = ref.split(':')[3:7]
print int(s, 16) << 48 | int(t, 16) << 32 | int(o, 16) << 16 | int(n, 16) >> 16
print "%.6f" % (time.time() - start_time)
 
start_time = time.time()
ref = ref.split(':')[3:7]
if len(ref) > 3:
        print int(ref[0], 16) << 48 | int(ref[1], 16) << 32 | int(ref[2], 16) << 16 | int(ref[3], 16) >> 16
print "%.6f" % (time.time() - start_time)
 


['1139', '2AF8', '13E', '820000']
0.000148
1241070416978378882
0.000083
1241070416978378882
0.000047

[littlesat]

So we can get something like

def strippedref(self, ref)
    ref = ref.split(':')[3:7]
    if len(ref) > 3:
        return int(ref[0], 16) << 48 | int(ref[1], 16) << 32 | int(ref[2], 16) << 16 | int(ref[3], 16) >> 16
 
or -might be even faster without the check)-
 
def strippedref(self, ref)
    ref = ref.split(':')[3:7]
    try:
        return int(ref[0], 16) << 48 | int(ref[1], 16) << 32 | int(ref[2], 16) << 16 | int(ref[3], 16) >> 16
    except:
        return

[Huevos]

Looks good.