80 replies to this topic

[WanWizard]

@wanwizard the filter was not applied to a comment but to a valid XML tag called "channel id". If is not a regex applied to the full line but only to the field it must be applied

 

I see it now. Even weirder imho, as that means you potentionally have to filter hunderds of channels (obviously depending on the source).

[littlesat]

I cannot do it in my situation... so I’m now afraid you are angry because you did contribute something that might not be accepted and that is why you respond like this.... note e2 ha already language support for epg why didn’t you use and adapt it?
Instead you created another patchy work-a-round sorry to say that. The tags in the xmls are in the first place not dvb compatible.... this is wrong!

Edited by littlesat, 9 April 2021 - 17:46.

[doglover]

The language tags are created following a standard.

And of course this standard is wrong for you.

So, I do nothing.

 

Very nice.

[jeepcook]

Hello,

 

As you said 90% of users have no knowledge about advanced e2 options or how to create or edit a file, ok. So what's the problem to implement options for advanced users ?

 

There are already a lot of specific configurations in e2 we can use, we, the 10% ! For epgimport, there are already specific options (custom.channels.xml, custom.sources.xml and an unlimited way to create specific sources and channels xml).

 

If you only want user friendly options, we can consider to destroy e2, because specificities are already a lot... by my side I make a lot of actions by script at boot, after restore or at defined time. The 90% don't know what is a crontab, so what's the e2 future, remove the crontab ? obviously not...

 

From years, e2 is in deep sleep mode, no real new functionalities and buggy ones that nobody want to fix. So I think you have to consider specific devs, and this one too.

 

Thanks

 

Regards

Edited by jeepcook, 10 April 2021 - 08:36.

[Pr2]

 

@wanwizard the filter was not applied to a comment but to a valid XML tag called "channel id". If is not a regex applied to the full line but only to the field it must be applied

 

I see it now. Even weirder imho, as that means you potentionally have to filter hunderds of channels (obviously depending on the source).

 

 

WanWizard,

 

Sorry but you already told me that you are not using EPGImport, so please don't  make assumption for something that you are not using.

We of course don't need to filter hundreds of channels, people select the EPG they want data on so this patch only fix a few situation where a channel with the same service reference is mentionned several times in the epg source file and it is always the latest one which is display in the EPG. This problem is very specific to a few channels only.

 

So when people ask Willy/Doglover to fix this he can simply ask them to create the channel_id_filter.conf and provide the string to put in, it is really easy and fix the problem without having to wait for months (years) to have skilled and intelligent developer to implement what he think is the ideal solution.  So far I propose a very simple and effective solution tested by people that really need it. 95% of the user will never have to create such filter file.

[Pr2]

I cannot do it in my situation... so I’m now afraid you are angry because you did contribute something that might not be accepted and that is why you respond like this.... note e2 ha already language support for epg why didn’t you use and adapt it?
Instead you created another patchy work-a-round sorry to say that. The tags in the xmls are in the first place not dvb compatible.... this is wrong!

 

So we are back in a dead end: a solution that don't please you but is working versus a theoretical solution that you won't implement? Is this a good summary of the current situation?

[WanWizard]

Sorry but you already told me that you are not using EPGImport, so please don't  make assumption for something that you are not using.

 

Where do you get this idea from? I do, every day. I only stopped using it for UK channels, since the OpenTV format is now supported which is much more reliable.

 

Besides that, I don't need to be using something to have a functional discussion about it.

 

All I say is that this discussion hasn't happened, and now you're angry we're not happy with the workaround hack you came up with, because doglover didn't specify the problem to solve but came up with a proposed implementation, and you didn't look at the big picture so you missed there is a difference between the two.

 

The problem with specifying solutions (like someone saying implement "this" feature "this" way) is that the scope is always limited to the the observation of that someone, and it is very likely something is missed...

[Pr2]

I check the rytecCH_Basic  which contains the EPG for Switzerland channels and they are all taggued as lang="fr"  even if they are in "de" or "it".

So the source is not yet fully reliable, I already inform Doglover about this. So a deep review of all the sources data must be done to ensure that the lang= is always correctly set.

 

By the way the implementation asked by Doglover was not bad that's why I implement it and we then don't have to care about the epg data source file quality, so forcing the usage of the lang= parameters can be worst than the currently proposed solution...

[littlesat]

I think we should consider to use the dvb style language strings for the languages, which is the standard in dvb.... when it is adapted in the xml sources we do not need additional conversion tables and are also able to use the current existing configs for it straight away..... this means the grabber need some recoding aswell and the code in e2 will be more straight away. It might be indeed a conversion table should be added to the epg grabbing instead.

when more languages are required for dvb default we can consider to add prefered epg languages.... maybe also change them all to unlimited numbers (most are no limited to 4 auto languages)

Edited by littlesat, 10 April 2021 - 17:21.

[Pr2]

The starting point module to change to support multilingual is the:

 

https://github.com/O...onverter.py#L29

 

There you will see that it focus on returning only the data, the language is totally omitted.

But the correct way to implement it is to support the language so load all the possible available languages into EPG data memory and let the program that visualized the EPG selec the right available language. 

EPGImport should not take into account the E2 settings it should just load data taking into account the defined lang=  parameter. But this parameter is repeated for the different possible fields.

 

 <programme start="... +0200" stop="... +0200" channel="channel_id.ext">
    <title lang="it">...</title>
    <sub-title lang="it">...</sub-title>
    <desc lang="it">...</desc>
</programme>

[doglover]

First the swis source:

 

Yes the swiss channels are mess in the language settings.

I already have fixed today a number of channels.

But there a big number to be fixed.

 

This originates from the fact that they are grabbed mosty from one website.  And language setting is per website.

I can however make 3 version each with a separate language setting.  That will fix it.

 

Secondly:

Are you testing how far you can go?

First the fix I asked for, is not good.

Now the language settings are not good, because not the DVB standard.

Do you know how much work is involved to get these all changed?

 

My enthusiasm for providing EPG is rapidly dwindling.

If you want to kill the EPG altogether, you are doing very well.

I am not a sado-masochist.

Edited by doglover, 10 April 2021 - 17:38.

[Pr2]

Hi Doglover,

 

1. Thanks for the explanation

 

2. I totally agree with you, you are doing a great job for the EPG community now if they want that the lang tag become valuable they should tell you which standard is used for the DVB EPG standard.

 

Google since to indicate that the value should be in the ISO 639.2 standard. For exemple is wikipedia is righ french can be either  fre ou fra  so which one to choose for the EPG Data? But fr is no longer valid since it is ISO 639.1

Should we already use the ISO 639.3 which is non ambiguous for french so it should then be fra. Same applies for Deutsch ger/deu

 

So please DVB EPG expert can you please tell Doglover what is the right reference for the EPG language code. How E2 is handling this double language reference code possibility?

 

Pr2

[doglover]

The WG++ program use the microsoft langusge codes.
I cannot easily replace these.

Of course it is possible, but rquires me to add approx 15 statements to each grabber.
And i have 400 of them.

So no thanks.
Is it so hard to include a translation table?

Besides the fact that anybody using the data on non-enigma machine suddenly does not understand these language codes.

If I am forced to do it, I will quit.
Clear?

[jeepcook]

No Doglover !! You make a hard work every day to propose us a complete epg, you fix every missing or ref changes as soon as you received them. You are a serious guy who give use a powerfull and correct grabbing.

 

So please don't quit this project ....

[demosat1]

I hope that Doglover is immortal, because the day or it will no longer be this world ... when will it be maintenance of the EPG.

[doglover]

May I remind the devs that the locales in enigma2 are in the language coding as I am using.

Including nb for Norwegian Bokmal

See screen shot.

Attached Files

•  Knipsel.JPG   28.59KB   3 downloads

Edited by doglover, 11 April 2021 - 08:19.

[Pr2]

Yes but the DVB standard is using the ISO 639.2 standard.

To avoid any confusion I am not working on implementing the usage of the XML Lang="XX" tag in EPGimport. I just point out where this is currently retrieved in code.

[Pr2]

Hello,

 

I propose a new PR with the channel id filtering. I better describe the constraints that lead to implement this solution, it is slightly improved compared to the first version to avoid some confusing entries in log.

 

https://github.com/O...gimport/pull/53

 

This doesn't prevent to work on implementing the standard DVB language support into EPGImport so please view it as a Quick Win solution both solutions can live together.

 

End-users doesn't have to learn regex a simple cut and paste of the channel id will do the job (risk of false positive is really low and the log is clearly displaying the filtered "channel id" so debugging is easy). Moreover the content of this file for standard endusers will be explained by Doglover to easily fix there problem. Advanced users that test the solution already found benefits out of it. People can easily turn on/off one particular filter in the file since comments are allowed (line starting with a # are ignored).

The option to enable it or not on the custom.channels.xml is a specific request from Doglover.

 

I hope that it will be accepted because Doglover (Willy) is really waiting for it.

 

Regards,

 

Pr2

[jeepcook]

Hello Pr2,

 

Thanks for your work.

 

In the github description you speak about /etc/epgimport/custom.channels.conf, but in the commited code you are using channel_id_filter.conf.txt.

 

Using custom.channels.conf will be confused (with custom.channels.xml), and I think we have to use channel_id_filter.conf instead if channel_id_filter.conf.txt.

 

Thanks

 

 

 

 

 

 

[Pr2]

Sorry it is a typo of course the file is:  channel_id_filter.conf