139 replies to this topic

[IanSav]

Hi Abu,

 

One important point that I missed in my previous post is that if a file has been saved within the repository in Windows, CRLF, format then I believe that the file will be downloaded to all targets as Windows format and the line ending corrections will not be made.  I am not sure about this but it does seem to be what happens.  I have also seen where conversions are made turning a CRLF line ending into a CRCRLF ending!

 

Any way you look at this it is bad news.  As I suggested, in the previous post, always use the Notepad++ option to convert the line endings of ALL Enigma2 based files into UNIX based LF files and use appropriate tools on your Windows PC to manipulate them.

 

If you have any other questions or concerns please ask.

 

Regards,

Ian.

[Pr2]

...or use Linux. 

[IanSav]

Hi Pr2,

 

...or use Linux. 

 

I accept your comment.  I need, and prefer, to use Windows so I make it my responsibility to ensure that all my contributions to Enigma2 are appropriate and correct.  If I ever do make a mistake then please point it out to me and I will make immediate corrections. 

 

Regards,

Ian.

[WanWizard]

Satellites.xml is a complete mess in PLi. Apart from not being human friendly it contains problems. e.g  why are the input streams listed in reverse order? Seeing as this order is used directly in the GUI it affects the user experience.

 

As I wrote before, it is generated as-is from Lyngsat, I can see no post-processing in the script, so it is in the xml in whatever sequence it appears on the website.

 

I've quickly looked at it (it is written in python which isn't my thing), it seems to generate the xml as it parses the data, without intermediate storage, so changing this means quite a rewrite.

[IanSav]

Hi WanWizard,

 

 

Satellites.xml is a complete mess in PLi. Apart from not being human friendly it contains problems. e.g  why are the input streams listed in reverse order? Seeing as this order is used directly in the GUI it affects the user experience.

 

As I wrote before, it is generated as-is from Lyngsat, I can see no post-processing in the script, so it is in the xml in whatever sequence it appears on the website.

 

I've quickly looked at it (it is written in python which isn't my thing), it seems to generate the xml as it parses the data, without intermediate storage, so changing this means quite a rewrite.

 

 

Rather than outputting the lines directly to the XML file could the data be held in a list / tuple / dict that can then be numerically sorted and the written out to the file in its sorted order?  (Obviously there is a little bit more to it than this but this is a quick suggestion to move forward.)

 

Regards,

Ian.

[IanSav]

Hi WanWizard,
 
Another alternative is to add an XML file sort step to the scipt used for the build process completes.  See: https://stackoverflo...e-and-modifying or  https://www.codeproject.com/Questions/386286/How-to-sort-an-xml-Based-on-Attribute-Name or https://www.oxygenxm.../topic1322.html
 
Regards,
Ian.

Edited by WanWizard, 9 August 2018 - 11:54.
moved

[WanWizard]

Let me try to give a resume of all this, to avoid another 6 pages of ping-pong:

• PLi predates OE-A, and our satellites.xml generator is that old. We have our own tuxbox-common repo because we always had, it also predates OE-A.
• OE-A is (as far as we are concerned) not neutral ground. No point going into the history about why OE-A exists, just take this as a fact.
• We have never generated or maintained the cable.xml or the terrestrial.xml, all updates to those files come from "third parties".
• The reason we traditionally never use third-party repo's is that they tended to break our builds quite often, which we find unacceptable.
• I personally have no problem using the tuxbox-common data from OE-A, but though a sync and validation process with out repo, not directly.
• I am not OpenPLi, far from it, so I don't make the final decision.

[Erik Slagter]

Completely agree.

 

We've had endless discussions whether we should keep Lyngsat's (sometimes a bit off) positions, the actual ones, or the ones from e.g. KingOfSat. Whichever we choose, it will always cause breakage for a part of the user base. And that's why simply keep using the one's from Lyngsat. Not because it's the best, but because changing it will break things. Besides that, I feel the satellites.xml is quite good, not many complaints there afaik.

 

The cables.xml and terrestrial.xml is another story. I can't really care as long as

- if it breaks, it only breaks one provider

- whenever it breaks, it's solved quickly

 

And indeed, I don't want to be dependend on a third party for our builds to succeed. We've had too many bad experiences with those.

[littlesat]

+1...

 

And beside a cable.xml I prefer to have some kind of cablescan.xml -or- a section within the cable.xml, which includes a list of towns (by country) that points to the specific town's 'home' transponder parameters. So when you have cable you can select the country, town and have the home transponder parameters for 'free' without entering frequency or network IDs...

Edited by littlesat, 9 August 2018 - 12:12.

[Huevos]

 

Satellites.xml is a complete mess in PLi. Apart from not being human friendly it contains problems. e.g  why are the input streams listed in reverse order? Seeing as this order is used directly in the GUI it affects the user experience.

 

As I wrote before, it is generated as-is from Lyngsat, I can see no post-processing in the script, so it is in the xml in whatever sequence it appears on the website.

 

I'm not trying to play a game of one-up-man-ship.I just want to point out the errors so they can be corrected. This is exactly why I made the cables and terrestrial regions in OpenPLi be sorted alphabetically by the .py code in the image. That way if data in the xml file is in a strange order it doesn't matter, and the regions are sorted alphabetically whatever language the user has configured.

 

In the case of the OpenPLi satellites.xml the data is not in the same order as on Lyngsat. Lyngsat is in the natural order. OpenPLi satelites.xml is in reverse order. So the OpenPLi fetcher script must be manipulating the data. This is also the case with the (unused) t2mi_plp_id values (which currently serve no purpose and just add unnecessary padding to the file).

 

I wrote my fetcher because at the time OpenPLi had all the feeds transponders and various other junk in satellites.xml and because the satellite positions used to wander about by a few tenths of a degree from build to build, breaking everyone's settings.

 

My method uses 2 steps.

1) fetch the data and put it in the DB.

2) send a query to the DB and build the xml file.

 

That way it doesn't matter what order the data is on Lyngsat or what order we read it in. The only thing that governs how the data is presented in the XML file is the DB query. So if there is ever a problem with the sort order all I need to do is update the DB query. Right now it looks like this:

ORDER BY 
 transponders.orb_pos, transponders.frequency, transponders.polarisation %2, transponders.symbol_rate, transponders.fec_inner, transponders.DVB_system, transponders.modulation, transponders.is_id, transponders.pls_code, transponders.pls_mode 

[mrvica]

@IanSav, for doing DVB-T(2) scan you may try

https://github.com/O...terrestrialscan

it does not need any .xml, a kind of blind scan for DVB-T

[Erik Slagter]

In the case of the OpenPLi satellites.xml the data is not in the same order as on Lyngsat. Lyngsat is in the natural order. OpenPLi satelites.xml is in reverse order. So the OpenPLi fetcher script must be manipulating the data. This is also the case with the (unused) t2mi_plp_id values (which currently serve no purpose and just add unnecessary padding to the file).

There is definitely some manipulation going on. And after that, apparently the natural way of Python when adding an array entry, is to push it to the front, which would explain the "reverse" order. This is not on purpose.

[IanSav]

Hi Mrvica,

 

There is no need for any other sort of scan.  The data I uploaded makes the current OpenPLi scanner work well enough.  There are other issues involved with DVB-T scans in Australia but that is well beyond the scope of this thread.  Without custom programming neither OpenViX nor OpenPLi can perform a guaranteed successful DVB-T scan.  Beyonwiz has a task specific and customised scanner (IniLCNScanner).

 

Regards,

Ian.

[IanSav]

Hi All,

 

Now that this topic is in its own thread I am happy to let it takes its own course.  I am prepared to supply any more information or contribute if asked.  Other than that I have no interest in participating in any more "image wars", particularly on topics that are of no real interest or benefit to me.

 

I hope that the OpenPLi developers reach out to me when there is a workable and sustainable development build that I can use.  Until then I hope that someone from OpenPLi will step up and offer to be a development partner with me.  I will cut the new Enigma2 code and pass it over to the partner for OpenPLi testing.  If I can't get any help then I may as well abandon trying to include OpenPLi in my development planning.  I will leave it to OpenPLi developers to pick the enhancements from one of the other images.

 

Regards,

Ian.

[Huevos]

 

In the case of the OpenPLi satellites.xml the data is not in the same order as on Lyngsat. Lyngsat is in the natural order. OpenPLi satelites.xml is in reverse order. So the OpenPLi fetcher script must be manipulating the data. This is also the case with the (unused) t2mi_plp_id values (which currently serve no purpose and just add unnecessary padding to the file).

There is definitely some manipulation going on. And after that, apparently the natural way of Python when adding an array entry, is to push it to the front, which would explain the "reverse" order. This is not on purpose.

 

The code is closed source so it is impossible to offer anything more than pointing out faults in the output, otherwise I would offer suggestions how to fix the code.

[WanWizard]

I fixed the sat position order already, it was indeed sorting alfanumerically, I changed sorted(list) by sorted (list, key=int).

 

The code is closed source so it is impossible to offer anything more than pointing out faults in the output, otherwise I would offer suggestions how to fix the code.

 

Thanks, I'll shout when I'm stuck. For now, this is one of those very rare opportunities for me to learn a little python.

[Huevos]

I fixed the sat position order already, it was indeed sorting alfanumerically, I changed sorted(list) by sorted (list, key=int).

 

The code is closed source so it is impossible to offer anything more than pointing out faults in the output, otherwise I would offer suggestions how to fix the code.

 

Thanks, I'll shout when I'm stuck. For now, this is one of those very rare opportunities for me to learn a little python.

Don't laugh, mine is PHP.

[WanWizard]

Don't laugh, mine is PHP.

 

I'd rather have that, I've been doing PHP for over 20 years.

 

It seems to be a bit more complex, the sat sort order seems to be like this on purpose, all E's first, then all W's, instead of using the orbital value. Perhaps because it is (or was) not sorted when meant we had to scroll all the way down to get to "our" positions? I personally wouldn't be to happy if every sat list started at 177W and I have to scroll 100 sats down to find 19.2E. 

 

edit: also found and fixed the order of the entries per transponder. Haven't pushed it, since I don't know the why of this specific way of sorting.

[Huevos]

 

Don't laugh, mine is PHP.

 

I'd rather have that, I've been doing PHP for over 20 years.

 

It seems to be a bit more complex, the sat sort order seems to be like this on purpose, all E's first, then all W's, instead of using the orbital value. Perhaps because it is (or was) not sorted when meant we had to scroll all the way down to get to "our" positions? I personally wouldn't be to happy if every sat list started at 177W and I have to scroll 100 sats down to find 19.2E. 

 

edit: also found and fixed the order of the entries per transponder. Haven't pushed it, since I don't know the why of this specific way of sorting.

 

My file starts at orbital -1771. When I do a first configuration of the tuner the first satellite that shows is 1.9E. So as far as I can see the xml file is not dictating the which satellite should be displayed first.

Edited by Huevos, 9 August 2018 - 16:36.

[WanWizard]

That is good to know. In that case, no clue why it is explicitly ordered this way. Legacy perhaps?