Jump to content


Photo

satellites.xml is wrong !


  • Please log in to reply
64 replies to this topic

Re: satellites.xml is wrong ! #21 Huevos

  • PLi® Contributor
  • 4,621 posts

+161
Excellent

Posted 21 August 2018 - 23:50

BTW Ku-band needs to have the positions correct for USALS, whereas C-band is more likely to be using a positioner that requires manually setting up on each position.



Re: satellites.xml is wrong ! #22 athoik

  • PLi® Core member
  • 8,458 posts

+327
Excellent

Posted 22 August 2018 - 05:44

Hi,

First of all I am not complaining!

I just wanted to understand the logic behind the Lyngsat parser.

The +-1 for C and +-2 for Ka is totally understandable now, indeed that way won't break xml (by moving sat's around when TP's are changing).

Now regarding satellite change about 0.25º, does this change the Lyngsat grouping?

https://www.lyngsat.com/europe.html

So sometimes we get 4.8E and 4.9E separate? Or it's always 4.9E and we change it to 4.8E!

I am experimenting writting a parser myself, using python with requests and beautifulsoup.


If something "good" comes up, I can upload it in a repository and with some magic from travis-ci, run it from there.

But on the other hand, if everyone is building his own "satellites.xml" it might cause problems to Lyngsat (resulting in captchas).

Anyway as I said, still experimenting ;)

PS. Here is the band check.

    @property
    def isKa(self):
        return self.freq > 13000000

    @property
    def isKu(self):
        return self.freq > 10000000 and self.freq < 13000000

    @property
    def isC(self):
        return self.freq > 3000000 and self.freq < 5000000

    @property
    def isS(self):
        return self.freq > 2000000 and self.freq < 3000000

    @property
    def isL(self):
        return self.freq > 1000000 and self.freq < 1700000

Edited by athoik, 22 August 2018 - 05:46.

Wavefield T90: 0.8W - 1.9E - 4.8E - 13E - 16E - 19.2E - 23.5E - 26E - 33E - 39E - 42E - 45E on EMP Centauri DiseqC 16/1
Unamed: 13E Quattro - 9E Quattro on IKUSI MS-0916

Re: satellites.xml is wrong ! #23 Huevos

  • PLi® Contributor
  • 4,621 posts

+161
Excellent

Posted 22 August 2018 - 14:44

I am against offering code to individual users that will increase load on Lyngsat. My script does the download over several hours to try to avoid causing any load issues on that website. If for some reason a captcha is added to that site it will be a nightmare for everyone. People that want an individual customised xml can easily download it from satellite-xml.org. And anyway the file offered by OpenPLi and OE-Alliance is a pretty good file for the majority of uses.

 

I swap the position data before putting it in the database. That way the positions in the database will always be the same.

 

The basics of my code to pull the positions is as follows. Note there are no collisions in this list but the complete code has a workaround just in case a new satellite causes an unexpected one (unlikely due to allocations, e.g. 0.8ºW and 1.0ºW share the same page so will show as one satellite).

 

List of fixed sats:

$fixed_slots = array(
						'57.0E', '56.0E', '54.9E', '53.0E', '49.0E', '45.0E', '42.0E', '39.0E', '36.0E',  
						'31.5E', '30.5E', '28.2E', '26.0E', '25.5E', '23.5E', '21.6E', '20.0E', '19.2E', 
						'17.0E', '16.0E', '13.0E', '10.0E', '9.0E', '7.0E', '4.8E', '3.0E', '0.8W', '4.0W', 
						'5.0W', '7.0W', '8.0W', '11.0W', '12.5W', '15.0W', '18.0W', '20.0W', '22.0W', 
						'24.5W', '27.5W', '30.0W', '31.5W', '34.5W', '37.5W', '40.5W', '43.1W', '45.0W', 
						'50.0W', '53.0W', '55.5W', '58.0W'
					);

Convert the slots to positions:

function slot_2_orb_pos($slot)
{
	$orb_pos = intval(substr($slot, 0, strlen($slot)-1)*10);
	if(substr($slot, -1) === 'W')
	{
		$orb_pos *= -1;
	}
	return $orb_pos;
}

And then to check if the current position is in the fixed slots and migrate if necessary:

function check_against_fixed_orbitals($orb_pos, $orbitals)
{
	$tolerance = 3;
	foreach($orbitals as $pos)
	{
		if($orb_pos <= ($pos+$tolerance) and $orb_pos >= ($pos-$tolerance))
		{
			$orb_pos = $pos;
			break;
		}
	}
	return $orb_pos;
}

If you want the whole script I'm happy to send it but would like it kept private.



Re: satellites.xml is wrong ! #24 Persian Prince

  • Senior Member
  • 1,982 posts

+247
Excellent

Posted 22 August 2018 - 15:19

I am against offering code to individual users that will increase load on Lyngsat. My script does the download over several hours to try to avoid causing any load issues on that website. If for some reason a captcha is added to that site it will be a nightmare for everyone. People that want an individual customised xml can easily download it from satellite-xml.org. And anyway the file offered by OpenPLi and OE-Alliance is a pretty good file for the majority of uses.


If you want the whole script I'm happy to send it but would like it kept private.

 

Well you are against open source, simple as that.

 

It's a complete waste of time for anyone other than PLi and OE-A members to talk about satellites.xml as we can see none of you want to publish this tool or let others help you.

 

Just take a look how many posts should we see about this matter and still there is no real good repo for this as nobody accpets the other one.

 

Just imagine if the script was on github how we could imporve it together!

 

Even my post is pointless and I feel sorry for myself to waste another post on this matter.


Open Vision sources: https://github.com/OpenVisionE2


Re: satellites.xml is wrong ! #25 Pr2

  • PLi® Contributor
  • 6,165 posts

+261
Excellent

Posted 22 August 2018 - 15:56

Hi,

 

Here I go for my useless post too.   ;)

 

Isn't time to think to a new satellite.xml format (or field extension) that will cover the current constraints and avoid some tricks to work-around the current limitation?

 

Did you already try to contact Lyngsat to discuss with them if there is not a better way than grabbing their site to have access to there database? I am quite sure that they can create an API that you can call to get a dump in for exemple a csv format (or xml) and this URL can remain private or secured.

 

Pr2


NO SUPPORT by PM, it is a forum make your question public so everybody can benefit from the question/answer.
If you think that my answer helps you, you can press the up arrow in bottom right of the answer.

Wanna help with OpenPLi Translation? Please read our Wiki Information for translators

Sat: Hotbird 13.0E, Astra 19.2E, Eutelsat5A 5.0W
VU+ Solo 4K: 2*DVB-S2 + 2*DVB-C/T/T2 (used in DVB-C) & Duo 4K: 2*DVB-S2X + DVB-C (FBC)

AB-Com: PULSe 4K 1*DVB-S2X (+ DVB-C/T/T2)
Edision OS Mio 4K: 1*DVB-S2X + 1*DVB-C/T/T2
 


Re: satellites.xml is wrong ! #26 athoik

  • PLi® Core member
  • 8,458 posts

+327
Excellent

Posted 22 August 2018 - 16:17

I am against offering code to individual users that will increase load on Lyngsat. My script does the download over several hours to try to avoid causing any load issues on that website. If for some reason a captcha is added to that site it will be a nightmare for everyone. People that want an individual customised xml can easily download it from satellite-xml.org. And anyway the file offered by OpenPLi and OE-Alliance is a pretty good file for the majority of uses.
 
I swap the position data before putting it in the database. That way the positions in the database will always be the same.
 
The basics of my code to pull the positions is as follows. Note there are no collisions in this list but the complete code has a workaround just in case a new satellite causes an unexpected one (unlikely due to allocations, e.g. 0.8ºW and 1.0ºW share the same page so will show as one satellite).
 
List of fixed sats:

$fixed_slots = array(
						'57.0E', '56.0E', '54.9E', '53.0E', '49.0E', '45.0E', '42.0E', '39.0E', '36.0E',  
						'31.5E', '30.5E', '28.2E', '26.0E', '25.5E', '23.5E', '21.6E', '20.0E', '19.2E', 
						'17.0E', '16.0E', '13.0E', '10.0E', '9.0E', '7.0E', '4.8E', '3.0E', '0.8W', '4.0W', 
						'5.0W', '7.0W', '8.0W', '11.0W', '12.5W', '15.0W', '18.0W', '20.0W', '22.0W', 
						'24.5W', '27.5W', '30.0W', '31.5W', '34.5W', '37.5W', '40.5W', '43.1W', '45.0W', 
						'50.0W', '53.0W', '55.5W', '58.0W'
					);
Convert the slots to positions:
function slot_2_orb_pos($slot)
{
	$orb_pos = intval(substr($slot, 0, strlen($slot)-1)*10);
	if(substr($slot, -1) === 'W')
	{
		$orb_pos *= -1;
	}
	return $orb_pos;
}
And then to check if the current position is in the fixed slots and migrate if necessary:
function check_against_fixed_orbitals($orb_pos, $orbitals)
{
	$tolerance = 3;
	foreach($orbitals as $pos)
	{
		if($orb_pos <= ($pos+$tolerance) and $orb_pos >= ($pos-$tolerance))
		{
			$orb_pos = $pos;
			break;
		}
	}
	return $orb_pos;
}
If you want the whole script I'm happy to send it but would like it kept private.


Thanks for the fixed positions. I'll see how I can add them. :)
Wavefield T90: 0.8W - 1.9E - 4.8E - 13E - 16E - 19.2E - 23.5E - 26E - 33E - 39E - 42E - 45E on EMP Centauri DiseqC 16/1
Unamed: 13E Quattro - 9E Quattro on IKUSI MS-0916

Re: satellites.xml is wrong ! #27 Huevos

  • PLi® Contributor
  • 4,621 posts

+161
Excellent

Posted 22 August 2018 - 16:38

 

I am against offering code to individual users that will increase load on Lyngsat. My script does the download over several hours to try to avoid causing any load issues on that website. If for some reason a captcha is added to that site it will be a nightmare for everyone. People that want an individual customised xml can easily download it from satellite-xml.org. And anyway the file offered by OpenPLi and OE-Alliance is a pretty good file for the majority of uses.


If you want the whole script I'm happy to send it but would like it kept private.

 

Well you are against open source, simple as that.

 

It's a complete waste of time for anyone other than PLi and OE-A members to talk about satellites.xml as we can see none of you want to publish this tool or let others help you.

No, I am not against open source. I share lots of code publicly and even push code to other images. In this case PLi had already stated they have no interest in sharing their code.

 

I wrote this tool for my own personal use and I don't have a moral responsibility to put every piece of code I write in the public domain. Up until this point no one has been interested in this code.



Re: satellites.xml is wrong ! #28 Huevos

  • PLi® Contributor
  • 4,621 posts

+161
Excellent

Posted 22 August 2018 - 16:42

The real problem though is PLi don't feel comfortable building from the repo where the data is maintained meaning we have 2 copies of the data and all the problems that causes.



Re: satellites.xml is wrong ! #29 athoik

  • PLi® Core member
  • 8,458 posts

+327
Excellent

Posted 22 August 2018 - 17:15

PLi grabber didn't cause problems to majority of people till now.

I can make my version public, if that will improve things.

Maybe problem was the absence of code, that lead to another fork under non PLi repositories.

Anyway, I am having a look on it, so please be patient.
Wavefield T90: 0.8W - 1.9E - 4.8E - 13E - 16E - 19.2E - 23.5E - 26E - 33E - 39E - 42E - 45E on EMP Centauri DiseqC 16/1
Unamed: 13E Quattro - 9E Quattro on IKUSI MS-0916

Re: satellites.xml is wrong ! #30 Pr2

  • PLi® Contributor
  • 6,165 posts

+261
Excellent

Posted 22 August 2018 - 18:07

Except once when the satellite.xml was missing I lose all my tuner setup because the name is retrieved from the satellite.xml so a corrupted or wrong one lead to lost of settings.


NO SUPPORT by PM, it is a forum make your question public so everybody can benefit from the question/answer.
If you think that my answer helps you, you can press the up arrow in bottom right of the answer.

Wanna help with OpenPLi Translation? Please read our Wiki Information for translators

Sat: Hotbird 13.0E, Astra 19.2E, Eutelsat5A 5.0W
VU+ Solo 4K: 2*DVB-S2 + 2*DVB-C/T/T2 (used in DVB-C) & Duo 4K: 2*DVB-S2X + DVB-C (FBC)

AB-Com: PULSe 4K 1*DVB-S2X (+ DVB-C/T/T2)
Edision OS Mio 4K: 1*DVB-S2X + 1*DVB-C/T/T2
 


Re: satellites.xml is wrong ! #31 WanWizard

  • PLi® Core member
  • 70,220 posts

+1,798
Excellent

Posted 22 August 2018 - 18:37

The real problem though is PLi don't feel comfortable building from the repo where the data is maintained meaning we have 2 copies of the data and all the problems that causes.

 

I've already mentioned dozens of pages ago in the other thread that I personally don't have a problem using "a" satellites.xml, I do have a problem with using it without knowing if it is ok or not, but that can be dealt with.

 

As mentioned before, I'm not home and therefore not able to do anything that requires some else than a smartphone...


Currently in use: VU+ Duo 4K (2xFBC S2), VU+ Solo 4K (1xFBC S2), uClan Usytm 4K Ultimate (S2+T2), Octagon SF8008 (S2+T2), Zgemma H9.2H (S2+T2)

Due to my bad health, I will not be very active at times and may be slow to respond. I will not read the forum or PM on a regular basis.

Many answers to your question can be found in our new and improved wiki.


Re: satellites.xml is wrong ! #32 Huevos

  • PLi® Contributor
  • 4,621 posts

+161
Excellent

Posted 22 August 2018 - 19:00

BTW, i

 

 

The real problem though is PLi don't feel comfortable building from the repo where the data is maintained meaning we have 2 copies of the data and all the problems that causes.

 

I've already mentioned dozens of pages ago in the other thread that I personally don't have a problem using "a" satellites.xml, I do have a problem with using it without knowing if it is ok or not, but that can be dealt with.

 

As mentioned before, I'm not home and therefore not able to do anything that requires some else than a smartphone...

 

That repo is well maintained, and it is used by a lot of teams, so if something were to go wrong (although this has never happened in the past) it would be fixed very fast.

 

What doesn't make sense to me is 2 sources of data (2 repos) being maintained by one person (AbuBaniaz, cables and terrestrial). This just means the data gets fragmented after pull requests and has to be re-sync'ed manually with the possibility of human error. And it is not just a case of dropping the xml from one repo into another because different pull requests arrive at different repos. So the files need to be merged manually.



Re: satellites.xml is wrong ! #33 WanWizard

  • PLi® Core member
  • 70,220 posts

+1,798
Excellent

Posted 22 August 2018 - 19:07

Merging can happen automatically, we can deal with that, that shouldn't be something that should worry the maintainer of an upstream repo.


Currently in use: VU+ Duo 4K (2xFBC S2), VU+ Solo 4K (1xFBC S2), uClan Usytm 4K Ultimate (S2+T2), Octagon SF8008 (S2+T2), Zgemma H9.2H (S2+T2)

Due to my bad health, I will not be very active at times and may be slow to respond. I will not read the forum or PM on a regular basis.

Many answers to your question can be found in our new and improved wiki.


Re: satellites.xml is wrong ! #34 Huevos

  • PLi® Contributor
  • 4,621 posts

+161
Excellent

Posted 22 August 2018 - 19:16

By the way, OE-Alliance has the xmlUpdate plugin. This allows users to update satellites.xml, terrestrial.xml and cables.xml direct from the OE-A tuxbox repo without waiting for a build update.

 

Attached File  1.jpg   65.55KB   2 downloads

 

Attached File  2.jpg   57.33KB   1 downloads

 

Attached File  3.jpg   69.92KB   1 downloads

 

 



Re: satellites.xml is wrong ! #35 Huevos

  • PLi® Contributor
  • 4,621 posts

+161
Excellent

Posted 22 August 2018 - 20:21

Merging can happen automatically, we can deal with that, that shouldn't be something that should worry the maintainer of an upstream repo.

Yes but commits to your repo need to get to the upstream repo. That is the problem I am getting at when maintaining 2 sources of data... unless there is something I don't understand here.



Re: satellites.xml is wrong ! #36 WanWizard

  • PLi® Core member
  • 70,220 posts

+1,798
Excellent

Posted 22 August 2018 - 20:27

Yes but commits to your repo need to get to the upstream repo. That is the problem I am getting at when maintaining 2 sources of data... unless there is something I don't understand here.

 

No, there is always only one upstream repo, the one upstream. ;) Never maintain seperate versions if you can't help it, and as this is just about data, we CAN help it. ;) 


Currently in use: VU+ Duo 4K (2xFBC S2), VU+ Solo 4K (1xFBC S2), uClan Usytm 4K Ultimate (S2+T2), Octagon SF8008 (S2+T2), Zgemma H9.2H (S2+T2)

Due to my bad health, I will not be very active at times and may be slow to respond. I will not read the forum or PM on a regular basis.

Many answers to your question can be found in our new and improved wiki.


Re: satellites.xml is wrong ! #37 littlesat

  • PLi® Core member
  • 57,062 posts

+698
Excellent

Posted 23 August 2018 - 07:03

Such thing as xmlUpdate without build needed can’t really work in a structured way... as doing an update without a build was running will overwrite the XML’s.. when you put it on the ‘override’ folder a normal update does not update it...

Edited by littlesat, 23 August 2018 - 07:05.

WaveFrontier 28.2E | 23.5E | 19.2E | 16E | 13E | 10/9E | 7E | 5E | 1W | 4/5W | 15W


Re: satellites.xml is wrong ! #38 Rob van der Does

  • Senior Member
  • 7,766 posts

+184
Excellent

Posted 23 August 2018 - 07:32

So you comment without even looking at the internals.......

Attached File  1_0_19_1B1D_802_2_11A0000_0_0_0_20180823083101.jpg   81.13KB   3 downloads

Re: satellites.xml is wrong ! #39 littlesat

  • PLi® Core member
  • 57,062 posts

+698
Excellent

Posted 23 August 2018 - 09:01

I read it... also that don't really work... as during an image update these xmls are also not updates as you took the latest downloaded manual one.

 

The feeds should be simply made good! And when this is good we don't need plugins what work-a-round this stuff... Actually you just want to have a correct file.

 

And when you want to have an option to upgrade satellites.xml (etc) only it can be done like we have the option to upgrade settings only...


Edited by littlesat, 23 August 2018 - 09:02.

WaveFrontier 28.2E | 23.5E | 19.2E | 16E | 13E | 10/9E | 7E | 5E | 1W | 4/5W | 15W


Re: satellites.xml is wrong ! #40 Abu Baniaz

  • PLi® Contributor
  • 2,494 posts

+64
Good

Posted 23 August 2018 - 14:38

And when you want to have an option to upgrade satellites.xml (etc) only it can be done like we have the option to upgrade settings only...

 

Doesn't that needs a build to be run?

 

If so, say a build is run on Monday, a change to xml is made on Tuesday. The file will not get updated until next build. Isn't this why you still run the "abandoned" branches?




4 user(s) are reading this topic

0 members, 4 guests, 0 anonymous users