64 replies to this topic

[Huevos]

BTW Ku-band needs to have the positions correct for USALS, whereas C-band is more likely to be using a positioner that requires manually setting up on each position.

[athoik]

Hi,

First of all I am not complaining!

I just wanted to understand the logic behind the Lyngsat parser.

The +-1 for C and +-2 for Ka is totally understandable now, indeed that way won't break xml (by moving sat's around when TP's are changing).

Now regarding satellite change about 0.25º, does this change the Lyngsat grouping?

https://www.lyngsat.com/europe.html

So sometimes we get 4.8E and 4.9E separate? Or it's always 4.9E and we change it to 4.8E!

I am experimenting writting a parser myself, using python with requests and beautifulsoup.


If something "good" comes up, I can upload it in a repository and with some magic from travis-ci, run it from there.

But on the other hand, if everyone is building his own "satellites.xml" it might cause problems to Lyngsat (resulting in captchas).

Anyway as I said, still experimenting

PS. Here is the band check.

    @property
    def isKa(self):
        return self.freq > 13000000

    @property
    def isKu(self):
        return self.freq > 10000000 and self.freq < 13000000

    @property
    def isC(self):
        return self.freq > 3000000 and self.freq < 5000000

    @property
    def isS(self):
        return self.freq > 2000000 and self.freq < 3000000

    @property
    def isL(self):
        return self.freq > 1000000 and self.freq < 1700000

Edited by athoik, 22 August 2018 - 05:46.

[Huevos]

I am against offering code to individual users that will increase load on Lyngsat. My script does the download over several hours to try to avoid causing any load issues on that website. If for some reason a captcha is added to that site it will be a nightmare for everyone. People that want an individual customised xml can easily download it from satellite-xml.org. And anyway the file offered by OpenPLi and OE-Alliance is a pretty good file for the majority of uses.

 

I swap the position data before putting it in the database. That way the positions in the database will always be the same.

 

The basics of my code to pull the positions is as follows. Note there are no collisions in this list but the complete code has a workaround just in case a new satellite causes an unexpected one (unlikely due to allocations, e.g. 0.8ºW and 1.0ºW share the same page so will show as one satellite).

 

List of fixed sats:

$fixed_slots = array(
						'57.0E', '56.0E', '54.9E', '53.0E', '49.0E', '45.0E', '42.0E', '39.0E', '36.0E',  
						'31.5E', '30.5E', '28.2E', '26.0E', '25.5E', '23.5E', '21.6E', '20.0E', '19.2E', 
						'17.0E', '16.0E', '13.0E', '10.0E', '9.0E', '7.0E', '4.8E', '3.0E', '0.8W', '4.0W', 
						'5.0W', '7.0W', '8.0W', '11.0W', '12.5W', '15.0W', '18.0W', '20.0W', '22.0W', 
						'24.5W', '27.5W', '30.0W', '31.5W', '34.5W', '37.5W', '40.5W', '43.1W', '45.0W', 
						'50.0W', '53.0W', '55.5W', '58.0W'
					);

Convert the slots to positions:

function slot_2_orb_pos($slot)
{
	$orb_pos = intval(substr($slot, 0, strlen($slot)-1)*10);
	if(substr($slot, -1) === 'W')
	{
		$orb_pos *= -1;
	}
	return $orb_pos;
}

And then to check if the current position is in the fixed slots and migrate if necessary:

function check_against_fixed_orbitals($orb_pos, $orbitals)
{
	$tolerance = 3;
	foreach($orbitals as $pos)
	{
		if($orb_pos <= ($pos+$tolerance) and $orb_pos >= ($pos-$tolerance))
		{
			$orb_pos = $pos;
			break;
		}
	}
	return $orb_pos;
}

If you want the whole script I'm happy to send it but would like it kept private.

[Persian Prince]

I am against offering code to individual users that will increase load on Lyngsat. My script does the download over several hours to try to avoid causing any load issues on that website. If for some reason a captcha is added to that site it will be a nightmare for everyone. People that want an individual customised xml can easily download it from satellite-xml.org. And anyway the file offered by OpenPLi and OE-Alliance is a pretty good file for the majority of uses.

If you want the whole script I'm happy to send it but would like it kept private.

 

Well you are against open source, simple as that.

 

It's a complete waste of time for anyone other than PLi and OE-A members to talk about satellites.xml as we can see none of you want to publish this tool or let others help you.

 

Just take a look how many posts should we see about this matter and still there is no real good repo for this as nobody accpets the other one.

 

Just imagine if the script was on github how we could imporve it together!

 

Even my post is pointless and I feel sorry for myself to waste another post on this matter.

[Pr2]

Hi,

 

Here I go for my useless post too.  

 

Isn't time to think to a new satellite.xml format (or field extension) that will cover the current constraints and avoid some tricks to work-around the current limitation?

 

Did you already try to contact Lyngsat to discuss with them if there is not a better way than grabbing their site to have access to there database? I am quite sure that they can create an API that you can call to get a dump in for exemple a csv format (or xml) and this URL can remain private or secured.

 

Pr2

[athoik]

I am against offering code to individual users that will increase load on Lyngsat. My script does the download over several hours to try to avoid causing any load issues on that website. If for some reason a captcha is added to that site it will be a nightmare for everyone. People that want an individual customised xml can easily download it from satellite-xml.org. And anyway the file offered by OpenPLi and OE-Alliance is a pretty good file for the majority of uses.
 
I swap the position data before putting it in the database. That way the positions in the database will always be the same.
 
The basics of my code to pull the positions is as follows. Note there are no collisions in this list but the complete code has a workaround just in case a new satellite causes an unexpected one (unlikely due to allocations, e.g. 0.8ºW and 1.0ºW share the same page so will show as one satellite).
 
List of fixed sats:

$fixed_slots = array(
						'57.0E', '56.0E', '54.9E', '53.0E', '49.0E', '45.0E', '42.0E', '39.0E', '36.0E',  
						'31.5E', '30.5E', '28.2E', '26.0E', '25.5E', '23.5E', '21.6E', '20.0E', '19.2E', 
						'17.0E', '16.0E', '13.0E', '10.0E', '9.0E', '7.0E', '4.8E', '3.0E', '0.8W', '4.0W', 
						'5.0W', '7.0W', '8.0W', '11.0W', '12.5W', '15.0W', '18.0W', '20.0W', '22.0W', 
						'24.5W', '27.5W', '30.0W', '31.5W', '34.5W', '37.5W', '40.5W', '43.1W', '45.0W', 
						'50.0W', '53.0W', '55.5W', '58.0W'
					);

Convert the slots to positions:

function slot_2_orb_pos($slot)
{
	$orb_pos = intval(substr($slot, 0, strlen($slot)-1)*10);
	if(substr($slot, -1) === 'W')
	{
		$orb_pos *= -1;
	}
	return $orb_pos;
}

And then to check if the current position is in the fixed slots and migrate if necessary:

function check_against_fixed_orbitals($orb_pos, $orbitals)
{
	$tolerance = 3;
	foreach($orbitals as $pos)
	{
		if($orb_pos <= ($pos+$tolerance) and $orb_pos >= ($pos-$tolerance))
		{
			$orb_pos = $pos;
			break;
		}
	}
	return $orb_pos;
}

If you want the whole script I'm happy to send it but would like it kept private.

Thanks for the fixed positions. I'll see how I can add them.

[Huevos]

 

I am against offering code to individual users that will increase load on Lyngsat. My script does the download over several hours to try to avoid causing any load issues on that website. If for some reason a captcha is added to that site it will be a nightmare for everyone. People that want an individual customised xml can easily download it from satellite-xml.org. And anyway the file offered by OpenPLi and OE-Alliance is a pretty good file for the majority of uses.

If you want the whole script I'm happy to send it but would like it kept private.

 

Well you are against open source, simple as that.

 

It's a complete waste of time for anyone other than PLi and OE-A members to talk about satellites.xml as we can see none of you want to publish this tool or let others help you.

No, I am not against open source. I share lots of code publicly and even push code to other images. In this case PLi had already stated they have no interest in sharing their code.

 

I wrote this tool for my own personal use and I don't have a moral responsibility to put every piece of code I write in the public domain. Up until this point no one has been interested in this code.

[Huevos]

The real problem though is PLi don't feel comfortable building from the repo where the data is maintained meaning we have 2 copies of the data and all the problems that causes.

[athoik]

PLi grabber didn't cause problems to majority of people till now.

I can make my version public, if that will improve things.

Maybe problem was the absence of code, that lead to another fork under non PLi repositories.

Anyway, I am having a look on it, so please be patient.

[Pr2]

Except once when the satellite.xml was missing I lose all my tuner setup because the name is retrieved from the satellite.xml so a corrupted or wrong one lead to lost of settings.

[WanWizard]

The real problem though is PLi don't feel comfortable building from the repo where the data is maintained meaning we have 2 copies of the data and all the problems that causes.

 

I've already mentioned dozens of pages ago in the other thread that I personally don't have a problem using "a" satellites.xml, I do have a problem with using it without knowing if it is ok or not, but that can be dealt with.

 

As mentioned before, I'm not home and therefore not able to do anything that requires some else than a smartphone...

[Huevos]

BTW, i

 

 

The real problem though is PLi don't feel comfortable building from the repo where the data is maintained meaning we have 2 copies of the data and all the problems that causes.

 

I've already mentioned dozens of pages ago in the other thread that I personally don't have a problem using "a" satellites.xml, I do have a problem with using it without knowing if it is ok or not, but that can be dealt with.

 

As mentioned before, I'm not home and therefore not able to do anything that requires some else than a smartphone...

 

That repo is well maintained, and it is used by a lot of teams, so if something were to go wrong (although this has never happened in the past) it would be fixed very fast.

 

What doesn't make sense to me is 2 sources of data (2 repos) being maintained by one person (AbuBaniaz, cables and terrestrial). This just means the data gets fragmented after pull requests and has to be re-sync'ed manually with the possibility of human error. And it is not just a case of dropping the xml from one repo into another because different pull requests arrive at different repos. So the files need to be merged manually.

[WanWizard]

Merging can happen automatically, we can deal with that, that shouldn't be something that should worry the maintainer of an upstream repo.

[Huevos]

By the way, OE-Alliance has the xmlUpdate plugin. This allows users to update satellites.xml, terrestrial.xml and cables.xml direct from the OE-A tuxbox repo without waiting for a build update.

 

 1.jpg   65.55KB   2 downloads

 

 2.jpg   57.33KB   1 downloads

 

 3.jpg   69.92KB   1 downloads

 

 

[Huevos]

Merging can happen automatically, we can deal with that, that shouldn't be something that should worry the maintainer of an upstream repo.

Yes but commits to your repo need to get to the upstream repo. That is the problem I am getting at when maintaining 2 sources of data... unless there is something I don't understand here.

[WanWizard]

Yes but commits to your repo need to get to the upstream repo. That is the problem I am getting at when maintaining 2 sources of data... unless there is something I don't understand here.

 

No, there is always only one upstream repo, the one upstream. Never maintain seperate versions if you can't help it, and as this is just about data, we CAN help it.  

[littlesat]

Such thing as xmlUpdate without build needed can’t really work in a structured way... as doing an update without a build was running will overwrite the XML’s.. when you put it on the ‘override’ folder a normal update does not update it...

Edited by littlesat, 23 August 2018 - 07:05.

[Rob van der Does]

So you comment without even looking at the internals.......

 1_0_19_1B1D_802_2_11A0000_0_0_0_20180823083101.jpg   81.13KB   3 downloads

[littlesat]

I read it... also that don't really work... as during an image update these xmls are also not updates as you took the latest downloaded manual one.

 

The feeds should be simply made good! And when this is good we don't need plugins what work-a-round this stuff... Actually you just want to have a correct file.

 

And when you want to have an option to upgrade satellites.xml (etc) only it can be done like we have the option to upgrade settings only...

Edited by littlesat, 23 August 2018 - 09:02.

[Abu Baniaz]

And when you want to have an option to upgrade satellites.xml (etc) only it can be done like we have the option to upgrade settings only...

 

Doesn't that needs a build to be run?

 

If so, say a build is run on Monday, a change to xml is made on Tuesday. The file will not get updated until next build. Isn't this why you still run the "abandoned" branches?