173 replies to this topic

[WanWizard]

Maybe the System-Time Plugin should be removed from the plugin list if it is not working at all.

 

You still seem to have missed the actual issues that are being discussed here.

 

The plugin didn't manage to correct the time because the time of your box was outside the 120s delta window, so the correction was refused. For the same reason, using the command Dimitrij posted, or using the "date" linux command, won't work either. The plugin works fine if you let it set the time on cold boot.

[SpaceRat]

a.) If we are trying to find a real solution to the core problem, we should split this discussion off into some development thread.
b.) To make sure we all are discussing the same, we should point out some differences between OpenPLi on the one hand and other distros on the other.

b.)
This is the boot process of OpenPLi (and older OpenViX/OpenATV/OpenHDF/...):

From power off:
1. Yocto starts to boot. System time is 1970-01-01 0:00 UTC
2. Stuff that uses certs fails in init with "certificate not valid yet"
3. One-shot NTP sync (in yocto) fails even on networked boxes due to ntpsync called too fast after ifup -> Time remains to be 1970-01-01 0:00 UTC plus current runtime
4. E2 GUI starts, restores the pseudo-RTC from front panel (which also contains 1970-01-01), checks "now < 2004?" -> True, so do a full transponder sync ONCE, no matter if that time is good or not

From reboot/deep standby:
1. Yocto starts to boot. System time is 1970-01-01 0:00 UTC
2. Stuff that uses certs fails in init with "certificate not valid yet"
3. One-shot NTP sync (in yocto) fails even on networked boxes due to ntpsync called too fast after ifup -> Time remains to be 1970-01-01 0:00 UTC plus current runtime
4. E2 GUI starts, restores the pseudo-RTC from front panel, checks "now < 2004?" -> False, so NEVER do a full transponder sync, even if the current system time has been borked by drift before

Problems:
- Cert based stuff on boot fails
- box time can drift wherever it wants to drift to (See the links from one of my previous posts), ONLY recoverable by FULLY POWERING the box OFF
- if the box powers on on a bad transponder, it will get a bad time right from the start and never fix it, also ONLY recoverable by FULLY POWERING the box OFF



This is the boot process in recent OpenViX/OpenATV/OpenHDF/...:

From any state:
1. Yocto starts to boot. System time is 1970-01-01 0:00 UTC
2. In early boot, fake-hwclock sets the system time to image build time or last shutdown
3. A bit later in boot (After the box drivers were loaded), stb-hwclock restores the pseudo-RTC from front panel as long as it is ahead of the time restored by fake-hwclock and sets system time accordingly
4. Stuff that uses certs succeeds in most cases (Unless it's happening between 2. and 3. AND the cert was issued between image build time/last shutdown and the real "now", very unlikely)
5. E2 GUI starts -> ALWAYS does a full transponder sync ONCE, no matter if that time is good or not

Problems:
- box time can drift wherever it wants to drift to (See the links from one of my previous posts), but will be recovered by ANY E2 restart, even only GUI restart
- if the box powers on on a bad transponder, it will get a bad time right from the start and never fix it, also recoverable only by E2 restart (but GUI restart is sufficient)

If you look carefully into any decent Linux distro and/or the yocto core, you will notice that the current OpenATV/OpenHDF/OpenViX/... behaviour is inline with normal Linux behaviour and yocto design.

It was previously (and still is in OpenPLi) behaving differently due to quirks and bugs in oe-a and OpenPLi core:

• The ntpdate always added a cronjob for periodic time sync https://github.com/o...10.bb#L157-L163
  This is only failing due to busybox-cron using a Debian like location for its cronjobs while this recipe adds the cron job into a Fedora like location
• yocto contains something like fake-hwclock itself
  yocto uses save-rtc.sh to store the shutdown time: https://github.com/o...1.0/save-rtc.sh
  ... and restores it in bootmisc: https://github.com/o...misc.sh#L63-L80
  That code was simply invalidated through bbappend (= workaround to keep the initial workaround in E2 working)

I would say it's rather safe to assume that having a decently proper system time at boot is a good idea, as any Linux distro out there does its best in order to maintain this.
In systemd, advancing the system clock to at least systemd build time even is a built-in feature.

Tell me whatever you want, but I can't consider a workaround in E2 that relies on everything around it to fail (NTP one shot, systemd) or to be patched to death (save-rtc.sh) as "best practice".

For that reason, I removed the bad workaround in E2 instead ...
https://github.com/O...e341aaae6a07f77
https://github.com/o...151b4ac04616692
https://github.com/o...d63387c1889eb1f
... because the problem is not that Linux distros give their best to maintain some decent system time but E2 making false assumptions about it.

Tests so far have proven that it's the correct direction, boxes without network (NTP) now also get a time sync (transponder) again ... and we got rid of one more bad workaround.

What's not evaluated so far is, if this might result in one "jump" of the clock under some conditions:
NTP time at boot -> transponder sync on GUI start -> back to NTP time again

The behaviour here might differ between OpenPLi and others though, as from this thread I have learned that OpenPLi users need yet another plugin for NTP, while NTP sync is a built in feature in at least OpenATV, but probably also OpenViX, OpenBH and others:
OpenATV users can toggle between NTP and transponder sync without the need for additional plugins, so selecting NTP disables transponder sync anyways, but only conditional (if the time makes sense).


Some thoughts:
The difference between transponder and box time should be "reconsidered" from time to time. It doesn't make sense to ask the transponder about its time, only to ignore it later.
One idea would to use modulo: Subtract full half hours from the difference, e.g. if the difference is 3800 seconds, it appears more probable to me that the transponder delivers a correct time that is simply in local time, rather than it being entirely wrong.
Subtracting 3600 sec = 1 hr would result in a difference of 200 seconds, which we could still sync with.

If transponder times really are that bad, why sync with them at all? Wouldn't it be more honest then to say "Your cheap-a$$ box vendor was too niggard to include a 20 Cent RTC in your box, you need network for NTP time!"?

If only some transponders deliver a good time, wouldn't it be more elegant to let E2 shortly tune to them at boot, rather than to advise the users to make the box come up with some - maybe unwanted - channel on such a transponder, just to get some good time?

Also: Why not include a list of "known-good" transponders/sat positions for time sync, so that E2 at least re-syncs when switching to them, rather than to preserve the bad offset it stored on first sync on a bad one?
From what I learned so far, bad time sync is a problem that doesn't really exist on Astra or Hotbird = most of our users.
Why does the whole world need to suffer from bad code written for down under (It was mentioned that Aussie sat positions DO deliver bad times)?

Edited by SpaceRat, 27 December 2017 - 15:22.

[WanWizard]

To make it clear, I never said that whatever happens now is good, I have just been explaining what the current process is, and where it's flaws are.

 

What exactly do you mean with "One-shot NTP sync"? The standard Yocto NTP package?

 

The systemtime plugin doesn't have that problem, it always sets the NTP time without issues. It also allows you to define a delay to address this issue on systems with wifi interfaces (which do take longer to setup a connection). I prefer this above anything that requires commandline access, which is useless for most users. The plugin also has the option to disable transponder time, so you don't bump into the problem that Enigma "corrects" the time already set by NTP. 

 

As to the transponder times, it would not be a bad idea to add some logging to the code, so we can have an idea about the exact issue, and the scale of the issue. Because currently we're guessing, nobody knows what "a known good" transponder is, I've personally picked BBC One HD, but that is also a guess...

 

The difference between transponder and box time should be "reconsidered" from time to time. It doesn't make sense to ask the transponder about its time, only to ignore it later.

 

It isn't ignored, it is skipped if its delta to the current time is too big. This is because the mechanism is created to make small adjustments to deal with a drifting hardware clock, while ignoring transponders with an incorrect time. It makes the assumption that the current time was set correctly at boot time, if not, this mechanism does the opposite of what it should do, and the box will never get the correct time.

 

Wether your idea is good or not depends on what the actual issue is, for which we need to log transponder times, to avoid guesswork.

 

If transponder times really are that bad, why sync with them at all? Wouldn't it be more honest then to say "Your cheap-a$$ box vendor was too niggard to include a 20 Cent RTC in your box, you need network for NTP time!"?

 

Problem is that this is also a guess. For starters, I assume that most transponders are correct. But to know the scale of the problem, see the previous point.

 

If only some transponders deliver a good time, wouldn't it be more elegant to let E2 shortly tune to them at boot, rather than to advise the users to make the box come up with some - maybe unwanted - channel on such a transponder, just to get some good time?

 

That would not be a bad idea, designate a transponder on every sat that enigma tunes into upon cold start to get the correct time.

 

But before we start with something, I still would like to understand where the clock drift comes from. All my boxes use transponder time, and I have personally never experienced a clock drift. And some drifts reported are too large to blame it on a bad hardware clock. If we know what causes it, we can design a better solution.

[SpaceRat]

The plugin didn't manage to correct the time because the time of your box was outside the 120s delta window, so the correction was refused. For the same reason, using the command Dimitrij posted, or using the "date" linux command, won't work either. The plugin works fine if you let it set the time on cold boot.

I don't know about your plugin, but ntp on the command line doesn't care about this bad code.

The only reason why normal NTP previously failed (OpenATV/OpenViX/...) was or still fails (OpenPLi):
- added to the wrong crontab (Fedora location while busybox-cron uses Debian locations)
- one shot sync invoked too fast after ifup -> Network not settled/ready yet -> No connection possible -> No sync

That's one of my main development goals:
Get rid of quirky E2 code where things that Linux can do very well are broken first just to rewrite them badly inside E2 later.

With limited ressources (developers), E2 development should concentrate on being a STB GUI and let Linux or existing 3rd party tools do what they can do better.

There is one thing in OpenPLi where this is done exactly right:
You simply use existing system mechanisms to start CAMs (/etc/init.d/softcam and /etc/init.d/cardserver -> start through SysVinit) rather than (badly) re-inventing the wheel using Python CrapStart from inside E2.

If we would focus on getting rid of useless code in E2, rather than to rewrite bad useless code into good but still useless code, there would be much more time left to maintain and extend the useful code.

One other example:
Rather than having "E2 mount" (mount invoked from Python, yikes) and silly automounts.xml, E2 should simply take care that auto.network and fstab are correctly filled. No need to re-invent the wheel here, the Linux mechanisms work (especially if it later even comes to systemd mount units).

In short: As long as the OS already can do it, only let E2 modify existing configs, the part you would do using nano/joe/mcedit/vi on a normal Linux.

[WanWizard]

From a technical point I absolutely agree with you.

 

Problem (or should I call it challenge? ) is that we have something called "end-users". And I already find it very annoying that a lot of answers to questions here contain the words "go to the commandline" or "login to the box", which is very off-putting for the average end-user.

 

So to do this properly, we need proper GUI frontends for OS features (like this one, but also networking, storage management, etc). For which you also need developers.

 

Given the power of the current hardware, it might be better moving forward to switch to a light-weight window manager, and use the already existing GUI tools.

[littlesat]

/usr/sbin/ntpdate -s -u pool.ntp.org

 

I prefer  ntpd -p 192.168.1.1 (or another IP that points to your gateway) to not 'spray' the WWW....

Edited by littlesat, 27 December 2017 - 16:14.

[Abu Baniaz]

@SpaceRat

 

You have omitted this in your description. It has worked fine for over two years until the recent batch of commits

https://github.com/O...6752e4d5a6bb160

Edited by Abu Baniaz, 27 December 2017 - 16:14.

[littlesat]

And why using /usr/sbin/ntpdate while the image has ntpd...?

This also quits after the clock is set...

 

 ntpd -qp 192.168.1.1

 

So I think no && pause 2 required...???
 

Edited by littlesat, 27 December 2017 - 16:32.

[SpaceRat]

What exactly do you mean with "One-shot NTP sync"? The standard Yocto NTP package?

Yes.

It is adding a link to if-up.d:
ln -s ${bindir}/ntpdate-sync ${D}/${sysconfdir}/network/if-up.d
https://github.com/o...4.2.8p10.bb#L92

The desired behaviour obviously is to get a proper NTP time at boot as soon as networking is upped.
This fails however, because there isn't even one millisecond of delay and DHCP isn't instant in assigning addresses and routes, but probably even when using static IPs the routing table wouldn't be ready that quick, so the one shot can not contact an NTP server at all.

I have addressed this issue here: https://github.com/o...e80b9fd99b18362
If invoked as /network/if-up.d/ntpdate-sync , ntpdate-sync waits 5 seconds before doing the sync (Maybe 2, 3 or 4 seconds would be sufficient, but it doesn't matter as the process is backgrounded anyways).

The second part of the commit calls stb-hwclock rather than the non-existant hwclock, in order to also adjust the pseudo-RTC of the box if it has one.
stb-hwclock is a simple shell script that understands the two hwclock parameters --systohc and --hctosys for saving/restoring the pseudo-RTC as if it was a real RTC.

 

The difference between transponder and box time should be "reconsidered" from time to time. It doesn't make sense to ask the transponder about its time, only to ignore it later.

It isn't ignored, it is skipped if its delta to the current time is too big.

That's actually the same:
The transponder time is only used to confirm the - possibly already bad - system time, but never used to adjust it ... resp. only within a 240sec windows (+/- 120sec).

That's the same kind of question as if your wife asks you if she's too fat: "No, of course not!!!!!!!!" is the only valid answer, neither "yes" nor "nah, not fat enough" nor "I love every ton on you." will do as an alternative.
E2 is asking if it has the right time and will only accept an ACK, but ignore any NAK.

 

This is because the mechanism is created to make small adjustments to deal with a drifting hardware clock, while ignoring transponders with an incorrect time.

We are going in circles:
If the box time is that good, we should use that as a reference and upload it to the transponder
If we believe that the transponder can experience such a heavy drift that we need to ignore it, we will also need to face the fact that it can as well experience slow drift.

What we know for sure is:

• Our STBs don't have a real RTC and are guaranteed to have a bad time after power off
• Obviously STBs don't have high precision clocks, else we wouldn't need to allow transponders to re-adjust the time at all and else we wouldn't have plenty of boxes drifting away due to syncs beings de-facto disabled
• NTP is a reliable time source, all current OSes use it by default to adjust the system clock and RTC
• although a non-networked E2 box doesn't make much sense, there are some operated that way, so they do not have access to NTP
• as user reports indicate, there are rarely ever problems with transponder time sync on serious sat positions like 19.2°E, 13°E, 23.5°E, 28.x°E, but a lot without

I'm happy with adding debug code, but I bet it will only prove the above points.

My predicted outcome:
For known-good positions/transponders it would be best practice to whitelist them, allowing them to always adjust the clock, and if technically possible to use them for a first sync on startup if NTP is not available.
For known-bad positions, the existing quirks can prevent the clock from changing all the time, but they can never ever ensure a proper time, especially as long as they allow E2 to adjust the clock totally wrong on boot.

I see no real solution for boxes that
- aren't networked
and
- don't have a real RTC and don't allow manual time adjust
and
- can only receive from bad sats/transponders
but I see ways to get much better results for the mainly European audience that has known-good transponders and currently suffers from quirks for bad positions.

Edited by SpaceRat, 27 December 2017 - 16:36.

[SpaceRat]

@SpaceRat
 
You have omitted this in your description. It has worked fine for over two years until the recent batch of commits
https://github.com/O...6752e4d5a6bb160

I don't monitor OpenViX changes.

And honestly, it's just yet another quirk ... you could have fixed that in OE core (= for all distros) instead.
Why keep it broken in yocto and add a workaround if one can as well fix it?

You shouldn't be proud but instead ashamed of it.
I would.

[littlesat]

And honestly, it's just yet another quirk ... you could have fixed that in OE core (= for all distros) instead.

 

+1

Why keep it broken in yocto and add a workaround if one can as well fix it?

Until now I do not see any real fix... I only see work-a-rounds...

Edited by littlesat, 27 December 2017 - 16:39.

[SpaceRat]

Until now I do not see any real fix... I only see work-a-rounds...

I just linked to it ... giving the network some time to settle before doing the sync makes the one-shot ntp-sync work.

[SpaceRat]

So to do this properly, we need proper GUI frontends for OS features (like this one, but also networking, storage management, etc).

I totally agree on this.

The point is: It should remain limited to exactly that, a front-end.

[Abu Baniaz]

 

@SpaceRat
 
You have omitted this in your description. It has worked fine for over two years until the recent batch of commits
https://github.com/O...6752e4d5a6bb160

I don't monitor OpenViX changes.

And honestly, it's just yet another quirk ... you could have fixed that in OE core (= for all distros) instead.
Why keep it broken in yocto and add a workaround if one can as well fix it?

You shouldn't be proud but instead ashamed of it.
I would.

 

 

You definitely broke something that was working. Maybe not perfect, but working.

 

As usual. no concern for what you break.

[SpaceRat]

You definitely broke something that was working. Maybe not perfect, but working.

As I already mentioned on the OpenViX forum:
Why not build your pixel-shifter distro from oe-a 4.0?
Nobody will mess around with it anymore.

Either do "releases" right, the way OpenPLi does it, or not at all.

OpenPLi builds release 6.0 from frozen code and doesn't even fix known bugs or limitations there, but at least you know which bugs or limitations you get (The same as yesterday and the day before).
6.1 rc is built from moving code which is also subject to possible new bugs due to code changes.

You however build from moving code and just call it a release.
In fact, what you do is just "irregular nightlies".
As a matter of fact, you built the 5.1.006 OpenViX, even tested it for some time and then released it ... and now you are complaining there is a bug inside?
Why did you release a buggy build as release?
As my changes were that crappy, you should have seen instantly that your OpenViX 5.1.006 is crappy too and ditched it.
You didn't.
Why?
Because you either didn't really test it before calling it a "release" or maybe the problems where just slightly more hidden, so that even the almighty and fault-free OpenViX didn't notice before it was in the wild.

All very strange ...

No pun intended:
If you want no changes, learn to use git and use an own branch, tagged releases or simply stop doing "make update" before build.

But please accept that not every team only wants to move pixels around to call it a new version.

As usual. no concern for what you break.

No?
Why am I writing here then?
Didn't I provide a pull request to make E2 sync time on startup again?

Yes, I make a lot of mistakes.
Why?
Because I make a lot of commits.
The only way to never make mistakes is to do nothing.

"Doing nothing" is what comments like yours make devs do.
No wonder all teams have less and lesser developers.

But here something to make you smile:
People like you are close to success. My frustration has reached a level where I more and more consider if a hobby should really be frustrating.
So it's not entirely unlikely that some time in the near future you can shift pixels again without any distortions by real changes.

[Abu Baniaz]

Time is still wrong on startup. Was fine until your commits.

So you are now going on your high horse while others miss recordings.

[Abu Baniaz]

There is nothing wrong with the transponder time for those affected.

[SpaceRat]

So you are now going on your high horse while others miss recordings.

Bullshit.

If you build with all changes (fake-hwclock, stb-hwclock, working NTP-one-shot and E2 change), everything is working fine, exactly as before, just that we now have the best possible time at all times during boot.
If you build your pseudo-release, you are missing the E2 change, as it hasn't been merged from Dev branch to master.

[Abu Baniaz]

I built the Dev image which includes the changes. Calling it bullshit does not detract from the problem.

Why must you be so vulgar?

[Erik Slagter]

/usr/sbin/ntpdate -s -u pool.ntp.org

I prefer  ntpd -p 192.168.1.1 (or another IP that points to your gateway) to not 'spray' the WWW....

Do not use ntpd, exactly for that reason. The program ntpdate does a one shot, which isn't that big an issue.