214 replies to this topic

[serdeliuk]

Somebody should change the thread title because -»- truncate the link on the notification email and this is annoying

See the below image...

 Screenshot 2019-10-02 at 11.03.43.png   66.03KB   3 downloads

Edited by serdeliuk, 2 October 2019 - 10:09.

[WanWizard]

Somebody should change the thread title because -»- truncate the link on the notification email and this is annoying

 

Done.

[serdeliuk]

Thanks

[cmatte]

Please could a mod remove the previous message (also reported)?
Edits were not finished and no permission was available to edit anymore!

Thanks a lot for the input! EPG sources for CrossEPG and channels to refresh for EPGRefresh shall be a decent number, now There are a few doubts still floating around.

• First, just wondering why OOM happens at all: free RAM is around 300M without loading EPG. The current epg.dat is around 20M, CrossEPG reports the whole DB to be below 50M, when loaded the free RAM drops to 180M and that's fine. Then, why would the RAM be randomly used so much to then just finish up? It's true that EPGRefresh and CrossEPG are refreshing in the background, but you would not expect any of that to triple its usage and get to an OOM as they both just, literally, refresh something already there, mainly! 
• Resolved - single epg cache is okay, no need for epg retrievers to save it, really.
• Resolved - swapon bug - post #1109508 has a fix.

Cheers

 

The OOM matter feels like there is some serious leak somewhere that soft-bricks the vuduo4k, requiring a re-image. Specifically:

• 786m swap enabled on usb
• 56m CrossEPG DB on usb, running every hr only in standby
• 16hr ago the last EPGRefresh run ended with success
• 0 options set to mess with epg.dat from EPGRefresh 
• 0 options set to reload epg.dat at boot
• 4m epg.dat on usb
• 22h uptime
• terribly slowed down box, still usable, all working fine

What was found:

• 47m free mem reported (see below)
• 20m freeded up by clearing the memory from the menu under information/blue button
• 0 crashlogs available
• 21m available / 786m total swap
• init 4 killed the ui while enigma2 kept being running and consuming 1003m (below)
• init 3 did not bring enigma2 to life
• reboot did not make it either
• a manual switch off/on saw the box booting with the VU+ logo, to then send the TV and the LCD a black screen and keep the buttons lighted and working, responsive to remote and touch. This state happened a few times already, however still with no crashlogs and no ssh access. Ended up re-imagining it all several times before, as the same would just happen again and again, hence the soft-brick.

Workaround found to avoid re-flashing:

• pressing the remote power button for 10 seconds triggers some sort of soft-reboot that succeeds and gets the box back to life.

After the boot, all works fine, 301m free mem is available, no EPG is loaded however it is successfully restored either manually by CrossEPG, or within the hr. After restoring, 158m free mem stays available with 0m swap use and with enigma2 at 180m usage, versus 1003m before. The EPGRefresh work would instead be lost until the next run.
 
Why
I am not sure this is a faulty box, I guess not, it seems memory use just keeps growing for reasons beyond my understanding, though, and software makes it worse with the soft-brick.
Top pinpointed the following situation before the brick, with enigma2 at 1003m being the offender. The other two major mem users are just the drivers and they consume exactly the same when the box is in a healthy state.
 

Mem: 521016K used, 46728K free, 30528K shrd, 2456K buff, 47332K cached
CPU:   0% usr   0% sys   0% nic  99% idle   0% io   0% irq   0% sirq
Load average: 0.43 0.81 0.66 1/170 24738
  PID  PPID USER     STAT   VSZ %VSZ %CPU COMMAND
 2187  1668 root     S    1003m 181%   0% /usr/bin/enigma2
23753     2 root     SW       0   0%   0% [kdvb-ad-0-fe-0]
 1012     1 root     S     551m  99%   0% /usr/bin/dvb_server
1014     1 root     S     537m  97%   0% /usr/bin/init_client /lib/modules/4.1.45-1.17/extra/dvb-bcm.ko /lib/modules/4.1.45-1.17/extra/brcmfb.ko
20646 20632 root     S    31076   5%   0% /usr/sbin/smbd
20632     1 root     S    30028   5%   0% /usr/sbin/smbd
20634 20632 root     S    28572   5%   0% {smbd-notifyd} /usr/sbin/smbd
20635 20632 root     S    28564   5%   0% {cleanupd} /usr/sbin/smbd
20637     1 root     S    19556   3%   0% /usr/sbin/nmbd

Resolution?
Is there anything that could be done to further troubleshoot and fully resolve this?

Edited by cmatte, 4 October 2019 - 01:24.

[betacentauri]

How often do you reboot your box?
Do you really import EPG data every hour? Or was it only for testing?

[WanWizard]

Not sure what you're doing, but on my Duo 4K, Enigma constantly hovers around 250m.

 

And I import EPG once a day, both XML and via EPGRefresh, for all of Europe excl. Nordics.

[cmatte]

How often do you reboot your box?
Do you really import EPG data every hour? Or was it only for testing?

The box is never rebooted and yes, CrossEPG runs every hour only if the box is in standby, however it does only with one specific DVB EPG, which it updates with the latest from the provider not available with XML, instead. Let me point out its own EPG DB does not grow out of control but stays constant~ish as it evidently does not just duplicate the data, but updates it.

 

Not sure what you're doing, but on my Duo 4K, Enigma constantly hovers around 250m.

 

And I import EPG once a day, both XML and via EPGRefresh, for all of Europe excl. Nordics.

Interesting build up, as I have only European sport/movie XMLs and EPGRefresh pulls only about 1.8k channels covering my own country and another country, and ends in a couple of hours in the morning.

EPGRefresh is set this way:

• runs automatically
• runs even if the box is active, as there's always the second tuner available because there are x2 DVB-S2 and x2 DVB-T2
• runs through a fake recording
• stays on a channel for only 20 seconds, set both in the main config and the EPG extra one
• does not load/save any EPG in cache

After my post the swap was bumped up to 2gb for testing. Now, after 12h, the box shows 

• 224m swap used
• 66m free mem
• 60m CrossEPG DB, counting 4k channels 592k events and 180k keys(not sure what's this!)
• 3 hrs ago last update from CrossEPG
• 26m epg.dat in place, last updated 3 hrs ago by CrossEPG
• 7 hrs ago last EPGRefresh run ended
• 644m mem used by enigma2
• non-functional samba shares, restarting smbd worked

I have no other background tasks or particular plugins running, two days ago following the advice only two languages are loaded in enigma2 (saves ~60m), however it all does seem to go towards the same outcome in the long run, unfortunately. Any suggestion to troubleshoot further is welcome! Thanks.

Edited by cmatte, 4 October 2019 - 15:20.

[littlesat]

Crazy to run it every-hour... once per day is sufficient and does eg not the epg servers unnecessarily! It is a ‘shame’ crossepg offers this option!

Edited by littlesat, 4 October 2019 - 15:41.

[betacentauri]

Like littlesat said don’t start EPG load every hour and reboot your box regularly.

[cmatte]

Crazy to run it every-hour... once per day is sufficient and does eg not the epg servers unnecessarily! It is a ‘shame’ crossepg offers this option!

Hey, it seems the explanation above wasn't too clear

CrossEPG does *not* download anything every hour, does *not* access any internet service or server every hour.

It *does tune* to an OpenEPG provider channel which is not just craziness, it simply catches any programme guide changes made by the provider within the day, which XML could never catch  

The EPG result after 12 hrs is still matching the initial EPG size.

 

Hope this explains it. However, okay, I will test not to tune and load OpenEPG and see if this helps at all.

Edited by cmatte, 4 October 2019 - 16:05.

[cmatte]

[...] reboot your box regularly.

Had missed this part! How do you suggest to reboot the box regularly? Why would you? Also, how to ensure, for example, the EPG runs are not interrupted? How to re-load epg at boot, both CrossEPG and EPGRefresh runs data? It seems they both try to race to modify the same cache file, and generate an enigma2 crash, which is why EPGRefresh is set not to touch the cache and since then there were no crashes and no box in standby randomly, anymore. If there's a way, happy to try!

 

I've just tried manually switching the box off from the menu and it actually does soft-brick it. This confirms that when enigma2 has grown that much in memory use, this happens, so I guess a reboot would do the same as attempted last night.

 

So, for now, CrossEPG hourly run is disabled to identify whether that is the culprit.

The workaround will be used to boot, then clear any EPG, download the XML+DVB one only, daily, with CrossEPG, keep EPGRefresh running at a different time, and see if this works

Edited by cmatte, 4 October 2019 - 16:19.

[betacentauri]

You should be able to reboot the box via a cronjob.

But you have a quite complex configuration. I only use EPG refresh and nothing else. Can't give you any hints how to setup everything.

[WanWizard]

Might also be an issue with CrossEPG, given the fact I don't have your problem. For importing and refreshing it's not needed.

[cmatte]

Yup good point, unfortunately no other plugin that I know processes both XML giving you a few days/a week of epg and OpenEPG that gives you a reliable, comprehensive, weekly, data, in just a couple of minutes, which XML can't really provide. However I agree, it shall be considered to just switch to EPGImport for XML (never tried, is it the best choice?) and EPGRefresh for missing data. This is the next step if we can identify that's the root cause but can't resolve it.

By the way, is CrossEPG development active?

Edited by cmatte, 4 October 2019 - 18:41.

[WanWizard]

We don't do anything with CrossEPG, don't have a clue.

 

The EPGimport plugin imports XMLTV format files, and comes with a long list of sources (which I bet are the same as in CrossEPG). EPGRefresh does what it says on the tin.

 

Not sure what you mean with OpenEPG, if you mean OpenTV EPG (aka SKY EPG), that is going to be supported in 7.2-release, so you can use EPGRefresh to update that as well.

[cmatte]

...and this comes right after I thought to test 7.2, wow  

 

Ok, thanks for the input! I'll keep testing without the hourly for a while more, for now I have good feelings with e2 still at 230m, and then when possible test 7.2 and, well, probably forget about CrossEPG!

[cmatte]

Heads up, 7.2 now under testing, as enigma2 had still bumped up to 500m usage for unknown reasons, even without CrossEPG running hourly. On the bright side, it did reboot normally and is back at 230m as it should.

Anyway, with 7.2, I will drop CrossEPG and use EPGImport, then report back. 

 

Just a thought, might picons be affecting enigma2 ram usage this much? Like it might keep loading more and more in memory until it's too much? Well, even if the receiver is barely used, as in the last 48 hrs. On the other hand, they are updated and quite a lot are there available to use since when I installed 7.1, so, just wondering...

Edited by cmatte, 5 October 2019 - 23:30.

[WanWizard]

Yes, Enigma caches images it loads. But unless you have some bad-ass images, I doubt that will cost you 270m.

 

Mine has an update of 6 days now, and is using 337m. And I'm using picons too...

[cmatte]

Not sure, the biggest picon package in the feeds is installed, both versions srp snp, however picons have been piled up over the years in this folder and it sums up to a nonsense 42k files with 374MB size and 41.2 GB size on disk - wow - why this crazy difference? Maybe this one needs some proper housekeeping

[Erik Slagter]

About memory usage: RAM is always shared between the operating system (Linux / Enigma) and the DVB part of the SoC. No single receiver has all RAM available for Linux. It's not required too, enough RAM is left to comfortably run enigma, unless memory gets fragmented, which has been an issue since Linux kernel 2.6. It's not the kernel that's at fault, but apparently it has gotten quiet a bit more difficult to write proper functioning DVB drivers for those kernels, which can handle physically fragmented memory.

 

The SoC uses the RAM for the demuxers (temporary storage of elementary streams to line them up), the OSD layer, the frame buffer and storage for the decoders. Especially for UHD receivers A LOT of memory is required. And A LOT is left for the kernel. Don't compare it to your PC.