Jump to content


Photo

OpenPli 7 and DM8000: out of memory


  • Please log in to reply
4 replies to this topic

#1 stdemonac

  • Senior Member
  • 38 posts

0
Neutral

Posted 15 September 2019 - 14:54

Hello,

Since OpenPli 7, my DM8000 (now under OpenPLI 7.1 release 20190902) has frequent instability problems.

After a reboot, everything runs fine, but after one or two days, some problems are coming up.
For example:

  • When I check system updates, the progress bar at the bottom of the screen becomes jerky, stops and the box reboots.
  • During nightly EPGImport updates, the box often crashes and reboot.
  • Same problen when I take a screenshot (with a telnet 'grab -d -p /tmp/screenshot.png' or OpenWebif), or when I start to watch or start a recording.

No enigma2_crash.log file is generated. Very few plugins are installed.

Here are the /var/log/messages entries for a failed grab:

... a lot of lines like these ones (1200 in 10 minutes) ...
Sep 11 22:41:22 dm8000 user.warn kernel: audio_status_isr: 47 callbacks suppressed
Sep 11 22:41:22 dm8000 user.warn kernel: eModeChange Isr.. set source_changed_event!
Sep 11 22:41:22 dm8000 user.warn kernel: eModeChange Isr.. set source_changed_event!
Sep 11 22:41:23 dm8000 user.warn kernel: eModeChange Isr.. set source_changed_event!
Sep 11 22:41:29 dm8000 user.warn kernel: eModeChange Isr.. set source_changed_event!
Sep 11 22:41:29 dm8000 user.warn kernel: eModeChange Isr.. set source_changed_event!
...
Sep 11 22:42:27 dm8000 user.warn kernel: audio_status_isr: 22 callbacks suppressed
Sep 11 22:42:27 dm8000 user.warn kernel: eModeChange Isr.. set source_changed_event!
Sep 11 22:42:27 dm8000 user.warn kernel: eModeChange Isr.. set source_changed_event!
Sep 11 22:42:27 dm8000 user.warn kernel: eModeChange Isr.. set source_changed_event!
Sep 11 22:42:27 dm8000 user.warn kernel: eModeChange Isr.. set source_changed_event!
Sep 11 22:42:27 dm8000 user.warn kernel: eModeChange Isr.. set source_changed_event!
Sep 11 22:42:27 dm8000 user.warn kernel: eModeChange Isr.. set source_changed_event!
Sep 11 22:42:42 dm8000 user.warn kernel: eModeChange Isr.. set source_changed_event!
Sep 11 22:42:42 dm8000 user.warn kernel: eModeChange Isr.. set source_changed_event!
Sep 11 22:42:44 dm8000 user.warn kernel: enigma2 invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
Sep 11 22:42:44 dm8000 user.warn kernel: Call Trace:
Sep 11 22:42:44 dm8000 user.warn kernel: [<804b8a58>] dump_stack+0x8/0x34
Sep 11 22:42:44 dm8000 user.warn kernel: [<80084c48>] dump_header.isra.4+0x84/0x1a0
Sep 11 22:42:44 dm8000 user.warn kernel: [<80084f98>] oom_kill_process.isra.9+0xb4/0x2b8
Sep 11 22:42:44 dm8000 user.warn kernel: [<800855e8>] out_of_memory+0x230/0x398
Sep 11 22:42:44 dm8000 user.warn kernel: [<80088d3c>] __alloc_pages_nodemask+0x6c4/0x6d8
Sep 11 22:42:44 dm8000 user.warn kernel: [<800834e4>] filemap_fault+0x240/0x4d8
Sep 11 22:42:44 dm8000 user.warn kernel: [<8009e8d4>] __do_fault+0x8c/0x4d8
Sep 11 22:42:44 dm8000 user.warn kernel: [<800a1d1c>] handle_pte_fault+0x9c/0x870
Sep 11 22:42:44 dm8000 user.warn kernel: [<800a2580>] handle_mm_fault+0x90/0xe8
Sep 11 22:42:44 dm8000 user.warn kernel: [<8000e788>] do_page_fault+0xb8/0x390
Sep 11 22:42:44 dm8000 user.warn kernel: [<80004600>] ret_from_exception+0x0/0x28
Sep 11 22:42:44 dm8000 user.warn kernel: Mem-Info:
Sep 11 22:42:44 dm8000 user.warn kernel: Normal per-cpu:
Sep 11 22:42:44 dm8000 user.warn kernel: CPU    0: hi:   90, btch:  15 usd:  22
Sep 11 22:42:44 dm8000 user.warn kernel: CPU    1: hi:   90, btch:  15 usd:  14
Sep 11 22:42:44 dm8000 user.warn kernel: active_anon:30116 inactive_anon:571 isolated_anon:0
Sep 11 22:42:44 dm8000 user.warn kernel:  active_file:42 inactive_file:20 isolated_file:32
Sep 11 22:42:44 dm8000 user.warn kernel:  unevictable:0 dirty:0 writeback:0 unstable:0
Sep 11 22:42:44 dm8000 user.warn kernel:  free:506 slab_reclaimable:772 slab_unreclaimable:2377
Sep 11 22:42:44 dm8000 user.warn kernel:  mapped:2 shmem:769 pagetables:184 bounce:0
Sep 11 22:42:44 dm8000 user.warn kernel: Normal free:2024kB min:2036kB low:2544kB high:3052kB active_anon:120464kB inactive_anon:2284kB active_file:96kB inactive_file:208kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:260096kB mlocked:0kB dirty:0kB wr
Sep 11 22:42:44 dm8000 user.warn kernel: lowmem_reserve[]: 0 0
Sep 11 22:42:44 dm8000 user.warn kernel: Normal: 214*4kB 72*8kB 13*16kB 12*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2024kB
Sep 11 22:42:44 dm8000 user.warn kernel: 862 total pagecache pages
Sep 11 22:42:44 dm8000 user.warn kernel: 0 pages in swap cache
Sep 11 22:42:44 dm8000 user.warn kernel: Swap cache stats: add 0, delete 0, find 0/0
Sep 11 22:42:44 dm8000 user.warn kernel: Free swap  = 0kB
Sep 11 22:42:44 dm8000 user.warn kernel: Total swap = 0kB
Sep 11 22:42:44 dm8000 user.warn kernel: 65536 pages RAM
Sep 11 22:42:44 dm8000 user.warn kernel: 28213 pages reserved
Sep 11 22:42:44 dm8000 user.warn kernel: 796 pages shared
Sep 11 22:42:44 dm8000 user.warn kernel: 35753 pages non-shared
Sep 11 22:42:44 dm8000 user.info kernel: [ pid ]   uid  tgid total_vm      rss cpu oom_adj oom_score_adj name
Sep 11 22:42:44 dm8000 user.info kernel: [  537]   999   537      809       42   0       0             0 dbus-daemon
Sep 11 22:42:44 dm8000 user.info kernel: [  544]     0   544      684       28   0       0             0 dropbear
Sep 11 22:42:44 dm8000 user.info kernel: [  548]   998   548      643       49   1       0             0 rpcbind
Sep 11 22:42:44 dm8000 user.info kernel: [  553]   996   553      795      146   1       0             0 rpc.statd
Sep 11 22:42:44 dm8000 user.info kernel: [  557]     0   557      857       29   1       0             0 inetd
Sep 11 22:42:44 dm8000 user.info kernel: [  577]     0   577      855      120   0       0             0 rpc.mountd
Sep 11 22:42:44 dm8000 user.info kernel: [  595]     0   595     8299      369   1       0             0 smbd
Sep 11 22:42:44 dm8000 user.info kernel: [  597]     0   597     7934      356   1       0             0 smbd-notifyd
Sep 11 22:42:44 dm8000 user.info kernel: [  598]     0   598     7932      356   1       0             0 cleanupd
Sep 11 22:42:44 dm8000 user.info kernel: [  600]     0   600     5384      282   0       0             0 nmbd
Sep 11 22:42:44 dm8000 user.info kernel: [  607]     0   607      527       19   0       0             0 wsdd
Sep 11 22:42:44 dm8000 user.info kernel: [  612]     0   612      759       19   0       0             0 syslogd
Sep 11 22:42:44 dm8000 user.info kernel: [  614]     0   614      759       22   0       0             0 klogd
Sep 11 22:42:44 dm8000 user.info kernel: [  621]   997   621      975       63   0       0             0 avahi-daemon
Sep 11 22:42:44 dm8000 user.info kernel: [  622]   997   622      943       40   1       0             0 avahi-daemon
Sep 11 22:42:44 dm8000 user.info kernel: [  625]     0   625     1747       61   0       0             0 oscam
Sep 11 22:42:44 dm8000 user.info kernel: [  626]     0   626     3721      686   0       0             0 oscam
Sep 11 22:42:44 dm8000 user.info kernel: [23619]     0 23619      759       20   1       0             0 enigma2.sh
Sep 11 22:42:44 dm8000 user.info kernel: [31574]     0 31574      748       51   1       0             0 dropbear
Sep 11 22:42:44 dm8000 user.info kernel: [31628]     0 31628      823       33   0       0             0 sh
Sep 11 22:42:44 dm8000 user.info kernel: [ 2990]     0  2990    38646    22625   1       0             0 enigma2
Sep 11 22:42:44 dm8000 user.info kernel: [13126]     0 13126      759       19   1       0             0 tail
Sep 11 22:42:44 dm8000 user.info kernel: [13169]     0 13169      748       48   1       0             0 dropbear
Sep 11 22:42:44 dm8000 user.info kernel: [13190]     0 13190      823       32   0       0             0 sh
Sep 11 22:42:44 dm8000 user.info kernel: [16679]     0 16679    12705     5099   0       0             0 grab
Sep 11 22:42:44 dm8000 user.err kernel: Out of memory: Kill process 2990 (enigma2) score 589 or sacrifice child
Sep 11 22:42:44 dm8000 user.err kernel: Killed process 2990 (enigma2) total-vm:154584kB, anon-rss:90496kB, file-rss:4kB
Sep 11 22:42:44 dm8000 user.warn kernel: stop pcr0 on dmx 0 pid:a3
Sep 11 22:42:44 dm8000 user.warn kernel: stop audio on dmx 0 pid:5c
Sep 11 22:42:44 dm8000 user.warn kernel: audio: xpt stop(1)
Sep 11 22:42:44 dm8000 user.warn kernel: audio: decoder stop
Sep 11 22:42:44 dm8000 user.warn kernel: CI stopped...
Sep 11 22:42:44 dm8000 user.warn kernel: CI stopped...
Sep 11 22:42:44 dm8000 user.warn kernel: bcm_audio_channel_stop(1) done
Sep 11 22:42:44 dm8000 user.warn kernel: stop video0 on dmx 0 pid:a3
Sep 11 22:42:44 dm8000 user.warn kernel: VIDEO0: stop decode
Sep 11 22:42:44 dm8000 user.warn kernel: no decoder left in sync handling so disable xpt_sync
Sep 11 22:42:45 dm8000 user.warn kernel: VIDEO0: set streamtype!
Sep 11 22:42:45 dm8000 user.warn kernel: set H264
Sep 11 22:42:45 dm8000 user.warn kernel: sw rave disabled!
Sep 11 22:42:45 dm8000 user.warn kernel: VIDEO0: set blank to 0
Sep 11 22:42:45 dm8000 user.warn kernel: sync is off so we enable it
Sep 11 22:42:45 dm8000 user.warn kernel: playback!
Sep 11 22:42:45 dm8000 user.warn kernel: total video delay: 0 (0 ms)
Sep 11 22:42:45 dm8000 user.warn kernel: VIDEO0: start decode
Sep 11 22:42:45 dm8000 user.warn kernel: flush video decoder
Sep 11 22:42:45 dm8000 user.warn kernel: VIDEO0 use saved pts 0xbcd997f4 (ret 0)
Sep 11 22:42:45 dm8000 user.warn kernel: VIDEO0: first pts 0x0
Sep 11 22:42:45 dm8000 user.warn kernel: VIDEO0: resolution changed (1920 1080)
Sep 11 22:42:45 dm8000 user.warn kernel: VIDEO0: set blank to 0
Sep 11 22:42:45 dm8000 user.warn kernel: VIDEO0: stop decode
Sep 11 22:42:45 dm8000 user.warn kernel: VIDEO0: set blank to 0
Sep 11 22:42:45 dm8000 user.warn kernel: no decoder left in sync handling so disable xpt_sync
Sep 11 22:42:47 dm8000 user.err kernel: bcm_accel: invalid opcode! 00000080
Sep 11 22:42:47 dm8000 user.info kernel: bcmfb: enable manual blit
Sep 11 22:42:47 dm8000 user.warn kernel: tsmux set cisource reg 00 to val 44
Sep 11 22:42:47 dm8000 user.warn kernel: tsmux set cisource reg 00 to val 44
Sep 11 22:42:47 dm8000 user.warn kernel: tsmux set cisource reg 01 to val 34
Sep 11 22:42:47 dm8000 user.warn kernel: tsmux set cisource reg 01 to val 44
Sep 11 22:42:47 dm8000 user.warn kernel: tsmux set tsports reg 02 to val dc
Sep 11 22:42:47 dm8000 user.warn kernel: tsmux set tsports reg 02 to val dc
Sep 11 22:42:47 dm8000 user.warn kernel: tsmux set tsports reg 03 to val fe
Sep 11 22:42:47 dm8000 user.warn kernel: tsmux set tsports reg 03 to val fe
...

Here are those from another crash :

Sep 15 15:19:11 dm8000 user.info kernel: bcm4501: channel 1 is sleeping now
Sep 15 15:19:13 dm8000 user.warn kernel: grab invoked oom-killer: gfp_mask=0x200da, order=0, oom_adj=0, oom_score_adj=0
Sep 15 15:19:14 dm8000 user.warn kernel: Call Trace:
Sep 15 15:19:14 dm8000 user.warn kernel: [<804b8a58>] dump_stack+0x8/0x34
Sep 15 15:19:14 dm8000 user.warn kernel: [<80084c48>] dump_header.isra.4+0x84/0x1a0
Sep 15 15:19:14 dm8000 user.warn kernel: [<80084f98>] oom_kill_process.isra.9+0xb4/0x2b8
Sep 15 15:19:14 dm8000 user.warn kernel: [<800855e8>] out_of_memory+0x230/0x398
Sep 15 15:19:14 dm8000 user.warn kernel: [<80088d3c>] __alloc_pages_nodemask+0x6c4/0x6d8
Sep 15 15:19:14 dm8000 user.warn kernel: [<800a2364>] handle_pte_fault+0x6e4/0x870
Sep 15 15:19:14 dm8000 user.warn kernel: [<800a2580>] handle_mm_fault+0x90/0xe8
Sep 15 15:19:14 dm8000 user.warn kernel: [<8000e788>] do_page_fault+0xb8/0x390
Sep 15 15:19:14 dm8000 user.warn kernel: [<80004600>] ret_from_exception+0x0/0x28
Sep 15 15:19:14 dm8000 user.warn kernel: Mem-Info:
Sep 15 15:19:14 dm8000 user.warn kernel: Normal per-cpu:
Sep 15 15:19:14 dm8000 user.warn kernel: CPU    0: hi:   90, btch:  15 usd:  22
Sep 15 15:19:14 dm8000 user.warn kernel: CPU    1: hi:   90, btch:  15 usd:  11
Sep 15 15:19:14 dm8000 user.warn kernel: active_anon:30708 inactive_anon:28 isolated_anon:0
Sep 15 15:19:14 dm8000 user.warn kernel:  active_file:34 inactive_file:41 isolated_file:0
Sep 15 15:19:14 dm8000 user.warn kernel:  unevictable:0 dirty:0 writeback:0 unstable:0
Sep 15 15:19:14 dm8000 user.warn kernel:  free:497 slab_reclaimable:773 slab_unreclaimable:2371
Sep 15 15:19:14 dm8000 user.warn kernel:  mapped:0 shmem:76 pagetables:187 bounce:0
Sep 15 15:19:14 dm8000 user.warn kernel: Normal free:1988kB min:2036kB low:2544kB high:3052kB active_anon:122832kB inactive_anon:112kB active_file:136kB inactive_file:164kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:260096kB mlocked:0kB dirty:0kB wr
Sep 15 15:19:14 dm8000 user.warn kernel: lowmem_reserve[]: 0 0
Sep 15 15:19:14 dm8000 user.warn kernel: Normal: 121*4kB 22*8kB 35*16kB 14*32kB 3*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1988kB
Sep 15 15:19:14 dm8000 user.warn kernel: 146 total pagecache pages
Sep 15 15:19:14 dm8000 user.warn kernel: 0 pages in swap cache
Sep 15 15:19:14 dm8000 user.warn kernel: Swap cache stats: add 0, delete 0, find 0/0
Sep 15 15:19:14 dm8000 user.warn kernel: Free swap  = 0kB
Sep 15 15:19:14 dm8000 user.warn kernel: Total swap = 0kB
Sep 15 15:19:14 dm8000 user.warn kernel: 65536 pages RAM
Sep 15 15:19:14 dm8000 user.warn kernel: 28213 pages reserved
Sep 15 15:19:14 dm8000 user.warn kernel: 787 pages shared
Sep 15 15:19:14 dm8000 user.warn kernel: 35772 pages non-shared
Sep 15 15:19:14 dm8000 user.info kernel: [ pid ]   uid  tgid total_vm      rss cpu oom_adj oom_score_adj name
Sep 15 15:19:14 dm8000 user.info kernel: [  537]   999   537      809       42   1       0             0 dbus-daemon
Sep 15 15:19:14 dm8000 user.info kernel: [  544]     0   544      684       28   1       0             0 dropbear
Sep 15 15:19:14 dm8000 user.info kernel: [  548]   998   548      643       49   0       0             0 rpcbind
Sep 15 15:19:14 dm8000 user.info kernel: [  553]   996   553      795      146   1       0             0 rpc.statd
Sep 15 15:19:14 dm8000 user.info kernel: [  557]     0   557      857       28   0       0             0 inetd
Sep 15 15:19:14 dm8000 user.info kernel: [  577]     0   577      855      121   1       0             0 rpc.mountd
Sep 15 15:19:14 dm8000 user.info kernel: [  595]     0   595     8298      369   0       0             0 smbd
Sep 15 15:19:14 dm8000 user.info kernel: [  597]     0   597     7934      355   1       0             0 smbd-notifyd
Sep 15 15:19:14 dm8000 user.info kernel: [  598]     0   598     7932      355   1       0             0 cleanupd
Sep 15 15:19:14 dm8000 user.info kernel: [  600]     0   600     5384      283   0       0             0 nmbd
Sep 15 15:19:14 dm8000 user.info kernel: [  607]     0   607      527       21   1       0             0 wsdd
Sep 15 15:19:14 dm8000 user.info kernel: [  611]     0   611      759       19   0       0             0 syslogd
Sep 15 15:19:14 dm8000 user.info kernel: [  614]     0   614      759       22   1       0             0 klogd
Sep 15 15:19:14 dm8000 user.info kernel: [  621]   997   621      943       54   1       0             0 avahi-daemon
Sep 15 15:19:14 dm8000 user.info kernel: [  622]   997   622      943       40   0       0             0 avahi-daemon
Sep 15 15:19:14 dm8000 user.info kernel: [  625]     0   625     1747       61   0       0             0 oscam
Sep 15 15:19:14 dm8000 user.info kernel: [  626]     0   626     3510      544   1       0             0 oscam
Sep 15 15:19:14 dm8000 user.info kernel: [32009]     0 32009      759       20   1       0             0 enigma2.sh
Sep 15 15:19:14 dm8000 user.info kernel: [32015]     0 32015    43770    25137   0       0             0 enigma2
Sep 15 15:19:14 dm8000 user.info kernel: [14241] 65534 14241      671       36   1       0             0 vsftpd
Sep 15 15:19:14 dm8000 user.info kernel: [14319]     0 14319      680       48   0       0             0 vsftpd
Sep 15 15:19:14 dm8000 user.info kernel: [15077]     0 15077    13922     3575   1       0             0 grab
Sep 15 15:19:14 dm8000 user.err kernel: Out of memory: Kill process 32015 (enigma2) score 654 or sacrifice child
Sep 15 15:19:14 dm8000 user.err kernel: Killed process 15077 (grab) total-vm:55688kB, anon-rss:14300kB, file-rss:0kB

Do other Dreambox 8000 users of this forum experience the same problems ?
Do I need to go back to the more stable OpenPli 6 ?

What tools can I use to get more details of what's going wrong ?

 

Thanks.
 


DM8000 - 2 DVB-S2 tuners - 1 DVB-C tuner - OpenPLi 7.1


Re: OpenPli 7 and DM8000: out of memory #2 littlesat

  • PLi® Core member
  • 49,224 posts

+527
Excellent

Posted 15 September 2019 - 15:00

This old ‘lady’ is reaching the limits. I can imagine when eg you load too many epg and/or install too many plug-ins the memory is full quickly...
Maybe arrange a new box for around 100 box or so is an alternative solution....

Edited by littlesat, 15 September 2019 - 15:00.

WaveFrontier 28.2E | 23.5E | 19.2E | 16E | 13E | 10/9E | 7E | 5E | 1W | 4/5W | 15W


Re: OpenPli 7 and DM8000: out of memory #3 WanWizard

  • Forum Moderator
    PLi® Core member
  • 48,468 posts

+824
Excellent

Posted 15 September 2019 - 15:36

oom-killer = box out of memory.

 

Limit the amount of EPG you import might be a good start, as all EPG is stored in memory.


Currently in use: VU+Duo 4K (2xFBC S2), Amiko Viper T2C (T2), SAB Alpha Triple HD (S2+T2), Zgemma H3.T2C (T/C), Zgemma H6 (fallback), VU+Zero (fallback)

Many answers to your question can be found in our new and improved wiki.

note: I do not provide support via PM !

 


Re: OpenPli 7 and DM8000: out of memory #4 Erik Slagter

  • PLi® Core member
  • 45,353 posts

+498
Excellent

Posted 20 September 2019 - 18:11

The newest driver / kernel have memory fragmentation issues. A daily reboot works quite well to avoid these.


* Wavefrontier T90 with 28E/23E/19E/13E/9E/4.8E/0.8W/5W via SCR switches 2 x 2 x 6 user bands
* Ziggo digital cable TV (FTA)
I don't read PM -> if you have something to ask or to report, do it in the forum so others can benefit. I don't take freelance jobs.
Ik lees geen PM -> als je iets te vragen of te melden hebt, doe het op het forum, zodat anderen er ook wat aan hebben.

Re: OpenPli 7 and DM8000: out of memory #5 stdemonac

  • Senior Member
  • 38 posts

0
Neutral

Posted 21 September 2019 - 10:16

A daily reboot works quite well to avoid these.

 
That's exactly what I've done.
 
Installed busybox-cron, ...

root@dm8000:~# opkg install busybox-cron

...and setup a reboot just before the loading of EPGImport scheduled @7:33 :

root@dm8000:~# crontab -l
30 7 * * * init 4 && sleep 10 && rm /media/cf/epg.dat && sleep 10 && init 3
32 7 * * * wget -O /dev/null -q http://root:<password>@localhost/web/powerstate?newstate=0

The situation seems more stable. To be confirmed over time.


DM8000 - 2 DVB-S2 tuners - 1 DVB-C tuner - OpenPLi 7.1





0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users