Hello,
I was talking to phunkyfish earlier about Kodi addon for enigma2 and seems there's a problem in all Open images and that's in MovieList.py
Please check https://github.com/E...ebif/issues/751
Geplaatst op 30 november 2020 - 16:26
Hello,
I was talking to phunkyfish earlier about Kodi addon for enigma2 and seems there's a problem in all Open images and that's in MovieList.py
Please check https://github.com/E...ebif/issues/751
Open Vision sources: https://github.com/OpenVisionE2
Geplaatst op 30 november 2020 - 16:30
Veranderd door Abu Baniaz, 30 november 2020 - 16:33
Geplaatst op 30 november 2020 - 16:45
Well I don't have access to that section as I'm not a developer/contributor probably but lets focus on this:
I have made a copy of MovieList.py from enigma2 and then i have removed all the GUI parts.
I think the leak is in the GUI part of MovieList.py.
by jbleyel
Open Vision sources: https://github.com/OpenVisionE2
Geplaatst op 30 november 2020 - 16:47
@Abu Baniaz,
Is this related to the issue Erik and Simon are looking at?
Currently in use: VU+ Duo 4K (2xFBC S2), VU+ Solo 4K (1xFBC S2), uClan Usytm 4K Pro (S2+T2), Octagon SF8008 (S2+T2), Zgemma H9.2H (S2+T2)
Due to my bad health, I will not be very active at times and may be slow to respond. I will not read the forum or PM on a regular basis.
Many answers to your question can be found in our new and improved wiki.
Geplaatst op 30 november 2020 - 17:04
Not sure, maybe I should have kept quiet . filepush is a denominator.
I don't think any of the regular E2 testers use their E2 boxes as "Kodi box" sources (setup that Phunkyfish uses regularly) so do not experience the issue regularly. The issue seems to affect streaming to other boxes too, which is more likely to affect more E2 testers.
Thread on Vix here where Phunkyfish started a thread:
https://www.world-of-satellite.com/showthread.php?63317-Possible-memory-leak-need-help-on-how-to-figure-it-out
He has confirmed leak is in movielist.py
The commits Simon has made to ViX have not been pushed to Public image, so Phunkyfish cant test those.
Geplaatst op 30 november 2020 - 17:37
The issue we're working on is a deadlock, not a memory leak.
* Wavefrontier T90 with 28E/23E/19E/13E via SCR switches 2 x 2 x 6 user bands
I don't read PM -> if you have something to ask or to report, do it in the forum so others can benefit. I don't take freelance jobs.
Ik lees geen PM -> als je iets te vragen of te melden hebt, doe het op het forum, zodat anderen er ook wat aan hebben.
Geplaatst op 6 december 2020 - 09:33
Since MovieList is all Python code, memory "leak" can only be caused by some client keeping a reference to some object alive.
I'm unfamiliar with the webinterface, I lost track of how it works long ago.
It would be really heplful to know which call is causing it.
Geplaatst op 6 december 2020 - 10:24
I'd very interested in how was established that there is memory and it's exactly there, in MoveList.py. I am a bit sceptical for the moment.
* Wavefrontier T90 with 28E/23E/19E/13E via SCR switches 2 x 2 x 6 user bands
I don't read PM -> if you have something to ask or to report, do it in the forum so others can benefit. I don't take freelance jobs.
Ik lees geen PM -> als je iets te vragen of te melden hebt, doe het op het forum, zodat anderen er ook wat aan hebben.
Geplaatst op 6 december 2020 - 11:20
Veranderd door betacentauri, 6 december 2020 - 11:21
Geplaatst op 6 december 2020 - 11:55
How is determined that memory usage increases? As most, well-known sources of memory usage, are actually not accurate at all (I think of "free" and "ps" statistics).
* Wavefrontier T90 with 28E/23E/19E/13E via SCR switches 2 x 2 x 6 user bands
I don't read PM -> if you have something to ask or to report, do it in the forum so others can benefit. I don't take freelance jobs.
Ik lees geen PM -> als je iets te vragen of te melden hebt, doe het op het forum, zodat anderen er ook wat aan hebben.
Geplaatst op 6 december 2020 - 17:49
That could very well be buggy drivers, they're all over the place.
Simple test: restart enigma every day and see if the memory no longer runs out (not by consulting top, but by the lack of (hopefully) hangs). I bet the issue remains, even when enigma is restarted frequently (which would release all of it's memory).
* Wavefrontier T90 with 28E/23E/19E/13E via SCR switches 2 x 2 x 6 user bands
I don't read PM -> if you have something to ask or to report, do it in the forum so others can benefit. I don't take freelance jobs.
Ik lees geen PM -> als je iets te vragen of te melden hebt, doe het op het forum, zodat anderen er ook wat aan hebben.
Geplaatst op 12 december 2020 - 12:23
Here a MovieList object is created:
https://github.com/E.../movies.py#L150
A few lines later load() is called. It fills the movielist.list.
The movielist.list contains pointers to info(created in c++ code) objects. The info objects are ePtr.
At the end of the procedure "del movielist" is called. From my point of view this should also implicitly call something like this "del movielist.list". Or?
But when I replace at the end of the procedure "del movielist" with ""del movielist.list" memory consumption (according to top) doesn't grow much any longer. So it looks like a big difference.
Why does a "del movielist" don't free the objects in the movielist.list?
(If really needed, I can add debug statements in the constructor/destructor of the info object to confirm my observation)
Geplaatst op 12 december 2020 - 13:56
MovieList is a GUI thingy, so that might make it behave strange when there's no actual GUI to display. Probably deleting that "list" object makes it release things a bit earlier.
The "del movielist" at the end is a no-op, it doesn't actually do anything. The method ends there, so all variables go out of scope there anyway and the whole local namespace gets deleted there anyway.
"del someobject" does not call the objects destructor, it just removes the local reference "someobject" from the namespace. It will call the destructor then if this happens to be the last reference to it.
(Example: a=[1,2]; b=a; del a; This will delete "a" but not delete the actual array since b still has a reference to it)
Usually objects not getting freed when you expect it are a result of cyclic references. In Python, the garbage collector will find and destroy such objects. but it's hard to predict when that will run. Sometimes, something is keeping a reference alive. Tricky to find.
Geplaatst op 12 december 2020 - 14:16
Yes, that's what I also thought.
I don't think that waiting until the garbage collector has made his job, helps here. Even when I wait minutes there is no difference. So most likely there is still a reference alive. But I currently don't see where.
Geplaatst op 12 december 2020 - 14:49
I made a little change in MovieList.py. I changed this line https://github.com/O...vieList.py#L635 to
info = None
That means justStubInfo object is used everytime.
Then I have added a
print("RefCount", sys.getrefcount(justStubInfo))
here: https://github.com/E.../movies.py#L237
(and an import for sys and justStubInfo is needed)
I can see that refcount grows and grows with each call of the website.
When change it to "del movielist.list" ref count stays the same. So there is really a problem.
Geplaatst op 12 december 2020 - 16:54
I found the problem. MovieList creates c++ object
self.l = eListboxPythonMultiContent()
and gives a reference to self.buildMovieListEntry to it
self.l.setBuildFunc(self.buildMovieListEntry)
MovieList will not be destroyed until the reference is gone, but that will never happen as it references to itself.
As confirmation I used this and refcount don't grow anymore:
movielist.l.setBuildFunc(None)
How to solve this without manually call "movielist.l.setBuildFunc(None)" ???
Veranderd door betacentauri, 12 december 2020 - 16:54
Geplaatst op 12 december 2020 - 17:29
Usually the best way to break a self-ref chain is to have a third level, e.g. typically with subscriptions:
a.parent = b
b.subscriptions.append(a)
Now there's a cyclic dependency. But 'a' must store 'b' to be able to unsubscribe.
One way to solve:
a.parent = b.subscriptions
This way, a doesn't increase the refcount to b, and a can still unsubscribe when it goes out of scope.
In this particular case though, it's not as easy to break. A weak ref would help yes. Another thing might be to call self.l.setBuildFunc(...) only when needed, and set it back to None again when not building a list (in the load() function maybe?)
(I'm not a big fan of weakref, but in this case, it's the lesser evil compared to keeping that cylic dependency in. Probably the issue here is the refcounting done in the C++ part, which does not have a garbage-collector or any other way to detect cyclic dependencies)
Veranderd door MiLo, 12 december 2020 - 17:32
Geplaatst op 12 december 2020 - 18:00
Another thing might be to call self.l.setBuildFunc(...) only when needed, and set it back to None again when not building a list (in the load() function maybe?)
Don't know whether this is possible easily. I guess build function is also called when the list changes (when it's called e.g. via the MovieSelection screen). And that can also happen in the c++ part.
So in this specific case it would maybe work, but in general it's not my favorite solution as also other components are affected: EpgList, TimerList, GraphMultiEpg
So I will create a patch tomorrow or on Monday which uses weakrefs.
0 leden, 1 bezoekers, 0 anonieme gebruikers