Jump to content


Photo

Problem of using threading with enigma


  • Please log in to reply
17 replies to this topic

#1 mfaraj57

  • Senior Member
  • 1,605 posts

+286
Excellent

Posted 29 April 2019 - 10:53

There are precautions about using threading with enigma and enigma experts advice to avoid using threading but if we follow some rules as experts said regarding using threading outside main thread we can use threading safely.

The following code demonstrates not real plugin to run simple counter thread just to simplify the problem.

although the counter runs well and can stop it at any moment but has problem in showing another screen if i want in opensource images and start enigma without error notification in dreamos enigma

what is the proper way to use the following simple thread?

screenshot-1113.png

from Screens.Screen import Screen
from Screens.MessageBox import MessageBox
from Components.ActionMap import NumberActionMap,ActionMap
from Components.Pixmap import Pixmap
from Components.Label import Label
from Plugins.Plugin import PluginDescriptor
import threading
import time


####

class Test(Screen):
	skin = """
		<screen name="myscreen" position="center,center" size="1000,700" title="Test thread" >
			<widget name="text" position="250,100" size="300,100" font="Regular;22" />			
                        <eLabel  position = "250,370"   zPosition = "4"   size = "300,25"  halign = "center"  font = "Regular;24"  transparent = "0" foregroundColor = "white" backgroundColor = "green" text = "start counter" />
			<eLabel  position = "250,470"   zPosition = "4"   size = "300,25"  halign = "center"  font = "Regular;24"  transparent = "0" foregroundColor = "white" backgroundColor = "red" text = "stop counter" />
			
		</screen>"""
	
	def __init__(self, session):

		Screen.__init__(self, session)
		
	        self['text']=Label('Press green to start counter ')
		self["actions"] = ActionMap(['ColorActions',"WizardActions" ],
		{			
			"back": self.close,
                        "green": self.startthread,                        
                        "red": self.stopcounter,
			"ok": self.close,
		}, -1)
               
                self.exitcounter=False
                
        def startthread(self):
                 
                self.active_thread = threading.Thread(target=self.startcounter, args=())
                self.active_thread.daemon = True
                self.active_thread.start()
                
        def startcounter(self):
          self['text'].setText('Counter started')
          i=0
          while i<60:
              i=i+1
              print str(i)
              time.sleep(1)
              
              self['text'].setText(str(i))
              if self.exitcounter:
                  self.exitcounter=False
                  
                  break
          self['text'].setText('Counter stopped')
            
        def stopcounter(self):
            self.exitcounter=True  

def main(session, **kwargs):
	session.open(Test)
	
def menu(menuid, **kwargs):
    if menuid == 'mainmenu':
        return [(_('testPlugin'),
          main,
          'testPlugin_mainmenu',
          1)]
    return []

def Plugins(**kwargs):
    list = []
    try:
        
            list.append(PluginDescriptor(icon='plugin.png', name='testPlugin', description='test enigma', where=PluginDescriptor.WHERE_MENU, fnc=menu))
    except:
        list.append(PluginDescriptor(icon='plugin.png', name='testPlugin', description='test enigma', where=PluginDescriptor.WHERE_MENU, fnc=menu))

    list.append(PluginDescriptor(icon='plugin.png', name='testPlugin', description='test enigma', where=PluginDescriptor.WHERE_PLUGINMENU, fnc=main))
    return list



Re: Problem of using threading with enigma #2 betacentauri

  • PLi® Core member
  • 7,185 posts

+323
Excellent

Posted 29 April 2019 - 18:37

Afaik (I’m no GUI expert) is calculating something in a thread totally fine. But you shouldn’t access GUI elements from the thread.
So you shouldn’t do this in the thread:
self['text'].setText(str(i))
Xtrend ET-9200, ET-8000, ET-10000, OpenPliPC on Ubuntu 12.04

Re: Problem of using threading with enigma #3 mfaraj57

  • Senior Member
  • 1,605 posts

+286
Excellent

Posted 29 April 2019 - 19:26

Thanks for reply

I know this limitation but i want to know how to pass the data from the thread to the GUI element.also to know when completed

my code runs the thread  asynchronous so i did not read about thread_completed event if exists also using while loop is not practical and loosing asynchronous property of the thread

as this code can check for thread completed or not but again the image spinner appears during running the thread which i do not want  and also can not cancel the counter

        def startthread(self):
                 
                self.active_thread = threading.Thread(target=self.startcounter, args=())
                self.active_thread.daemon = True
                self.active_thread.start()
                while self.active_thread.isAlive():
                  time.sleep(0.5)
                self['text'].setText('Counter stopped')


Re: Problem of using threading with enigma #4 betacentauri

  • PLi® Core member
  • 7,185 posts

+323
Excellent

Posted 29 April 2019 - 19:37

You can use a normal timer which fires every 0,5 second. This timer checks whether thread is still running. And this timer could also write the data into the GUI element.
But it might need some lock mechanism so that both threads don’t access the data at the same time.

Edited by betacentauri, 29 April 2019 - 19:39.

Xtrend ET-9200, ET-8000, ET-10000, OpenPliPC on Ubuntu 12.04

Re: Problem of using threading with enigma #5 mfaraj57

  • Senior Member
  • 1,605 posts

+286
Excellent

Posted 29 April 2019 - 20:35

yes,you are right and could be good solution for my simple counter example but  a part from it looks unprofisional to use two threads but also with my real plugin which assigned to do complicated and resources consuming processes will slow getting the data output as well as may  produce unpredictable results.

 

I



Re: Problem of using threading with enigma #6 betacentauri

  • PLi® Core member
  • 7,185 posts

+323
Excellent

Posted 29 April 2019 - 21:50

What are you doing in your plugin? Eg downloading things from the internet can be done in background without extra threads.

And yes, in general it’s not a good solution, but I don’t have any other.
Xtrend ET-9200, ET-8000, ET-10000, OpenPliPC on Ubuntu 12.04

Re: Problem of using threading with enigma #7 littlesat

  • PLi® Core member
  • 56,123 posts

+685
Excellent

Posted 29 April 2019 - 22:13

Or twisted

WaveFrontier 28.2E | 23.5E | 19.2E | 16E | 13E | 10/9E | 7E | 5E | 1W | 4/5W | 15W


Re: Problem of using threading with enigma #8 mfaraj57

  • Senior Member
  • 1,605 posts

+286
Excellent

Posted 29 April 2019 - 22:19

yes,it is for downloading data from the net and doing parsing tasks and sending the output data again to the plugin.

yes,there are many choices to do this but each with some disadvantages.

twisted web can download data from the net but failed with some secured links as well as it is not flexiable like requests as i can post params in the later and can apply some anticloudflare protection.

i used for some time the shell execution of the working module by econsole application but discovered it is much slower than threading approach.

I am sure there are good ways to get the available data from the running thread and also to fire thread completed event

 

 

 



Re: Problem of using threading with enigma #9 samsamsam

  • Senior Member
  • 2,024 posts

+146
Excellent

Posted 29 April 2019 - 23:26

Hello,

 

I already wrote about this in your thread:

https://forums.openp...127#entry936647

 

You can use:

reactor.callFromThread(...)

 

In the mentioned post I provide you explanation + example.

 

This is really strange that you forgot about this.



Re: Problem of using threading with enigma #10 mfaraj57

  • Senior Member
  • 1,605 posts

+286
Excellent

Posted 30 April 2019 - 07:07

oh,shit,that,s bad of me,searching everywhere for the solution and the solution exists in my thread.

mfaraj57

 

Thanks samsamsam
The informations and ideas looks interesting,hope get some time to apply and see.

 

 

samsamsam

This is really strange that you forgot about this

Yes strange for me also,it is unusual to forget these things,but i do not have other than blaming the high dose cortisone received for some health problems lately,apologize



Re: Problem of using threading with enigma #11 mfaraj57

  • Senior Member
  • 1,605 posts

+286
Excellent

Posted 30 April 2019 - 07:13

Thanks samsamsam

now i read the subject and the example and include what i am looking for

import threading
import urllib
import urllib.request
import time
from twisted.internet import reactor
 
def getTextLabel(url, callback):
    # becaouse of GIL you should not made a long 
    # blocking operation in this thread 
    
    # for example, this is OK
    # sleep will release GIL so main thread will not be blocked, sleep in this thread will not cause spinner
    time.sleep(100000) 
    
    # this is NOT OK, such code will cause spinner
    agg = 0
    for i in range(1000000):
        agg *= i
        for j in range(1000000):
            agg *= j
            for k in range(1000000):
                agg *= k
    
    # this is OK - connection will release GIL
    response = urllib.request.urlopen(url)
    
    html = response.read()
    
    # !!!!!!!!!!!!!!!!!!!!!!
    # you should NOT modify eLabel component from here
    # like that:
    # self["console"].setText(html)
    # you must delegate this task to main thread
    
    reactor.callFromThread(callback, html)
 
def setTextToLabel(html):
    # this will be executed from main thread
    # so you can modify for example eLabel component from here
    self["console"].setText(html)
 
def callInWorkThread(target, *args, **kwargs)
    thread = Thread = threading.Thread(target = target, name = target.Callable.__name__, args = args, kwargs = kwargs)
    thread = Thread.start()
    return thread
 
def SomeMethodOfYourScreen():
    # here we are in the main thread
    callInWorkThread(target = getTextLabel, 'http://python.org/', setTextToLabel)

updating my simple counter demo plugin accordinglly and it is working nicely wihout problems

        def startcounter_thread(self):
                self['text'].setText('Counter started') 
                maxcounter=8
                self.counter_thread= threading.Thread(target =self.startcounter , args = (10,self.setTextToLabel))
                
                self.counter_thread.start()
                
        def startcounter(self,maxcounter, callback):
          
          i=0
          while i<maxcounter:
              i=i+1
              print str(i)
              time.sleep(1)
              msg=str(i)
              reactor.callFromThread(callback, msg)
              if self.exitcounter:
                  self.exitcounter=False
                  
                  break
          msg="counter completed successfully"
          reactor.callFromThread(callback, msg)
            
        def stopcounter_thread(self):
            self.exitcounter=True

        def setTextToLabel(self,msg):
            # this will be executed from main thread
            # so you can modify for example eLabel component from here
            self["text"].setText(msg)
         


Re: Problem of using threading with enigma #12 betacentauri

  • PLi® Core member
  • 7,185 posts

+323
Excellent

Posted 30 April 2019 - 15:34

Ah, interesting. Thanks for sharing it?
Xtrend ET-9200, ET-8000, ET-10000, OpenPliPC on Ubuntu 12.04

Re: Problem of using threading with enigma #13 mfaraj57

  • Senior Member
  • 1,605 posts

+286
Excellent

Posted 1 May 2019 - 16:46

struggling with enigma threading i found that not always twisted reactor fire the outside thread callback function

this is part of my real code

import threading
from twisted.internet import reactor

    def getdata(self)
            txt = 'from Plugins.Extensions.TSmedia.addons.' + host + '.default import start'
            try:
                exec txt
            except:
                trace_error()
                
                return

           
            self.active_thread = threading.Thread(target=self.threadon, args=(start))
            self.active_thread.daemon = True
            self.active_thread.start()



    def threadon(self, start):
        datalist = []
        try:
            datalist = start()#data output
        except:
            trace_error()
        
        reactor.callFromThread(self.publish, datalist)

    def publish(self, datalist):
        ##process the datalist

    def run(self):
        self.getdata()

in my code i can log the datalist and always returned by the threadon function but not always passed to publish function to populate the datalist.

i noticed this behavior in few images and not all,it is working with openatv6.3 and dreamos cvs images and not working with blackhole 3.0.9 image and not yet tested with openpli



Re: Problem of using threading with enigma #14 samsamsam

  • Senior Member
  • 2,024 posts

+146
Excellent

Posted 1 May 2019 - 18:28

@mfaraj57

 

 

i noticed this behavior in few images and not all,it is working 

 

 

I also wrote about this:

https://forums.openp...127#entry936659


Edited by samsamsam, 1 May 2019 - 18:28.


Re: Problem of using threading with enigma #15 mfaraj57

  • Senior Member
  • 1,605 posts

+286
Excellent

Posted 2 May 2019 - 09:27

 

samsamsam, on 21 Sept 2018 - 13:00, said:

snapback.png

As I wrote I do not use twisted because it is not available in all Enigma2 distribution.

I use Timer component and callbacks Queue, to delegate function to the main thread.

 

This is poll technique which is less effective than reactor.callFromThread(...) but I made a choice to support users even with very old Enigma2 distributions were twisted module is not available.

Ok,I think usage of twisted reactor not dependent only if included with the enigma distribution or not but also related to version of twisted web included.From my small experiment with 4 images i noticed that

vu solo2 blackhole 3.0.9, twisted web version 13.2.0 -reactor is not working.

dreambox 820 openatv 6.2,twisted web version 17.0.0 reactor is working.

dreambox 820 newnigma2(dreamos image) twisted web version 14.0.0 ,reactor is working.

dreambox dm900 openatv 6.3 twisted web version 18.0.0 ,reactor is working.

i can conclude from above that twisted reactor will not work with enigma distribution has twisted web version less than 14.0.0.

because usage of reactor improves performance as well as can download multiple images at same time it is not good decision not to use reactor because will not work with some old images.

to solve the problem with old images we can do  with simple routine to check for availability of twisted and validity of version for reactor,if ok we can use it else we can use other methods like timer to delegate function to main thread.


        
        use_reactor=False
        try:
           import twisted
           tversion=twisted.__version__.split(".")[0]
           tversion=int(tversion)
          
           if tversion<14:
               use_reactor=False
              
           else:
               use_reactor=True
               
        except:
              
               use_reactor=False

Edited by mfaraj57, 2 May 2019 - 09:27.


Re: Problem of using threading with enigma #16 Huevos

  • PLi® Contributor
  • 4,231 posts

+158
Excellent

Posted 9 May 2019 - 15:27

Afaik (I’m no GUI expert) is calculating something in a thread totally fine. But you shouldn’t access GUI elements from the thread.
So you shouldn’t do this in the thread:
self['text'].setText(str(i))

Why not?



Re: Problem of using threading with enigma #17 betacentauri

  • PLi® Core member
  • 7,185 posts

+323
Excellent

Posted 9 May 2019 - 17:02

Only an theoretical example!
The GUI thread is rendering a string. The current string is “Long string”. In the GUI Thread there is a local variable with the length of the string. This variable was set during rendering and before continuing the other thread gets the focus. It changes the string to “test”. Now the GUI thread continues. I thinks string is 11 characters long and tries to read 11 byte from the current string. But it’s now only 4 bytes long ...
I don’t think there are any locking mechanisms which can prevent this.

Edited by betacentauri, 9 May 2019 - 17:03.

Xtrend ET-9200, ET-8000, ET-10000, OpenPliPC on Ubuntu 12.04

Re: Problem of using threading with enigma #18 Erik Slagter

  • PLi® Core member
  • 46,951 posts

+541
Excellent

Posted 9 May 2019 - 19:08

I think betacentauri has a strong point here, it sounds very plausible.


* Wavefrontier T90 with 28E/23E/19E/13E via SCR switches 2 x 2 x 6 user bands
I don't read PM -> if you have something to ask or to report, do it in the forum so others can benefit. I don't take freelance jobs.
Ik lees geen PM -> als je iets te vragen of te melden hebt, doe het op het forum, zodat anderen er ook wat aan hebben.



0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users