Jump to content


Photo

XMLTV Import - %CPU

cpu xmltv

  • Please log in to reply
9 replies to this topic

#1 Robinson

  • Senior Member
  • 2,621 posts

+30
Good

Posted 25 October 2016 - 21:57

Hi,

I don't know if this place is correct but I will ask here.

What %CPU usage do you have on your boxes during Rytec XMLTV Import?

On my ET9000 it is about 50% CPU, which can be seen with "top" command. Isn't it too much? It really slows down using a remote.

 

 


ET9000, OpenPLi 4.0, 13E, 19E

HD51, OpenPLi 6.2, 75E - 30W


Re: XMLTV Import - %CPU #2 WanWizard

  • PLi® Core member
  • 70,523 posts

+1,810
Excellent

Posted 25 October 2016 - 22:16

As long as your CPU isn't running at 100%, I shouldn't worry to much. It shouldn't really slow the box down, it processes the data in chunks in a timeslice. Are you not importing too much, so the box has to swap (it will have to wait for I/O)?


Currently in use: VU+ Duo 4K (2xFBC S2), VU+ Solo 4K (1xFBC S2), uClan Usytm 4K Ultimate (S2+T2), Octagon SF8008 (S2+T2), Zgemma H9.2H (S2+T2)

Due to my bad health, I will not be very active at times and may be slow to respond. I will not read the forum or PM on a regular basis.

Many answers to your question can be found in our new and improved wiki.


Re: XMLTV Import - %CPU #3 Robinson

  • Senior Member
  • 2,621 posts

+30
Good

Posted 25 October 2016 - 22:31

Yes, I do import quite a lot. But the issue of 50% CPU is only during import, which takes perhaps 15-20 minutes, later no problem.


ET9000, OpenPLi 4.0, 13E, 19E

HD51, OpenPLi 6.2, 75E - 30W


Re: XMLTV Import - %CPU #4 littlesat

  • PLi® Core member
  • 57,166 posts

+698
Excellent

Posted 25 October 2016 - 22:32

Just a qustion, why do you need to download such a tons of epg?

WaveFrontier 28.2E | 23.5E | 19.2E | 16E | 13E | 10/9E | 7E | 5E | 1W | 4/5W | 15W


Re: XMLTV Import - %CPU #5 Robinson

  • Senior Member
  • 2,621 posts

+30
Good

Posted 25 October 2016 - 23:36

I know this is probably not so common among standard users but I import UK / Germany / Poland / Italy / France and Scandinavia EPG.

Is this really too much for ET9000? If I import just one or two, will I have less then 50% CPU occupation? I don't think so but I can test that.


ET9000, OpenPLi 4.0, 13E, 19E

HD51, OpenPLi 6.2, 75E - 30W


Re: XMLTV Import - %CPU #6 WanWizard

  • PLi® Core member
  • 70,523 posts

+1,810
Excellent

Posted 26 October 2016 - 09:33

The CPU % is not really relevant, as long as it doesn't hit 100%.

 

The problem is that plugin processes don't run using real multitasking, they all run in the same enigma main loop. If one doesn't release quick enough so other processes can get a timeslice, you can experience slow response, or if it takes too long, a spinner.

 

If the import process takes up too much time, the question is why. XML files are processed a node at the time, so that can't cause it. epg.dat files are loaded in one go, so that is more likely. Another possibility is that you have slow storage in your box, and the importer uses that to download and store temp files, in which case slow I/O could be a bottleneck.

 

If you check "top" when you think it is slow, what is the procentage "wa"? And what are the load averages?


Currently in use: VU+ Duo 4K (2xFBC S2), VU+ Solo 4K (1xFBC S2), uClan Usytm 4K Ultimate (S2+T2), Octagon SF8008 (S2+T2), Zgemma H9.2H (S2+T2)

Due to my bad health, I will not be very active at times and may be slow to respond. I will not read the forum or PM on a regular basis.

Many answers to your question can be found in our new and improved wiki.


Re: XMLTV Import - %CPU #7 MiLo

  • PLi® Core member
  • 14,055 posts

+298
Excellent

Posted 26 October 2016 - 12:08

Hi,
On my ET9000 it is about 50% CPU, which can be seen with "top" command. Isn't it too much? It really slows down using a remote.


The ET9000 has two CPUs, hence 50% means that it's fully using a single CPU. The other CPU is idle.

The underlying problem is that Python doesn't actually do multi-threading, and the parser spends most time in Python code. That's why your remote slows down.

If it were to do two imports in parallel, you'd only see slightly more than 50% CPU usage, but the effect on your remote would be doubled.

To actually solve this, one could run the import in another process, or re-write the code in C++. Patches to do so are welcome.

If you run xmltv import on something else than OpenPLi, the effects on your remote are much more pronounced too, since the Python code will have to do much more work there.

Edited by MiLo, 26 October 2016 - 12:12.

Real musicians never die - they just decompose

Re: XMLTV Import - %CPU #8 Robinson

  • Senior Member
  • 2,621 posts

+30
Good

Posted 26 October 2016 - 22:10

If you check "top" when you think it is slow, what is the procentage "wa"? And what are the load averages?
Mem: 231952K used, 58776K free, 0K shrd, 552K buff, 77356K cached
CPU:  53% usr   3% sys   0% nic  42% idle   0% io   0% irq   1% sirq
Load average: 1.03 0.85 0.45 2/89 23551
  PID  PPID USER     STAT   VSZ %VSZ %CPU COMMAND
12450 12444 root     S     174m  61%  47% /usr/bin/enigma2

I have just made an extreme test with Denmark, France, Germany, Italy, Norway, Middle East, Poland, Portugal, Spain, Sweden and UK XMLTV files.


ET9000, OpenPLi 4.0, 13E, 19E

HD51, OpenPLi 6.2, 75E - 30W


Re: XMLTV Import - %CPU #9 Robinson

  • Senior Member
  • 2,621 posts

+30
Good

Posted 29 October 2016 - 13:34

 

Hi,
On my ET9000 it is about 50% CPU, which can be seen with "top" command. Isn't it too much? It really slows down using a remote.


To actually solve this, one could run the import in another process, or re-write the code in C++. Patches to do so are welcome.

 

This is definitely outside my skills.


ET9000, OpenPLi 4.0, 13E, 19E

HD51, OpenPLi 6.2, 75E - 30W


Re: XMLTV Import - %CPU #10 Pr2

  • PLi® Contributor
  • 6,181 posts

+261
Excellent

Posted 30 October 2016 - 13:21

Hi,

 

I agree with Robinson, when running XMLTV while using the box it slow it down during the import process.

 

This is also true on the Solo4K, launch an import and go in the channel list, it becames nearly unresponsive but I can live with it, I prefer a plugin that slow down the box during import than a plugin that totally stuck it and that we need to wait for the process to end before getting back access to the box.

 

On a Solo4K start a manual import with delete EPG data before importing set to on (I also have EIT EPG current/next enables), and then browse the channel list. It is really slow and the Solo4K has a powerfull CPU.  I think that it is the delete current EPG process that slow down the channel list browsing. 

 

Pr2


NO SUPPORT by PM, it is a forum make your question public so everybody can benefit from the question/answer.
If you think that my answer helps you, you can press the up arrow in bottom right of the answer.

Wanna help with OpenPLi Translation? Please read our Wiki Information for translators

Sat: Hotbird 13.0E, Astra 19.2E, Eutelsat5A 5.0W
VU+ Solo 4K: 2*DVB-S2 + 2*DVB-C/T/T2 (used in DVB-C) & Duo 4K: 2*DVB-S2X + DVB-C (FBC)

AB-Com: PULSe 4K 1*DVB-S2X (+ DVB-C/T/T2)
Edision OS Mio 4K: 1*DVB-S2X + 1*DVB-C/T/T2
 




1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users