9 replies to this topic

[Robinson]

Hi,

I don't know if this place is correct but I will ask here.

What %CPU usage do you have on your boxes during Rytec XMLTV Import?

On my ET9000 it is about 50% CPU, which can be seen with "top" command. Isn't it too much? It really slows down using a remote.

 

 

[WanWizard]

As long as your CPU isn't running at 100%, I shouldn't worry to much. It shouldn't really slow the box down, it processes the data in chunks in a timeslice. Are you not importing too much, so the box has to swap (it will have to wait for I/O)?

[Robinson]

Yes, I do import quite a lot. But the issue of 50% CPU is only during import, which takes perhaps 15-20 minutes, later no problem.

[littlesat]

Just a qustion, why do you need to download such a tons of epg?

[Robinson]

I know this is probably not so common among standard users but I import UK / Germany / Poland / Italy / France and Scandinavia EPG.

Is this really too much for ET9000? If I import just one or two, will I have less then 50% CPU occupation? I don't think so but I can test that.

[WanWizard]

The CPU % is not really relevant, as long as it doesn't hit 100%.

 

The problem is that plugin processes don't run using real multitasking, they all run in the same enigma main loop. If one doesn't release quick enough so other processes can get a timeslice, you can experience slow response, or if it takes too long, a spinner.

 

If the import process takes up too much time, the question is why. XML files are processed a node at the time, so that can't cause it. epg.dat files are loaded in one go, so that is more likely. Another possibility is that you have slow storage in your box, and the importer uses that to download and store temp files, in which case slow I/O could be a bottleneck.

 

If you check "top" when you think it is slow, what is the procentage "wa"? And what are the load averages?

[MiLo]

Hi,
On my ET9000 it is about 50% CPU, which can be seen with "top" command. Isn't it too much? It really slows down using a remote.

The ET9000 has two CPUs, hence 50% means that it's fully using a single CPU. The other CPU is idle.

The underlying problem is that Python doesn't actually do multi-threading, and the parser spends most time in Python code. That's why your remote slows down.

If it were to do two imports in parallel, you'd only see slightly more than 50% CPU usage, but the effect on your remote would be doubled.

To actually solve this, one could run the import in another process, or re-write the code in C++. Patches to do so are welcome.

If you run xmltv import on something else than OpenPLi, the effects on your remote are much more pronounced too, since the Python code will have to do much more work there.

Edited by MiLo, 26 October 2016 - 12:12.

[Robinson]

If you check "top" when you think it is slow, what is the procentage "wa"? And what are the load averages?

Mem: 231952K used, 58776K free, 0K shrd, 552K buff, 77356K cached
CPU:  53% usr   3% sys   0% nic  42% idle   0% io   0% irq   1% sirq
Load average: 1.03 0.85 0.45 2/89 23551

  PID  PPID USER     STAT   VSZ %VSZ %CPU COMMAND
12450 12444 root     S     174m  61%  47% /usr/bin/enigma2

I have just made an extreme test with Denmark, France, Germany, Italy, Norway, Middle East, Poland, Portugal, Spain, Sweden and UK XMLTV files.

[Robinson]

 

Hi,
On my ET9000 it is about 50% CPU, which can be seen with "top" command. Isn't it too much? It really slows down using a remote.

To actually solve this, one could run the import in another process, or re-write the code in C++. Patches to do so are welcome.

 

This is definitely outside my skills.

[Pr2]

Hi,

 

I agree with Robinson, when running XMLTV while using the box it slow it down during the import process.

 

This is also true on the Solo4K, launch an import and go in the channel list, it becames nearly unresponsive but I can live with it, I prefer a plugin that slow down the box during import than a plugin that totally stuck it and that we need to wait for the process to end before getting back access to the box.

 

On a Solo4K start a manual import with delete EPG data before importing set to on (I also have EIT EPG current/next enables), and then browse the channel list. It is really slow and the Solo4K has a powerfull CPU.  I think that it is the delete current EPG process that slow down the channel list browsing. 

 

Pr2