21 replies to this topic

[WanWizard]

There seems to be another moron on the loose.

 

I've getting enormous hits since a few days, a few times a day (last one at 13:40 GMT+1), all for the Polish EPG file, so someone is distributing something scheduled again...

 

It looks like a genuine image, so difficult to block.

[doglover]

These morons will make sure this service will end one day.

 

It is now widely know in the Kodi community that the Kodi plugin "IPTV Simple PVR" in its native state causes severe download problems on our websites.  Nevertheless some people simply do not care.

Even the author of the plug-in "afedchin" denies there is a problem, although there is ample evidence.  So he will not do anything about it.  And users can still configure the enormous downloads volumes.

 

There was a valid alternative. The JIN plug-in.

http://www.kodinerds...Addon/?pageNo=1

 

However the author has retracted the plug-in because 95 % of the users where using it to watch illegally streamed content.

 

But nobody cares about using this. 

 

Willy

Edited by doglover, 6 December 2015 - 07:52.

[doglover]

@WanWizard

 

I have removed the Polish file from your website.  This will stop the enormous download from your website.

The XMLTVimporter will simply go to another website.

 

Crossepg users, will need to refresh the Rytec providers, if they do not have EPG import.

 

Willy

[doglover]

This is a hobby, and it should remain a hobby.

If this sort of harassment keeps on appearing,  the fun stops.  And I will do something else with my spare time.

 

Willy

[littlesat]

I think we should consider to stop this service for a while just to give a statement to these leechers.... Without source their plugin is wurthless.... Then you can tell them that we consider to continue the service when they adapt.... (And add some stuff so old plugins cannot get data).

Edited by littlesat, 6 December 2015 - 09:41.

[doglover]

A Google search revealed several websites which promote the use of the XMLTV file in Kodi - IPTV Simple PVR.

But since I do not speak (or write) Polish I cannot ask them to configure their software properly.

 

Could one of our Polish OpenPLi users help out here?

 

Willy

http://www.pkteam.pl/viewtopic.php?f=36&t=3687&start=60
http://www.iptvlinks.com/2013/11/epg-para-simpletv.html?m=0
http://www.debian.pl/threads/32906-skrypt-modyfikuj%C4%85cy-xmltv
http://stream-recorder.pl/viewtopic.php?t=144
http://satforum.biz/originalni-fw/getepg-v-2-2-by-morser!!!/?wap2

[macnuts]

A Google search revealed several websites which promote the use of the XMLTV file in Kodi - IPTV Simple PVR.
But since I do not speak (or write) Polish I cannot ask them to configure their software properly.

Could one of our Polish OpenPLi users help out here?

Willy

http://www.pkteam.pl/viewtopic.php?f=36&t=3687&start=60
http://www.iptvlinks.com/2013/11/epg-para-simpletv.html?m=0
http://www.debian.pl/threads/32906-skrypt-modyfikuj%C4%85cy-xmltv
http://stream-recorder.pl/viewtopic.php?t=144
http://satforum.biz/originalni-fw/getepg-v-2-2-by-morser!!!/?wap2 

Please state your statement, I will try and post it on those sites (translated).
I have no knowledge nor use Kodi plugin so I don't know how to phrase it right myself, but I am eager to help you.
Regards.

[macnuts]

(Should be EDIT of my previous message but not available).

 

I looked at the sites quoted I am quite sure that it will be no good just to post there anything, I am afraid. Most of them are users to users advices how to configure their not working components - they will not read them again till new problems arise.

 

Moreover, there will always be some morons and abuse the usage. Example:

a link

http://stream-recorder.pl/viewtopic.php?t=144

is a thread for configuring a SimpleTV (whatever piece of stream recording code it is) and that SimpleTV allows user to put EPG source http address directly. No control what a user put there whatsoever.

It looks like you need to ban any connection from not-intended sources somehow.

 

Regards.

[doglover]

(Should be EDIT of my previous message but not available).

 

I looked at the sites quoted I am quite sure that it will be no good just to post there anything, I am afraid. Most of them are users to users advices how to configure their not working components - they will not read them again till new problems arise.

 

Moreover, there will always be some morons and abuse the usage. Example:

a link

http://stream-recorder.pl/viewtopic.php?t=144

is a thread for configuring a SimpleTV (whatever piece of stream recording code it is) and that SimpleTV allows user to put EPG source http address directly. No control what a user put there whatsoever.

It looks like you need to ban any connection from not-intended sources somehow.

 

Regards.

 

Which is exactly the problem.  Putting the link to the XMLTV file directly in the IPTV simple PVR software leads to problems.

This piece of software, downloads the XMNLTVfile every 5 minutes, over and over.

 

I figured this out, and it is confirmed by Kodi specialists.  But the coder (afedchin) refuses to do anything about it.

see:  http://forum.kodi.tv...highlight=rytec

 

Although it looked that afedchein tried to fix something: https://github.com/k...vsimple/pull/12

 

There are subbranches of the IPTV simple PVR which are floating around which still uses the old method.

 

Willy

[doglover]

Ned Scott on Kodi noticed this:

 

Trying to access the EPG every 5 minutes is likely an issue even for local networks. I just checked my own NAS tonight and found out that this 5 minute check (due to one channel never having EPG data) prevents one of my hard drives from ever sleeping. Apparently it has been doing this for at least a month.

 

 

Willy

[WanWizard]

Thanks Willy.

[openpli4me]

What kind of webserver are you using? with nginx it is kindof easy to limit each ip to x requests per hour/day.

[WanWizard]

It is limited on the firewall.

 

Problem is not a single IP, problems is hunderds at the same time. So it becomes a DDOS since each file request is 2-5MB minimum...

[openpli4me]

Ok but the next time they all get a 503 within an hour/day until they float out of the cache.

[WanWizard]

You can't ban them without also banning legitimate users, there is no way to make a distinction between the two.

 

You can only limit the amount of request in a certain timespan, which is already done. But that doesn't help much, you must allow enough requests for someone with 5 boxes each loading 5 EPG files (= 25 requests per day). So limiting is pointless too...

[openpli4me]

Limiting doesn't have to be pointless but you do have to start being more aggressive about abuse and if that means those with more then 4 requests per hour have to wait a bit or reschedule some boxes for a different time window then so be it. It is of-course all up to you but whats happening now is not going to be the last 'thing' which is impacting a 'hobby'. I'd suggest a max of 4 requests / 8 hours for one ip, a full block for all when epg is being updated, and see how it goes. As for multiple boxes at the same location getting the 'same' epg maybe enigma2 can be enhanced by acting as a proxy for other boxes, thats how I did it here, one box gets it all and the other boxes get a copy. Its then just one request a day for all boxes.

[WanWizard]

That doesn't work. One box downloading 5 EPG "zones" does 6 downloads. And if you have multiple boxes at home, you can multiply that. You can't make the good suffer because of a few rotten eggs. Most users have no clue about the technical things, you will never be able to explain to them to have them download EPG at 4 different times of the day, it is just not practical.

 

Again, this specific issue is about a box that downloads once a day, one single EPG file (the Poland one). Not something you would catch with any of your proposals without blocking legitimate users too. The problem is that hunderds and hunderds of boxes all download at the same time, so it's probably some illegal scheme where box and image are sold as part of a service, and that image has this setting. For the same reasons we were forced to take OpenPLi 3 offline last year.

 

The only way to solve this, is to require authentication, so only people with a valid account will be able to download, and you will know why is abusing the system. But that is very against the ideology of open source...

[adri]

Wouldn't you be able to deny requests based on the USER_AGENT or some other header, to catch the rotten eggs?

[doglover]

From what I understand it is this time an enigma image which does the damage.

 

How is this possible: 

 

Somebody has produced an enigma2 image which is preconfigured.  The XMLTVimport plug-in is activated and set at a certain time.  Now everybody who installed this image (or buy's a preconfigured receiver) starts downloading at exact the same time.  To prevent this, the XMLTVimport plug-in when first configured proposes a random download time somewhere in the morning.  You are free to alter this.  But once you alter this the import takes place at that time.  If the configuration file is distributed with the image, all receivers installing this will download at the same time.

 

In order to find out where it comes from, I removed the Polish file on the WanWizard server.  The OpenPLi version of XMLTVimport would then automatically go to another server and download the file from that server.  The OE-Alliance version (it is called EPGimport in OpenATV and OpenVIX) works slightly different.  When it cannot download the file from a server, it loads a small config file (hidden) from which it picks at random one server, downloads the sources file from that server, and subsequently downloads all the XMLTV and channel files from that server.  I have taken the Wanwizard server out of the small config file.

 

After the removal of the WanWizard server, you should expect that traffic on the other servers would increase, because the automatic server selection procedure.  This has not happened.

And this puzzled me for a while.

 

However, it is possible that an old version of the XMLTVimport plug-in still floats around and is installed (pre-installed) on less official images.  Also enigma2 images are produced for unusual receivers with scraps of code gathered together from the net.  Because no increased traffic has been seen on other servers, it looks like an old version of XMLTVimport (without the automatic server selection) is the culprit here.

There is nothing wrong with the EPG import done by this old version.  And it can be used if the user want it.  The problem is they all start downloading on a predefined fixed and the same time.

 

The solution to this is simple:  Anyone downloading the Polish file should check their XMLTVimport configuration.  And change the time of import to something not nice (use a time like 12h23 etc..).  One time we have seen as especially bad is 13h40

Problem to this is:  most of the people which installed this image with predefined XMLTVimport would not even know they are importing EPG.  Maybe they do not even need the Polish EPG.  But the import is taken place without them even knowing it.  Why should they start reading forums and read about it?  Let's alone fix it.

 

Also anyone which publishes or distributes images for enigma2 receiver, should not activate (pre-installing is fine) the XMLTVimport module.  Leave the activation up to the user pls.

 

Willy

Edited by doglover, 7 December 2015 - 09:50.

[Ampersand]

I have already posted your info on one of polish tvsat forum. I hope people read this and change download time.