24 replies to this topic

[ian1095]

Ok what has changed in the last few days regarding https connections ?

 

Its now not possible to connect to certain sites,which used to work perfectly.

 

here is the debug when trying to connect to afdah after the plugin exception has been removed in order to create a gsod to see whats occurring.

 

<   677.587707> [eDVBServicePlay] timeshift
<   682.281688> action ->  SetupActions ok
<   682.593927> [eDVBServicePlay] timeshift
<   683.475815> action ->  OkCancelActions ok
<   687.598974> [eDVBServicePlay] timeshift
<   690.802681> action ->  OkCancelActions ok
<   690.809516> url = https://afdah.org/?page=1
<   691.487781> Traceback (most recent call last):
<   691.488986>   File "/usr/lib/enigma2/python/Components/ActionMap.py", line 4
6, in action
<   691.516401>   File "/usr/lib/enigma2/python/Plugins/Extensions/pl
ugin.py", line 1713, in go
<   691.541482>     self.session.open(afdah2, returnValue, returnTitle, value)
<   691.543162>   File "/usr/lib/enigma2/python/mytest.py", line 289, in open
<   691.552127>     dlg = self.current_dialog = self.instantiateDialog(screen, *
arguments, **kwargs)
<   691.553632>   File "/usr/lib/enigma2/python/mytest.py", line 232, in instant
iateDialog
<   691.556902>     return self.doInstantiateDialog(screen, arguments, kwargs, s
elf.desktop)
<   691.558397>   File "/usr/lib/enigma2/python/mytest.py", line 249, in doInsta
ntiateDialog
<   691.561315>     dlg = screen(self, *arguments, **kwargs)
<   691.562813>   File "/usr/lib/enigma2/python/Plugins/Extensions/pl
ugin.py", line 1773, in __init__
<   691.578964>     links = re.compile ('<h3><a title="(.+?)" href="\/watch\?v=(
.+?)#', re.DOTALL).findall(html)
<   691.580734> TypeError: expected string or buffer
<   691.582273> [ePyObject] (CallObject(<bound method ActionMap.action of <Compo
nents.ActionMap.ActionMap instance at 0x38ba490>>,('OkCancelActions', 'ok')) fai
led)
<   694.080635> [gRC] main thread is non-idle! display spinner!
<   701.841142> connectionLost? [Failure instance: Traceback (failure with no fr
ames): <class 'twisted.internet.error.ConnectionLost'>: Connection to the other
side was lost in a non-clean fashion: Connection lost.
]<   701.842482>

 

Is this a certs problem ?

 

adding this to the mytest.py though prevents the image from booting

 

import ssl
context = ssl._create_unverified_context()
urllib.urlopen("https://no-valid-cert", context=context)

Edited by ian1095, 9 February 2016 - 09:56.

[ian1095]

Forgot to add and I cannot edit.

 

You can see from the debug that the image is not even attempting to make a connection,hence me asking about have any certs changed.

 

Up until a few days ago this worked perfectly.

 

Ian.

Edited by ian1095, 9 February 2016 - 10:00.

[ian1095]

Anyone ?

 

The devs have to have changed something which requires fixing.

 

Whats the point in reporting new bugs if no one replies ?

 

Ian.

[mx3L]

links = re.compile ('<h3><a title="(.+?)" href="\/watch\?v=(
.+?)#', re.DOTALL).findall(html)
<   691.580734> TypeError: expected string or buffer

Looks like html variable is not a string or buffer, which RegexObject.findall expects. How do you get 'html'? Can you show source?

[ian1095]

Thanks for replying.

 

req = urllib2.Request(url)
            req.add_header('User-Agent', 'Mozilla/5.0 (Windows NT 5.1; rv:23.0) Gecko/20100101 Firefox/23.0')
            response = urllib2.urlopen(req)
            htmldoc = str(response.read())
            response.close()
            return htmldoc

 

The regex is fine and still works using BH,ATV or ViX and was working with Pli too up until a few days ago,so something has definitely changed in the Pli image.

 

Ian.

[mx3L]

It should work in normal circumstances, however it's really only part of the source. I don't see how you handle exceptions if there is some problem with connection. What do you assign then to html? Or do you crash?

 

Try to see what you're passing to findall function:

print type(html), html

[ian1095]

Thanks.

 

The exception has been purposely removed in order to force a crash to try to ascertain just whats going on,thats why you cannot see it.

 

Result from ATV which is working.

14:19:18.625 Plugins/Extensions/myplugin/plugin.py:71 __init__ <type 'str'> <!DOCTYPE html><html lang="en" xmlns="http://www.w3.org/1999/xhtml" xmlns:og="http://ogp.me/ns#" xmlns:fb="https://www.facebook.com/2008/fbml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
<meta http-equiv="Cache-Control" content="public, max-age=86400">
<meta property="fb:app_id" content="316405681889610"/>
<link rel="canonical" href="https://afdah.org/?page=1">
<link rel="alternate" media="handheld" href="https://m.afdah.org/?page=1">
<link rel="alternate" media="only screen and (max-width: 640px)" href="https://m.afdah.org/?page=1">
<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon">
<link rel="icon" href="/favicon.ico" type="image/x-icon">
<meta name="description" content=""/>
<title>AfDah</title>
<link rel="stylesheet" href="/css/style.min.css"/>
<script>(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)})(window,document,'script','//www.google-analytics.com/analytics.js','ga');ga('create','UA-53827788-1','auto');ga('send','pageview');</script><script>function loadScript(url,callback){var head=document.getElementsByTagName('head')[0];var s=document.createElement('script');s.type='text/javascript';s.src=url;if(typeof callback=='function'){if(s.addEventListener){s.addEventListener("load",callback,false);}else if(s.readyState){s.onreadystatechange=callback;}}
head.appendChild(s);}
function loadCSS(url){var head=document.getElementsByTagName('head')[0];var s=document.createElement('link');s.rel='stylesheet';s.type='text/css';s.href=url;s.media='all';head.appendChild(s);}</script>
</head>
<body class="container">
<div class="top_menu">
<div class="center">
<div id="cssmenu">
<ul>
<li><a href="/"><span>Home</span></a></li><li><a href="/most_viewed"><span>Most Viewed</span></a></li><li><a href="/recent_movies"><span>Recent Movies</span></a></li><li class="has-sub"><a href="#">

Result from Pli which is not working

<   416.409638> <type 'NoneType'> None
<   416.414116> Traceback (most recent call last):
<   416.414306>   File "/usr/lib/enigma2/python/Components/ActionMap.py", line 46, in action
<   416.414705>   File "/usr/lib/enigma2/python/Plugins/Extensions/myplugin/plugin.py", line 13, in go
<   416.417573>     self.session.open(afdah2, returnValue, returnTitle, value)
<   416.417859>   File "/usr/lib/enigma2/python/mytest.py", line 289, in open
<   416.418476>     dlg = self.current_dialog = self.instantiateDialog(screen, *arguments, **kwargs)
<   416.418747>   File "/usr/lib/enigma2/python/mytest.py", line 232, in instantiateDialog
<   416.419312>     return self.doInstantiateDialog(screen, arguments, kwargs, self.desktop)
<   416.419581>   File "/usr/lib/enigma2/python/mytest.py", line 249, in doInstantiateDialog
<   416.420158>     dlg = screen(self, *arguments, **kwargs)
<   416.420411>   File "/usr/lib/enigma2/python/Plugins/Extensions/myplugin/plugin.py", line 73, in __init__
<   416.423398>     links = re.compile ('<h3><a title="(.+?)" href="\/watch\?v=(.+?)#', re.DOTALL).findall(html)
<   416.423791> TypeError: expected string or buffer
<   416.424136> [ePyObject] (CallObject(<bound method ActionMap.action of <Components.ActionMap.ActionMap instance at 0x434ecd8>>,('OkCancelActions', 'ok')) failed)
<   419.599525> [gRC] main thread is non-idle! display spinner!
<   426.468181> connectionLost? [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionLost'>: Connection to the other side was lost in a non-clean fashion: Connection lost.

As stated this has only recently started to happen and for it not to work in Pli

 

Ian.


 

Edited by ian1095, 10 February 2016 - 15:38.

[Frenske]

Anyone ?
 
The devs have to have changed something which requires fixing.
 
Whats the point in reporting new bugs if no one replies ?
 
Ian.

FYI. All members have, besides their daily job, a life like wife, children, holiday and then there is the hobby. And besides the previous arguments we also have priorities fixing things.

[mx3L]

@ian1095

It's clear why is it happening, you're passing None to findall, that's why it crashes.

I don't know why is 'html' set to None, you didn't post relevant sources, only part of your request routine.

[ian1095]

Thank You.

 

I will look further into what you have said later as I have to go out on a job now,but honestly the code has not been touched in months and used to work perfectly up unitl a few days ago on Pli and still does work perfectly on all other images.

 

This is why I suspected something has changed in the Pli image recently, perhaps regarding strings ?

 

Ian.

[saintomer1866]

Hi mx3L,

here's the error from gethtml;

<   259.312557>   response = urllib2.urlopen(req)
<   259.312843>   File "/usr/lib/python2.7/urllib2.py", line 126, in urlopen
<   259.313286>   File "/usr/lib/python2.7/urllib2.py", line 400, in open
<   259.313612>   File "/usr/lib/python2.7/urllib2.py", line 418, in _open
<   259.313915>   File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain
<   259.314322>   File "/usr/lib/python2.7/urllib2.py", line 1215, in https_open
<   259.314667>   File "/usr/lib/python2.7/urllib2.py", line 1177, in do_open
<   259.315367> urllib2.URLError: <urlopen error [Errno 1] _ssl.c:506: error:14077438:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert internal error>
<   259.315693> [ePyObject] (CallObject(<bound method ActionMap.action of <Components.ActionMap.ActionMap instance at 0x435ac88>>,('OkCancelActions', 'ok')) failed)
<   262.263853> [gRC] main thread is non-idle! display spinner!
<   269.349428> connectionLost? [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionLost'>: Connection to the other side was lost in a non-clean fashion: Connection lost.

St.O

 

[WanWizard]

What is the version of openssl used in your non-OpenPLi images?

openssl version

And which version of pyOpenSSL do these images use?

cd / && find . -xdev -name "pyOpenSSL*"

[saintomer1866]

Hi WanWizard,

 

ATV is using OpenSSL 1.0.2d 9 Jul 2015
and BlackHole is using OpenSSL 1.0.2a 19 Mar 2015

[WanWizard]

Thanks.

 

Can you also check the pyOpenSSL version, and check if the package python-cryptography is present, and if installed, which version?

opkg info python-cryptography

[saintomer1866]

Thanks WanWizard,

ATV is using;

 

pyOpenSSL - 0.13.1-r3

python-cryptography is not present but python-crypt - 2.7.9-r1 is.
 

[WanWizard]

We're using pyOpenSSL 0.13, and python-crypt 2.0.1.

 

It is quite possible that ATV is using a newer OE, we will introduce one when we release OpenPLi 5.

[athoik]

I guess it is the known python SNI issue. Master-next has newer python with SNI included.

[ian1095]

So why has any of this only stopped it from working only for the last few days ?

 

Up until then it worked perfectly,so unless these have changed recently it cannot be this.

 

Ian.

Edited by ian1095, 10 February 2016 - 19:22.

[saintomer1866]

Thanks WanWizard,

whilst this is possibly true it still doesn't explain why this error was not present in PLi4 until very recently

It was working fine on your existing OE, so something has changed in the image to cause this

 

St.O

[WanWizard]

From what I understand it happens with a very specific version combination of OpenSSL, pyOpenSSL and/or the python cryptography libraries.

 

I saw a kernel update when I updated earlier this week, so perhaps new packages where introduced that exposed this problem.