371 replies to this topic

[40H3X]

Thanks at @foxbox and @blzr, I could but an update of my local repositories would remove the patch?

[blzr]

1) I suppose this plugin you'll build just once (unless you'd like to build image from scratch for some reason), so you can even patch sources in tmp/work directory
2) if you add a patch in your local vu bsp repo it won't be affected by openpli updates until vu bsp is updated upstream

[rantanplan]

 

 

You need to patch your local source to fix py3 syntax errors...

Full 2to3 diff for this plugin attached

 

(not sure how we'll proceed with vendors specific plugins like this? additional branch in bsp?)

 

for py3 all bsp changed

[athoik]

Yes, it makes sense to be unhappy, we used to compile all those years .py files to .pyo files.

 

With python3, now files are still compiled and moved to __pycache__ folder.

 

There is still a way to bring back py to pyc on same folder: python -m compileall -b .

 

https://stackoverflo...s-in-python-3-4

OE-A already used "compileall -b"

 

https://github.com/o...91066e69cb9fR25

 

Edit1. I guess we need a small patch to make legacy by default = True, that will force generating .pyc without big changes.

 

https://github.com/p...b/compileall.py

Edited by athoik, 21 January 2022 - 20:52.

[Huevos]

Yes...

Attached Files

•  5.jpg   129.99KB   6 downloads

[WanWizard]

Funny to see that when I brought it up, I was beaten to within an inch of my life for such a stupid idea...

[Huevos]

Funny to see that when I brought it up, I was beaten to within an inch of my life for such a stupid idea...

It has been done on OE-A so it is still possible to shoehorn the image into small flash boxes.

 

No idea how others have resources to build all those boxes. They must have a bank of build servers.

[blzr]

Funny to see that when I brought it up, I was beaten to within an inch of my life for such a stupid idea...

yup, posts #67-#73 from this very thread come to mind somehow...

//anyway, all's well that ends well   

[athoik]

Here is the patch for python3 to bring back the .pyc on the same folder.

 

diff --git a/Lib/importlib/_bootstrap_external.py b/Lib/importlib/_bootstrap_external.py
index fe31f43..ba972c8 100644
--- a/Lib/importlib/_bootstrap_external.py
+++ b/Lib/importlib/_bootstrap_external.py
@@ -389,7 +389,7 @@ def cache_from_source(path, debug_override=None, *, optimization=None):
     tag = sys.implementation.cache_tag
     if tag is None:
         raise NotImplementedError('sys.implementation.cache_tag is None')
-    almost_filename = ''.join([(base if base else rest), sep, tag])
+    almost_filename = ''.join([(base if base else rest)])
     if optimization is None:
         if sys.flags.optimize == 0:
             optimization = ''
@@ -426,7 +426,7 @@ def cache_from_source(path, debug_override=None, *, optimization=None):
             head.lstrip(path_separators),
             filename,
         )
-    return _path_join(head, _PYCACHE, filename)
+    return _path_join(head, filename)


 def source_from_cache(path):

 

root@osmio4k:~/test# ls
libpython3.9-1.0_3.9.9-r0_armv7ahf-neon.ipk  python3-core_3.9.9-r0_armv7ahf-neon.ipk      test.py
root@osmio4k:~/test# python -c "import test"
hello
root@osmio4k:~/test# ls
libpython3.9-1.0_3.9.9-r0_armv7ahf-neon.ipk  python3-core_3.9.9-r0_armv7ahf-neon.ipk      test.py                                      test.pyc

 

I guess we can even change it to use .pyo instead of .pyc! But nope, let's keep .pyc to recognize 3or2

[athoik]

Also need to bring back the default optimization level to 1 as we used to, because it seems that we are getting multiple files now.

 

Eg.

WARNING: enigma2-plugin-systemplugins-servicemp3-gitAUTOINC+aa60f9a1d0-r0 do_package: QA Issue: enigma2-plugin-systemplugins-servicemp3: Files/directories were installed but not shipped in any package:
  /usr/lib/enigma2/python/Plugins/SystemPlugins/ServiceMP3/__init__.pyc
  /usr/lib/enigma2/python/Plugins/SystemPlugins/ServiceMP3/__init__.opt-1.pyc

 

diff --git a/Lib/importlib/_bootstrap_external.py b/Lib/importlib/_bootstrap_external.py
index fe31f43..6ce18db 100644
--- a/Lib/importlib/_bootstrap_external.py
+++ b/Lib/importlib/_bootstrap_external.py
@@ -389,13 +389,13 @@ def cache_from_source(path, debug_override=None, *, optimization=None):
     tag = sys.implementation.cache_tag
     if tag is None:
         raise NotImplementedError('sys.implementation.cache_tag is None')
-    almost_filename = ''.join([(base if base else rest), sep, tag])
+    almost_filename = ''.join([(base if base else rest)])
     if optimization is None:
         if sys.flags.optimize == 0:
             optimization = ''
         else:
             optimization = sys.flags.optimize
-    optimization = str(optimization)
+    optimization = ''
     if optimization != '':
         if not optimization.isalnum():
             raise ValueError('{!r} is not alphanumeric'.format(optimization))
@@ -426,7 +426,7 @@ def cache_from_source(path, debug_override=None, *, optimization=None):
             head.lstrip(path_separators),
             filename,
         )
-    return _path_join(head, _PYCACHE, filename)
+    return _path_join(head, filename)


 def source_from_cache(path):
diff --git a/Python/initconfig.c b/Python/initconfig.c
index 116ee33..0e85d62 100644
--- a/Python/initconfig.c
+++ b/Python/initconfig.c
@@ -627,7 +627,7 @@ _PyConfig_InitCompatConfig(PyConfig *config)
     config->bytes_warning = -1;
     config->inspect = -1;
     config->interactive = -1;
-    config->optimization_level = -1;
+    config->optimization_level = 1;
     config->parser_debug= -1;
     config->write_bytecode = -1;
     config->verbose = -1;
@@ -658,7 +658,7 @@ config_init_defaults(PyConfig *config)
     config->bytes_warning = 0;
     config->inspect = 0;
     config->interactive = 0;
-    config->optimization_level = 0;
+    config->optimization_level = 1;
     config->parser_debug= 0;
     config->write_bytecode = 1;
     config->verbose = 0;
@@ -1996,7 +1996,7 @@ config_parse_cmdline(PyConfig *config, PyWideStringList *warnoptions,
         /* case 'J': reserved for Jython */

         case 'O':
-            config->optimization_level++;
+            config->optimization_level = 1;
             break;

         case 'B':

[athoik]

What will be the plan guys? Go for .pyc?

[WanWizard]

Instead of .py you mean?

 

Yes, absolutely, identical to how we do it now with the .pyo's in the image, and the .py's in a -src package.

[athoik]

That means extra job for hardknott-py3 branch. From .pyo to .py and now back to .pyc

 

 

 

@Hains are you ok with that? Send the patch for .pyc ?

[WanWizard]

Why are those patches needed?

 

In the current  bb's (in develop) we already run a "compileall" to make the pyo's for python2, and split the py's off into a separate package. That would make pyc's for python3, so that part is covered.

 

Python3 will use the pyc by default, if no py exists in the same directory (and never is the py does exist, which is different from python2, but something I have no issue with, only developers need source on their box).

[Beeker]

That means extra job for hardknott-py3 branch. From .pyo to .py and now back to .pyc

 

 

 

@Hains are you ok with that? Send the patch for .pyc ?

 

To save boxes with small flash memory I agree.

[athoik]

Why are those patches needed?

 

In the current  bb's (in develop) we already run a "compileall" to make the pyo's for python2, and split the py's off into a separate package. That would make pyc's for python3, so that part is covered.

 

Python3 will use the pyc by default, if no py exists in the same directory (and never is the py does exist, which is different from python2, but something I have no issue with, only developers need source on their box).

We already changed logic on several packages to include .py. Now we have to move .py back to -src and deploy .pyc files.

 

Also we need to change some plugins to check for .pyc file existance. Used to check .pyo, changed to .py and now to .pyc

 

Also we used to delete them! So now we need them back.

        find ${D}${libdir}/enigma2/python/ -name '*.pyc' -exec rm {} \;

Edited by athoik, 23 January 2022 - 16:16.

[athoik]

@WanWizard, could you please create a python3 branch from current master and provide push,merge rights for Hains, Nautilus7, Athoik (me)  for the following:

 

https://github.com/O...enigma2-plugins

https://github.com/OpenPLi/servicemp3

[WanWizard]

Hains and you should have access now, I've send an invitation to Nautilus7.

[athoik]

Great! Thanks!

[nautilus7]

Hains and you should have access now, I've send an invitation to Nautilus7.

 

 

Just accepted. Thanks!