+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Development (https://forum.kodi.tv/forumdisplay.php?fid=32)
+--- Forum: Add-ons (https://forum.kodi.tv/forumdisplay.php?fid=26)
+---- Forum: Python 3 migration (https://forum.kodi.tv/forumdisplay.php?fid=281)
+---- Thread: Library dependencies to Update to Python 3 (/showthread.php?tid=348400)
Library dependencies to Update to Python 3 - docwra - 2019-10-14
Please reply below if you find any Library modules that still need to be be updated to Python 3.
Feel free to discuss ones that are complete as well.
RE: Dependencies to Update to Python 3 - docwra - 2019-10-14
List of Python Modules that need to be upgraded
.......
PIL - https://forum.kodi.tv/showthread.php?tid=348192 https://github.com/xbmc/xbmc/pull/17143 *Fixed thx paxxi*
BeautifulSoup4 - https://forum.kodi.tv/showthread.php?tid=347007 *Fixed*
script.common.plugin.cache - https://github.com/anxdpanic/script.common.plugin.cache *Fixed thx to anxdpanic*
script.module.mechanize - http://mirrors.kodi.tv/addons/matrix/script.module.mechanize
pycryptodomex - https://pypi.org/project/pycryptodomex/
script.module.parsedom
script.module.elementtree
script.module.inputstreamhelper
script.module.routing *Fixed thx to sualfred*
script.module.youtube.dl
script.extendedinfo -> discontinued. Skinners should use https://forum.kodi.tv/showthread.php?tid=345847
RE: Library dependencies to Update to Python 3 - Roman_V_M - 2019-10-14
Wrong list. Usually, general-purpose libraries have been supporting Python 3 for a long time, so your list will be long. It is better to list libraries that have not been updated yet.
RE: Library dependencies to Update to Python 3 - docwra - 2019-10-14
(2019-10-14, 19:51)Roman_V_M Wrote: Wrong list. Usually, general-purpose libraries have been supporting Python 3 for a long time, so your list will be long. It is better to list libraries that have not been updated yet.
Ha! yes good point. I've changed it.
RE: Library dependencies to Update to Python 3 - Snapcase - 2019-10-14
A couple of us need PIL as mentioned in this thread
RE: Library dependencies to Update to Python 3 - docwra - 2019-10-15
Thanks updated.
RE: Library dependencies to Update to Python 3 - Varstahl - 2019-10-15
Mechanize needs an upgrade, since earlier this year it received Py3 support. Adding MechanicalSoup could be an option too.
Also, as far as I could tell, while BeautifulSoup4 has been upgraded, the optional dependency for CSS selection (soupsieve) is not available, that might also be a good addition. Sorry, this was wrong, there is a 4.6.3 in the matrix branch.
RE: Library dependencies to Update to Python 3 - ronie - 2019-10-16
script.module.mechanize 0.4.3 has been added to the matrix repo.
RE: Library dependencies to Update to Python 3 - Varstahl - 2019-10-16
@ronie I don't think mechanize will work with Kodi as it is, I've been tinkering with it up until now (it's one of those modules I didn't mention in the other thread). Here in line 189 it requires html_parser to function. Now, html_parser is provided by html5-parser, which is a binary+python script module. If Kodi wants to support binary-free modules, this just won't work.
On my end I bit the bullet and crafted the MechanicalSoup v0.12.0 module for Kodi which offers similar functionalities but is py only. It will require some refactoring but at least it's binary free. Will test it some more tomorrow.
RE: Library dependencies to Update to Python 3 - ronie - 2019-10-16
it's a try/except.. it will fallback to html5lib if html5_parser isn't available.
we have html5lib in our repo, so this shouldn't be a problem.
RE: Library dependencies to Update to Python 3 - Varstahl - 2019-10-16
Strange, it's failing on my end, I'll have to recheck the logs.
RE: Library dependencies to Update to Python 3 - ronie - 2019-10-16
problem might be on our end, i forgot to add the html5lib requirement to the addon.xml file of script.module.mechanize
will fix that asap
RE: Library dependencies to Update to Python 3 - Varstahl - 2019-10-16
So, while testing, this happened:
log:EXCEPTION Thrown (PythonToCppException) : -->Python callback/script returned the following error<--
- NOTE: IGNORING THIS CAN LEAD TO MEMORY LEAKS!
Error Type: <class 'TypeError'>
Error Contents: parse() got an unexpected keyword argument 'transport_encoding'
Traceback (most recent call last):
File "…\script.module.mechanize\lib\mechanize\_html.py", line 189, in content_parser
from html5_parser import parse
ModuleNotFoundError: No module named 'html5_parser'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "…\plugin.video.amazon-test\resources\lib\network.py", line 460, in LogIn
if 'signIn' not in [i.name for i in br.forms()]:
File "…\script.module.mechanize\lib\mechanize\_mechanize.py", line 526, in forms
return self._factory.forms()
File "…\script.module.mechanize\lib\mechanize\_html.py", line 320, in forms
self._current_forms, self._current_global_form = self._get_forms()
File "…\script.module.mechanize\lib\mechanize\_html.py", line 335, in _get_forms
if self.root is None:
File "…\script.module.mechanize\lib\mechanize\_html.py", line 300, in root
transport_encoding=transport_encoding)
File "…\script.module.mechanize\lib\mechanize\_html.py", line 195, in content_parser
return parse(data, **kw)
TypeError: parse() got an unexpected keyword argument 'transport_encoding'
-->End of Python script error report<--
From what I read, Mechanize needs a specific version of html5lib which is not available from the repo (0.999.0 won't work), and (take it with a bit of salt as I haven't confirmed this) installing that specific version breaks (or used to break) BeautifulSoup. In order for mechanize to work it might be needed to embed that specific version of html5lib into it, instead of using the one from the repos. I didn't test this yet as my "partner in crime" already completed the migration to MechanicalSoup, but I'm dropping these info here in case someone needs them.
RE: Library dependencies to Update to Python 3 - arvvoid - 2019-10-17
I done more testing on different platforms with the hbo go eu add-on (https://github.com/arvvoid/plugin.video.hbogoeu) and found out there is a problem with pycryptodomex (script.module.pycryptodome) on windows.
This is te import that fail:
python:
try:
from Cryptodome import Random
from Cryptodome.Cipher import AES
from Cryptodome.Util import Padding
except ImportError:
# no Cryptodome gracefully fail with an informative message
log:
2019-10-16 23:51:18.628 T:13504 DEBUG: [plugin.video.hbogoeu] MISSING Cryptodome dependency...exiting...Traceback (most recent call last):
File "D:\KodiPython3\Kodi\portable_data\addons\plugin.video.hbogoeu\hbogolib\handler.py", line 32, in <module>
from Cryptodome import Random
ModuleNotFoundError: No module named 'Cryptodome'
The situation on different platforms kodi/versions is as follows:
Kodi 18 Python 2
Windows - work out of the box no issues
LibreElec, CoreElec - work out of the box no issues
OSMC - work after installing pycryptodomex using pip
Various Linux - work after installing pycryptodomex using pip
MacOS - work out of the box no issues
Kodi 19 Python 2
Windows - work out of the box no issues
Generic Linux - work after installing pycryptodomex using pip
MacOS - work out of the box no issues
Kodi 19 Python 3
Windows - import fail even tho the module seams to be installed/present
Generic Linux - work after installing pycryptodomex using pip
MacOS - work after installing pycryptodomex using pip
RE: Library dependencies to Update to Python 3 - Roman_V_M - 2019-10-17
(2019-10-17, 09:00)arvvoid Wrote: I done more testing on different platforms with the hbo go eu add-on (https://github.com/arvvoid/plugin.video.hbogoeu) and found out there is a problem with pycryptodomex (script.module.pycryptodome) on windows.
Kodi on generic Linuxes share packages with system Python installations.
For Kodi on Windows all binary dependencies are pre-packaged and downloaded before compilation. Somebody with access to our mirrors needs to update Pycryptodome (and Pillow for that matter) with correct .pyd binaries compiled against Python 3.7.