[Deepin - Pycharm调试记录] Pyinstaller索引系统库问题

Deepin - Pycharm调试记录 - Pyinstaller索引不到系统库

现象

在Pycharm的Terminal中执行Pyinstaller指令时候遇到如下报错提示

OSError: Python library not found: libpython3.7mu.so.1.0, libpython3.7.so.1.0, libpython3.7.so, libpython3.7m.so.1.0, libpython3.7m.so

分析原因

step1

分析其原因是pyinstaller搜索不到libpython3.7.so.1.0

于是find了一下/usr/lib下所有安装的系统库，发现只有python2.7和python3.5两个版本。

./usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
./usr/lib/x86_64-linux-gnu/libpython2.7.so
./usr/lib/x86_64-linux-gnu/libpython3.5m.so
./usr/lib/x86_64-linux-gnu/libpython3.5m.so.1
./usr/lib/x86_64-linux-gnu/libpython2.7.a
./usr/lib/x86_64-linux-gnu/libpython2.7.so.1
./usr/lib/x86_64-linux-gnu/libpython3.5m.a
./usr/lib/x86_64-linux-gnu/libpython3.5m.so.1.0

step2

这是因为我的python3.7版本安装是通过源码编译的，之前执行的命令如下

./configure --prefix=/usr/local/python-3.7.4 --enable-shared && make -j8 && make install

于是我在/usr/local路径下能找到libpython3.7.so.1.0

进一步分析

step3

因此可以推断出，pyinstaller的源码中，应该有一个地方是用于记录搜索系统库的地方。因此从完整的log出发

3959 INFO: Python library not in binary dependencies. Doing additional searching...
Traceback (most recent call last):
  File "/home/rodney/ProjectCode/pythonProject/py_demo/venv/bin/pyinstaller", line 8, in <module>
    sys.exit(run())
  File "/home/rodney/ProjectCode/pythonProject/py_demo/venv/lib/python3.7/site-packages/PyInstaller/__main__.py", line 124, in run
    run_build(pyi_config, spec_file, **vars(args))
  File "/home/rodney/ProjectCode/pythonProject/py_demo/venv/lib/python3.7/site-packages/PyInstaller/__main__.py", line 58, in run_build
    PyInstaller.building.build_main.main(pyi_config, spec_file, **kwargs)
  File "/home/rodney/ProjectCode/pythonProject/py_demo/venv/lib/python3.7/site-packages/PyInstaller/building/build_main.py", line 783, in main
    build(specfile, kw.get('distpath'), kw.get('workpath'), kw.get('clean_build'))
  File "/home/rodney/ProjectCode/pythonProject/py_demo/venv/lib/python3.7/site-packages/PyInstaller/building/build_main.py", line 715, in build
    exec(code, spec_namespace)
  File "/home/rodney/ProjectCode/pythonProject/py_demo/py_ui/build/ui_test.spec", line 19, in <module>
    noarchive=False)
  File "/home/rodney/ProjectCode/pythonProject/py_demo/venv/lib/python3.7/site-packages/PyInstaller/building/build_main.py", line 277, in __init__
    self.__postinit__()
  File "/home/rodney/ProjectCode/pythonProject/py_demo/venv/lib/python3.7/site-packages/PyInstaller/building/datastruct.py", line 155, in __postinit__
    self.assemble()
  File "/home/rodney/ProjectCode/pythonProject/py_demo/venv/lib/python3.7/site-packages/PyInstaller/building/build_main.py", line 501, in assemble
    self._check_python_library(self.binaries)
  File "/home/rodney/ProjectCode/pythonProject/py_demo/venv/lib/python3.7/site-packages/PyInstaller/building/build_main.py", line 605, in _check_python_library
    python_lib = bindepend.get_python_library_path()
  File "/home/rodney/ProjectCode/pythonProject/py_demo/venv/lib/python3.7/site-packages/PyInstaller/depend/bindepend.py", line 926, in get_python_library_path
    raise IOError(msg)
OSError: Python library not found: libpython3.7mu.so.1.0, libpython3.7.so.1.0, libpython3.7.so, libpython3.7m.so.1.0, libpython3.7m.so

可以看出在python_lib = bindepend.get_python_library_path()这句话抛出了异常，也就意味着进这个文件的上下文去寻找，定位该函数的所在文件为

./Pyinstaller/depend/bindepend.py。

step4

接下来就是阅读源码的过程，是个比较繁琐的工作，最终定位到一个叫find_Library的函数，大概位置在708行左右。函数比较长，但是其中有一个叫paths的变量引起了我们的注意，paths是个数组，正好有含有我们/usr/lib的系统库路径。

因此我们在后面添加我们实际的python3.7库的路径/usr/local/python-3.7.4/lib

def findLibrary(name):
    """
    Look for a library in the system.

    Emulate the algorithm used by dlopen. `name` must include the prefix, e.g., ``libpython2.4.so``.
    """
    assert compat.is_unix, "Current implementation for Unix only (Linux, Solaris, AIX, FreeBSD)"

    # Look in the LD_LIBRARY_PATH according to platform.
    if compat.is_aix:
        lp = compat.getenv('LIBPATH', '')
    elif compat.is_darwin:
        lp = compat.getenv('DYLD_LIBRARY_PATH', '')
    else:
        lp = compat.getenv('LD_LIBRARY_PATH', '')
    lib = _which_library(name, filter(None, lp.split(os.pathsep)))

    # Look in /etc/ld.so.cache
    # Solaris does not have /sbin/ldconfig. Just check if this file exists.
    if lib is None:
        utils.load_ldconfig_cache()
        lib = utils.LDCONFIG_CACHE.get(name)
        if lib:
            assert os.path.isfile(lib)

    # Look in the known safe paths.
    if lib is None:
        # Architecture independent locations.
        # paths = ['/lib', '/usr/lib']
        paths = ['/lib', '/usr/lib', '/usr/local/python-3.7.4/lib']
        # Architecture dependent locations.
        if compat.architecture == '32bit':
            paths.extend(['/lib32', '/usr/lib32'])
        else:
            paths.extend(['/lib64', '/usr/lib64'])
        # Machine dependent locations.
        if compat.machine == 'intel':
            if compat.architecture == '32bit':
                paths.extend(['/usr/lib/i386-linux-gnu'])
            else:
                paths.extend(['/usr/lib/x86_64-linux-gnu'])

        # On Debian/Ubuntu /usr/bin/python is linked statically with libpython. Newer Debian/Ubuntu with multiarch
        # support puts the libpythonX.Y.so in paths like /usr/lib/i386-linux-gnu/.
        try:
            # Module available only in Python 2.7+
            import sysconfig

            # 'multiarchsubdir' works on Debian/Ubuntu only in Python 2.7 and 3.3+.
            arch_subdir = sysconfig.get_config_var('multiarchsubdir')
            # Ignore if None is returned.
            if arch_subdir:
                arch_subdir = os.path.basename(arch_subdir)
                paths.append(os.path.join('/usr/lib', arch_subdir))
            else:
                logger.debug('Multiarch directory not detected.')
        except ImportError:
            logger.debug('Multiarch directory not detected.')

        if compat.is_aix:
            paths.append('/opt/freeware/lib')
        elif compat.is_hpux:
            if compat.architecture == '32bit':
                paths.append('/usr/local/lib/hpux32')
            else:
                paths.append('/usr/local/lib/hpux64')
        elif compat.is_freebsd or compat.is_openbsd:
            paths.append('/usr/local/lib')
        lib = _which_library(name, paths)

    # Give up :(
    if lib is None:
        return None

    # Resolve the file name into the soname
    if compat.is_freebsd or compat.is_aix or compat.is_openbsd:
        # On FreeBSD objdump does not show SONAME, and on AIX objdump does not exist, so we just return the lib we
        # have found.
        return lib
    else:
        dir = os.path.dirname(lib)
        return os.path.join(dir, _get_so_name(lib))

重新执行/调试

pyinstaller -F -w test.py