python import at runtime

1. problem

对于python新手来说，可能常会遇到import找不到module的问题。尤其在使用相对引用时，可能会被这样的错误整的摸不着头脑，如ValueError: attempted relative import with no known parent package。

2. introduction

在python中，import分为两种：绝对引用和相对引用。
相对引用以前导点（”.”或者”..”）开始， “.”表示引用当前包的某些模块，”..”或”…”表示引入父包的某个子模块。注意，在python中我们需要使用init.py来注明package（貌似在python3的sub package可以不需要此文件了）。
比如，我们的项目结构如下:

在moduleC.py中引入其他模块，则需要这样写：

from . import moduleD
from ..subpackage1 import moduleB
from .. import moduleA

而绝对引用则相对直接，以包名开始，如：import x.y 或者from x import y。
那么在使用relative import的时候，为什么会出现no known parent package的import error?先来看下官方的说法PEP 328[2]。

Relative imports use a module’s name attribute to determine that module’s position in the package hierarchy. If the module’s name does not contain any package information (e.g. it is set to ‘main‘) then relative imports are resolved as if the module were a top level module, regardless of where the module is actually located on the file system.

可以看到，当我们越过top level(test, 见图2.1) 直接运行子模块的某个module（如: python -m subpackage2.moduleC）时，subpackage2被作为一个parent package(top level)来处理了，所以subpackage2是找不到其parent package的。
以下是在windows 7上运行的结果:

1> 相对引用(relative import)

moduleC.py:

print('__file__ = %s, __name__ = %s, __package__ = %s' % (__file__, __name__, __package__))

from . import moduleD
from ..subpackage1 import moduleB
from .. import moduleA

error log:

C:\Users\Future\PycharmProjects\test>python -m subpackage2.moduleC
__file__ = C:\Users\Future\PycharmProjects\test\subpackage2\moduleC.py, __name__ = __main__, __package__ = subpackage2
__file__ = C:\Users\Future\PycharmProjects\test\subpackage2\moduleD.py, __name__ = subpackage2.moduleD, __package__ = subpackage2
Traceback (most recent call last):
  File "C:\Users\Future\AppData\Local\Programs\Python\Python36\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Users\Future\AppData\Local\Programs\Python\Python36\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\Future\PycharmProjects\test\subpackage2\moduleC.py", line 4, in <module>
    from ..subpackage1 import moduleB
ValueError: attempted relative import beyond top-level package

可以看到moduleB.py已经超出了subpackage2的“控制范围”。

2> 绝对引用(absolute import)

moduleC.py(version 1):

print('__file__ = %s, __name__ = %s, __package__ = %s' % (__file__, __name__, __package__))

from test.subpackage2 import moduleD
from test.subpackage1 import moduleB
from test import moduleA

error log:

C:\Users\Future\PycharmProjects\test>python -m subpackage2.moduleC
__file__ = C:\Users\Future\PycharmProjects\test\subpackage2\moduleC.py, __name__ = __main__, __package__ = subpackage2
Traceback (most recent call last):
  File "C:\Users\Future\AppData\Local\Programs\Python\Python36\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Users\Future\AppData\Local\Programs\Python\Python36\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\Future\PycharmProjects\test\subpackage2\moduleC.py", line 3, in <module>
    from test.subpackage2 import moduleD
ModuleNotFoundError: No module named 'test.subpackage2'

moduleC.py(version 2):

print('__file__ = %s, __name__ = %s, __package__ = %s' % (__file__, __name__, __package__))

from subpackage2 import moduleD
from test.subpackage1 import moduleB
from test import moduleA

error log:

C:\Users\Future\PycharmProjects\test>python -m subpackage2.moduleC
__file__ = C:\Users\Future\PycharmProjects\test\subpackage2\moduleC.py, __name__ = __main__, __package__ = subpackage2
__file__ = C:\Users\Future\PycharmProjects\test\subpackage2\moduleD.py, __name__ = subpackage2.moduleD, __package__ = subpackage2
Traceback (most recent call last):
  File "C:\Users\Future\AppData\Local\Programs\Python\Python36\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Users\Future\AppData\Local\Programs\Python\Python36\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\Future\PycharmProjects\test\subpackage2\moduleC.py", line 4, in <module>
    from test.subpackage1 import moduleB
ModuleNotFoundError: No module named 'test.subpackage1'

从上面的error log中可以明显地看出，subpackage2被视为top level， 超出其范围的模块已不被识别。

3. solution

当然，遇到这种问题也不难解决， 我们只需改进下项目运行结构，从test层运行模块，目的是让test下的模块在运行的时候以test为parent package。

moduleC.py的运行结果：

C:\Users\Future\PycharmProjects>python -m  test.subpackage2.moduleC
__file__ = C:\Users\Future\PycharmProjects\test\subpackage2\moduleC.py, __name__ = __main__, __package__ = test.subpackage2
__file__ = C:\Users\Future\PycharmProjects\test\subpackage2\moduleD.py, __name__ = test.subpackage2.moduleD, __package__ = test.subpackage2
__file__ = C:\Users\Future\PycharmProjects\test\subpackage1\moduleB.py, __name__ = test.subpackage1.moduleB, __package__ = test.subpackage1
__file__ = C:\Users\Future\PycharmProjects\test\moduleA.py, __name__ = test.moduleA, __package__ = test

4. conclusion

正如python的官方文档[1]所说，python的package和module仅仅是类似于文件系统中的directory和file，但其本质并非这种简单的映射关系。所以在运行python模块时，需要从最外层package一级一级地深入。就好像，python的模块被包含在一个“包域”中，我们需要先找到这个域的最大边界，而后才能顺利地运行其中的模块。

5. references

[1] https://docs.python.org/3/reference/import.html
[2] https://www.python.org/dev/peps/pep-0328/
[3] https://napuzba.com/a/import-error-relative-no-parent