Ren'Py引擎源代码解读(1)——脚本文件加载

因为想要尝试把Ren'Py移植到Cocos上，尽可能的使用原来的rpy文件，这就难免要解析rpy文件，因此就参考了一下Ren'Py自己是怎么解析脚本的。

文件加载

那么从哪里看起呢？先简要看一下Ren'Py的启动过程。启动脚本肯定是根目录下的renpy.py了，前面是一堆设置路径的方法，我们暂且跳过，直接看最后一行

if __name__ == "__main__":
    main()

这里调用了main()函数，算是入口了，main()函数在上面一段

def main():

    renpy_base = path_to_renpy_base()

    # Add paths.
    if os.path.exists(renpy_base + "/module"):
        sys.path.append(renpy_base + "/module")

    sys.path.append(renpy_base)

    # This is looked for by the mac launcher.
    if os.path.exists(renpy_base + "/renpy.zip"):
        sys.path.append(renpy_base + "/renpy.zip")

    # Ignore warnings that happen.
    warnings.simplefilter("ignore", DeprecationWarning)

    # Start Ren'Py proper.
    try:
        import renpy.bootstrap
    except ImportError:
        print >>sys.stderr, "Could not import renpy.bootstrap. Please ensure you decompressed Ren'Py"
        print >>sys.stderr, "correctly, preserving the directory structure."
        raise

    if android:
        renpy.linux = False
        renpy.android = True

    renpy.bootstrap.bootstrap(renpy_base)

前面依然是设置路径导入package，最关键的还是在最后一行，调用了bootstrap下面的bootstrap()函数，并传入了renpy的根目录。

好那么我们移步bootstrap.py，在renpy文件夹下面，前面又是一堆设定目录的代码，直到第289行

                renpy.main.main()

又调用了main的main()，继续跳转。

我们要找的在main.py的第239行

    # Load the script.
    renpy.game.exception_info = 'While loading the script.'
    renpy.game.script = renpy.script.Script()

    # Set up error handling.
    renpy.exports.load_module("_errorhandling")
    renpy.style.build_styles() # @UndefinedVariable
    renpy.display.screen.prepare_screens()

    # Load all .rpy files.
    renpy.game.script.load_script() # sets renpy.game.script.

把一个Script对象赋给了renpy.game.scritp，然后调用load_script函数。

Script.py这个类是处理脚本的主要类，前面有这么一段说明注释：

    This class represents a Ren'Py script, which is parsed out of a
    collection of script files. Once parsing and initial analysis is
    complete, this object can be serialized out and loaded back in,
    so it shouldn't change at all after that has happened.

意思是说这个类代表了Ren'Py里面的脚本，在初始化完成之后可以进行序列化和反序列化，应该就是和存盘有关了。

def __init__(self):
        """
        Loads the script by parsing all of the given files, and then
        walking the various ASTs to initialize this Script object.
        """

        # Set us up as renpy.game.script, so things can use us while
        # we're loading.
        renpy.game.script = self

        if os.path.exists(renpy.config.renpy_base + "/lock.txt"):
            self.key = file(renpy.config.renpy_base + "/lock.txt", "rb").read()
        else:
            self.key = None

        self.namemap = { }
        self.all_stmts = [ ]
        self.all_pycode = [ ]

        # A list of statements that haven't been analyzed.
        self.need_analysis = [ ]

        self.record_pycode = True

        # Bytecode caches.
        self.bytecode_oldcache = { }
        self.bytecode_newcache = { }
        self.bytecode_dirty = False

        self.translator = renpy.translation.ScriptTranslator()

        self.init_bytecode()
        self.scan_script_files()

        self.translator.chain_translates()

        self.serial = 0

__init__里面进行了初始化操作，声明了一些成员变量。

namemap：AST节点及其名称的map，AST指的是抽象语法树，是对代码分析过后生成的，可以用于jump等操作。name有可能是用户自定义的string或者是自动生成的序号。

all_stmts：保存了所有文件里面的所有语句。

然后进行了一系列初始化操作：

    def init_bytecode(self):
        """
        Init/Loads the bytecode cache.
        """

        # Load the oldcache.
        try:
            version, cache = loads(renpy.loader.load("bytecode.rpyb").read().decode("zlib"))
            if version == BYTECODE_VERSION:
                self.bytecode_oldcache = cache
        except:
            pass

这部分主要是从打包的bytecode文件中读取cache，我们从这里也可以看出来用的是zlib来做的压缩编码。因为本文主要是讲解脚本读取这部分，而二进制文件解析这一块关系不大，所以先跳过了。

然后是scan_script_files()这个函数，这里开始进入正戏了，看名字就知道是要扫描脚本文件。

    def scan_script_files(self):
        """
        Scan the directories for script files.
        """

        # A list of all files in the search directories.
        dirlist = renpy.loader.listdirfiles()

        # A list of directory, filename w/o extension pairs. This is
        # what we will load immediately.
        self.script_files = [ ]

        # Similar, but for modules:
        self.module_files = [ ]

        for dir, fn in dirlist: #@ReservedAssignment

            if fn.endswith(".rpy"):
                if dir is None:
                    continue

                fn = fn[:-4]
                target = self.script_files
            elif fn.endswith(".rpyc"):
                fn = fn[:-5]
                target = self.script_files
            elif fn.endswith(".rpym"):
                if dir is None:
                    continue

                fn = fn[:-5]
                target = self.module_files
            elif fn.endswith(".rpymc"):
                fn = fn[:-6]
                target = self.module_files
            else:
                continue

            if (fn, dir) not in target:
                target.append((fn, dir))

其中还会用到loader的listdirfiles，我们放在一起看

def listdirfiles(common=True):
    """
    Returns a list of directory, file tuples known to the system. If
    the file is in an archive, the directory is None.
    """

    rv = [ ]

    seen = set()

    if common:
        list_apks = apks
    else:
        list_apks = game_apks

    for apk in list_apks:

        for f in apk.list():

            # Strip off the "x-" in front of each filename, which is there
            # to ensure that aapt actually includes every file.
            f = "/".join(i[2:] for i in f.split("/"))

            if f not in seen:
                rv.append((None, f))
                seen.add(f)

    for i in renpy.config.searchpath:

        if (not common) and (renpy.config.commondir) and (i == renpy.config.commondir):
            continue

        i = os.path.join(renpy.config.basedir, i)
        for j in walkdir(i):
            if j not in seen:
                rv.append((i, j))
                seen.add(j)

    for _prefix, index in archives:
        for j in index.iterkeys():
            if j not in seen:
                rv.append((None, j))
                seen.add(j)


    return rv

和apk相关的都是为andorid平台准备的，我们先跳过去，从第二个for循环开始看，遍历searchpath里面的路径，然后通过walkdir递归调用去获取里面所有文件的路径。看完searchpath再去处理archives里面的文件，最后返回<路径,文件名>这样的tuple组成的数组。

然后我们回到scan_script_files()这个函数里面，获取到文件列表之后开始根据扩展名分类，把所有文件分为script_file和module_file两类。

到此为止Script类的初始化工作就完成了，让我们回到main.py之后运行的是这段代码

    # Load all .rpy files.
    renpy.game.script.load_script() # sets renpy.game.script.

开始正式加载script文件

    def load_script(self):

        script_files = self.script_files

        # Sort script files by filename.
        script_files.sort()

        initcode = [ ]

        for fn, dir in script_files: #@ReservedAssignment
            self.load_appropriate_file(".rpyc", ".rpy", dir, fn, initcode)

        # Make the sort stable.
        initcode = [ (prio, index, code) for index, (prio, code) in
                     enumerate(initcode) ]

        initcode.sort()

        self.initcode = [ (prio, code) for prio, index, code in initcode ]

先对之前获取到的脚本文件数组做个排序，然后要调用load_appropriate_file选择从哪里加载文件，

    def load_appropriate_file(self, compiled, source, dir, fn, initcode): #@ReservedAssignment
        # This can only be a .rpyc file, since we're loading it
        # from an archive.
        if dir is None:

            rpyfn = fn + source
            lastfn = fn + compiled
            data, stmts = self.load_file(dir, fn + compiled)

            if data is None:
                raise Exception("Could not load from archive %s." % (lastfn,))

        else:

            # Otherwise, we're loading from disk. So we need to decide if
            # we want to load the rpy or the rpyc file.
            rpyfn = dir + "/" + fn + source
            rpycfn = dir + "/" + fn + compiled

            renpy.loader.add_auto(rpyfn)

            if os.path.exists(rpyfn) and os.path.exists(rpycfn):

                # Use the source file here since it'll be loaded if it exists.
                lastfn = rpyfn

                rpydigest = md5.md5(file(rpyfn, "rU").read()).digest()

                data, stmts = None, None

                try:
                    f = file(rpycfn, "rb")
                    f.seek(-md5.digest_size, 2)
                    rpycdigest = f.read(md5.digest_size)
                    f.close()

                    if rpydigest == rpycdigest and \
                        not (renpy.game.args.command == "compile" or renpy.game.args.compile): #@UndefinedVariable

                        data, stmts = self.load_file(dir, fn + compiled)

                        if data is None:
                            print "Could not load " + rpycfn

                except:
                    pass

                if data is None:
                    data, stmts = self.load_file(dir, fn + source)

            elif os.path.exists(rpycfn):
                lastfn = rpycfn
                data, stmts = self.load_file(dir, fn + compiled)

            elif os.path.exists(rpyfn):
                lastfn = rpyfn
                data, stmts = self.load_file(dir, fn + source)

        if data is None:
            raise Exception("Could not load file %s." % lastfn)

        # Check the key.
        if self.key is None:
            self.key = data['key']
        elif self.key != data['key']:
            raise Exception( fn + " does not share a key with at least one .rpyc file. To fix, delete all .rpyc files, or rerun Ren'Py with the --lock option.")

        self.finish_load(stmts, initcode, filename=rpyfn)

如果目录为空就是从archive里面加载，否则就从磁盘文件读取，这里要判断一下是加载原始脚本文件rpy还是编译后的字节码rpyc，如果两种文件同时存在的话就计算rpy文件的md5值，与rpyc里面提取出来的md5比较，相同的话就说明没有变化，直接加载rpyc，从中读取数据，如果没能读取到数据或是抛出异常则尝试从rpy文件读取。具体加载的方法在load_file()里面

    def load_file(self, dir, fn): #@ReservedAssignment

        if fn.endswith(".rpy") or fn.endswith(".rpym"):

            if not dir:
                raise Exception("Cannot load rpy/rpym file %s from inside an archive." % fn)

            fullfn = dir + "/" + fn

            stmts = renpy.parser.parse(fullfn)

            data = { }
            data['version'] = script_version
            data['key'] = self.key or 'unlocked'

            if stmts is None:
                return data, [ ]

            # See if we have a corresponding .rpyc file. If so, then
            # we want to try to upgrade our .rpy file with it.
            try:
                self.record_pycode = False
                old_data, old_stmts = self.load_file(dir, fn + "c")
                self.merge_names(old_stmts, stmts)
                del old_data
                del old_stmts
            except:
                pass
            finally:
                self.record_pycode = True

            self.assign_names(stmts, fullfn)

            try:
                rpydigest = md5.md5(file(fullfn, "rU").read()).digest()
                f = file(dir + "/" + fn + "c", "wb")
                f.write(dumps((data, stmts), 2).encode('zlib'))
                f.write(rpydigest)
                f.close()
            except:
                pass

        elif fn.endswith(".rpyc") or fn.endswith(".rpymc"):

            f = renpy.loader.load(fn)

            try:
                data, stmts = loads(f.read().decode('zlib'))
            except:
                return None, None

            if not isinstance(data, dict):
                return None, None

            if self.key and data.get('key', 'unlocked') != self.key:
                return None, None

            if data['version'] != script_version:
                return None, None

            f.close()
        else:
            return None, None

        return data, stmts

加载文件时候也需要根据文件类型分别处理，如果加载的是原始脚本文件，就需要进行parse操作，因为parse相对于文件加载是独立的一块，我们在下一章单独讲，这里parse之后获取到了若干语句，然后把之前编译过的同名的rpyc文件加载进来，新旧两组语句做merge操作，就是更新编译结果，更新之后还要调用assign_names来给语句编号，然后将rpy的md5值写入到更新后的rpyc的结尾。如果直接加载编译后的rpyc文件就不需要parse操作，直接load进来解码，返回语句集和相应的version，key就可以了。

加载工作完成后还有收尾工作finish_load