《CPython Internals》阅读笔记：p97-p117

《CPython Internals》学习第 7 天，p97-p117 总结，总计 21 页。

一、技术总结

1.词法分析(lexical analysis)

根据《Compilers-Principles, Techniques, and Tools》(《编译原理》第2版)第 5 页：The first phase of a compiler is called lexical analysis or scanning. The lexcical analyzer reads the stream of characters making up the source program and groups the characters into meaningful sequences called lexemes. For each lexeme, the lexical analyzer produces as output a token of the form <token-name, attribute-value> that is passes on to the subsequent phase, syntax analysis。

执行词法分析(lexical analysis)的 component 称为 lexical analyzer（有时候也称为lexer, scanner, tokenizer）。

2.关于parser-tokenizer的一疑问

p92, Creating a concrete syntax tree using a parser-tokenizer, or lexer.

p92，The CST is