make_func.y源码分析

定义部分

该部分包括直接进入输出文件的c代码(2行到53行),以及用来建立分析程序的有关记号、数据结构以及文法规则的信息(56行到66行)。

 

16 – 23 这里声明的全局变量将在解析的同时被赋值,这些值,在执行edit_source -build_efuns时用来生成LPC语法的仿函数列表,同时也自动生成编译MudOS所需要的一些头文件。

37 – 51 LPC支持的各种数据类型。

56 – 59 默认情况下,actions和词法分析的返回值为整数,为了让解析程序支持多种数据类型,定义一个union,在bison程序处理后的文件make_func.tab.c中,这个union定义成:

typedef union {

    int number;

    char *string;

} yystype;

62     定义EBNF终结符号。

64     定义非终结符号。其返回值相关数据类型为intID在终结符号声明中出现,仍为终结符。

66     非终结符号。其返回值相关数据类型为char *NUM在终结符号声明中出现,仍为终结符。

如果61行到65行按照下面这种方式声明,用bison处理后的文件make_func.tab.c将一模一样:

%token <string> ID

%token <number> NUM

%token DEFAULT OPERATOR

%type <number> type arg_list basic typel arg_type typel2

%type <string> optional_ID optional_default

词法分析

该部分定义解析程序如何从源文本中分析出单词的规则。这项工作可以交给flex或者lex来完成,但是由于这个解析程序比较简单,因此自定义词法分析代码(328行到414行)。

 

330      词法分析函数必须声明为 int yylex()

334      yyin声明于edit_source.c,为func_spec.cpp的文件流。

338 – 340 记录源文件的行号。

341 – 347 当源文件某行以”!”开头时,表示解析文件失败。这种情况可能出现在链接了两个或三个内存分配函数库的情况下,见macros.h93行到95行:

#if (defined(SYSMALLOC) + defined(SMALLOC) + defined(BSDMALLOC)) > 1

!Only one malloc package should be defined

#endif

执行edit_source –build_func_spec “gcc –E”指令获得func_spec.c的源代码列表时,可能将 !Only one malloc package should be defined”这行包含到func_spec.cpp中。

348 – 363 当源文件某行以 #” 开头时,文件名和行号将保存到current_filecurrent_line中,这类似于预编译指令#line,默认情况下,发现错误的时候,报告的是func_spec.cpp的文件名称及行号,如果解析文本解析到以”#”开头的行时,比如” #1 "op_spec.c" 1” ,紧接着解析出现错误,将报告的是”op_spec.c”第一行,这将保证报告出错位置的正确性。

364 – 366 遇到文件结束标志,解析完毕,返回-1

368 – 364 解析到数字,返回NUM终结符号。

385 – 387 解析到字符串,返回IDDEFAULTOPERATOR的其中之一。

397 – 403标示符过长的错误处理。

406 – 407 如果匹配字符串”default”,则返回DEFAULT的值,这个值在bison处理后的文件make_fucn.tab.c中出现:”#define DEFAULT 259”

408 – 409如果匹配字符串”operator”,则返回OPERATOR的值,这个值在bison处理后的文件make_fucn.tab.c中出现:”#define OPERATOR 260”

411 – 413 其他字符串都将被视为ID,它们本身的值,比如”to_int”被压入到yylval.string中,这个值在action代码里的表现形式为$n,见81行和194行。在定义部分,ID相关的数据类型为char *

规则部分

该部分包括EBNF文法规则,以及在识别出相关的文法规则时被执行的action代码部分(69行到231行)。

待解析的句型实例:

operator add, subtract, multiply, divide, mod, and, or, xor, lsh, rsh;

int call_out(string | function, int,...);

 

除去相关的action代码,余下的EBNF文法规则整理如下:

specs à /* empty */

 | specs spec ;

 

spec à operator

| func;

 

operator à OPERATOR op_list ';' ;

 

op_list à op

| op_list ',' op ;

 

op à ID ;

 

func à type ID optional_ID '(' arg_list optional_default ')' ';' ;

 

optional_ID à ID

| /* empty */ ;

 

optional_default à /* empty */

| DEFAULT ':' NUM

| DEFAULT ':' ID ;

 

type à basic

| basic '*' ;

 

basic à ID ;

 

arg_list à /* empty */

| typel2

| arg_list ',' typel2 ;

 

typel2 à typel ;

 

arg_type à type ;

 

typel à arg_type

| typel '|' arg_type

| '.' '.' '.' ;

 

73 – 91   寻找句型:”OPERATOR ID( ,ID)* /;” ID的值将保存到oper_codes中,同时记录operator的个数op_code

93 – 231  寻找句型:”(ID|ID /*) ID [ID]{0,1} /(…”(不知道它的正则表达式该怎么写L,路漫漫其修远兮)

93       empty时,optional_ID的值为””,否则直接返回ID的值。

95       empty时,optional_default的值为”DEFAULT_NONE”

96 – 101  匹配句型”DEFAULT : NUM”,将NUM的值转换成string,令optional_default的值为这个string

102 – 107 匹配句型”DEFAULT : ID”,如果ID的值不为”F_THIS_OBJECT”,解析失败,否则,令optinal_default的值为”DEFAULT_THIS_OBJECT”

187 – 231 描述了arg_list的规则,这些代码比较复杂,整理为下面这张表,表中的第一列表示arg_list的各种句型,第一行为程序中出现全局变量和arg_list本身的值,其他行列的值表示这些全局变量在arg_list匹配不同句型的情况下的值,之所以将它们总结出来,是因为在funcaction函数里要用到:

 

arg_list

min_arg

limit_max

curr_arg_type_size

/*empty*/

0

-1

0

0

ID

(ID != VOID)

1

-1

0

1

ID

(ID == VOID)

1

0

0

0

ID | ID

1

-1

0

2

ID | ID

(某个ID == VOID)

1

0

0

1

ID , ID

2

-1

0

3

ID , ID

(某个ID == VOID)

1

0

0

2

1

-1

1

0

ID, …

2

1

1

1

……(省略)

 

 

 

 

现在来分析一下各个全局变量的意义。

arg_list:函数参数的个数。

min_arg:函数中参数的最小数量,这个数量由参数中出现”VOID””…”来决定。

limit_max:表示是否限制参数个数,为1的时候不限制,为0的时候有最大参数限制,这个限制为arg_list的个数。

curr_arg_type_size:这个参数和arg_list有关,如果是逗号相隔的参数列表,那么两个参数之间必须有个0作为分隔符,因此如果是ID, IDcurr_arg_type_size的值为3;如果某个IDVOID,将不计算在内。

 

114 – 116 如果在arg_list的词法分析中,min_arg的值一直保持为-1,那么将arg_list的值赋给它,min_arg的最大值为4,超过则报错,程序退出。

117 – 139 如果optional_idNULL,则对f_nameefun1_code或者efun_code赋值,区别在于其参数的个数,如果是单参数则赋值到efun1_code,其他将赋值到efun_code

139 – 149 如果optional_id不为NULL,例如:object this_user this_player( int default: 0); 这表示这个函数是一个aliasf_name = “F_THIS_PLAYER | F_ALIAS_FLAG”

150 – 167 将参数列表curr_arg_types赋值到arg_types中,但是如果之前已经有curr_arg_types 中的连续元素存在,将不进行赋值操作。这一点,保证,参数列表的唯一性,见curr_arg_typesarg_types的声明注释:Store the types of the current efun. They will be copied into the arg_types list if they were not already there (to save memory).

167 – 170 符合条件的源文件只有:unknown call_other(object | string | object *, string | mixed *,...); 如果遇到这样的句型,将type的值,也就是$1改为MIXED

171 – 174 buff赋值,其赋值语句可解释为:ID,f_name,0,0,min_arg,limit_max ? -1 : arg_list,type != VOID ? ctype(type) : "TYPE_NOVALUE", etype(0), etype(1), eytpe(2), etype(3), i, optional_default

178 – 180 追加buffbuf数组中。

235 – 261 取得efuns的返回值数据类型。

264 – 326 取得参数的数据类型。由于efuns的参数最大值为4,因此,在171 – 174行对buff赋值的时候只需要执行4etype函数,如果参数个数不足4个,则etype返回值为T_ANY

 

 

 

参考资料:

[1]      Kenneth C. Louden. 冯博琴,冯岚(译). 《编译原理及实践》. 机械工业出版社

[2]      Stephen C. Johnson. Yacc: Yet Another Compiler-Compiler

[3]      Yacclex快速入门:

 http://www-900.ibm.com/developerWorks/cn/linux/sdk/lex/index.shtml

[4]      Bison manual:

http://tinf2.vub.ac.be/~dvermeir/courses/compilers/bison-1.25/bison_toc.html

PS D:\vscode\python\网易云歌曲评论采集与分析> & D:/python/python3.13t.exe d:/vscode/python/网易云歌曲评论采集与分析/代码/test.py Traceback (most recent call last): File "D:\python\Lib\site-packages\numpy\_core\__init__.py", line 22, in <module> from . import multiarray File "D:\python\Lib\site-packages\numpy\_core\multiarray.py", line 11, in <module> from . import _multiarray_umath, overrides File "D:\python\Lib\site-packages\numpy\_core\overrides.py", line 5, in <module> from numpy._core._multiarray_umath import ( ...<3 lines>... ) ModuleNotFoundError: No module named 'numpy._core._multiarray_umath' The above exception was the direct cause of the following exception: Traceback (most recent call last): File "d:\vscode\python\网易云歌曲评论采集与分析\代码\test.py", line 11, in <module> from wordcloud import WordCloud File "D:\python\Lib\site-packages\wordcloud\__init__.py", line 1, in <module> from .wordcloud import (WordCloud, STOPWORDS, random_color_func, get_single_color_func) File "D:\python\Lib\site-packages\wordcloud\wordcloud.py", line 19, in <module> import matplotlib File "D:\python\Lib\site-packages\matplotlib\__init__.py", line 161, in <module> from . import _api, _version, cbook, _docstring, rcsetup File "D:\python\Lib\site-packages\matplotlib\cbook.py", line 24, in <module> import numpy as np File "D:\python\Lib\site-packages\numpy\__init__.py", line 125, in <module> from numpy.__config__ import show_config File "D:\python\Lib\site-packages\numpy\__config__.py", line 4, in <module> from numpy._core._multiarray_umath import ( ...<3 lines>... ) File "D:\python\Lib\site-packages\numpy\_core\__init__.py", line 79, in <module> raise ImportError(msg) from exc ImportError: IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE! Importing the numpy C-extensions failed. This error can happen for many reasons, often due to issues with your setup or how NumPy was installed. The following compiled module files exist, but seem incompatible with with either python 'cpython-313' or the platform 'win32': * _multiarray_umath.cp313-win_amd64.lib * _multiarray_umath.cp313-win_amd64.pyd We have compiled some common reasons and troubleshooting tips at: https://numpy.org/devdocs/user/troubleshooting-importerror.html Please note and check the following: * The Python version is: Python 3.13 from "D:\python\python3.13t.exe" * The NumPy version is: "2.4.0.dev0+git20250527.ba27d95" and make sure that they are the versions you expect. Please carefully study the information and documentation linked above. This is unlikely to be a NumPy issue but will be caused by a bad install or environment on your machine. Original error was: No module named 'numpy._core._multiarray_umath'
05-29
(rasa) unitree@ubuntu:~/rasa_ws$ rasa run --enable-api /home/unitree/rasa/lib/python3.8/site-packages/rasa/core/tracker_store.py:1044: MovedIn20Warning: Deprecated API features detected! These feature(s) are not compatible with SQLAlchemy 2.0. To prevent incompatible upgrades prior to updating applications, ensure requirements files are pinned to "sqlalchemy<2.0". Set environment variable SQLALCHEMY_WARN_20=1 to show all deprecation warnings. Set environment variable SQLALCHEMY_SILENCE_UBER_WARNING=1 to silence this message. (Background on SQLAlchemy 2.0 at: https://sqlalche.me/e/b8d9) Base: DeclarativeMeta = declarative_base() /home/unitree/rasa/lib/python3.8/site-packages/rasa/shared/utils/validation.py:134: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html import pkg_resources /home/unitree/rasa/lib/python3.8/site-packages/pkg_resources/__init__.py:3117: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('mpl_toolkits')`. Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages declare_namespace(pkg) /home/unitree/rasa/lib/python3.8/site-packages/pkg_resources/__init__.py:3117: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('ruamel')`. Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages declare_namespace(pkg) /home/unitree/rasa/lib/python3.8/site-packages/sanic_cors/extension.py:39: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead. SANIC_VERSION = LooseVersion(sanic_version) Traceback (most recent call last): File "/home/unitree/rasa/lib/python3.8/site-packages/sklearn/__check_build/__init__.py", line 48, in <module> from ._check_build import check_build # noqa ImportError: /home/unitree/rasa/lib/python3.8/site-packages/sklearn/__check_build/../../scikit_learn.libs/libgomp-d22c30c5.so.1.0.0: cannot allocate memory in static TLS block During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/unitree/rasa/bin/rasa", line 8, in <module> sys.exit(main()) File "/home/unitree/rasa/lib/python3.8/site-packages/rasa/__main__.py", line 133, in main cmdline_arguments.func(cmdline_arguments) File "/home/unitree/rasa/lib/python3.8/site-packages/rasa/cli/run.py", line 93, in run rasa.run(**vars(args)) File "/home/unitree/rasa/lib/python3.8/site-packages/rasa/api.py", line 36, in run import rasa.core.run File "/home/unitree/rasa/lib/python3.8/site-packages/rasa/core/run.py", line 25, in <module> from rasa import server, telemetry File "/home/unitree/rasa/lib/python3.8/site-packages/rasa/server.py", line 59, in <module> from rasa.core.agent import Agent File "/home/unitree/rasa/lib/python3.8/site-packages/rasa/core/agent.py", line 22, in <module> from rasa.core.policies.policy import PolicyPrediction File "/home/unitree/rasa/lib/python3.8/site-packages/rasa/core/policies/policy.py", line 26, in <module> from rasa.core.featurizers.tracker_featurizers import TrackerFeaturizer File "/home/unitree/rasa/lib/python3.8/site-packages/rasa/core/featurizers/tracker_featurizers.py", line 31, in <module> from rasa.core.featurizers.single_state_featurizer import SingleStateFeaturizer File "/home/unitree/rasa/lib/python3.8/site-packages/rasa/core/featurizers/single_state_featurizer.py", line 8, in <module> from rasa.nlu.extractors.extractor import EntityTagSpec File "/home/unitree/rasa/lib/python3.8/site-packages/rasa/nlu/extractors/extractor.py", line 30, in <module> import rasa.utils.train_utils File "/home/unitree/rasa/lib/python3.8/site-packages/rasa/utils/train_utils.py", line 32, in <module> from rasa.utils.tensorflow.data_generator import RasaBatchDataGenerator File "/home/unitree/rasa/lib/python3.8/site-packages/rasa/utils/tensorflow/data_generator.py", line 10, in <module> from rasa.utils.tensorflow.model_data import RasaModelData, Data, FeatureArray File "/home/unitree/rasa/lib/python3.8/site-packages/rasa/utils/tensorflow/model_data.py", line 21, in <module> from sklearn.model_selection import train_test_split File "/home/unitree/rasa/lib/python3.8/site-packages/sklearn/__init__.py", line 81, in <module> from . import __check_build # noqa: F401 File "/home/unitree/rasa/lib/python3.8/site-packages/sklearn/__check_build/__init__.py", line 50, in <module> raise_build_error(e) File "/home/unitree/rasa/lib/python3.8/site-packages/sklearn/__check_build/__init__.py", line 31, in raise_build_error raise ImportError( ImportError: /home/unitree/rasa/lib/python3.8/site-packages/sklearn/__check_build/../../scikit_learn.libs/libgomp-d22c30c5.so.1.0.0: cannot allocate memory in static TLS block ___________________________________________________________________________ Contents of /home/unitree/rasa/lib/python3.8/site-packages/sklearn/__check_build: __pycache__ setup.py __init__.py _check_build.cpython-38-aarch64-linux-gnu.so ___________________________________________________________________________ It seems that scikit-learn has not been built correctly. If you have installed scikit-learn from source, please do not forget to build the package before using it: run `python setup.py install` or `make` in the source directory. If you have used an installer, please check that it is suited for your Python version, your operating system and your platform. 这是我出现的错误
07-14
评论 8
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值