JavaCC SPECIAL_TOKEN option

最新推荐文章于 2025-09-19 20:05:31 发布

原创最新推荐文章于 2025-09-19 20:05:31 发布 · 737 阅读

0 ·

CC 4.0 BY-SA版权

JavaCC 专栏收录该内容

7 篇文章

订阅专栏

本文探讨了如何定义和处理语法中的特殊标记及注释，包括单行和多行注释，并展示了这些元素如何与主要的语法元素如TOKEN关联。

SPECIAL_TOKEN are special tokens, that are bound to a token; they are referred from token.specialToken fields.

For example:

Grammar define comments as SPECIAL_TOKEN.

SKIP :
{
  " "
| "\t"
| "\n"
| "\r"
}

/* COMMENTS */
SPECIAL_TOKEN :
{
  <SINGLE_LINE_COMMENT: "--" (~["\n","\r"])* ("\n"|"\r"|"\r\n")>
| <MULTI_LINE_COMMENT: "/*" (~["*"])* "*" ("*" | (~["*","/"] (~["*"])* "*"))* "/">
}


SimpleNode Input() :
{}
{
 P()
 <EOF>
 { return jjtThis; }
}

void P() :
{}
{
  ( A() | B() | C() )+
}

So for this input:

-- line comment 1
A /* field comment 1 */ B
-- line comment 2
/* field comment 2 */
C
-- line comment 3

These inputs will generate 3 TOKEN, A, B, C

A.next = B;

B.next = C;

C.next = null;

And 5 comment special tokens; the comment tokens are bound to A, B, or C

A.specialToken = "--line comment 1"

"--line comment 1".next = A

B.specialToken = "/* field comment 1 */"

"/* field comment 1 */".next = B

C.specialToken = "/* field comment 2 */"

"/* field comment 2 */".next = C

"/* field comment 2 */".specialToken = "--line comment 2"

"--line comment 2".next = "/* field comment 2 */"

But the problem is where is "--line comment 3", there is no Token after it ?

Fortunately, there is a <EOF> token at the end, <EOF>.next = "--line comment 3".

So the final token train is:

lc = line comment

fc = field comment