[Parser] Build: PL By: JS

Java后端实现Markdown与HTML双向转换:Flexmark-java实战指南 在内容管理系统和博客平台开发中,处理富文本与轻量级标记文档的相互转换是常见需求。Markdown作为一种轻量级标记语言,以其简洁的语法和良好的可读性被广泛用于文档编写,而HTML则是网页内容展示的标准格式。理解两者间的语义映射关系是实现高效转换的基础,这涉及到对文档结构树的解析与重构。从技术价值看,可靠的双向转换能力能大幅提升内容迁移、数据导入和编辑体验的一致性,尤其在需要维护内容多版本格式的场景下至关重要。在Java生态中,Flexmark-java凭借其模块化设计和原生支持HTML转Markdown的特 阅读详情

origin: http://lisperator.net/pltut/

Summary

var ast = parse(TokenStream(InputStream(code)));

Writing a parser is, depending on the language, a moderately complex task.

In essence, it must transform a piece of code (which we inspect by looking at the characters) into an “abstract syntax tree” (AST).

The AST is a structured in-memory representation of the program, and it’s “abstract” in the sense that it does not care exactly what characters is the source code made of, but it faithfully represents the semantics of it.

For example, for the following program text:

sum = lambda(a, b) {
  a + b;
};
print(sum(1, 2));

our parser will generate the following AST, as a JavaScript object:

{
  type: "prog",
  prog: [
    // first line:
    {
      type: "assign",
      operator: "=",
      left: { type: "var", value: "sum" },
      right: {
        type: "lambda",
        vars: [ "a", "b" ],
        body: {
          // the body should be a "prog", but because
          // it contains a single expression, our parser
          // reduces it to the expression itself.
          type: "binary",
          operator: "+",
          left: { type: "var", value: "a" },
          right: { type: "var", value: "b" }
        }
      }
    },
    // second line:
    {
      type: "call",
      func: { type: "var", value: "print" },
      args: [{
        type: "call",
        func: { type: "var", value: "sum" },
        args: [ { type: "num", value: 1 },
                { type: "num", value: 2 } ]
      }]
    }
  ]
}

The main difficulty in writing a parser consists in a failure to properly organize the code.

The parser should operate at a higher level than reading characters from a string. A few advices on how to keep complexity manageable:

  • Write many functions and keep them small. In every function, do one thing and do it well.

  • Do not try to use regexps for parsing. They don’t work. Regexps can be helpful in the lexer though, but I suggest to limit them to very simple things.

  • Don’t attempt to guess. When unsure how to parse something, throw an error and make sure the message contains the error location (line/column).

To keep it simple I’ve split my code in three parts, which are further divided into many small functions:

  • The character input stream
  • The token input stream (lexer)
  • The parser

Input stream

We’re going to create a “stream object” which provides operations to read characters from a string. A stream object has 4 methods:

  • peek() — returns the next value but without removing it from the stream.

  • next() — returns the next value and also discards it from the stream.

  • eof() — returns true if and only if there are no more values in the stream.

  • croak(msg) — does throw new Error(msg).

The reason why I’m including the last one is that the stream can easily keep track of the current location (i.e. line/column), which is important to display in the case of an error message.

Feel free to add more methods here, depending on your needs.

The character input stream deals with characters, so the values that next() / peek() return are chars (well, since JS doesn’t have a char type, they’re strings containing one character).

Here is the full code of this object, which I will call “InputStream”. It’s small enough and you should have no problem to understand it:

function InputStream(input) {
    var pos = 0, line = 1, col = 0;
    return {
        next  : next,
        peek  : peek,
        eof   : eof,
        croak : croak,
    };
    function next() {
        var ch = input.charAt(pos++);
        if (ch == "\n") line++, col = 0; else col++;
        return ch;
    }
    function peek() {
        return input.charAt(pos);
    }
    function eof() {
        return peek() == "";
    }
    function croak(msg) {
        throw new Error(msg + " (" + line + ":" + col + ")");
    }
}

Note that it’s not a standard object (the kind you create with new). You just do var stream = InputStream(string) to get a stream object.

Next we’re going to write another abstraction on top of this object: the tokenizer.

Token Stream

The tokenizer (also called “lexer”) operates on a character input stream and returns a stream object with the same interface, but the values returned by peek() / next() will be tokens. A token is an object with two properties: type and value. Here are some examples with supported tokens:

{ type: "punc", value: "(" }           // punctuation: parens, comma, semicolon etc.
{ type: "num", value: 5 }              // numbers
{ type: "str", value: "Hello World!" } // strings
{ type: "kw", value: "lambda" }        // keywords
{ type: "var", value: "a" }            // identifiers
{ type: "op", value: "!=" }            // operators

Whitespace and comments are skipped over, no tokens are returned.

In order to write the tokenizer we need to look more closely into the syntax of our language. The idea is to notice that depending on the current character (as returned by input.peek()) we can decide what kind of token to read:

  • First off, skip over whitespace.
  • If input.eof() then return null.
  • If it’s a sharp sign (#), skip comment (retry after - the end of line).
  • If it’s a quote then read a string.
  • If it’s a digit, then we proceed to read a number.
  • If it’s a “letter”, then read an identifier or a keyword token.
  • If it’s one of the punctuation characters, return a punctuation token.
  • If it’s one of the operator characters, return an operator token.
  • If none of the above, error out with input.croak().

So here’s the “read_next” function — the “core” of the tokenizer — which implements the above:

function read_next() {
    read_while(is_whitespace);
    if (input.eof()) return null;
    var ch = input.peek();
    if (ch == "#") {
        skip_comment();
        return read_next();
    }
    if (ch == '"') return read_string();
    if (is_digit(ch)) return read_number();
    if (is_id_start(ch)) return read_ident();
    if (is_punc(ch)) return {
        type  : "punc",
        value : input.next()
    };
    if (is_op_char(ch)) return {
        type  : "op",
        value : read_while(is_op_char)
    };
    input.croak("Can't handle character: " + ch);
}

This is a “dispatcher” function and it’s what next() will call in order to fetch the next token.
Note it uses many utilities that are focused on particular token types, like read_string(), read_number() etc. There’s no point to complicate the dispatcher with code from those functions, even if we never call them elsewhere.

Another thing to notice is that we don’t consume all the input stream in one step. Each time the parser will call for next token, we read one token. In case of a parse error we don’t even reach the end of the stream.


read_ident() will read characters as long as they are allowed as part of an identifier (is_id). Identifiers must start with a letter, or λ or _, and can contain further such characters, or digits, or one of the following: ?!-<>=. Therefore, foo-bar will not be read as three tokens but as a single identifier (a “var” token). The reason for this rule is that I’d like to be able to define functions named is-pair? or string>= (sorry, it’s the Lisper in me).


Also, the read_ident() function will check the identifier against the list of known keywords, and if it’s there it will return a “kw” token, instead of a “var” one.

I think the code pretty much speaks for itself now, so here is the complete tokenizer for our language. Couple of small other notes below.

function TokenStream(input) {
    var current = null;
    var keywords = " if then else lambda λ true false ";
    return {
        next  : next,
        peek  : peek,
        eof   : eof,
        croak : input.croak
    };
    function is_keyword(x) {
        return keywords.indexOf(" " + x + " ") >= 0;
    }
    function is_digit(ch) {
        return /[0-9]/i.test(ch);
    }
    function is_id_start(ch) {
        return /[a-zλ_]/i.test(ch);
    }
    function is_id(ch) {
        return is_id_start(ch) || "?!-<>=0123456789".indexOf(ch) >= 0;
    }
    function is_op_char(ch) {
        return "+-*/%=&|<>!".indexOf(ch) >= 0;
    }
    function is_punc(ch) {
        return ",;(){}[]".indexOf(ch) >= 0;
    }
    function is_whitespace(ch) {
        return " \t\n".indexOf(ch) >= 0;
    }
    function read_while(predicate) {
        var str = "";
        while (!input.eof() && predicate(input.peek()))
            str += input.next();
        return str;
    }
    function read_number() {
        var has_dot = false;
        var number = read_while(function(ch){
            if (ch == ".") {
                if (has_dot) return false;
                has_dot = true;
                return true;
            }
            return is_digit(ch);
        });
        return { type: "num", value: parseFloat(number) };
    }
    function read_ident() {
        var id = read_while(is_id);
        return {
            type  : is_keyword(id) ? "kw" : "var",
            value : id
        };
    }
    function read_escaped(end) {
        var escaped = false, str = "";
        input.next();
        while (!input.eof()) {
            var ch = input.next();
            if (escaped) {
                str += ch;
                escaped = false;
            } else if (ch == "\\") {
                escaped = true;
            } else if (ch == end) {
                break;
            } else {
                str += ch;
            }
        }
        return str;
    }
    function read_string() {
        return { type: "str", value: read_escaped('"') };
    }
    function skip_comment() {
        read_while(function(ch){ return ch != "\n" });
        input.next();
    }
    function read_next() {
        read_while(is_whitespace);
        if (input.eof()) return null;
        var ch = input.peek();
        if (ch == "#") {
            skip_comment();
            return read_next();
        }
        if (ch == '"') return read_string();
        if (is_digit(ch)) return read_number();
        if (is_id_start(ch)) return read_ident();
        if (is_punc(ch)) return {
            type  : "punc",
            value : input.next()
        };
        if (is_op_char(ch)) return {
            type  : "op",
            value : read_while(is_op_char)
        };
        input.croak("Can't handle character: " + ch);
    }
    function peek() {
        return current || (current = read_next());
    }
    function next() {
        var tok = current;
        current = null;
        return tok || read_next();
    }
    function eof() {
        return peek() == null;
    }
}
  • The next() function doesn’t always call read_next(), because it might have been peeked before (in which case read_next() was already called and the stream advanced). Therefore we need a current variable which keeps track of the current token.

  • We only support decimal numbers with the usual notation (no 1E5 stuff, no hex, no octal). But if we ever need more, the changes go only in read_number() and are pretty easy to do.

  • Unlike JavaScript, the only characters that cannot appear unquoted in a string are the quote character itself and the backslash. You need to backslash them. Otherwise strings can contain hard newlines, tabs, and whatnot. We don’t interpret the usual escapes like \n, \t etc. though again, the changes would be pretty trivial (in “read_string”).

We now have sufficiently powerful tools to easily write the parser, but first I’d recommend you to look at the description of the AST.

AST

num { type: "num", value: NUMBER }
str { type: "str", value: STRING }
bool { type: "bool", value: true or false }
var { type: "var", value: NAME }
lambda { type: "lambda", vars: [ NAME... ], body: AST }
call { type: "call", func: AST, args: [ AST... ] }
if { type: "if", cond: AST, then: AST, else: AST }
assign { type: "assign", operator: "=", left: AST, right: AST }
binary { type: "binary", operator: OPERATOR, left: AST, right: AST }
prog { type: "prog", prog: [ AST... ] }
let { type: "let", vars: [ VARS... ], body: AST }

Example:

let (a = 10, b = a * 10) {
  a + b;
}

To:

{
  "type": "let",
  "vars": [
    {
      "name": "a",
      "def": { "type": "num", "value": 10 }
    },
    {
      "name": "b",
      "def": {
        "type": "binary",
        "operator": "*",
        "left": { "type": "var", "value": "a" },
        "right": { "type": "num", "value": 10 }
      }
    }
  ],
  "body": {
    "type": "binary",
    "operator": "+",
    "left": { "type": "var", "value": "a" },
    "right": { "type": "var", "value": "b" }
  }
}

The parser

The parser creates AST nodes that are described in the AST section.

Thanks to the work we did in the tokenizer, the parser operates on a stream of tokens instead of dealing with individual characters. It still defines many helpers to keep complexity down. I’ll discuss here the main functions that comprise the parser. Let’s start with a high level one, the lambda parser:

function parse_lambda() {
    return {
        type: "lambda",
        vars: delimited("(", ")", ",", parse_varname),
        body: parse_expression()
    };
}

This function will be invoked when the lambda keyword has already been seen and “eaten” from the input, so all it cares for is to parse the argument names; but they’re in parentheses and delimited by commas. Rather than placing that code in parse_lambda, I preferred to write a delimited function that takes these arguments: the start token, the end token, the separator, and a function which parses whatever must be between those start/end tokens. In this case, it’s parse_varname, which takes care to throw an error if it encounters anything which doesn’t look like a variable. The body of the function is an expression, so we get it with parse_expression.

delimited is a bit lower-level:

function delimited(start, stop, separator, parser) {
    var a = [], first = true;
    skip_punc(start);
    while (!input.eof()) {
        if (is_punc(stop)) break;
        if (first) first = false; else skip_punc(separator);
        if (is_punc(stop)) break; // the last separator can be missing
        a.push(parser());
    }
    skip_punc(stop);
    return a;
}

As you can see, it uses more utilities: is_punc and skip_punc. The former will return true if the current token is the given punctuation sign (without “eating” it), while skip_punc will ensure that the current token is that punctuation (throws an error otherwise) and will discard it from the input.

The function that parses the whole program is probably the simplest:

function parse_toplevel() {
    var prog = [];
    while (!input.eof()) {
        prog.push(parse_expression());
        if (!input.eof()) skip_punc(";");
    }
    return { type: "prog", prog: prog };
}

Since we have no statements, we simply call parse_expression() and read expressions until we get to the end of the input. Using skip_punc(";") we demand semicolons between these expressions.

Another simple example: parse_if():

function parse_if() {
    skip_kw("if");
    var cond = parse_expression();
    if (!is_punc("{")) skip_kw("then");
    var then = parse_expression();
    var ret = { type: "if", cond: cond, then: then };
    if (is_kw("else")) {
        input.next();
        ret.else = parse_expression();
    }
    return ret;
}

It skips over the if keyword with skip_kw (and this throws an error if the current token is not the given keyword), reads the condition using parse_expression(). Next, if the consequent branch doesn’t start with a { we require the keyword then to be present (I feel like the syntax is too scarce without it). The branches are just expressions, so again we use parse_expression() for them. The else branch is optional so we need to check if the keyword is present before parsing it.

Having many small utilities helps a lot in keeping the code simple. We almost write the parser like we had a high level language dedicated for parsing. All these functions are “mutually recursive”, e.g.: there’s a parse_atom() function which is the main dispatcher — based on the current token it calls other functions. One of them is parse_if() (called when the current token is if) and that in turn calls parse_expression(). But parse_expression() calls parse_atom(). The reason why there’s no infinite loop is that at each step, one function or another will advance at least one token.

This kind of parser is called a “recursive descent parser” and it’s probably the easiest kind to write manually.

Lower level: parse_atom() and parse_expression()

parse_atom() does the main dispatching job, depending on the current token:

function parse_atom() {
    return maybe_call(function(){
        if (is_punc("(")) {
            input.next();
            var exp = parse_expression();
            skip_punc(")");
            return exp;
        }
        if (is_punc("{")) return parse_prog();
        if (is_kw("if")) return parse_if();
        if (is_kw("true") || is_kw("false")) return parse_bool();
        if (is_kw("lambda") || is_kw("λ")) {
            input.next();
            return parse_lambda();
        }
        var tok = input.next();
        if (tok.type == "var" || tok.type == "num" || tok.type == "str")
            return tok;
        unexpected();
    });
}

If it sees an open paren, then it must be a parenthesized expression — thus, skip over paren, call parse_expression() and expect a closing paren. If it sees some keyword, it calls the appropriate parser function. If it sees a constant or an identifier, it’s just returned as is. And if nothing works, unexpected() will throw an error.

When an atomic expression is expected and it sees {, it calls parse_prog to parse a sequence of expressions. That’s defined below. It will do some minor optimization at this point — if the prog is empty, then it just returns FALSE. If it has a single expression, it is returned instead of a “prog” node. Otherwise it returns a “prog” node containing the expressions.

// we're going to use the FALSE node in various places,
// so I'm making it a global.
var FALSE = { type: "bool", value: false };

function parse_prog() {
    var prog = delimited("{", "}", ";", parse_expression);
    if (prog.length == 0) return FALSE;
    if (prog.length == 1) return prog[0];
    return { type: "prog", prog: prog };
}

Here’s the parse_expression() function. Contrary to parse_atom(), this one will extend an expression as much as possible to the right using maybe_binary(), which is explained below.

function parse_expression() {
    return maybe_call(function(){
        return maybe_binary(parse_atom(), 0);
    });
}

The maybe_* functions

These functions check what follows after an expression in order to decide whether to wrap that expression in another node, or just return it as is.

maybe_call() is very simple. It receives a function that is expected to parse the current expression. If after that expression it sees a ( punctuation token, then it must be a “call” node, which is what parse_call() makes (included below). Notice again how delimited() comes in handy for reading the argument list.

function maybe_call(expr) {
    expr = expr();
    return is_punc("(") ? parse_call(expr) : expr;
}

function parse_call(func) {
    return {
        type: "call",
        func: func,
        args: delimited("(", ")", ",", parse_expression)
    };
}
Operator precedence

maybe_binary(left, my_prec) is used to compose binary expressions like 1 + 2 * 3. The trick to parse them properly is to correctly define the operator precedence, so we’ll start with that:

var PRECEDENCE = {
    "=": 1,
    "||": 2,
    "&&": 3,
    "<": 7, ">": 7, "<=": 7, ">=": 7, "==": 7, "!=": 7,
    "+": 10, "-": 10,
    "*": 20, "/": 20, "%": 20,
};

This says that * binds tighter than +, so an expression like 1 + 2 * 3 must be read as (1 + (2 * 3)) instead of ((1 + 2) * 3), which would be the normal left-to-right order in which the parser operates.

The trick is to read an atomic expression (only the 1) and pass it to maybe_binary() (the left argument), along with the current precedence (the my_prec). maybe_binary will look at what follows. If it doesn’t see an operator, or if it has a smaller priority, then left is returned as is.

If it’s an operator that has a higher precedence than ours, then it wraps left in a new “binary” node, and for the right side it repeats the trick at the new precedence level (*):

function maybe_binary(left, my_prec) {
    var tok = is_op();
    if (tok) {
        var his_prec = PRECEDENCE[tok.value];
        if (his_prec > my_prec) {
            input.next();
            var right = maybe_binary(parse_atom(), his_prec) // (*);
            var binary = {
                type     : tok.value == "=" ? "assign" : "binary",
                operator : tok.value,
                left     : left,
                right    : right
            };
            return maybe_binary(binary, my_prec);
        }
    }
    return left;
}

Note that before returning the binary expression we must also call maybe_binary at the old precedence level (my_prec), in order to wrap the expression in another one, should an operator with a higher precedence follow. If all this is confusing, read the code again and again (perhaps try to execute it mentally on some input expressions) until you get it.

Finally, since my_prec is initially zero, any operator will trigger the building of a “binary” node (or “assign” when the operator is =).

There are a few more functions in the parser, so I’m including the whole parse function below. Click “Show code” to display it (about 150 lines).

var FALSE = { type: "bool", value: false };
function parse(input) {
    var PRECEDENCE = {
        "=": 1,
        "||": 2,
        "&&": 3,
        "<": 7, ">": 7, "<=": 7, ">=": 7, "==": 7, "!=": 7,
        "+": 10, "-": 10,
        "*": 20, "/": 20, "%": 20,
    };
    return parse_toplevel();
    function is_punc(ch) {
        var tok = input.peek();
        return tok && tok.type == "punc" && (!ch || tok.value == ch) && tok;
    }
    function is_kw(kw) {
        var tok = input.peek();
        return tok && tok.type == "kw" && (!kw || tok.value == kw) && tok;
    }
    function is_op(op) {
        var tok = input.peek();
        return tok && tok.type == "op" && (!op || tok.value == op) && tok;
    }
    function skip_punc(ch) {
        if (is_punc(ch)) input.next();
        else input.croak("Expecting punctuation: \"" + ch + "\"");
    }
    function skip_kw(kw) {
        if (is_kw(kw)) input.next();
        else input.croak("Expecting keyword: \"" + kw + "\"");
    }
    function skip_op(op) {
        if (is_op(op)) input.next();
        else input.croak("Expecting operator: \"" + op + "\"");
    }
    function unexpected() {
        input.croak("Unexpected token: " + JSON.stringify(input.peek()));
    }
    function maybe_binary(left, my_prec) {
        var tok = is_op();
        if (tok) {
            var his_prec = PRECEDENCE[tok.value];
            if (his_prec > my_prec) {
                input.next();
                return maybe_binary({
                    type     : tok.value == "=" ? "assign" : "binary",
                    operator : tok.value,
                    left     : left,
                    right    : maybe_binary(parse_atom(), his_prec)
                }, my_prec);
            }
        }
        return left;
    }
    function delimited(start, stop, separator, parser) {
        var a = [], first = true;
        skip_punc(start);
        while (!input.eof()) {
            if (is_punc(stop)) break;
            if (first) first = false; else skip_punc(separator);
            if (is_punc(stop)) break;
            a.push(parser());
        }
        skip_punc(stop);
        return a;
    }
    function parse_call(func) {
        return {
            type: "call",
            func: func,
            args: delimited("(", ")", ",", parse_expression),
        };
    }
    function parse_varname() {
        var name = input.next();
        if (name.type != "var") input.croak("Expecting variable name");
        return name.value;
    }
    function parse_if() {
        skip_kw("if");
        var cond = parse_expression();
        if (!is_punc("{")) skip_kw("then");
        var then = parse_expression();
        var ret = {
            type: "if",
            cond: cond,
            then: then,
        };
        if (is_kw("else")) {
            input.next();
            ret.else = parse_expression();
        }
        return ret;
    }
    function parse_lambda() {
        return {
            type: "lambda",
            vars: delimited("(", ")", ",", parse_varname),
            body: parse_expression()
        };
    }
    function parse_bool() {
        return {
            type  : "bool",
            value : input.next().value == "true"
        };
    }
    function maybe_call(expr) {
        expr = expr();
        return is_punc("(") ? parse_call(expr) : expr;
    }
    function parse_atom() {
        return maybe_call(function(){
            if (is_punc("(")) {
                input.next();
                var exp = parse_expression();
                skip_punc(")");
                return exp;
            }
            if (is_punc("{")) return parse_prog();
            if (is_kw("if")) return parse_if();
            if (is_kw("true") || is_kw("false")) return parse_bool();
            if (is_kw("lambda") || is_kw("λ")) {
                input.next();
                return parse_lambda();
            }
            var tok = input.next();
            if (tok.type == "var" || tok.type == "num" || tok.type == "str")
                return tok;
            unexpected();
        });
    }
    function parse_toplevel() {
        var prog = [];
        while (!input.eof()) {
            prog.push(parse_expression());
            if (!input.eof()) skip_punc(";");
        }
        return { type: "prog", prog: prog };
    }
    function parse_prog() {
        var prog = delimited("{", "}", ";", parse_expression);
        if (prog.length == 0) return FALSE;
        if (prog.length == 1) return prog[0];
        return { type: "prog", prog: prog };
    }
    function parse_expression() {
        return maybe_call(function(){
            return maybe_binary(parse_atom(), 0);
        });
    }
}

Credit

The moment I understood how to write a non-trivial parser occurred while studying Marijn Haverbeke’s parse-js library (Common Lisp). The parser above, although for a much simpler language, is modeled after his code.

Unexpected token c in JSON at position 0 报错信息及解决 问题产生:mpvue编译过程报错 Unexpected token c in JSON at position 0,且是websocket返回实时数据时。 问题产生原因:js中json  解析失败 解决:添加判断字符串是否是json格式方法isJson,并调用一下 isJson(str) { if (typeof str == 'string') { try { var obj = JSON.parse(str); if (typeof obj == 'object' && obj) { return true; } el 立即下载

相关推荐

MOSS-TTS开源语音合成家族:从架构解析到生产部署实战

语音合成(TTS)技术旨在让计算机将文本转换为自然流畅的语音,其核心原理通常涉及文本分析、声学模型和声码器等模块。近年来,基于深度学习的端到端TTS模型,特别是结合残差向量量化(RVQ)和Transformer架构的方案,显著提升了合成语音的自然度和效率。这类技术的价值在于能够广泛应用于有声内容创作、智能助手、无障碍服务以及实时交互等场景。本文聚焦于开源的MOSS-TTS家族,它通过模块化设计,针对高保真合成、多角色对话、实时交互等不同需求提供了专用模型。文章深入解析了其底层的Delay与Local两种架构

weixin_28745565的博客 422

qml报错error: Expected token “)“.

QML 项目打开的时候,问题栏中会出现很多这种错误提示,但是程序编译、运行都没有问题,只要在文件中任何位置按下空格,然后保存一下,这个界面上所有的这种问题就消失了;这个时候文件的编码就变成了编译器选择的默认编码格式 编码格式问题, 在Windows 上,加 BOM 保平安, Linux 上,不要加! 如果你也出现了这个问题,可以尝试以下解决办法: 1. Qt ->工具 -> 选项 -> 文本编辑器 ->文件编码 修改默认编码为 UTF-8(也许其他的也可以,有时间可以试试看)

u013352076的专栏 4175

从零到一:Ollama工具在RK3588上的深度优化实践

本文详细介绍了如何在RK3588开发板上深度优化Ollama工具,以高效运行DeepSeek模型。通过CPU与NPU负载平衡、模型量化、温度控制等策略,显著提升性能并降低功耗,适用于边缘计算场景。文章提供了具体的配置命令和优化方案,帮助开发者充分发挥RK3588的硬件潜力。

rgv23456789的博客 315

解决error: Expected token ",".

QML 项目打开的时候,问题栏中会出现很多这种错误提示,但是程序编译、运行都没有问题,只要在文件中任何位置按下空格,然后保存一下,这个界面上所有的这种问题就消失了;但是,当再次打开这个工程、或者重启 Qt IDE 之后,这个问题还是会出现,让人很不开心~ 在 QTCN 上提问之后,有网友觉得是编码格式问题,之后,我就在 UTF-8 加不加 BOM (Byte Order Mark)的问题上百度了一...

W__Alvin的博客 1万+

【Pandas总结】第二节 Pandas 的数据读取_pd.read_csv()的使用详解(非常全面,推荐收藏)

pandas对纯文本的读取提供了非常强力的支持,参数有四五十个。这些参数中,有的很容易被忽略,但是在实际工作中却用处很大。

weixin_47139649的博客 11万+

基于 YOLO26 的车辆实时检测与计数系统

本文提出了一种基于YOLO26的车辆实时检测与计数系统。系统采用CSPDarknet53作为主干网络,结合特征金字塔和多尺度预测实现高精度车辆检测。通过DeepSORT算法进行目标跟踪,并统计车辆数量、行驶方向和交通流量。实验表明,该系统在保持实时性能的同时,检测精度达到92.3%,可有效应用于智能交通管理。系统采用Python实现,支持GPU加速,适用于多种交通场景下的车辆监测需求。

一键难忘的博客 911

[Error] expected ‘;‘ before ‘)‘ token报错

可能此句中应该使用“;

2302_80213617的博客 1074

[QMT量化交易小白入门]-三十五、如何将聚宽策略信号转为QMT实盘交易

Flask是一个轻量级的Web应用框架,它提供了简单而灵活的方式来构建Web应用程序。在本案例中,将使用Flask来搭建一个本地服务,用于监听来自jq2qmt.py脚本的交易信号,并根据信号执行相应的下单操作。

木头大左的博客 2697

YT-DLP Web Player API参考:20个核心端点完全文档

**YT-DLP Web Player** 是一款基于 yt-dlp 的在线视频播放服务,它的 Flask 后端对外暴露了 20 个实用的 HTTP API 端点,覆盖播放、下载、搜索、嵌入、直播流等场景。本文带你快速看懂这些端点的用途与参数,方便二次开发与集成。 服务启动后默认运行在 `http://localhost:5000`(可用 `PORT` 环境变量修改),所有端点定义集中在 [sr

gitblog_00330的博客 604

STM32F4 SPI FLASH(W25Q128)驱动详解(含DMA+完整读写擦除+低功耗)

本文实现了一款基于STM32 HAL库的SPI Flash驱动,以W25Q128为例,提供了完整的初始化、读写、擦除、ID读取等功能。驱动集成了DMA传输、超时保护、页对齐写入等特性,支持扇区擦除、整片擦除、页写入及跨页缓冲写入。代码结构清晰,易于移植到其他SPI Flash型号(如W25Q64、W25Q16等),适用于数据存储、固件升级、文件系统等嵌入式场景。

kevin1499的博客 98

Playwright实战:打造绕过全维度前端反爬的无头浏览器爬虫(2026实战版)

Playwright绕过前端反爬的核心是模拟真实:真实的设备指纹、真实的行为轨迹、真实的网络请求;2025年前端反爬的核心检测维度是设备指纹+行为轨迹+网络请求,三者缺一不可;实战中需重点关注WebGL/Canvas指纹模拟、行为随机性、请求头完整性,这是绕过反爬的关键;性能优化需以“不触发反爬”为前提,优先选择浏览器复用、资源拦截,而非无限制并发。

专注于Python爬虫开发,分享爬虫技巧、项目实战与反爬经验,使用Scrapy、BeautifulSoup等工具,解决数据抓取难题。 2097

Boa 引擎性能剖析实战指南:用 Flamegraph 与 Callgrind 定位 JavaScript 执行瓶颈

本文聚焦于用 Rust 编写的可嵌入 JavaScript 引擎 [Boa](https://link.gitcode.com/i/32cfa3c99759abcff3020484aae4c9f6) 的性能剖析(Profiling)方法。Boa 的执行链路包含词法分析(lexer)、语法解析(parser)、字节码编译(bytecompiler)与虚拟机执行(VM)多个阶段,脚本耗时可能分布在任意

gitblog_01187的博客 295

【AI大模型应用开发】3.2 RAG实战 - RAG应用+UI实现加载本地文件并对话

前面我们实现了RAG基本流程,今天我们在此基础上给它加个UI界面,在浏览器打开,实现上传本地文件、解析,并用大模型实现与该文档的对话。也就是一个阉割版demo版的ChatPDF或知识库对话系统。

同学小张的博客 3056

微信红包助手实战指南:自动抢红包不拼手速,3 分钟配置到位还不露馅

群红包弹出的那一秒,你的手总是慢半拍。装上开源抢红包插件 WeChatLuckyMoney(微信红包助手)之后,我不再跟手速较劲了。它基于 Android 无障碍服务,自动识别通知栏和聊天页里的红包并替你拆开,不用 ROOT,也不碰微信本体。 ## 先划清界限:自动抢红包能替你做什么 原理不神秘:无障碍服务读取屏幕上的文字,靠特征词认出红包,再模拟点击拆开——这是系统给辅助工具的标准能力,不是

gitblog_00045的博客 214

汉书HE8,好用的汇编代码编写工具

汉书HE8,好用的汇编代码编写工具早期编写汇编代码所用功能强大

Camera2Demo

在Camera2 的基础上进行封装,使得Camera2一堆初始化逻辑与Activity分离,降低耦合度,完成预览拍照并保存的操作

preserve-cd 快速校验:3 步确认绝版游戏 ISO 是否完整

下载条跑满之后,先别急着点开游戏——怎么在一分钟内判断这份镜像完整、没被改动?preserve-cd 是一个以 ISO 镜像形式保存绝版游戏的保护工程,两张主碟分别收录 64 款 DOS 经典游戏和 100 个 Windows/Flash 小游戏。这个体量的文件,下载中断、内容被动的情况并不少见,校验就是在游玩之前该补上的那一步。 ## 先摸清资料:两张主碟和一本增刊里都有什么 动手校验前,先

gitblog_00578的博客 196

LinePhone3.2.7(android端最新版本) sip客户端

SIP客户端For android:LinePhone3.2.7(最新版本),官网地址:http://www.linphone.org/

基于 Next.js App Router 搭建 `@plasmicpkgs` 开发测试宿主:`plasmicpkgs-dev` 实战指南

`plasmicpkgs-dev` 是 Plasmic 开源仓库(gh_mirrors/pl/plasmic)中用于**在开发阶段验证 `@plasmicpkgs/*` 组件包**的专用 App Host 应用:它本身是一个基于 `@plasmicapp/loader-nextjs` SDK、使用 Next.js App Router 编写的最小 Next.js 应用,并把 Plasmic Stu

gitblog_00776的博客 355

深度学习工具包 DeepLearningToolbox

深度学习工具包Deprecation notice.-----This toolbox is outdated and no longer maintained.There are much better tools available for deep learning than this toolbox, e.g. [Theano](http://deeplearning.net/software/theano/), [torch](http://torch.ch/) or [tensorflow](http://www.tensorflow.org/)I would suggest you use one of the tools mentioned above rather than use this toolbox.Best, Rasmus.DeepLearnToolbox================A Matlab toolbox for Deep Learning.Deep Learning is a new subfield of machine learning that focuses on learning deep hierarchical models of data.It is inspired by the human brain's apparent deep (layered, hierarchical) architecture.A good overview of the theory of Deep Learning theory is[Learning Deep Architectures for AI](http://www.iro.umontreal.ca/~bengioy/papers/ftml_book.pdf)For a more informal introduction, see the following videos by Geoffrey Hinton and Andrew Ng.* [The Next Generation of Neural Networks](http://www.youtube.com/watch?v=AyzOUbkUf3M) (Hinton, 2007)* [Recent Developments in Deep Learning](http://www.youtube.com/watch?v=VdIURAu1-aU) (Hinton, 2010)* [Unsupervised Feature Learning and Deep Learning](http://www.youtube.com/watch?v=ZmNOAtZIgIk) (Ng, 2011)If you use this toolbox in your research please cite [Prediction as a candidate for learning deep hierarchical models of data](http://www2.imm.dtu.dk/pubdb/views/publication_details.php?id=6284)```@MASTERSTHESIS\{IMM2012-06284, author = "R. B. Palm", title = "Prediction as a candidate for learning deep hierarchical models of data", year = "2012",}```Contact: rasmusbergpalm at gmail dot comDirectories included in the toolbox-----------------------------------`NN/` - A library for Feedforward Backpropagation Neural Networks`CNN/` - A library for Convolutional Neural Networks`DBN/` - A library for Deep Belief Networks`SAE/` - A library for Stacked Auto-Encoders`CAE/` - A library for Convolutional Auto-Encoders`util/` - Utility functions used by the libraries`data/` - Data used by the examples`tests/` - unit tests to verify toolbox is workingFor references on each library check REFS.mdSetup-----1. Download.2. addpath(genpath('DeepLearnToolbox'));Example: Deep Belief Network---------------------```matlabfunction test_example_DBNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit RBM and visualize its weightsrand('state',0)dbn.sizes = [100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);figure; visualize(dbn.rbm{1}.W'); % Visualize the RBM weights%% ex2 train a 100-100 hidden unit DBN and use its weights to initialize a NNrand('state',0)%train dbndbn.sizes = [100 100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);%unfold dbn to nnnn = dbnunfoldtonn(dbn, 10);nn.activation_function = 'sigm';%train nnopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.10, 'Too big error');```Example: Stacked Auto-Encoders---------------------```matlabfunction test_example_SAEload mnist_uint8;train_x = double(train_x)/255;test_x = double(test_x)/255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit SDAE and use it to initialize a FFNN% Setup and train a stacked denoising autoencoder (SDAE)rand('state',0)sae = saesetup([784 100]);sae.ae{1}.activation_function = 'sigm';sae.ae{1}.learningRate = 1;sae.ae{1}.inputZeroMaskedFraction = 0.5;opts.numepochs = 1;opts.batchsize = 100;sae = saetrain(sae, train_x, opts);visualize(sae.ae{1}.W{1}(:,2:end)')% Use the SDAE to initialize a FFNNnn = nnsetup([784 100 10]);nn.activation_function = 'sigm';nn.learningRate = 1;nn.W{1} = sae.ae{1}.W{1};% Train the FFNNopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.16, 'Too big error');```Example: Convolutional Neural Nets---------------------```matlabfunction test_example_CNNload mnist_uint8;train_x = double(reshape(train_x',28,28,60000))/255;test_x = double(reshape(test_x',28,28,10000))/255;train_y = double(train_y');test_y = double(test_y');%% ex1 Train a 6c-2s-12c-2s Convolutional neural network %will run 1 epoch in about 200 second and get around 11% error. %With 100 epochs you'll get around 1.2% errorrand('state',0)cnn.layers = { struct('type', 'i') %input layer struct('type', 'c', 'outputmaps', 6, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %sub sampling layer struct('type', 'c', 'outputmaps', 12, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %subsampling layer};cnn = cnnsetup(cnn, train_x, train_y);opts.alpha = 1;opts.batchsize = 50;opts.numepochs = 1;cnn = cnntrain(cnn, train_x, train_y, opts);[er, bad] = cnntest(cnn, test_x, test_y);%plot mean squared errorfigure; plot(cnn.rL);assert(er<0.12, 'Too big error');```Example: Neural Networks---------------------```matlabfunction test_example_NNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);% normalize[train_x, mu, sigma] = zscore(train_x);test_x = normalize(test_x, mu, sigma);%% ex1 vanilla neural netrand('state',0)nn = nnsetup([784 100 10]);opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samples[nn, L] = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.08, 'Too big error');%% ex2 neural net with L2 weight decayrand('state',0)nn = nnsetup([784 100 10]);nn.weightPenaltyL2 = 1e-4; % L2 weight decayopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex3 neural net with dropoutrand('state',0)nn = nnsetup([784 100 10]);nn.dropoutFraction = 0.5; % Dropout fraction opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex4 neural net with sigmoid activation functionrand('state',0)nn = nnsetup([784 100 10]);nn.activation_function = 'sigm'; % Sigmoid activation functionnn.learningRate = 1; % Sigm require a lower learning rateopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex5 plotting functionalityrand('state',0)nn = nnsetup([784 20 10]);opts.numepochs = 5; % Number of full sweeps through datann.output = 'softmax'; % use softmax outputopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex6 neural net with sigmoid activation and plotting of validation and training error% split training data into training and validation datavx = train_x(1:10000,:);tx = train_x(10001:end,:);vy = train_y(1:10000,:);ty = train_y(10001:end,:);rand('state',0)nn = nnsetup([784 20 10]); nn.output = 'softmax'; % use softmax outputopts.numepochs = 5; % Number of full sweeps through dataopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, tx, ty, opts, vx, vy); % nntrain takes validation set as last two arguments (optionally)[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');```[![Bitdeli Badge](https://d2weczhvl823v0.cloudfront.net/rasmusbergpalm/deeplearntoolbox/trend.png)](https://bitdeli.com/free "Bitdeli Badge")

沉浸式翻译网页翻译不工作?3 分钟按层排查快速修复清单

沉浸式翻译(immersive-translate)支持网页、PDF、字幕与输入框双语翻译。若你遇到网页停在英文不出译文、图标点了没反应、PDF 翻译一片空白,本文按「配置层→环境层→网络/引擎层」递进排查,全程约 3 分钟,照做即可自救。 ## 先对号入座 🧭 问自己三个问题,直接跳到对应章节: - 网页还是英文、不出译文吗?→ [删掉排除列表再验脚本](#网页不出译文删掉排除列表再验脚

gitblog_00246的博客 946

karate-js 的 test262 一致性测试架构:可复现的 ES 通过率矩阵、跳过清单与迭代路线图

`karate-js-test262` 是 karate 仓库中专为 karate-js(一个从零实现的 JVM 轻量级 JavaScript 引擎)打造的 ECMAScript test262 一致性测试 harness。它以 tc39/test262 官方向量为"记分牌",以"能否运行真实世界(尤其是 LLM 编写的)JavaScript"为合格线,通过声明式跳过清单 [etc/expecta

gitblog_00938的博客 171

大麦自动抢票脚本本地部署指南:3条命令跑通、参数速查与报错自查

Automatic_ticket_purchase 是一个用于学习研究的大麦网自动抢票脚本:首次登录交给 Selenium,之后的查询与下单全部由 requests 完成,V2.1 新增选座购买。读完本文,你可以用 3 条命令完成本地部署、配好 5 个核心参数,并看懂登录到下单的完整链路。 ## 📦 环境依赖清单:Python、Chrome 与 ChromeDriver 依赖固定在 [req

gitblog_00752的博客 768

Android 随笔集 By tuliyuan

1.反射机制调用系统服务EthernetManager private void getNetInfo(Context context) { try { // Log.d(TAG, "dddd getNetInfo is " ); Method getServiceMethod = Class.forName("android.os.ServiceManager").getDeclaredMethod("getService", new Cla

qq_18906227的博客 2286

BambuStudio 设备管理页(Device Page)解析:React 19 + TanStack Router 嵌入式 WebView 架构与实战指南

本指南围绕 BambuStudio 中嵌入桌面客户端(wxWidgets + wxWebView)的 **Device Page** 前端工程展开,系统讲解其技术选型、构建加载机制、多标签路由、JS↔C++ 双向通信协议、功能模块组织、国际化和 `file://` 兼容构建等核心内容。读完本文,你将掌握如何在 BambuStudio 仓库内独立开发、调试并新增该 Web 设备管理页面,也能理解它与

gitblog_01053的博客 172

手写解释器入门指南:Crafting Interpreters 导论与 jlox/clox 双实现路线图

本篇技术指南围绕《Crafting Interpreters》一书的开篇导论展开,结合本仓库中完整可运行的 jlox(Java 树遍历解释器)与 clox(C 字节码虚拟机)源码,梳理"为什么要学习语言实现、全书如何组织、两个解释器各自主攻什么"的核心脉络。读完本文,你将掌握这本书的章节结构与学习路线,理解代码片段、旁注、挑战与设计笔记四种章节组件的阅读方式,并能在本仓库中实际构建、运行和测试这两

gitblog_00864的博客 342

【纯白嫖】7YUV:快速验证 YUV 原始帧一键完成

解码完 dump 出一个 YUV 文件,系统看图工具完全打不开。7YUV 让你直接双击打开 YUV、RGB 等原始图像数据,分辨率、色空间、位深一眼读到。适合做视频编解码和图像处理的工程师。 ## YUV 文件打不开时的麻烦 解码程序跑完,落盘往往是单个几百 MB 的 .yuv 文件:没有头信息,打开是灰屏或花屏。想确认色度子采样格式,只能拿文件大小反推分辨率;想对比同一帧转换前后的效果,还得

gitblog_00240的博客 723

foobox-cn:foobox 中文 DUI 皮肤,一个快速设置面板切换 foobar2000 的多主题布局

foobox-cn 是 foobar2000 的中文 DUI 皮肤配置,只重排主窗口界面,不改动播放内核。它在 foobar2000 之上搭起多播放列表、专辑封面、歌词和简介几个面板,提供深色与浅色两套配色,并在专辑缺少封面图时按流派标签回退到内置的流派图片。对已经习惯 foobar2000 播放与标签体系、却想换个观感的用户,这套配置装完即可用。 [![foobox-cn 中文 DUI 皮肤浅

gitblog_00611的博客 129
上一篇: [Initial] Build: PL By: JS
下一篇: SICP lec1b: # Computing process
ReZerou
博客等级 码龄10年 19粉丝 119原创
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付:10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值