- 博客(269)
- 收藏
- 关注
RSS订阅原创 LLM 模型量化推理速度评测
1、compile 整个模型后的模型确实更快了，如果只compile attention部分加速不大，因为compile优化的空间并不大。2、投机采样测试过程中发现多卡推理 int4 模型出现报错，int8模型没有问题，这个目前没空看了。公众号"小晨的AI工作室"回复: "vllm_test" 获得原图，测试不易，希望点点关注哈~3、量化确实会带来速度的巨大提升，但是模型效果截图不太方便，效果确实降低了一些。4、AWQ量化后的千问7B模型，效果巨差，暂不确定问题在哪，目测了效果。
2024-03-05 21:08:48
1327
原创 一张图系列 - “position_embedding”
总之,Positional encoding为transformer模型提供了顺序信息,是实现transformer表征能力的重要组成部分。在 transformer 模型中,由于attention机制是并行计算的,输入的词向量不包含顺序信息,需要positional encoding来表示单词位置。Positional encoding 是在 transformer 模型中用于表示单词位置信息的一种技术。通过在词向量中加入位置信息,可以帮助注意力机制学习句子中单词的相关程度。位置邻近的单词更可能有关联。
2023-11-15 20:52:36
320
原创 Headed-Span-Based Projective Dependency Parsing
学习ACL2022论文地址：https://arxiv.org/abs/2108.04750code地址：https://github.com/sustcsonglin/span-based-dependency-parsing这个论文确实不错建议阅读：We propose a new method for projective dependency parsing based on headed spans. In a projective dependency tre
2022-05-02 22:18:03
239
原创 sanic & flask & fastapi 对比
代码：from flask import Flaskapp = Flask(__name__)@app.route("/")def hello_world(): return "hello, flask!"if __name__ == "__main__": app.run(host="127.0.0.1", port=8080)# -----from sanic import Sanicfrom sanic.response import jsonap
2022-04-18 00:13:14
4247
原创 golang tollbooth 中间件 & 压测工具 vegeta
参考：1、[译] Go 中基于 IP 地址的 HTTP 限流2、Tollbooth - Fasthttp integration layer3、didip/tollbooth Simple middleware to rate-limit HTTP requests4、valyala / fasthttp Fast HTTP package for Go5、uber-go / ratelimit6、ulule / limiter-examples - 法国众筹公司🤦🏻♀️7、t
2022-04-16 17:01:20
855
原创 记录两个小工具 - Hydra & Wandb
【1】Getting started | Hydra【2】Weights & Biases - DocumentationGitHub - wandb/client: 🔥 A tool for visualizing and tracking your machine learning experiments. This repo contains the CLI and Python API.
2022-03-20 21:31:54
300
原创 Iterative Depth First Traversal of Graph
C++ -> DFS 【1】#include <list>#include <stack>#include <vector>#include <iostream>using namespace std;class Graph { int V; list<int> *adj;public: explicit Graph(int V); void addEdge(int v, int w)
2022-02-27 00:56:12
160
原创 explicit 用法提醒
参考链接：C++ 关键字 explicit 的使用隐式转换，如何消除隐式转换？Clion提示：Single-argument constructors must be marked explicitly to avoid unintentional implicit conversions 解法办法https://www.cplusplus.com/doc/tutorial/typecasting/什么时候会 explicit：C++面向对象的多态特性，就是通过父类的类型实现对子
2022-02-07 00:00:36
521
原创 Find if there is a path between two vertices in an undirected graph
如图：python bfs 预热 （简单粗暴）# 无向图的的两个节点是否相连 ~ ：from collections import dequedef addEdge(v, w): global adj adj[v].append(w) adj[w].append(v)def isReachable(s, d, V): if (s == d): return True visited = [False for i in range
2022-02-06 01:12:27
644
原创 Find if there is a path between two vertices in a directed graph
基础遍历预热#include <iostream>#include <vector>#include <stack>#include <algorithm>using namespace std;struct TreeNode { int val; TreeNode* left; TreeNode* right; TreeNode(int x): val(x), left(nullptr), right(..
2022-02-04 19:05:10
740
原创 7 Class Templates array and vector； Catching Exceptions
目录：array -> Sorting and searching arrays. vector -> Demonstrating C++ Standard Library class template vector.1. array -> Sorting and searching arrays.// array -> sorting and searching#include <iostream>#include <iomanip>
2021-12-18 10:58:55
891
原创 检查_有向图&无向图中的环
目录：有向图中检查环 python（递归，迭代） c++ 无向图中检查环 python（递归，迭代） c++ 1.1 python 递归： 有向图中环检测：核心思想是检测节点的邻居是否再已有的stack里面from collections import defaultdict# 有向图查环class UGraph: "undirected graph recursion" def __init__(self, vertices): s
2021-12-17 14:10:09
1883
原创 6 Functions and an Introduction to Recursion
目录：1、maximum function with a function prototype.2、Shifted, scaled integers produced by 1 + rand() % 6.3、Demonstrating srand (随机种子)4、Random Number （优点）5、Inline Functions6、References and Reference Parameters7、Using default arguments8、Unary sc
2021-12-14 13:47:15
172
原创 5 Control Statements: Part 2； Logical Operators
概述：Chapter 4 discussed if, if…else and while. This chapter demonstrated for, do…while and switch.You used the break statement to exit a switch statement and to immediately terminate a loop, and used a continue statement to terminate a loop’s current ..
2021-12-11 19:24:59
486
原创 4 A lgorithm Development and Control Statements: Part 1
1、Only three types of control statements—sequence, selection and iteration—are needed to develop any algorithmmain.cpp#include <iostream>#include <iomanip>using namespace std;int main() { int total{0}; unsigned int gradeCoun..
2021-12-11 16:31:13
479
原创 3 Introduction to Classes, Objects, Member Functions and Strings
目录：1、created your own classes and member functions2、create UML class diagrams that model the member functions, attributes and constructors of classesAccount.h#ifndef UNTITLED1_ACCOUNT_H#define UNTITLED1_ACCOUNT_H#include <string>#incl
2021-12-11 15:55:13
216
原创 cmake - 02 - subprojects
参考链接????：GCC全过程详解+剖析生成的.o文件CMAKE 里PRIVATE、PUBLIC、INTERFACE属性示例详解cmake：target_** 中的 PUBLIC，PRIVATE，INTERFACE1. 指令说明target_include_directories()：指定目标包含的头文件路径。官方文档target_link_libraries()：指定目标链接的库。官方文档target_compile_options()：指定目标的编译选项。官方文档目标 由
2021-10-21 07:31:54
200
原创 cmake - 01K - imported-targets
目录：文件：run_test.sh#!/bin/bashcmake_version=`cmake --version | grep version | cut -d" " -f3`[[ "$cmake_version" =~ ([3-9][.][5-9.][.][0-9]) ]] || exit 0echo "correct version of cmake"mkdir -p build && cd build && cmake .. &am
2021-10-21 06:25:12
300
原创 cmake - 01H - third_party_library
目录：文件：main.cpp#include <iostream>#include <boost/shared_ptr.hpp>#include <boost/filesystem.hpp>int main(int argc, char *argv[]){ std::cout << "Hello Third Party Include!" << std::endl; boost::shared_ptr
2021-10-16 16:42:14
174
原创 cmake - 01E - installing
目录：(base) ➜ cmaker_learning tree.├── CMakeLists.txt├── build├── cmake-examples.conf├── include│ └── installing│ └── Hello.h└── src ├── Hello.cpp └── main.cpp4 directories, 5 files文件：与之前相同重点是 CMakeLists.txtcmake_min...
2021-10-16 13:41:25
142
原创 cmake - 01D - shared-library
动态链接目录：文件：Hello.h#ifndef CMAKER_LEARNING_HELLO_H#define CMAKER_LEARNING_HELLO_Hclass Hello{public: static void print();};#endif //CMAKER_LEARNING_HELLO_HHello.cpp#include <iostream>#include "../include/shared/Hello.h"u.
2021-10-16 12:18:47
85
原创 cmake - 01C - static-library
目录结构文件：Hello.h#ifndef CMAKER_LEARNING_HELLO_H#define CMAKER_LEARNING_HELLO_Hclass Hello{public: void print();};#endif //CMAKER_LEARNING_HELLO_HHello.cpp#include <iostream>#include "static/Hello.h"using namespace std;vo
2021-10-16 11:49:22
82
原创 cmake - 01 - hello-headers
目录：cmake 复习一下文件结构：(两个实验)文件内容：Hello.h#ifndef CMAKER_LEARNING_HELLO_H#define CMAKER_LEARNING_HELLO_Hclass Hello{public: void print();};#endif //CMAKER_LEARNING_HELLO_HHello.cpp#ifndef CMAKER_LEARNING_HELLO_H#define CMAKE.
2021-10-16 11:09:48
112
原创 20 - searching and sorting
Binary search xx1. Binary searchBinarySearch.cpp//// Created by on 2021/7/28.//// BinarySearch#include <algorithm>#include <array>#include <ctime>#include <iostream>#include <random>using namespace std
2021-07-29 07:50:19
145
原创 19 - Custom Templatized Data Structure
目录：1. List1. List 定义//// Created by on 2021/7/23.//#ifndef PRACTICE15_LIST_H#define PRACTICE15_LIST_H#include <iostream>#include "ListNode.h"using namespace std;template<typename NODETYPE>class List{public: bool isE
2021-07-23 16:32:59
126
原创 17 Exception Handling
目录：example that throw exception Rethrowing an exception xxx xx xxx xxx xx1.example that throw exception// example that throw exception#include <iostream>#include "DivideByZeroException.h"using namespace std;double quotient(i...
2021-05-16 09:13:24
128
原创 16 Standard Library Algorithms
目录：lambda expressions xxxx xx xxx xxx1.lambda expressions// lambda expressions#include <iostream>#include <array>#include <algorithm>#include <iterator>using namespace std;int main() { const siz...
2021-05-04 16:05:49
140
原创 15 standard library containers and iterators
目录：demonstrating input and output with iterators xx xx xxdemonstrating input and output with iterators
2021-04-30 17:46:30
149
1
原创 14 file processing
目录：Creating a sequential file Reading and printing a sequential file credit inquiry program1）Creating a sequential file// Creating a sequential file#include <iostream>#include <string>#include <fstream>#include <cstdlib..
2021-04-22 02:06:02
141
原创 13 stream Input/Output -1
很久没写了，今天是4月13号，难得有时间安静的坐下来写写，挺开心的~目录： Using Member Functions eof, get and put Comparing cin and cin.get 1) Using Member Functions eof, get and put#include <iostream>using namespace std;int main() { int character; cout &l.
2021-04-13 22:53:15
184
空空如也
空空如也
TA创建的收藏夹 TA关注的收藏夹
TA关注的人