COMP3230 Principles of Operating Systems
Programming Assignment Two
Due date: November 17 , 2024, at 23:59
Total 12 points – RC2
Programming Exercise – Accelerate LLM Inference using Multi-Threading
Objectives
1. An assessment task related to ILO 4 [Practicability] – “demonstrate knowledge in applying system
software and tools available in the modern operating system for software development”.
2. A learning activity related to ILO 2.
3. The goals of this programming exercise are:
• to have hands-on practice in designing and developing multi-threading programs;
• to learn how to use POSIX Pthreads (and Semaphore) libraries to create, manage, and
coordinate multiple threads in a shared memory environment;
• to design and implementsynchronization schemes for multithreaded processes using
semaphores, or mutex locks and condition variables.
Tasks
Optimizing the throughput of GPTs is an important topic. Similar to other neural networks, GPT and
its variations utilize matrix-vector-multiplication, or called fully-connected/linear layer in Deep
Learning, to apply parameters learned. Meanwhile, GPT leverages multi-head attention, a
mechanism to adopt important information from history tokens. Thus, to accelerate GPT and get a
faster response, it’s critical to have faster matrix-vector-multiplication and faster multi-head
attention computation. Given the non-sequential property of the two algorithms, parallel
computing based on multi-threading is usually considered helpful.
Following PA1, we use Llama3, an open-source variation of GPT for this assignment. A single-thread C
implementation of the inference program, named seq.c, is provided as the starting point of your
work. You need to use POSIX Pthreads with either the semaphore or (mutex_lock + condition variable)
to implement a multi-threading version of the inference program, which parallelizes both matrix vector-multiplication and multi-head attention functions. This multi-threading version shall
significantly accelerate the inference task of the Large Language Model. Moreover, to reduce the
system workload in creating multiple threads, you need to reuse threads by formulating them into a
thread pool.
Acknowledgement: The inference framework used in this assignment is based on the open-source
project llama2.c by Andrej Karpathy. The LLM used in this assignment is based on SmolLM by
HuggingfaceTB. Thanks open-source!
GPT-based Large Language Model
At high-level, GPT is a machine that can generate words one by one based on previous words (also
known as prompts), and Figure 1a illustrates the basic workflow of GPT on generating “How are you”:
Figure 1. GPT Insight. a) GPT generates text one by one, and each output is the input of the next generation. b) GPT has
four major components: Tokenizer turns word (string) into a vector, Softmax + Sample gives the next token, and each layer
has Attention and FFN (Feed-Forward Network), consisting of many Matrix-Vector-Multiplication
Figure 1b showcases the inference workflow of each word like “You” in “How are you”: First, words
are transformed into token embeddings using the tokenizer, which is essentially a (python)
dictionary that assigns a unique vector to each word. Embedding vectors go through multiple layers,
which involve three steps.
• The first step is Multi-Head Attention, where the model first calculates attention scores based on
the cosine similarity between the current word's query embedding and embeddings of previous
words (keys). Then weighted average value embeddings are used to formulate output.
• The second step is a feed-forward network (FFN) that applies more learnable parameters
through Matrix-Vector-Multiplication.
• The third step is positional embedding, which takes into account the ordering of words in natural
language by adding positional information by RoPE (not required in this assignment).
After going through all the layers, the embeddings are classified to generate a specific word as the
output. This involves using a softmax function to convert the embeddings into a probability
distribution, and randomly sample a word from the distribution.
Task 1: Matrix-Vector-Multiplication
Figure 2. Matrix-Vector-Multiplication Algorithm.
As shown in Figure 2, Matrix-Vector-Multiplication can be illustrated as two iterations:
For Each Row i
For Column j, accumulate Matrix[i][j] * Vector[j] to Out[i]
More specifically, a sample single-thread C implementation isshown below:
void mat_vec_mul(float* out, float* vec, float* mat, int col, int row) {
for (int i = 0; i < row; i++) { // for each row i
float val = 0.0f;
for (int j = 0; j < col; j++) // for each column j
val += mat[i * col + j] * vec[j]; // mat[i * col + j] := mat[i][j]
out[i] = val;
}
}
Your 1st task in this assignment is to parallelize the outer iteration (at the 2nd line) by allocating
blocks of rows to threads. More specifically, in the case of a Matrix with 𝑑 rows and 𝑛 threads
working on the computation, assuming that 𝑑 is divisible by 𝑛, the k-th thread will handle the rows
from [𝑘 × 𝑑/𝑛] to [(𝑘+1) × 𝑑/𝑛−1]. To illustrate, if we have a 6-row matrix with 2 threads, the 0th
thread will handle rows 0 to 2, while the 1st thread will handle rows 3 to 5. If 𝑑 is not divisible by 𝑛,
we can assign 𝑛−1 threads with ⌈
𝑑/𝑛 ⌉ rows, while the last thread is assigned with the remaining
rows. More explanation on such a design can be found in Appendix a. Parallel Checking.
In this assignment, the model used is quantized, so there is a slight difference with the above C code
but the workflow is still the same.
Task 2: Multi-Head Attention
Figure 3. Attention and Multi-Head Attention Algorithm. The computation of each head is separated.
As shown in Figure 3, Multi-Head Attention can be illustrated by the following iterations:
For Each Head h,
For Each Timestep t,
For head item i, accumulate q[h][i] * keys[h][t][i] to score[h][t]
SoftMax(score)
For Each Timestep t,
For head item i, out[h][i] += score[h][t] * values[h][t][i]
More specifically, a sample single-thread C implementation isshown below:
void multi_head_attn(float* out, float* q, float* key_cache, float* value_cache,
float* att, int seq_len, int n_heads, int head_size, int kv_dim, int kv_mul) {
for (int h = 0; h < n_heads; h++) { // iterate over all heads, PARALLEL THIS LOOP
float* head_q = q + h * head_size; // query vector for this head
float* head_att = att + h * seq_len; // attention scores for this head
for (int t = 0; t <= pos; t++) { // iterate over all timesteps
// get the key vector for this head and at this timestep
float* head_k = key_cache + t * kv_dim + (h / kv_mul) * head_size;
float score = 0.0f;
for (int i = 0; i < head_size; i++)
score += head_q[i] * head_k[i]; // attention score := q dot k
score /= sqrtf(head_size); // normalize by head_size
head_att[t] = score; // save to the attention buffer
}
softmax(head_att, pos + 1); // THREADS-SAFE SoftMax to normalize scores to weight
float* head_out = out + h * head_size; // out vector for this head
memset(head_out, 0, head_size * sizeof(float)); // clear buffer
for (int t = 0; t <= pos; t++) {
// get the value vector for this head and at this timestep
float* head_v = value_cache + t * kv_dim + (h / kv_mul) * head_size;
float a = head_att[t]; // attention weight for this timestep
for (int i = 0; i < head_size; i++)
head_out[i] += a * head_v[i]; // accumulate the weighted sum to head out
}
}
}
Though looks complicated, it’s worth noticing that the computation involved for each head k is completely
independent of other heads.
Your 2nd task in this assignment is to parallelize the head-iteration (in 3rd line of the above sample
code) by assigning blocks of heads to different threads. More specifically, consider a model with h
heads and n threads. If h is divisible by n, then k-th thread will handle heads from [𝑘 × ℎ/𝑛] to [(𝑘+
1) × ℎ/𝑛−1]. And if h is not divisible by n, we can assign 𝑛−1 threads with ⌈
ℎ/𝑛⌉ heads, and the last
thread handles the remaining heads. For example, our model has 9 heads, and with 4 threads, they
will handle 0-2 (1st thread), 3-5 (2nd thread), 6-8 (3rd thread) and the last thread shall handle no
heads.
Note: Due to MQA, the no. of heads for a query might not be equal to the no. of heads for key /
value, but this is already handled correctly within the inner loop, just stick to n_heads.
Task 3: Thread Pool
Moreover, to reduce the performance overhead of frequent thread creation and cancellation, your 3rd
task is to create one set of N threads and reuse them for all mat_vec_mul() and multi_head_attn()
function calls, instead of creating N threads for each mat_vec_mul() or multi_head_attn() call. One
popular method is based on synchronization using a thread pool as shown in Figure 4.
Figure 4. Reference Synchronization Workflow, consisting of 3 functions: a) INIT_THR_POOL function: create N threads,
each thread falls asleep immediately; b) MAT_VEC_MUL or MULTI_HEAD_ATTN function: assign new parameters and
tasks, wake up all threads to work, and wait until threads to finish before returned; c) CLOSE_THR_POOL function: wake
up threads to collect system usage and exit, wait until all threads to exit and collect usage of terminated threads.
More specifically, the synchronization workflow in Figure 4 consists of 4 functions and a thread
function:
1. init_thr_pool(int thr_count): to be called by the main thread at the beginning of the
program, shall:
a. Create thr_count threads
b. Let threadsidentify themselves, i.e., thread knows I am the i-th thread
c. Let the created threads go to wait immediately
2. void mat_vec_mul(float* out, float* vec, float* mat,...): API exposed to do
Matrix-Vector-Multiplication, signature must be same as sequential version, shall:
a. Assign the mat_vec_mul computation and the parameters (out, vec, mat, ...) to
threads
b. Wake up threads to do the calculation
c. Main thread waits until all threads to complete the calculation
3. void multi_head_attn(float* out, float* q, ...): API exposed to do Multi-Head
Attention, signature must be same as sequential version, shall:
d. Assign the multi_head_attn computation and the parameters(out, q, ...) to threads
e. Wake up threads to do the calculation
f. Main thread waits until all threads to complete the calculation
4. close_thr_pool(): to be called at the end of the program, shall:
a. Wake up all threads and inform them to collect their system usage and terminate
start
├── common.h # common and helper macro definitions, read through first
├── seq.c # start point, single-thread inference engine
├── parallel_[UID].c # [your task] template including synchronization functions above
├── Makefile # makefile for the project, update [UID] on line 5
└── model.h # GPT model definition, modification not allowed
make prepare # will download if not existed
# or manually download via wget, will force repeated download, not recommended
wget -O model.bin https://huggingface.co/huangs0/smollm/resolve/main/model.bin
wget -O tokenizer.bin https://huggingface.co/huangs0/smollm/resolve/main/tokenizer.bin
make -B seq # -B := --always-make, force rebuild
gcc -o seq seq.c -O2 -lm # or manually make with gcc
b. Wait until all threads exit, and collect and print the system usage of all terminated threads
c. Collect and print the system usage of the main thread as well as the whole program
d. Release all resourcesrelated to multi-threading
5. void* thr_func(void* arg): thread function, shall:
a. Immediately wait for synchronization after initialization
b. Can be woken up by the main thread to work on the assigned computation (i.e., based on the
task and parameters)
c. After finishing the current workload, inform the main thread and go back to wait
d. Being able to terminate and collect its system usage
It’s worth noticing that you should create one thread pool that can properly handle both mat_vec_mul
and multi_head_attn instead of two pools (one for each) Thus, you may also need to implement the
following:
6. mat_vec_mul_task_func(int id, ...): function executed by each thread to perform (its
portion of) matrix-vector-multiplication. The first argument is the thread id in the pool (not tid
in OS), and the rest of the signature is left for you to design and implement.
7. multi_head_attn_task_func(int id, ...): function executed by each thread to
perform (its portion of) multi-head attention. The first argument is the thread id in the pool
(not tid in OS), and the rest of the signature is left for you to design and implement.
More details and reasons behind the design can be found in Appendix b. Context Design.
There might be other synchronization workflows, and we are open to your ideas. However, due to
the large class size, we can only accept submissions following the above design.
Specifications
a. Preparing Environment
Download start code – Download start.zip from the course’s Moodle, and unzip to a folder with:
Download the model files. There are two files required, model.bin for model weight and
tokenizer.bin for tokenizer. Please use the following instructions to download them:
Compile and run the inference program. The initial seq.c is a complete single-thread (sequential)
C inference program that can be compiled as follows:
Please use -lm flag to link the Math library and -O2 flag to apply level-2 optimization. Please use -O2
and don’t use other optimization for fairness.
Run the compiled seq program. The program can be executed with an integer specifying the
random seed and a quoted string specifying the prompt. For simplicity, we fixed it to a single
prompt.
./seq <seed> <prompt> # prompt must quoted with "", recommend using only one prompt
# examples, more to be found in bench.txt
./seq 42 "What’s Fibonacci Number?"
./seq 42 "Why didn't my parents invite me to their wedding?"
Upon invocation, the program will configure the random seed and begin sentence generation based
on the provided prompt. The program calls the forward function to call LLM and generate the
next token, and the printf() with fflush() to print the generated word to the terminal
immediately. A pair of utility time measurement functions time_in_ms will measure the time
with millisecond accuracy.
Finally, when generation isfinished, the length, average speed, and system usage will be printed:
$ ./seq 42 "What is Fibonacci Number?"
User: What is Fibonacci Number?
assistant
A Fibonacci sequence is a sequence of numbers in which each number is the sum of the two
preceding numbers (1, 1, 2, 3, 5, 8, 13, ...)
......
F(n) = F(n-1) + F(n-2) where F(n) is the nth Fibonacci number.
length: 266, speed (tok/s): 21.4759
main thread – user: 12.4495 s, system: 0.0240 s
By fixing the same machine (workbench2) and the same random seed, generated text can be
exactly replicated. For example, the above sample is from the test we conducted on workbench2
with random seed 42. Moreover, achieved tok/s represents the average number of tokens
generated within a second, and we use it as the metric for speed measurement. Due to the
fluctuating system load from time to time, the speed of the generation will fluctuate around some
level.
b. Implement the parallel mat_vec_mul and multi_head_attn by multi-threading
Open the parallel_[UID].c, rename [UID] with your UID, and open Makefile, rename [UID]
with your UID (make sure no blank space after). Implement the thread pool with the workflow
illustrated in Figure. 4 by completing the required five functions and adding appropriate global
variables or more functions if needed.
For multi-threading, please use pthread.h. For synchronization, please use either semaphore or
(mutex locks and conditional variables). You can only modify the code between specified // YOUR
CODE STARTS HERE at line 28 and // YOUR CODE ENDS HERE at line 88.
Here are some suggestions for the implementation:
1. How to assign new parameters to threads and inform threads to terminate?
i. Via variables in globalspace. Noted that thr_func can access global variables.
ii. Via pointers to parameters. The main thread changes pointers to params before wake-up
worker threads.
2. How to assign new tasks (mat_vec_mul or multi_head_attn)? Via function pointers.
3. Once the main thread invokes mat_vec_mul() or multi_head_attn(), it should wait for all
computations to complete before returning from the function.
make -B # applicable after renaming [UID]
gcc -o parallel parallel_[UID].c -O2 -lm -lpthread # or manually via gcc
./parallel 4 42 "What is Fibonacci Number?"
User: What is Fibonacci Number?
assistant
A Fibonacci sequence is a sequence of numbers in which each number is the sum of the two
preceding numbers (1, 1, 2, 3, 5, 8, 13, ...)
......
F(n) = F(n-1) + F(n-2) where F(n) is the nth Fibonacci number.
length: 266, speed (tok/s): 38.8889
Thread 0 has completed - user: 4.9396 s, system: 0.1620 s
Thread 1 has completed - user: 4.7195 s, system: 0.1806 s
Thread 2 has completed - user: 4.6274 s, system: 0.1843 s
Thread 3 has completed - user: 5.0763 s, system: 0.1702 s
main thread - user: 0.6361 s, system: 0.6993 s
Whole process - user: 20.0198 s, system: 1.3757 s
4. For collecting system usage upon finish, please use getrusage.
Your implementation shall be able to be compiled by the following command:
The program accepts three arguments, (1) thr_count , (2) seed, and (3) the prompt (enclosed by
""). Code related to reading arguments has been provided in parallel_[UID].c. You can use
thr_count to specify the number of threads to use.
./parallel <thr_count> <seed> <prompt>
If your implementation is correct, under the same random seed, the generated text shall be the
same as the sequential version, but the generation will be faster. Moreover, you should report on
the system usage for each thread respectively (including the main thread) and the whole program.
For example, this is the output of random seed 42 on workbench2 with 4 threads:
c. Measure the performance and report your findings
Benchmark your implementation (tok/s) on your environment (Linux or WSL or docker or Github
Codespaces) with different thread numbers and report metrics like the following table:
Thread Numbers Speed (tok/s) User Time System Time Use Time/System Time
0 (Sequential)
1
2
4
6
8
10
12
16
Regarding system usage (user time / system time), please report the usage of the whole process
instead of each thread. Then based on the above table, try to briefly analyze the relation between
performance and the number of threads and reason the relationship. Submit the table, your analysis,
and your reasoning in a one-page pdf document.
IMPORTANT: Due to the large number of students this year, please conduct the benchmark on your
own computer instead of the workbench2 server. The grading of your report is based on your
analysis and reasoning instead of the speed you achieved. When you’re working on workbench2,
please be reminded that you have limited maximum allowed thread numbers (128) and process
(512), so please do not conduct benchmarking on the workbench2 server.
Submission
Submit your program to the Programming # 2 submission page on the course’s Moodle website.
Name the program to parallel _[UID].c (replace [UID] with your HKU student number). As
the Moodle site may not accept source code submission, you can compress files to the zip format
before uploading. Submission checklist:
• Yoursource code parallel_[UID].c, must be self-contained with no dependencies other than
model.h and common.h) and Makefile.
• Your report must include the benchmark table, your analysis, and your reasoning.
• Your GenAI usage report contains GenAI models used (if any), prompts and responses.
• Please do not compress and submit the model and tokenizer binary files. (make clean_bin)
Documentation
1. At the head of the submitted source code, state the:
• File name
• Student’s Name and UID
• Development Platform (Please include compiler version by gcc -v)
• Remark – describe how much you have completed (See Grading Criteria)
2. Inline comments (try to be detailed so that your code can be understood by others easily)
Computer Platform to Use
For this assignment, you can develop and test your program on any Linux platform, but you must
make sure that the program can correctly execute on the workbench2 Linux server (as the tutors
will use this platform to do the grading). Your program must be written in C and successfully
compiled with gcc on the server.
It’s worth noticing that the only server for COMP3230 is workbench2.cs.hku.hk, and please do not
use any CS department server, especially academy11 and academy21, as they are reserved for
other courses. In case you cannot login to workbench2, please contact the tutor(s) for help.
Grading Criteria
1. Your submission will be primarily tested on the workbench2 server. Make sure that your
program can be compiled without any errors. Otherwise, we have no way to test your
submission, and you will get a zero mark.
2. As the tutor will check your source code, please write your program with good readability (i.e.,
with clear code and sufficient comments) so that you will not lose marks due to confusion.
3. You can only use pthread.h and semaphore.h (if needed), using other external libraries
like OpenMP, BLAS or LAPACK will lead to a 0 mark.
Detailed Grading Criteria
• Documentation -1 point if failed to do
• Include necessary documentation to explain the logic of the program
• Include the required student’s info at the beginning of the program
• Report: 1 point
• Measure the performance of the sequential program and your parallel program on your
computer with various No. of threads (0, 1, 2, 4, 6, 8, 10, 12, 16).
• Briefly analyze the relation between performance and No. of threads, and reason the relation
• Implementation: 11 points evaluated progressively
1. (+3 points = 3 points) Achieve correct result & use multi-threading. Correct means
generated text of multi-threading and sequential are identical with the same random
seed.
2. (+3 points = 6 points total) All in 1., and achieve >10% acceleration by multi-threading
compared with sequential under 4 threads. Acceleration measurement is based on tok/s,
acceleration must result from multi-threading instead of others like compiler (-O3), etc.
3. (+2 points = 8 points total) All in 2., and reuse threads in multi-threading. Reuse threads
means the number of threads created in the whole program must be constant as
thr_count.
4. (+3 points = 11 points total) All in 3., and mat_vec_mul and multi_head_attn use the same
thread pool. Reusing the same thread pool means there’s only one pool and one thread
group.
Plagiarism
Plagiarism is a very serious offense. Students should understand what constitutes plagiarism, the
consequences of committing an offense of plagiarism, and how to avoid it. Please note that we may
request you to explain to us how your program is functioning as well as we may also make use of
software tools to detect software plagiarism.
GenAI Usage Guide
Following course syllabus, you are allowed to use Generative AI to help completing the assignment,
and please clearly state the GenAI usage in GenAI Report, including:
• Which GenAI models you used
• Your conversations, including your prompts and the responses
Appendix
a. Parallelism Checking
To parallel by multi-threading, it’s critical to verify if the computation is independent to avoid race
conditions and the potential use of lock. More specifically, we need to pay special attention to
check and avoid writing to the same memory location while persisting the correctness.
For example, the 1st iteration (outer for-loop) matches the requirement of independence as the
computation of each row won’t affect others, and the only two writing is out[i] and val. Writing to
the same out[i] can be avoided by separating i between threads. val can be implemented as stack
variables for each thread respectively so no writing to the same memory.
Quite the opposite, 2nd iteration (inner for-loop) is not a good example for multi-threading, though
the only writing is val. If val is implemented as a stack variable, then each thread only holds a part
of the correct answer. If val is implemented as heap variables to be shared among threads, then
val requires a lock for every writing to avoid race writing, which is obviously costly.
b. Design of Context
A straightforward solution to the above problem is to let thr_func (thread function) do computation
and exit when finished and let original mat_vec_mul or multi_head_attn to create threads and wait
for threads to exit by pthread_join. This could provide the same synchronization.
However, this implementation is problematic because each function call to mat_vec_mul or
multi_head_attn will create 𝑛 new threads. Unfortunately, to generate a sentence, GPTs will call
mat_vec_mul or multi_head_attn thousands of times, so thousands of threads will be created and
destroyed, which leads to significant overhead to the operation system.
Noted that all the calls to mat_vec_mul function or multi_head_attn are doing the same task, i.e.,
Matrix-Vector-Multiplication, and the only difference between each function call is the parameters.
Thus, a straightforward optimization is to reuse the threads. In high-level, we can create N threads in
advance, and when mat_vec_mul or multi_head_attn is called, we assign new parameters for
thread functions and let threads work on new parameters.
Moreover, it’s worth noting that mat_vec_mul and multi_head_attn are only valid within the
context, i.e., between init_thr_pool and close_thr_pool, or there are no threads other than the
main (not yet created or has been destroyed). This kind of context provides efficient and robust
control over local variables and has been integrated with high-level languages like Python.
相关推荐
app逆向分析案例-95分
先把justtrusmeplus打开设置好95分,开始抓包,得到结果为: https://www.95fenapp.com/api_goods/list/v3.0?is_all=1&keywords=%E7%A7%91%E6%AF%94& page=1&page_size=20 &publish_timestamp=0 &scene=2&scene_type=95fen_android_search_personal&sn=SearchList
COMP3230 Principles of Operating Systems Programming TwoPython
llama2.cbyAndrejKarpathySmolLMby“Howi < row;j < col;h++) {//queryvector forthis headfor thisheadt <= pos;t++) {//iterateoveralltimestepstimestepi++):=qdotk// norma// saveto theto weight//out。
利用CiteSpace实现知网专利文献的高效计量与可视化分析
本文详细介绍了如何利用CiteSpace软件对中国知网(CNKI)的专利文献进行高效的文献计量与可视化分析。文章提供了从数据获取、格式转换、参数设置到图谱解读的完整工作流,旨在帮助研究者快速掌握该工具,从而洞察技术领域的研究热点、核心机构、学者合作网络及发展趋势,提升专利情报分析效率。
RNNoise-SpeexDSP-WebRTC使用说明
本文介绍RNNoise、SpeexDSP与WebRTC音频处理库的下载、调用及参数配置方法,适用于Android语音工程集成。RNNoise基于深度学习,适合非稳态噪声降噪,需48kHz输入;SpeexDSP支持多采样率与回声消除,适用于全双工场景;WebRTC提供轻量级窄带处理。三者可串联使用:麦克风信号先经前级降噪(RNNoise或SpeexDSP),再进入WebRTC媒体引擎进行编码。重点说明各库的典型应用场景、接口调用方式、关键参数调节策略及采样率转换注意事项,帮助开发者高效集成与优化语音处理链路。
别再盲调PID了!手把手教你用VOFA+实时可视化电机速度曲线(附STM32F4源码)
本文详细介绍了如何使用VOFA+工具实现PID参数的可视化调试,帮助开发者从盲调转向精准调优。通过实时可视化电机速度曲线,结合STM32F4源码示例,展示了如何高效调试Kp、Ki、Kd参数,解决响应慢、超调等常见问题,大幅提升电机控制系统的性能。
2025.06.24【R语言】|clusterProfiler安装与常见报错FAQ全解
本文介绍了clusterProfiler工具的安装方法及常见报错解决方案。安装可通过Bioconductor或Conda完成,需注意安装配套注释包。文中列举了10种常见报错,包括依赖包缺失、ID类型不匹配、物种不支持等问题,并给出具体解决方法。最后总结了GO/KEGG富集分析的标准流程,包括ID转换、富集分析和可视化步骤,为生物信息学分析提供实用参考。建议使用R≥4.2版本,并确保正确调用BiocManager进行安装。
Transformer 为什么能取代 RNN?从自注意力讲起
2017 年谷歌在《Attention Is All You Need》中提出 Transformer,核心是。它如今是 GPT、BERT、ChatGPT 的共同底座。
本地部署 Bolt.new - 小学生都能轻松搭建AI编程环境
本地部署 Bolt.new - 小学生都能轻松搭建AI编程环境
基于深度学习的商品评论情感分析系统-毕业设计
面向商品评论的中文情感分析系统,支持单条预测、批量分析、情感分布与关键词、报告导出。TextCNN、BiLSTM、Transformer 在同一词表、同一数据切分、同一种子与超参下公平对比,测试集 F1 分别为 0.9101、0.9089、0.9075。结论是这一规模下模型复杂度未带来收益,但误差结构不同。附技术栈、截图、文档预览与演示视频。
直播间人气协议算法:从批量操作到数据接口的自动化实现
本文深入解析直播间人气协议算法的实现原理与技术细节,涵盖批量账号管理、IP代理方案、用户行为模拟算法及平台数据接口调用策略。通过逆向工程分析通信协议,结合算法设计实现自动化流量提升,同时探讨风控规避与系统性能优化方案,为开发者提供合规的技术实践参考。
YOLOv8改进 | 主干篇 | 轻量级的低照度图像增强网络IAT改进YOLOv8暗光检测(全网独家首发)
本文给大家带来的改进机制是轻量级的变换器模型:Illumination Adaptive Transformer (IAT),用于图像增强和曝光校正。其基本原理是通过分解图像信号处理器(ISP)管道到局部和全局图像组件,从而恢复在低光或过/欠曝光条件下的正常光照sRGB图像。具体来说,IAT使用注意力查询来表示和调整ISP相关参数,例如颜色校正、伽马校正。模型具有约90k参数和约0.004s的处理速度,能够在低光增强和曝光校正的基准数据集上持续实现优于最新技术(State-of-The-Art, SOTA)
【原创深度学习项目】基于深度学习的建筑工地防护装备检测系统|安全帽与反光衣检测系统(RT-DETR-L 模型)
本文设计并实现了一套基于RT-DETR-L的建筑工地防护装备检测系统,采用Django+Vue前后端架构,实现对安全帽、反光背心等7类装备及“无PPE”“未戴安全帽”等缺失状态的精准识别。系统支持图片、视频、摄像头实时检测,具备参数可调、数据可视化与统计分析功能,有效解决小目标漏检、误检及管理难问题,助力智慧工地安全管理落地。文末可获取源码。
蓝牙模块HM-10手册
蓝牙模块HM-10手册,该手册对蓝牙模块HM-10进行了详细的介绍和使用方法以及一些AT指令。
图解循环神经网络(RNN)
RNN(循环神经网络)就是“带记忆的网络”:让网络拥有一块“记忆”,在处理当前数据时,能同时参考之前看过的内容;它不一次性看完全部输入,而是像人读书一样,一个字一个字地读,每读一个字就更新一次脑子里的记忆,然后用这份记忆去理解当前这个字。但便签一直在被改写,新信息不断挤占旧信息,传到第 40 步时,“巴黎”的痕迹已经淡得几乎没了。读到“贵”的时候,网络并没有重新去查“好看”这个词——但“好看”的影响已经融在了便签里(那个 1.5)。这三个困难的共同点:答案不在当前这一个字里,而在“前面那串东西”里。
LinePhone3.2.7(android端最新版本) sip客户端
SIP客户端For android:LinePhone3.2.7(最新版本),官网地址:http://www.linphone.org/
Camera2Demo
在Camera2 的基础上进行封装,使得Camera2一堆初始化逻辑与Activity分离,降低耦合度,完成预览拍照并保存的操作
神经网络入门指南:从 CNN、RNN 到 Attention
神经网络的学习,本质是“预测—比较—纠错”的循环;CNN 通过局部连接与权重共享理解图像;RNN 通过隐藏状态传递上下文;LSTM 和 GRU 用门控机制管理记忆;Encoder–Decoder 把理解与生成拆成两个阶段;Attention 则让模型在每一步都能主动寻找最相关的信息。到这里,我们已经走完了从普通神经网络到 Attention 的完整路线。下一篇将正式进入 Transformer,看看它如何摆脱 RNN 的顺序计算,又如何让每个 Token 直接找到与自己相关的信息。
Outlook Express 6 (OE6) for Win7/Win8/Win10
英文版 Outlook Express6.0, 兼容Win7/Win10/WinXP,解压就可用。
AI 学习第 15 课:变长序列不 pack 直接喂 RNN,隐状态全被 0 污染
NLP 的 batch 里句子长短不一,短句垫 0 补齐后直接喂 RNN,看似能跑,实际上每个 0 垫步都在更新隐状态——短序列的最后一步输出根本不是它真实句子的结果。这一课用 torch 2.15(
CNN 卷积神经网络深度解析:卷积、池化与特征提取的底层原理
卷积核(Kernel),也叫滤波器(Filter),本质上是一个可以训练的权重矩阵。1 0 -11 0 -11 0 -1这个卷积核可以对图像中的局部像素进行加权计算。不同的卷积核能够提取不同类型的特征:水平边缘特征垂直边缘特征纹理特征角点特征局部形状特征在传统图像处理中,滤波器通常由人工设计。而在 CNN 中,卷积核的权重通过训练数据和反向传播自动学习得到。这也是 CNN 能够自动提取图像特征的关键。# 第一层卷积padding=1# 第二层卷积padding=1。
图解常用深度学习优化器
来隐式构造 Hessian 逆的近似,从而直接估计出“谷底在哪”,而非仅仅判断“哪边最陡”。即SGD/Adam 主要看“当前坡度”,LBFGS 还会看“最近几步坡度怎么变”,像用局部地形图来规划路线。训练 Transformer 时,有些层梯度大,有些层梯度小。如果用 Adam + L2,梯度大的参数反而可能被“少衰减”,正则效果不稳定;:你要下山,但每次迈步前都要派侦察兵把整座山的每一个坡面都测量一遍,算出“绝对最陡”的方向才走一步。:还是下山,但这次你只问身边一个路人“哪边最陡”,然后就走。
【无标题】无人机视角道路坑洞坑洼目标检测数据集深度学习 目标检测 无人机航拍 道路坑洞 路面病害 YOLO格式 数据集发布9032期
(一)类别构成该数据集聚焦单一类别——坑洞(类别 ID 为 0),任务指向明确,围绕无人机俯视视角下的道路坑洞目标开展检测标注。类别体系精简,与公路路面病害巡检的实际需求高度契合,图像中的坑洞样本可直接作为路面破损识别的核心正样本,便于对接道路养护排查流程。(二)数据数量数据集规模为 3285 张无人机俯视航拍图像,采用标准 YOLO txt 标注格式,标注框总计 4457 个,其中带有效标注的图片 2007 张,全部图片均带有有效标注。
第 05/24 期:计算图与自动微分:深度学习如何计算梯度
本文系统阐述了神经网络训练中梯度计算的本质:损失对参数的敏感性。通过构建计算图,将复杂函数分解为基本运算,利用链式法则实现反向传播。核心在于反向模式自动微分——以一次前向计算为基础,通过传播输出对中间变量的敏感性,高效求得所有参数梯度。相比前向模式,反向模式在参数多、目标为标量的场景下更高效。计算图不仅适用于神经网络,也广泛适用于任意可微程序。关键思想是:梯度是局部导数与上游敏感性的乘积,分支路径的贡献需累加,从而保证导数计算的正确性与效率。
基于深度学习的猫图像目标检测方法研究:YOLOv8s模型海选、多种子对比训练与Django+ONNX部署实现详解
本课题基于深度学习研究猫图像目标检测问题。数据集含1000张猫图像、1188个标注框,按固定比例划分并冻结清单;训练前在多个候选族中短训海选,确定YOLOv8s为主模型,经学习率搜索与显存标定后完成150轮训练,验证集mAP50达到0.92,并以7个随机种子的对比实验验证稳定性。最优权重导出为ONNX接入Django系统,提供图片、视频、摄像头三种识别方式,配套识别记录管理与统计大屏,形成从训练到应用的完整闭环。
【深度学习项目实战·30】PM2.5短期预测:用LSTM构造没有未来信息的滑动窗口
本文基于北京PM2.5数据,使用LSTM实现从最近24小时历史观测预测下一小时PM2.5。重点解决时间切分、因果填补、滑动窗口构建与标准化等关键问题,确保模型仅依赖过去信息,避免未来泄漏。通过滚动单步预测与持续性基线对比,评估在原始单位下的MAE/RMSE,验证模型有效性。强调时间顺序与数据完整性,为时序预测提供可复现的实践框架。
深度学习 目标检测 无人机巡检 防震锤 电力设备 YOLO格式 数据集发布
深度学习 目标检测 无人机巡检 防震锤 电力设备 YOLO格式 数据集发布 9038期
无人机巡检道路路面异常检测数据集 深度学习 目标检测 无人机巡检 道路病害 路面异常 YOLO格式 数据集发布9028期
无人机巡检道路路面异常检测数据集 深度学习 目标检测 无人机巡检 道路病害 路面异常 YOLO格式 数据集发布9028期
AI 学习第 14 课:CNN 训 6 轮卡在 0.5352,数据没打乱
第 13 课把 Conv2d 的参数量和池化后的尺寸算清了,这课把卷积块拼成一个能训练的完整网络,跑满 6 个 epoch。第一版直接卡死:loss 从 2.6104 掉到 0.6989 就再也不动,
深度学习基本原理:从数学基础到神经网络训练
深度学习训练的核心,是先确定模型结构,再通过前向计算得到损失,利用链式法则和反向传播计算梯度,最后借助优化器与学习率不断更新参数。真正理解这条链路,后续学习更复杂的网络结构才有可靠的基础。
2747



被折叠的 条评论
为什么被折叠?



