Detecting Bias in Large Language Models: Fine-tuned KcBERT

本文是LLM系列文章,针对《Detecting Bias in Large Language Models: Fine-tuned KcBERT》的翻译。

检测大型语言模型中的偏差:微调KcBERT

摘要

大型语言模型(LLM)的快速发展使其具有与人类相似的自然语言处理能力,LLM正被广泛应用于教育和医疗保健等各个社会领域。虽然这些模型的多功能性有所提高,但它们有可能产生主观和规范的语言,导致社会群体中的歧视性待遇或结果,尤其是由于网络攻击性语言。在本文中,我们将此类危害定义为社会偏见,并通过基于模板的掩蔽语言建模(MLM),使用来自Transformer的双向编码器表示(KcBERT)和KOLD数据,在一个与韩语评论微调的模型中评估种族、性别和种族偏见。为了定量评估偏差,我们采用了LPBS和CBS指标。与KcBERT相比,微调后的模型显示种族偏见有所减少,但性别和种族偏见发生了显著变化。基于这些结果,我们提出了两种减轻社会偏见的方法。首先,预训练阶段的数据平衡方法通过调整特定单词的出现分布并将周围的有害单词转换为无害单词来调整数据的一致性。其次,在训练阶段,我们通过调整dropout和正则化来应用去偏正则化,证实了训练损失的减少。我们的贡献在于证明,由于语言依赖性特征,韩语模型中存在社会偏见。

1 引言

2 相关工作

3 方法

4 实验

5 实验结果

6 结论和未来工作

在本文中,研究了用KcBERT和KOLD数据微调的模型的社会偏见。最初,为了证实社会偏见,我们采用了基于模板的MLM方法,揭示了两个模型中社会偏见的发生。为了量化社会偏见,我们引入了L

船载 OpenHarmony AIS 航海工作台:告警与防碰撞预警系统 船载OpenHarmony AIS航海工作台的告警与防碰撞系统,集成十余类告警(超速、偏航、会遇、SOS等),通过网格空间索引实现高效邻近判定,保障实时性。告警管理器统一调度规则、去重、通知与日志,支持悬浮球、语音、跑马灯、弹窗多渠道触达,确保航行安全。SOS紧急通道独立运行,避免干扰。系统设计强调性能优化与多场景适配,为航海安全提供核心支撑。 阅读详情

相关推荐

AI 应用开发:四维架构落地实战——以 HR Agent 为例拆解感知·推理·记忆·执行【意图路由、规划编排、长期记忆、工具执行、面试题】

本文基于四维架构(感知·推理·记忆·执行),以HR Agent为例,系统拆解大模型应用从“聊天”到“生产可用”的落地路径。通过Spring AI实现意图路由、规划编排、RAG检索与工具调用,结合真实踩坑复盘,揭示架构化设计对安全、成本与可维护性的关键价值,为AI Agent工程化提供可复用的实战框架。

王大师企业官方博客 1323

PCB过孔电流承载解析:完整指南与数据参考

深入解析PCB过孔的电流承载能力,结合实际应用场景提供可靠设计参考。涵盖影响载流的关键因素,并整理实用的pcb过孔与电流对照一览表,帮助工程师优化布局与布线。

weixin_35677363的博客 634

西门子CPU与汇川伺服通信与控制

第三方驱动器的配置内容和西门子的配置内容基本一致,基本上会一种即可知道其他品牌的配置默认报文,需要激活硬限位才能收到限位信号默认报文,原点信号不能实时输出到PLC;自定义报文,需要伺服端设置DIN点位。

大小曲奇的博客 1万+

从零到一:STM32CubeMX虚拟串口开发中的常见陷阱与优化策略

本文深入探讨了STM32CubeMX在USB虚拟串口开发中的常见问题与优化策略,涵盖数据流吞吐量瓶颈、实时性约束等核心挑战。通过实战案例和代码示例,详细解析了波特率同步异常、数据丢失等问题的解决方案,并提供了DMA配置和中断优先级调优的高级技巧,帮助开发者提升STM32虚拟串口的性能和可靠性。

weixin_29164185的博客 370

TC387 MCMCAN Message RAM配置详解:Tx Buffer与Rx FIFO地址布局

在嵌入式CAN/CAN FD通信系统中,缓冲区和FIFO是数据收发的核心机制,而AURIX TC3xx系列单片机将这一机制集成在了MCMCAN模块专用的Message RAM中。这是一块由CAN控制器独占的内存区域,开发人员需要按照寄存器规范手动划分Tx Buffer、Rx FIFO、过滤元素等子区间。合理的RAM布局能确保不同功能区域互不重叠、有序排列,避免数据覆盖与错帧。通过掌握TXBC、RXF0C等配置寄存器的字段含义,结合实用的地址布局表,可以显著降低多节点CAN通信的调试成本。本内容面向使用TC3

weixin_34226182的博客 225

OneNet平台MQTT连接避坑指南:C语言Token计算实战(附完整代码)

本文详细解析了OneNet平台MQTT连接中的Token计算机制,提供了C语言实现的完整代码和实战指南。通过HMAC算法和Base64编码,开发者可以高效生成安全凭证,避免常见连接问题,适用于物联网设备开发。

weixin_29193359的博客 464

【RT-DETR涨点改进】损失函数改进篇 | RT-DETR引入SlideLoss损失函数,适合目标检测中面临的尺度变化、遮挡以及难易样本不平衡等复杂场景下的检测任务,含详细使用步骤

本文提出在RT-DETR目标检测模型中引入SlideLoss损失函数,以解决尺度变化、遮挡和难易样本不平衡等问题。SlideLoss通过自适应阈值划分正负样本,并对难样本赋予更高权重,使模型更关注分类边界样本。文章详细展示了SlideLoss的核心代码实现,并逐步指导如何在RT-DETR框架中集成该损失函数。实验表明,该方法能有效提升模型性能,特别适用于复杂场景下的目标检测任务。

Ai缝合怪、助力小伙伴高效发论文 265

基于Costas环的载波同步实现及Matlab仿真

其中,第一个相位环接收带噪声的待同步载波,并产生一个相位误差信号,这个误差信号再通过第二个相位环进行反馈控制,使得两个相位环的相位差保持为90度,最终输出同步载波信号。接下来,我们可以编写Costas环的仿真程序,并将上面生成的信号输入到Costas环中进行同步,最终输出同步后的载波信号。综上所述,本文介绍了如何使用Matlab实现基于Costas环的载波同步,并给出相应的源代码。上述代码中,我们初始化了Costas环的各个参数,然后通过循环迭代实现了相位估计和相位误差控制,最终输出了同步后的载波信号。

2301_78484069的博客 1049

TMS_WebGMaps_1.9.8.0_Full_Source

TMS_WebGMaps_1.9.8.0_Full_Source 全源码 delphi cbuilder

WINCC7.3破解文件

WINCC7.3硬件够破解文件,有需要的下载!WINCC7.3硬件够破解文件,有需要的下载!

深度学习工具包 DeepLearningToolbox

深度学习工具包Deprecation notice.-----This toolbox is outdated and no longer maintained.There are much better tools available for deep learning than this toolbox, e.g. [Theano](http://deeplearning.net/software/theano/), [torch](http://torch.ch/) or [tensorflow](http://www.tensorflow.org/)I would suggest you use one of the tools mentioned above rather than use this toolbox.Best, Rasmus.DeepLearnToolbox================A Matlab toolbox for Deep Learning.Deep Learning is a new subfield of machine learning that focuses on learning deep hierarchical models of data.It is inspired by the human brain's apparent deep (layered, hierarchical) architecture.A good overview of the theory of Deep Learning theory is[Learning Deep Architectures for AI](http://www.iro.umontreal.ca/~bengioy/papers/ftml_book.pdf)For a more informal introduction, see the following videos by Geoffrey Hinton and Andrew Ng.* [The Next Generation of Neural Networks](http://www.youtube.com/watch?v=AyzOUbkUf3M) (Hinton, 2007)* [Recent Developments in Deep Learning](http://www.youtube.com/watch?v=VdIURAu1-aU) (Hinton, 2010)* [Unsupervised Feature Learning and Deep Learning](http://www.youtube.com/watch?v=ZmNOAtZIgIk) (Ng, 2011)If you use this toolbox in your research please cite [Prediction as a candidate for learning deep hierarchical models of data](http://www2.imm.dtu.dk/pubdb/views/publication_details.php?id=6284)```@MASTERSTHESIS\{IMM2012-06284, author = "R. B. Palm", title = "Prediction as a candidate for learning deep hierarchical models of data", year = "2012",}```Contact: rasmusbergpalm at gmail dot comDirectories included in the toolbox-----------------------------------`NN/` - A library for Feedforward Backpropagation Neural Networks`CNN/` - A library for Convolutional Neural Networks`DBN/` - A library for Deep Belief Networks`SAE/` - A library for Stacked Auto-Encoders`CAE/` - A library for Convolutional Auto-Encoders`util/` - Utility functions used by the libraries`data/` - Data used by the examples`tests/` - unit tests to verify toolbox is workingFor references on each library check REFS.mdSetup-----1. Download.2. addpath(genpath('DeepLearnToolbox'));Example: Deep Belief Network---------------------```matlabfunction test_example_DBNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit RBM and visualize its weightsrand('state',0)dbn.sizes = [100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);figure; visualize(dbn.rbm{1}.W'); % Visualize the RBM weights%% ex2 train a 100-100 hidden unit DBN and use its weights to initialize a NNrand('state',0)%train dbndbn.sizes = [100 100];opts.numepochs = 1;opts.batchsize = 100;opts.momentum = 0;opts.alpha = 1;dbn = dbnsetup(dbn, train_x, opts);dbn = dbntrain(dbn, train_x, opts);%unfold dbn to nnnn = dbnunfoldtonn(dbn, 10);nn.activation_function = 'sigm';%train nnopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.10, 'Too big error');```Example: Stacked Auto-Encoders---------------------```matlabfunction test_example_SAEload mnist_uint8;train_x = double(train_x)/255;test_x = double(test_x)/255;train_y = double(train_y);test_y = double(test_y);%% ex1 train a 100 hidden unit SDAE and use it to initialize a FFNN% Setup and train a stacked denoising autoencoder (SDAE)rand('state',0)sae = saesetup([784 100]);sae.ae{1}.activation_function = 'sigm';sae.ae{1}.learningRate = 1;sae.ae{1}.inputZeroMaskedFraction = 0.5;opts.numepochs = 1;opts.batchsize = 100;sae = saetrain(sae, train_x, opts);visualize(sae.ae{1}.W{1}(:,2:end)')% Use the SDAE to initialize a FFNNnn = nnsetup([784 100 10]);nn.activation_function = 'sigm';nn.learningRate = 1;nn.W{1} = sae.ae{1}.W{1};% Train the FFNNopts.numepochs = 1;opts.batchsize = 100;nn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.16, 'Too big error');```Example: Convolutional Neural Nets---------------------```matlabfunction test_example_CNNload mnist_uint8;train_x = double(reshape(train_x',28,28,60000))/255;test_x = double(reshape(test_x',28,28,10000))/255;train_y = double(train_y');test_y = double(test_y');%% ex1 Train a 6c-2s-12c-2s Convolutional neural network %will run 1 epoch in about 200 second and get around 11% error. %With 100 epochs you'll get around 1.2% errorrand('state',0)cnn.layers = { struct('type', 'i') %input layer struct('type', 'c', 'outputmaps', 6, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %sub sampling layer struct('type', 'c', 'outputmaps', 12, 'kernelsize', 5) %convolution layer struct('type', 's', 'scale', 2) %subsampling layer};cnn = cnnsetup(cnn, train_x, train_y);opts.alpha = 1;opts.batchsize = 50;opts.numepochs = 1;cnn = cnntrain(cnn, train_x, train_y, opts);[er, bad] = cnntest(cnn, test_x, test_y);%plot mean squared errorfigure; plot(cnn.rL);assert(er<0.12, 'Too big error');```Example: Neural Networks---------------------```matlabfunction test_example_NNload mnist_uint8;train_x = double(train_x) / 255;test_x = double(test_x) / 255;train_y = double(train_y);test_y = double(test_y);% normalize[train_x, mu, sigma] = zscore(train_x);test_x = normalize(test_x, mu, sigma);%% ex1 vanilla neural netrand('state',0)nn = nnsetup([784 100 10]);opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samples[nn, L] = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.08, 'Too big error');%% ex2 neural net with L2 weight decayrand('state',0)nn = nnsetup([784 100 10]);nn.weightPenaltyL2 = 1e-4; % L2 weight decayopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex3 neural net with dropoutrand('state',0)nn = nnsetup([784 100 10]);nn.dropoutFraction = 0.5; % Dropout fraction opts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex4 neural net with sigmoid activation functionrand('state',0)nn = nnsetup([784 100 10]);nn.activation_function = 'sigm'; % Sigmoid activation functionnn.learningRate = 1; % Sigm require a lower learning rateopts.numepochs = 1; % Number of full sweeps through dataopts.batchsize = 100; % Take a mean gradient step over this many samplesnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex5 plotting functionalityrand('state',0)nn = nnsetup([784 20 10]);opts.numepochs = 5; % Number of full sweeps through datann.output = 'softmax'; % use softmax outputopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, train_x, train_y, opts);[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');%% ex6 neural net with sigmoid activation and plotting of validation and training error% split training data into training and validation datavx = train_x(1:10000,:);tx = train_x(10001:end,:);vy = train_y(1:10000,:);ty = train_y(10001:end,:);rand('state',0)nn = nnsetup([784 20 10]); nn.output = 'softmax'; % use softmax outputopts.numepochs = 5; % Number of full sweeps through dataopts.batchsize = 1000; % Take a mean gradient step over this many samplesopts.plot = 1; % enable plottingnn = nntrain(nn, tx, ty, opts, vx, vy); % nntrain takes validation set as last two arguments (optionally)[er, bad] = nntest(nn, test_x, test_y);assert(er < 0.1, 'Too big error');```[![Bitdeli Badge](https://d2weczhvl823v0.cloudfront.net/rasmusbergpalm/deeplearntoolbox/trend.png)](https://bitdeli.com/free "Bitdeli Badge")

OV9734 DATASHEET,完整规格书

OV9734完整规格书,包括电气参数,尺寸规格,寄存器配置表等

上一篇: Facilitating Pornographic Text Detection for Open Domain Dialogue Systems via Knowledge Distillation
下一篇: Optimizing Language Augmentation for Multilingual Large Language Models: A Case Study on Korean
UnknownBody
博客等级 码龄16年 5601粉丝 4754原创
评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付:10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

UnknownBody

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值