What is regularization in Machine Learning

本文探讨了正则化技术如何解决机器学习中的过拟合问题。通过引入惩罚项来限制模型复杂度,避免模型过分依赖训练数据,提高泛化能力。文中详细解释了L1(LASSO)和L2(Ridge)正则化的应用,并介绍了如何使用交叉验证选择最优的正则化参数。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

转自:https://www.quora.com/What-is-regularization-in-machine-learning

Regularization is a technique used in an attempt to solve the overfitting[1] problem in statistical models.*

First of all, I want to clarify how this problem of overfitting arises.

When someone wants to model a problem, let's say trying to predict the wage of someone based on his age, he will first try a linear regression model with age as an independent variable and wage as a dependent one. This model will mostly fail, since it is too simple.

Then, you might think: well, I also have the age, the sex and the education of each individual in my data set. I could add these as explaining variables.

Your model becomes more interesting and more complex. You measure its accuracy regarding a loss metric L(X,Y)

where X is your design matrix and Y

is the observations (also denoted targets) vector (here the wages).

You find out that your result are quite good but not as perfect as you wish.

So you add more variables: location, profession of parents, social background, number of children, weight, number of books, preferred color, best meal, last holidays destination and so on and so forth.

Your model will do good but it is probably overfitting, i.e. it will probably have poor prediction and generalization power: it sticks too much to the data and the model has probably learned the background noise while being fit. This isn't of course acceptable.

So how do you solve this?

It is here where the regularization technique comes in handy.

You penalize your loss function by adding a multiple of an L1

(LASSO[2]) or an L2 (Ridge[3]) norm of your weights vector w

(it is the vector of the learned parameters in your linear regression). You get the following equation:

L(X,Y)+λN(w)

( N

is either the L1 , L2

or any other norm)

This will help you avoid overfitting and will perform, at the same time, features selection for certain regularization norms (the L1

in the LASSO does the job).

Finally you might ask: OK I have everything now. How can I tune in the regularization term λ

?

One possible answer is to use cross-validation: you divide your training data, you train your model for a fixed value of λ

and test it on the remaining subsets and repeat this procedure while varying λ . Then you select the best λ

that minimizes your loss function.


I hope this was helpful. Let me know if there is any mistakes. I will try to add some graphs and eventually some R or Python code to illustrate this concept.

Also, you can read more about these topics (regularization and cross validation) here:

* Actually this is only one of the many uses. According to Wikipedia, it can be used to solve ill-posed problems. Here is the article for reference: Regularization (mathematics).

As always, make sure to follow me for more insights about machine learning and its pitfalls: http://quora.com/profile/Yassine...

Footnotes

[1] Overfitting

[2] Lasso (statistics)

[3] Tikhonov regularization


内容概要:文章详细介绍了ETL工程师这一职业,解释了ETL(Extract-Transform-Load)的概念及其在数据处理中的重要性。ETL工程师负责将分散、不统一的数据整合为有价值的信息,支持企业的决策分析。日常工作包括数据整合、存储管理、挖掘设计支持和多维分析展现。文中强调了ETL工程师所需的核心技能,如数据库知识、ETL工具使用、编程能力、业务理解能力和问题解决能力。此外,还盘点了常见的ETL工具,包括开源工具如Kettle、XXL-JOB、Oozie、Azkaban和海豚调度,以及企业级工具如TASKCTL和Moia Comtrol。最后,文章探讨了ETL工程师的职业发展路径,从初级到高级的技术晋升,以及向大数据工程师或数据产品经理的横向发展,并提供了学习资源和求职技巧。 适合人群:对数据处理感兴趣,尤其是希望从事数据工程领域的人士,如数据分析师、数据科学家、软件工程师等。 使用场景及目标:①了解ETL工程师的职责和技能要求;②选择适合自己的ETL工具;③规划ETL工程师的职业发展路径;④获取相关的学习资源和求职建议。 其他说明:随着大数据技术的发展和企业数字化转型的加速,ETL工程师的需求不断增加,尤其是在金融、零售、制造、人工智能、物联网和区块链等领域。数据隐私保护法规的完善也使得ETL工程师在数据安全和合规处理方面的作用更加重要。
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值