minFunc 2012:Huber loss_huber loss markdown-优快云博客

本文链接：https://blog.youkuaiyun.com/tianliangjay/article/details/51146141

本文详细介绍了Huber损失函数的定义、性质以及在实际应用中的重要性，包括其在最小二乘法和鲁棒统计学中的角色。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

minFunc 2012:Huber loss

本Markdown编辑器使用[StackEdit][6]修改而来，用它写博客，将会带来全新的体验哦：

Huber loss 定义

Huber loss 定义

The Huber loss function describes the penalty incurred by an estimation procedure f. Huber (1964) defines the loss function piecewise by[1]

L δ (a) = {- x δ （ | a | - 1 2 δ ） f o r | a | \leq δ, o t h e r w i s e .

$L_\delta(a)=\begin{cases} -x & for |a|\leq\delta,\\ \delta（|a|-\frac{1}{2}\delta） & otherwise. \end{cases}$

This function is quadratic for small values of a, and linear for large values, with equal values and slopes of the different sections at the two points where $|a| = \delta$ . The variable a often refers to the residuals, that is to the difference between the observed and predicted values $a=y-f(x)$ , so the former can be expanded to

L δ (y, f (x)) = {- 1 2 (y - f (x)) 2 δ （ | y - f (x) | - 1 2 δ 2 ） f o r | y - f (x) | \leq δ, o t h e r w i s e .

$L_\delta(y,f(x))=\begin{cases} -\frac{1}{2}(y-f(x))^2 & for |y-f(x)|\leq\delta,\\ \delta（|y-f(x)|-\frac{1}{2}\delta^2） & otherwise. \end{cases}$

$Huber loss (green, \delta=1) and squared error loss (blue) as a function of y - f(x)$
Huber loss (green, \delta=1) and squared error loss (blue) as a function of y - f(x)

<!--matlab-->
function [f,g,H,T] = HuberLoss(w,X,y,k)
% w(feature,1)
% X(instance,feature)
% y(instance,1)

r = X*w-y;

closeInd = abs(r) <= k;

f = (1/2)*sum(r(closeInd).^2) + k*sum(abs(r(~closeInd))) - (1/2)*sum(~closeInd)*k^2;
if nargout > 1
   g = X(closeInd,:)'*(X(closeInd,:)*w - y(closeInd)) + k*X(~closeInd,:)'*sign(r(~closeInd));
end

if nargout > 2
   H = X(closeInd,:)'*X(closeInd,:); 
end