design a regress function whose coeficients satisfy certain conditions

breeze_gao

于 2013-04-16 11:14:48 发布

阅读量680

点赞数

CC 4.0 BY-SA版权

分类专栏： statistic and data mining 文章标签： KKT conditions

本文链接：https://blog.youkuaiyun.com/breeze_gao/article/details/8807706

statistic and data mining 专栏收录该内容

4 篇文章

订阅专栏

本文介绍了一种基于KKT条件的正则化线性回归方法，该方法适用于带有特定约束条件的数据集，包括权重归一化及非负限制，并提供了R语言实现代码。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

Problem:

Given n points (x_1, x_2, ...x_n) as independent variables, where x_i can be a value or a vector, and the dependent variables (y_1, y_2, ... y_i), design a regression that satifies:

1. f(x) = t(w) * x (assume the the attribute of x corresponding w_0 is 1)

2. sigma(w) = 1

3. w_i >= 0

3. minimize sigma(y - f(x))^2

Solution:

The solution is based on KKT condition which is a generized from langrange muptiply method.

KKT conditions of this question:

1. delta(siagma(y - f(x))^2 + a * delta(w) + sigma(b_i * delta(w_i)) = 0

2. sigma(w) = 1

solve about equotions, we get:

b_i = 0

rbind(

cbind(2 * t(X) *X, c(1, ....),

c(w, 1)

) * c(w, a) = 2 * t(X) * Y

R code: ( assume x is a vector, whose first element is 1 correponding w_0)

regress <- function(x, y) {
  A <- 2 * t(x) %*% x
  B <- rep(1, nrow(A))
  C <- 2 * t(x) %*% y
  D <- c(rep(1, nrow(A)), 0)
  left <- rbind(cbind(A, B), D)
  coef <- solve(left, c(C, 1))
  coef[1:length(coef) - 1]
}