Winnow算法

最新推荐文章于 2024-05-31 09:39:58 发布

sulliy

最新推荐文章于 2024-05-31 09:39:58 发布

阅读量7.4k

点赞数

分类专栏： Algorithm 文章标签：算法 algorithm variables features each classification

本文链接：https://blog.youkuaiyun.com/sulliy/article/details/6627850

版权

Winnow算法是机器学习中用于从标记样例学习线性分类器的方法，类似于感知机算法。与感知机的加权更新不同，Winnow采用乘法权重更新，能更好地处理大量无关维度的情况。尽管不复杂，但它在高维空间中表现良好。训练过程中，Winnow通过正负样例学习决策超平面，并适用于在线学习场景。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

The winnow algorithm is a technique from machine learning for learning a linear classifier from labeled examples. It is very similar to the perceptron algorithm. However, the perceptron algorithm uses an additive weight-update scheme, but winnow uses a multiplicative weight-update scheme that allows it to perform much better when many dimensions are irrelevant (hence its name). It is not a sophisticated algorithm but it scales well to high-dimensional spaces. During training, winnow is shown a sequence of positive and negative examples. From these it learns a decision hyperplane. It can also be used in the online learning setting, where the learning and the classification phase are not clearly separated.

		15-859(B) Machine Learning Theory            01/20/10
Online learning contd
* The Winnow algorithm for disjunctions
* Winnow for k-of-r functions and general LTFs in terms of L_1 margin
* If time: Infinite-attribute model, string-valued features
=======================================================================

WINNOW ALGORITHM
================
If you think about the problem of learning an OR-function, we
saw an algorithm: "list all features and cross off bad ones on
negative examples" that makes at most n mistakes.  But, what if most
features are irrelevant?  E.g., if representing a document as vector
indicating which words appear in it and which don't, then n is pretty
large!  What if the target is an OR of r relevant features where r is
a lot smaller than n.  Can we get a better bound in that case?

What could we do if computation time were no object?  How many bits to
describe an OR of r variables, where r << n?  Ans: O(r log n).  So, in
principle, we'd like to obtain a bound like this.

Winnow will give us a bound of O(r log n) mistakes efficiently.

So, this means you only have a small penalty for "throwing lots of
features at the problem".  In general, will say that an algorithm with
only polylog dependence on n is "attribute-efficient".  

Winnow Algorithm: (basic version)

1. Initialize the weights

最低0.47元/天解锁文章