关于one-hot encoding思考

最新推荐文章于 2024-01-24 20:17:38 发布

weixin_30839881

最新推荐文章于 2024-01-24 20:17:38 发布

阅读量60

点赞数

CC 4.0 BY-SA版权

原文链接：http://www.cnblogs.com/ljygoodgoodstudydaydayup/p/6874046.html

本文探讨了机器学习中类别特征的不同编码方式，特别是针对线性模型的一热编码（one-hot encoding）的重要性。通过实例说明了一热编码如何帮助解决多分类问题，并对比了其与简单编码在距离计算上的差异。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

Many learning algorithms either learn a single weight per feature, or they use distances between samples. The former is the case for linear models such as logistic regression, which are easy to explain.

Suppose you have a dataset having only a single categorical feature "nationality", with values "UK", "French" and "US". Assume, without loss of generality, that these are encoded as 0, 1 and 2. You then have a weight w for this feature in a linear classifier, which will make some kind of decision based on the constraint w×x + b > 0, or equivalently w×x < b.

The problem now is that the weight w cannot encode a three-way choice. The three possible values of w×x are 0, w and 2×w. Either these three all lead to the same decision (they're all < b or ≥b) or "UK" and "French" lead to the same decision, or "French" and "US" give the same decision. There's no possibility for the model to learn that "UK" and "US" should be given the same label, with "French" the odd one out.（二分类问题，若dummy encoding，us和uk始终不能单独成为一类，而若one-hot encoding，则可以适应任何情况）

By one-hot encoding, you effectively blow up the feature space to three features, which will each get their own weights, so the decision function is now w[UK]x[UK] + w[FR]x[FR] + w[US]x[US] < b, where all the x's are booleans. In this space, such a linear function can express any sum/disjunction of the possibilities (e.g. "UK or US", which might be a predictor for someone speaking English).

Similarly, any learner based on standard distance metrics (such as k-nearest neighbors) between samples will get confused without one-hot encoding. With the naive encoding and Euclidean distance, the distance between French and US is 1. The distance between US and UK is 2. But with the one-hot encoding, the pairwise distances between [1, 0, 0], [0, 1, 0] and [0, 0, 1] are all equal to √2.

This is not true for all learning algorithms; decision trees and derived models such as random forests, if deep enough, can handle categorical variables without one-hot encoding.

dataframe one-hot encoding：pandas.get_dummies方法

参考：

https://gist.github.com/ramhiser/982ce339d5f8c9a769a0

http://pandas.pydata.org/pandas-docs/version/0.17.0/generated/pandas.get_dummies.html

转载于:https://www.cnblogs.com/ljygoodgoodstudydaydayup/p/6874046.html