Pyspark MLlib example of Word2Vec

最新推荐文章于 2025-12-09 16:47:37 发布

原创最新推荐文章于 2025-12-09 16:47:37 发布 · 152 阅读

0 ·

CC 4.0 BY-SA版权

文章标签：

#spark-ml #word2vec #人工智能

本文讲述了在使用SparkMLlib中的Word2Vec进行特征提取时，对输出向量的深入理解，揭示其实质是单层神经网络训练后隐藏层的加权值。参考链接：freecodecamp.org教程。

调试Spark MLlib自带的官方特征提取示例时，对于Word2Vec输出的结果不是很明白，参考其他资料后了解了输出结果的含义，记录一下：

Word2Vec输出的结果如下：

结果向量说明：

Word2Vec is a ML method of building a language model based on Deep Learning ideas.But a neural network that is used here is rather shallow(consists of only one hidden layer).

Behind the scenes we are actually training a simple neural network with a single hidden layer.But we are actually not going to use the neural network after training.Instead,the goal is to learn the weights of the hidden layer.These weights are essentially the word vectors that we're trying to learn.

结论：

word2Vec输出的向量实际是其背后三层神经网络隐藏层的加权值

参考：How to get started with Word2Vec — and then how to make it work (freecodecamp.org)