yangyu222的专栏

BERT Published by Google in 2018 Bidirectional Encoder Representation from Transformers Two Phrases: Pre-training, fine-turning Use Transformer proposed in Attention Is All You Need by Google 2017 to replace RNN BERT takes advantages of multiple mode.

ElMo Published in 2018 and named as Embedding from language Models Deep contextualized word representations that models complex characteristics of word use and how these uses vary across linguistic contexts. It enables models to better disambiguate betw

Glove Global Vectors for word Representation. Same model as Word2Vec Trainning is performed on aggregated global word-word. co-occurence statistics from a corpus. Must be trained offline.

Word2Vec Abstract The similar structure as NNLM, bug focus on Word Embedding Two learning approach: Continuous Bag of Word(CBOW) Continuous Skip-gram(Skip-gram) 1. CBow Given its context wi-N, wi-n+1,wi-n+2...

algorithm->machine learning->nlp->word embedding Abstract hard encode Bag of Word onehot embedding 1. Hard-coded Word is represent by ID. IDs arejust symbolic data. For example Enum, unicode string Cons Hard-code...