Reproducing "Efficient Estimation of Word Representations in Vector Space" (Mikolov et al., 2013) in PyTorch — Skip-gram with Negative Sampling, trained on the text8 corpus — and documenting why every ...

You shall know a word by the company it keeps. Train a tiny network to predict a word's neighbours — and the hidden layer it learns on the way becomes a map of meaning. Words used in similar contexts ...