Home

Awesome

Polish Word Embeddings Review

Evaluation of polish word embeddings prepared by various research groups. Evaluation is done by words analogy task

Used word embeddings

Word2Vec Format

IPIPAN Models:

Facebook FastText:

Embeddings from https://github.com/Kyubyong/wordvectors Fasttext

Word2Vec

Embeddings from SÅ‚awomir Dadas (embeddings in results with opi prefix):

Pre-Trained Models

Evaluation procedure

To evaluate the word embedding, we use the function evaluation_word_pairs and assessment_word_analogies provided by gensim.

For analogue testing, a file provided by facebook fastext is used, a link to it can be found in References. To evaluate the similarity in word embedding, the Polish version of SimLex999, made available by IPIPAN, is used, but we had to adjust it to the function by removing the column with the row identifier (first collumn) and the entire relatedness column (last column). A link to the version provided by IPIPAN can be found in References.

Results

namevocab sizevector sizeTotal analogy accuracypearsonspearmanout of vocab
facebook_fastext-300.txt20000003000,660,3690,39344,004
nkjp+wiki-lemmas-restricted-300-skipg-ns.txt14077623000,620,38750,44441,6016
nkjp+wiki-lemmas-all-300-skipg-ns.txt15493223000,610,39730,45671,6016
nkjp+wiki-lemmas-restricted-300-cbow-ns.txt14077623000,590,41320,45411,6016
nkjp-lemmas-restricted-300-skipg-ns.txt11628453000,580,40310,45081,6016
wiki-lemmas-restricted-300-skipg-ns.txt4466083000,580,33110,38932,1021
wiki-lemmas-all-300-skipg-ns.txt4730003000,560,34560,40252,1021
nkjp-lemmas-all-300-skipg-ns.txt12826213000,560,41270,46341,6016
wiki-lemmas-restricted-300-cbow-ns.txt4466083000,550,36470,40442,1021
wiki-lemmas-restricted-100-skipg-ns.txt4466081000,530,30080,3482,1021
nkjp+wiki-lemmas-all-300-cbow-ns.txt15493223000,530,40850,45781,6016
nkjp-lemmas-restricted-300-cbow-ns.txt11628453000,530,41080,45241,6016
nkjp+wiki-lemmas-restricted-300-skipg-hs.txt14077623000,520,31820,37851,6016
nkjp+wiki-lemmas-all-300-cbow-ns-50.txt2941823000,520,40920,461,6016
wiki-lemmas-restricted-300-skipg-hs.txt4466083000,520,28110,33742,1021
wiki-lemmas-all-300-cbow-ns.txt4730003000,510,36340,41042,1021
wiki-lemmas-all-300-skipg-hs.txt4730003000,510,28610,34252,1021
nkjp+wiki-lemmas-restricted-100-skipg-ns.txt14077621000,500,34360,39371,6016
nkjp+wiki-lemmas-pos-all-300-skipg-ns.txt16862083000,50-0,1073-0,399,4995
nkjp+wiki-lemmas-pos-all-300-cbow-hs.txt16862083000,50-0,1167-0,399,4995
wiki-lemmas-all-100-skipg-ns.txt4730001000,500,30540,35582,1021
nkjp+wiki-lemmas-all-300-skipg-hs.txt15493223000,500,33190,39371,6016
nkjp+wiki-lemmas-restricted-100-cbow-ns.txt14077621000,490,3680,41231,6016
nkjp-lemmas-restricted-300-skipg-hs.txt11628453000,480,34930,40421,6016
nkjp+wiki-forms-restricted-300-skipg-ns.txt19824793000,480,38330,44061,3013
wiki-lemmas-restricted-100-cbow-ns.txt4466081000,480,33210,37252,1021
nkjp-lemmas-all-300-cbow-ns.txt12826213000,480,40570,45441,6016
kyubyong_fasttext.txt500363000,480,31140,387941,4414
nkjp-lemmas-restricted-100-skipg-ns.txt11628451000,470,36130,4031,6016
nkjp+wiki-lemmas-all-100-skipg-ns.txt15493221000,460,35240,40541,6016
nkjp+wiki-forms-all-300-skipg-ns.txt21231323000,460,38810,43881,3013
nkjp-forms-restricted-300-skipg-ns.txt17137383000,460,39370,44461,3013
nkjp-lemmas-all-300-skipg-hs.txt12826213000,460,36030,41661,6016
nkjp-forms-all-300-skipg-ns.txt18327003000,450,39240,44051,3013
nkjp+wiki-forms-all-300-skipg-ns-50.txt5569323000,450,38560,4381,3013
nkjp-lemmas-all-100-skipg-ns.txt12826211000,440,36730,41111,6016
nkjp-lemmas-restricted-100-cbow-ns.txt11628451000,430,36610,41021,6016
wiki-lemmas-all-100-cbow-ns.txt4730001000,430,32920,37942,1021
wiki-lemmas-restricted-100-skipg-hs.txt4466081000,420,25470,30112,1021
nkjp+wiki-forms-restricted-300-cbow-ns.txt19824793000,420,36310,39171,3013
nkjp+wiki-forms-restricted-300-skipg-hs.txt19824793000,410,34150,40331,3013
nkjp+wiki-lemmas-restricted-100-skipg-hs.txt14077621000,400,28280,33581,6016
nkjp-forms-restricted-300-cbow-ns.txt17137383000,400,36230,38841,3013
nkjp+wiki-lemmas-all-100-cbow-ns.txt15493221000,400,36570,41661,6016
nkjp+wiki-forms-all-300-cbow-ns.txt21231323000,400,34520,37091,3013
wiki-lemmas-all-100-skipg-hs.txt4730001000,400,26460,31332,1021
nkjp+wiki-forms-all-300-skipg-hs.txt21231323000,390,340,39751,3013
nkjp+wiki-forms-all-300-cbow-ns-50.txt5569323000,390,34420,36831,3013
nkjp+wiki-forms-restricted-100-skipg-ns.txt19824791000,390,3330,38191,3013
nkjp-forms-restricted-300-skipg-hs.txt17137383000,380,34040,3871,3013
nkjp-forms-restricted-100-skipg-ns.txt17137381000,380,33990,38031,3013
nkjp-forms-all-300-cbow-ns.txt18327003000,380,34810,37141,3013
nkjp+wiki-forms-all-300-skipg-hs-50.txt5569323000,380,34480,40211,3013
nkjp+wiki-forms-all-100-skipg-ns.txt21231321000,370,34210,38541,3013
nkjp-forms-all-300-skipg-hs.txt18327003000,370,35030,39061,3013
nkjp-lemmas-restricted-100-skipg-hs.txt11628451000,370,31530,36121,6016
nkjp+wiki-lemmas-all-100-skipg-hs.txt15493221000,360,29310,34681,6016
nkjp-forms-all-100-skipg-ns.txt18327001000,360,35080,38791,3013
nkjp-lemmas-all-100-cbow-ns.txt12826211000,350,36240,41241,6016
nkjp+wiki-lemmas-restricted-300-cbow-hs.txt14077623000,350,4150,461,6016
nkjp+wiki-forms-restricted-100-cbow-ns.txt19824791000,330,31060,34211,3013
wiki-lemmas-restricted-300-cbow-hs.txt4466083000,320,3610,39862,1021
nkjp-lemmas-all-100-skipg-hs.txt12826211000,320,32270,37161,6016
wiki-forms-restricted-300-skipg-ns.txt6983043000,320,33880,40535,2052
wiki-forms-all-300-skipg-ns.txt7247563000,310,33210,38845,5055
nkjp-forms-restricted-100-cbow-ns.txt17137381000,310,3110,34091,3013
nkjp+wiki-forms-restricted-100-skipg-hs.txt19824791000,300,29890,3511,3013
wiki-forms-all-100-skipg-ns-30-it100.txt2263961000,300,27480,34768,6086
nkjp-forms-restricted-100-skipg-hs.txt17137381000,290,30570,34411,3013
nkjp+wiki-forms-all-100-skipg-hs.txt21231321000,280,30350,35441,3013
wiki-forms-restricted-300-skipg-hs.txt6983043000,280,26560,33795,2052
nkjp-lemmas-restricted-300-cbow-hs.txt11628453000,280,41210,45631,6016
nkjp+wiki-forms-all-100-cbow-ns.txt21231321000,280,29750,32631,3013
wiki-forms-restricted-100-skipg-ns.txt6983041000,280,30160,36575,2052
wiki-forms-all-100-skipg-ns.txt7247561000,280,30280,36875,5055
nkjp+wiki-lemmas-all-300-cbow-hs.txt15493223000,280,40710,45971,6016
wiki-forms-all-300-skipg-hs.txt7247563000,280,27670,3435,5055
wiki-lemmas-all-300-cbow-hs.txt4730003000,270,3620,40562,1021
wiki-forms-restricted-300-cbow-ns.txt6983043000,270,27950,3155,2052
nkjp-forms-all-100-skipg-hs.txt18327001000,260,30910,34361,3013
nkjp-forms-all-100-cbow-ns.txt18327001000,260,30190,33051,3013
wiki-lemmas-restricted-100-cbow-hs.txt4466081000,260,32150,35752,1021
wiki-forms-all-300-cbow-ns-30.txt2263963000,260,28280,32018,6086
nkjp-lemmas-all-300-cbow-hs.txt12826213000,240,40850,45821,6016
wiki-forms-all-300-cbow-ns.txt7247563000,240,27910,31385,5055
nkjp+wiki-forms-restricted-300-cbow-hs.txt19824793000,230,29740,31471,3013
nkjp-forms-restricted-300-cbow-hs.txt17137383000,230,30610,3221,3013
kyubyong_word2vec.txt500353000,230,25440,308741,4414
nkjp-lemmas-restricted-100-cbow-hs.txt11628451000,220,35030,39671,6016
wiki-forms-restricted-100-skipg-hs.txt6983041000,220,25940,32155,2052
nkjp-forms-all-300-cbow-hs.txt18327003000,220,29620,3111,3013
wiki-forms-all-100-cbow-ns-30-it100.txt2263961000,210,26430,30528,6086
wiki-forms-restricted-100-cbow-ns.txt6983041000,210,2590,29365,2052
nkjp+wiki-forms-all-300-cbow-hs.txt21231323000,210,29240,30651,3013
wiki-forms-all-100-skipg-hs.txt7247561000,210,26750,32765,5055
nkjp+wiki-lemmas-all-100-cbow-hs.txt15493221000,210,34710,39841,6016
nkjp+wiki-forms-all-300-cbow-hs-50.txt5569323000,200,29420,31041,3013
nkjp-lemmas-all-100-cbow-hs.txt12826211000,190,35560,40721,6016
wiki-lemmas-all-100-cbow-hs.txt4730001000,190,31890,36342,1021
wiki-forms-all-100-cbow-ns.txt7247561000,190,2550,29365,5055
nkjp+wiki-forms-restricted-100-cbow-hs.txt19824791000,180,2420,26391,3013
nkjp-forms-restricted-100-cbow-hs.txt17137381000,180,25370,27481,3013
nkjp+wiki-forms-all-100-cbow-hs.txt21231321000,150,24370,26881,3013
nkjp-forms-all-100-cbow-hs.txt18327001000,150,24930,27331,3013
wiki-forms-restricted-300-cbow-hs.txt6983043000,130,24960,28165,2052
wiki-forms-all-300-cbow-hs-30.txt2263963000,130,26260,29168,6086
wiki-forms-all-300-cbow-hs.txt7247563000,130,26360,29255,5055
wiki-forms-restricted-100-cbow-hs.txt6983041000,090,21620,24275,2052
wiki-forms-all-100-cbow-hs.txt7247561000,080,22940,26115,5055
opi_word2vec_100_3_polish.bin19340301000,330,39610,46161,1011
opi_word2vec_300_3_polish.bin19340283000,410,4420,50051,1011
opi_word2vec_500_3_polish.bin19340285000,420,45320,51071.1011
opi_word2vec_800_3_polish.bin19340288000.42610.45990.51631.1011
opi_fasttext_100_3_polish.bin24883061000,230,35320,40921,1011
opi_fasttext_300_3_polish.bin24883063000.270,39760,44471,1011
opi_fasttext_500_3_polish.bin24883065000.27180.40810.4511.1011
opi_fasttext_800_3_polish.bin24883068000.24290.41250.45191.1011
opi_glove_100_3_polish.txt19263201000,540,31280,37051,1011
opi_glove_300_3_polish.txt19263193000,620,3510,40271,1011
opi_glove_500_3_polish.txt19263195000,610,34930,39871,1011
opi_glove_800_3_polish.txt19263198000,570,34010,39111,1011

References

Polish Word Analogy:

https://dl.fbaipublicfiles.com/fasttext/word-analogies/questions-words-pl.txt

MSimLex999 Polish:

http://zil.ipipan.waw.pl/CoDeS?action=AttachFile&do=view&target=MSimLex999_Polish.zip