=========== cell 35 word corpus \ 0 visa Wiki control (same size) 1 visa Forum (same size) 2 salary Wiki control (same size) 3 salary Forum (same size) 4 doha Wiki control (same size) 5 doha Forum (same size) 6 family Wiki control (same size) 7 family Forum (same size) 8 king Wiki control (same size) 9 king Forum (same size) 10 paris Wiki control (same size) 11 paris Forum (same size) 12 computer Wiki control (same size) 13 computer Forum (same size) top 5 0 visas, passport, bdtc, passports, zwartendijk 1 rp, iqama, viza, residence, residance 2 salaries, repayment, pay, wages, paid 3 slary, salry, sallary, salery, salaray 4 manama, abbasiyyin, qatar, dubai, shaab 5 qatar, doah, qata, maseeid, alkhore 6 relatives, father, families, mother, grandfather 7 familly, rescidence, fmaily, pernament, famliy 8 prince, queen, monarch, harthacnut, harefoot 9 edshel, koil, ypsilanti, edsel, bhumibol 10 brussels, marseille, fontainebleau, universell... 11 amsterdam, france, cancun, london, tokyo 12 computers, software, mainframe, hardware, comp... 13 soundcard, desktop, hardware, pc, computers =========== cell 39 word model \ 0 king GloVe (pre-trained) 1 king Wiki 20% 2 cat GloVe (pre-trained) 3 cat Wiki 20% 4 apple GloVe (pre-trained) 5 apple Wiki 20% 6 bank GloVe (pre-trained) 7 bank Wiki 20% 8 good GloVe (pre-trained) 9 good Wiki 20% 10 taiwan GloVe (pre-trained) 11 taiwan Wiki 20% 12 computer GloVe (pre-trained) 13 computer Wiki 20% top 5 (similarity) 0 prince (0.77), queen (0.75), son (0.70), brother (0.70), monarch (0.70) 1 nangklao (0.74), prince (0.74), queen (0.72), suriyothai (0.72), throne (0.72) 2 dog (0.88), rabbit (0.74), cats (0.73), monkey (0.73), pet (0.72) 3 dog (0.74), rabbit (0.74), pet (0.72), sourpuss (0.71), mouse (0.71) 4 microsoft (0.74), ibm (0.68), intel (0.68), software (0.68), dell (0.67) 5 blackberry (0.77), iphone (0.70), tvos (0.68), raspberry (0.67), xelibri (0.66) 6 banks (0.81), banking (0.75), credit (0.70), investment (0.69), financial (0.68) 7 savings (0.74), bancorp (0.72), guaranty (0.71), dfcu (0.70), bancorporation (0.70) 8 better (0.89), sure (0.83), really (0.83), kind (0.83), very (0.83) 9 sure (0.74), bad (0.73), tough (0.72), better (0.72), decent (0.71) 10 mainland (0.86), china (0.83), taiwanese (0.79), taipei (0.79), hong (0.77) 11 china (0.85), guangdong (0.80), taipei (0.80), hainan (0.80), japan (0.78) 12 computers (0.88), software (0.84), technology (0.76), pc (0.74), hardware (0.73) 13 computers (0.82), software (0.81), computing (0.81), mainframe (0.79), hardware (0.76) =========== cell 41 top-1 top-3 top-5 top-10 GloVe (pre-trained) 63.11 73.68 77.73 82.01 Wiki 20% 48.29 62.27 67.01 72.77 =========== cell 42 wrong answers: 10107 of 19544 OOV (question word not in vocab): 0 gold answer was 2nd-10th: 4785 : capital-world Question gold pred rank 508 Abuja Nigeria Amman Jordan jordan kuwait NaN 510 Abuja Nigeria Antananarivo Madagascar madagascar senegal 8.0 513 Abuja Nigeria Asmara Eritrea eritrea indonesia 5.0 518 Abuja Nigeria Bamako Mali mali faso NaN 526 Abuja Nigeria Bern Switzerland switzerland aarau 2.0 527 Abuja Nigeria Bishkek Kyrgyzstan kyrgyzstan tajikistan 2.0 : currency Question gold pred rank 5030 Algeria dinar Angola kwanza kwanza banknote NaN 5032 Algeria dinar Armenia dram dram hryvnia NaN 5033 Algeria dinar Brazil real real peso NaN 5034 Algeria dinar Bulgaria lev lev hryvnia NaN 5035 Algeria dinar Cambodia riel riel kyat NaN 5036 Algeria dinar Canada dollar dollar toonie NaN : family Question gold pred rank 8363 boy girl brother sister sister cousin 4.0 8366 boy girl father mother mother stepmother 2.0 8376 boy girl nephew niece niece cousin 8.0 8381 boy girl stepbrother stepsister stepsister stepmother 5.0 8382 boy girl stepfather stepmother stepmother husband 2.0 8391 brother sister groom bride bride bridesmaid 2.0 : gram3-comparative Question gold pred rank 10673 bad worse big bigger bigger sooner 2.0 10677 bad worse cool cooler cooler better 6.0 10686 bad worse hot hotter hotter peaking NaN 10687 bad worse large larger larger smaller 2.0 10688 bad worse long longer longer shorter 4.0 10690 bad worse low lower lower higher NaN : gram6-nationality-adjective Question gold pred \ 14183 Albania Albanian Argentina Argentinean argentinean argentinian 14186 Albania Albanian Belarus Belorussian belorussian belarusian 14192 Albania Albanian Colombia Colombian colombian honduran 14201 Albania Albanian India Indian indian oriya 14217 Albania Albanian Slovakia Slovakian slovakian slovak 14224 Argentina Argentinean Belarus Belorussian belorussian belarusian rank 14183 8.0 14186 NaN 14192 2.0 14201 4.0 14217 5.0 14224 NaN : gram8-plural Question gold pred rank 17342 banana bananas bird birds birds waterfowl 7.0 17344 banana bananas building buildings buildings renovating 2.0 17345 banana bananas car cars cars truck 2.0 17346 banana bananas cat cats cats rabbits 5.0 17347 banana bananas child children children abusers NaN 17348 banana bananas cloud clouds clouds lidar NaN =========== cell 43 questions containing a stop word: 206 SubCategory : family 86 : gram1-adjective-to-adverb 62 : currency 58 Name: count, dtype: int64 ['Algeria dinar Korea won', 'Angola kwanza Korea won', 'Argentina peso Korea won', 'Armenia dram Korea won', 'Brazil real Korea won'] trained on wiki_sampled_5_nostop.txt: 6.8 min, vocab 278,810 Wiki 5% (stop words removed): overall 45.03% (OOV questions 284) Wiki 5% Wiki 5% (stop words removed) Overall 45.22 45.03 Semantic 53.93 54.59 Syntactic 37.99 37.09 : capital-common-countries 79.25 81.82 : capital-world 67.68 70.73 : currency 7.16 7.27 : city-in-state 37.21 36.93 : family 67.19 50.20 : gram1-adjective-to-adverb 13.21 12.90 : gram2-opposite 10.96 8.62 : gram3-comparative 43.09 37.46 : gram4-superlative 16.76 16.40 : gram5-present-participle 29.73 31.63 : gram6-nationality-adjective 76.99 82.05 : gram7-past-tense 44.87 40.64 : gram8-plural 37.24 42.27 : gram9-plural-verbs 38.16 27.01 OOV questions 107.00 284.00