=== cell 2 stdout downloaded questions-words.txt === cell 4 stdout : capital-common-countries Athens Greece Baghdad Iraq Athens Greece Bangkok Thailand Athens Greece Beijing China Athens Greece Berlin Germany Athens Greece Bern Switzerland Athens Greece Cairo Egypt Athens Greece Canberra Australia Athens Greece Hanoi Vietnam Athens Greece Havana Cuba === cell 5 stdout 19544 questions === cell 7 result Question Category SubCategory 0 Athens Greece Baghdad Iraq Semantic : capital-common-countries 1 Athens Greece Bangkok Thailand Semantic : capital-common-countries 2 Athens Greece Beijing China Semantic : capital-common-countries 3 Athens Greece Berlin Germany Semantic : capital-common-countries 4 Athens Greece Bern Switzerland Semantic : capital-common-countries === cell 12 stdout The Gensim model loaded successfully! === cell 13 stdout OOV questions: 0 === cell 14 stdout Category: Semantic, Accuracy: 65.3399481339497% Category: Syntactic, Accuracy: 61.255269320843084% Sub-Category: capital-common-countries, Accuracy: 93.87351778656127% Sub-Category: capital-world, Accuracy: 88.94783377541998% Sub-Category: currency, Accuracy: 14.203233256351039% Sub-Category: city-in-state, Accuracy: 30.806647750304013% Sub-Category: family, Accuracy: 81.62055335968378% Sub-Category: gram1-adjective-to-adverb, Accuracy: 24.39516129032258% Sub-Category: gram2-opposite, Accuracy: 20.073891625615765% Sub-Category: gram3-comparative, Accuracy: 79.12912912912913% Sub-Category: gram4-superlative, Accuracy: 54.278074866310156% Sub-Category: gram5-present-participle, Accuracy: 69.50757575757575% Sub-Category: gram6-nationality-adjective, Accuracy: 87.86741713570981% Sub-Category: gram7-past-tense, Accuracy: 55.44871794871795% Sub-Category: gram8-plural, Accuracy: 71.996996996997% Sub-Category: gram9-plural-verbs, Accuracy: 58.39080459770115% === cell 15 [image cell15_0.png] === cell 15 result
=== cell 18 stdout data\wiki_texts_part_0.txt.gz 1.51 GB data\wiki_texts_part_1.txt.gz 0.84 GB data\wiki_texts_part_2.txt.gz 0.67 GB data\wiki_texts_part_3.txt.gz 0.60 GB data\wiki_texts_part_4.txt.gz 0.57 GB data\wiki_texts_part_5.txt.gz 0.58 GB data\wiki_texts_part_6.txt.gz 0.56 GB data\wiki_texts_part_7.txt.gz 0.53 GB data\wiki_texts_part_8.txt.gz 0.55 GB data\wiki_texts_part_9.txt.gz 0.55 GB data\wiki_texts_part_10.txt.gz 0.00 GB === cell 19 stdout all parts downloaded === cell 20 stdout 11 parts: ['wiki_texts_part_0.txt.gz', 'wiki_texts_part_1.txt.gz', 'wiki_texts_part_2.txt.gz', 'wiki_texts_part_3.txt.gz', 'wiki_texts_part_4.txt.gz', 'wiki_texts_part_5.txt.gz', 'wiki_texts_part_6.txt.gz', 'wiki_texts_part_7.txt.gz', 'wiki_texts_part_8.txt.gz', 'wiki_texts_part_9.txt.gz', 'wiki_texts_part_10.txt.gz'] === cell 21 stdout total compressed size: 6.93 GB === cell 22 stdout 0 anarchism is political philosophy and movement that is against all forms of authority and seeks to abolish the institutions it claims maintain unnecessary coercion and hierarchy typically including th ... 1 albedo change in greenland the map shows the difference between the amount of sunlight greenland reflected in the summer of versus the average percent it reflected between and some areas reflect close ... 2 or is the first letter and the first vowel letter of the latin alphabet used in the modern english alphabet and others worldwide its name in english is pronounced plural aes it is similar in shape to ... 3 alabama is state in the southeastern region of the united states it borders tennessee to the north georgia to the east florida and the gulf of mexico to the south and mississippi to the west alabama i ... 4 in greek mythology achilles or achilleus was hero of the trojan war who was known as being the greatest of all the greek warriors the central character in homer iliad he was the son of the nereid thet ... 5 abraham lincoln february april was an american lawyer politician and statesman who served as the th president of the united states from until his assassination in he led the united states through the ... 6 aristotle aristotélēs bc was an ancient greek philosopher and polymath his writings cover broad range of subjects spanning the natural sciences philosophy linguistics economics politics psychology and ... 7 an american in paris is jazz influenced symphonic poem or tone poem for orchestra by american composer george gershwin first performed in it was inspired by the time that gershwin had spent in paris a ... 8 the academy award for best production design recognizes achievement for art direction in film the category original name was best art direction but was changed to its current name in for the th academ ... 9 the academy awards of merit commonly known as the oscars or academy awards are awards for artistic and technical merit given for excellence within the american and international film industry they are ... === cell 24 stdout articles: total 5,623,655, sampled 1,124,733 (20.00%) wiki_sampled_5.txt 1.04 GB wiki_sampled_10.txt 2.08 GB wiki_sampled_20.txt 4.16 GB === cell 25 stdout 20% wiki tokens after preprocessing: 666,053,126 === cell 25 stdout trained on wiki_sampled_20_clean.txt: 36.0 min, vocab 299,405 === cell 27 stdout OOV questions: 0 Category: Semantic, Accuracy: 56.0942609087834% Category: Syntactic, Accuracy: 41.658079625292736% Sub-Category: capital-common-countries, Accuracy: 77.4703557312253% Sub-Category: capital-world, Accuracy: 72.56852343059239% Sub-Category: currency, Accuracy: 9.237875288683602% Sub-Category: city-in-state, Accuracy: 34.252128090798536% Sub-Category: family, Accuracy: 74.1106719367589% Sub-Category: gram1-adjective-to-adverb, Accuracy: 16.028225806451612% Sub-Category: gram2-opposite, Accuracy: 16.871921182266007% Sub-Category: gram3-comparative, Accuracy: 45.94594594594595% Sub-Category: gram4-superlative, Accuracy: 26.203208556149733% Sub-Category: gram5-present-participle, Accuracy: 32.38636363636363% Sub-Category: gram6-nationality-adjective, Accuracy: 80.61288305190745% Sub-Category: gram7-past-tense, Accuracy: 43.84615384615385% Sub-Category: gram8-plural, Accuracy: 42.492492492492495% Sub-Category: gram9-plural-verbs, Accuracy: 41.839080459770116% === cell 28 [image cell28_0.png] === cell 28 result
=== cell 31 stdout GloVe (pre-trained): overall 63.11% (OOV questions 0) === cell 31 stdout Wiki 20%: overall 48.21% (OOV questions 0) === cell 32 stdout keys: ['RelComments', 'RelQuestion', 'THREAD_SEQUENCE'] | question keys: ['RELQ_DATE', 'RELQ_ID', 'RelQSubject', 'RELQ_USERNAME', 'RELQ_USERID', 'RelQBody', 'RELQ_CATEGORY'] subject: Thailand:IT Minsitry blocks CNN; Facebook; first comment: have they blocked porn??? === cell 33 stdout forum: 189,941 threads, 2,118,254 posts, 72,919,300 tokens === cell 33 stdout 5% wiki tokens: 166,194,401 === cell 33 stdout wiki control: 72,920,923 tokens === cell 34 stdout trained on forum_clean.txt: 3.6 min, vocab 100,310 === cell 34 stdout trained on wiki_control_clean.txt: 4.0 min, vocab 224,050 === cell 34 stdout Forum (same size): overall 26.01% (OOV questions 3510) === cell 34 stdout Wiki control (same size): overall 41.98% (OOV questions 169) === cell 34 result Wiki control (same size) Forum (same size) \ Overall 41.98 26.01 Semantic 46.95 12.88 Syntactic 37.85 36.92 : capital-common-countries 79.05 34.19 : capital-world 55.02 12.82 : currency 6.35 1.27 : city-in-state 36.12 2.59 : family 65.02 62.06 : gram1-adjective-to-adverb 9.38 12.30 : gram2-opposite 9.61 13.05 : gram3-comparative 52.70 66.82 : gram4-superlative 17.47 41.71 : gram5-present-participle 31.82 51.99 : gram6-nationality-adjective 76.42 20.95 : gram7-past-tense 41.67 34.81 : gram8-plural 35.51 40.62 : gram9-plural-verbs 33.45 44.48 OOV questions 169.00 3510.00 Wiki 20% Overall 48.21 Semantic 56.09 Syntactic 41.66 : capital-common-countries 77.47 : capital-world 72.57 : currency 9.24 : city-in-state 34.25 : family 74.11 : gram1-adjective-to-adverb 16.03 : gram2-opposite 16.87 : gram3-comparative 45.95 : gram4-superlative 26.20 : gram5-present-participle 32.39 : gram6-nationality-adjective 80.61 : gram7-past-tense 43.85 : gram8-plural 42.49 : gram9-plural-verbs 41.84 OOV questions 0.00 === cell 34 result coverage: wiki control coverage: forum SubCategory : capital-common-countries 100.0 100.0 : capital-world 98.3 48.6 : currency 86.8 39.0 : city-in-state 100.0 68.3 : family 100.0 91.3 : gram1-adjective-to-adverb 100.0 100.0 : gram2-opposite 100.0 100.0 : gram3-comparative 100.0 100.0 : gram4-superlative 94.1 100.0 : gram5-present-participle 100.0 100.0 : gram6-nationality-adjective 100.0 95.1 : gram7-past-tense 100.0 100.0 : gram8-plural 100.0 100.0 : gram9-plural-verbs 100.0 100.0 === cell 35 result word corpus \ 0 visa Wiki control (same size) 1 visa Forum (same size) 2 salary Wiki control (same size) 3 salary Forum (same size) 4 doha Wiki control (same size) 5 doha Forum (same size) 6 family Wiki control (same size) 7 family Forum (same size) 8 king Wiki control (same size) 9 king Forum (same size) 10 paris Wiki control (same size) 11 paris Forum (same size) 12 computer Wiki control (same size) 13 computer Forum (same size) top 5 0 visas, passport, passports, bdtc, retransfer 1 rp, viza, iqama, vissa, residence 2 salaries, refund, repayment, remuneration, paid 3 slary, salry, sallary, salery, salaray 4 manama, qatar, muharraq, abbasiyyin, dhabi 5 qatar, doah, qata, alkhore, masaieed 6 relatives, father, parents, grandparents, mother 7 familly, fmaily, rescidence, familiy, famliy 8 queen, prince, pretender, harefoot, harthacnut 9 edshel, ypsilanti, bhumibol, edsel, boorjer 10 ferrand, marseille, brussels, france, lausanne 11 amsterdam, france, bangkok, toronto, aires 12 computers, software, mainframe, hardware, comp... 13 desktop, soundcard, computers, hardware, pc