Practical_11_Unsupervised_Learning_and_Topic_modeling.ipynb

{"cells":[{"cell_type":"markdown","source":"# Unsupervised Machine Learning\n","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00000-58da4fa2-200e-4412-b39f-ef56293c36f3","output_cleared":false}},{"cell_type":"markdown","source":"### What is unsupervised learning and how is it used in NLP? ","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00001-5dafd085-0b40-496f-83ef-f439295682a7","output_cleared":false}},{"cell_type":"markdown","source":"Remember how we do supervised learning? Get data with labels, find a good data representation/do feature engineering, and train a model. But what if we don't have labelled data? The majority of textual data is, in fact, unlabelled. And labelling is expensive! Can we still mine such data for useful patterns?","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00002-98ecfec6-90a8-4432-87ba-46b6c132382d","output_cleared":false}},{"cell_type":"markdown","source":"**Unsupervised machine learning** looks for patterns in **unlabelled data** and with **no or minimal human supervision**.","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00002-41394717-088a-46a7-a2f3-b023e6f75b06","output_cleared":false}},{"cell_type":"markdown","source":"![] (https://miro.medium.com/max/2232/1*BKvYt1EHtzHOd_HWjm-Jrw.png)\n(Image source: https://towardsdatascience.com/unsupervised-machine-learning-example-in-keras-8c8bf9e63ee0)","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00003-9391cfa2-0145-4657-9a1f-8c565410e2d3","output_cleared":false}},{"cell_type":"markdown","source":"## Clustering ","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00017-d1b5d046-5d55-4a57-a21f-f2e38a86aba9","output_cleared":false}},{"cell_type":"markdown","source":"**Clustering** is an unsupervised machine learning technique for splitting a set of unlabelled data points into groups (clusters) of points considered by the algorithm as similar.","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00018-14c7660e-4570-4fd6-853c-bd6434497b25","output_cleared":false}},{"cell_type":"markdown","source":"In NLP, a common application of clustering is **document clustering**  -  uncovering hidden structure within a corpus and grouping similar documents into categories (without labels or human supervision!). \n\nSteps:\n- vectorise documents (e.g. count-based, TF-IDF)\n- apply a clustering algorithm (e.g. K-means)\n","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00019-9f9df5a9-9306-4ba4-958c-d999a71b818f","output_cleared":false}},{"cell_type":"markdown","source":"** Hard vs Soft Clustering **\n- hard clustering : Each sample belongs to exactly one cluster.\n- soft clustering: Each  sample can belong to one or more clusters.\n\n![](https://miro.medium.com/max/500/1*bBYxRxPveZ8khii6Ge2T5Q.png)\n\n","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00008-7f300262-6ae6-4637-a790-1695cfec0fde","output_cleared":false}},{"cell_type":"markdown","source":"#### K-Means clustering algorithm","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00020-f0d96ea3-e61c-45e9-9aef-47cf5ed842c7","output_cleared":false}},{"cell_type":"markdown","source":"It is one of the most commonly used ones. \nWatch this video (8m 30sec) to learn about how it works: https://www.youtube.com/watch?v=4b5d3muPQmA\n\n","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00021-4e40f457-2ee0-4a10-8b92-08f77ef46f76","output_cleared":false}},{"cell_type":"markdown","source":"Steps:\n- specify the number of clusters K (your choice! but there are some *smater* ways to choose this number)\n- initialize K centroids (randomly)\n- assign each data point to the cluster whose centroid is closest to it\n- recalculate the centroid location (find a new centre of the cluster) for each cluster as a mean of the points within the cluster\n- assign each vector data point to the centroid closest to it\n- keep iterating until centroids (and assignment of data points to clusters) do not chnage anymore ('converged')\n","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00022-a31fe72d-cea8-453d-86a3-017b37211a31","output_cleared":false}},{"cell_type":"markdown","source":"![](https://www.researchgate.net/profile/Andras_Kalmar/publication/321051036/figure/fig5/AS:563793287290885@1511430202869/Steps-of-the-K-mean-clustering-algorithm.png)\n(Image source: https://www.researchgate.net/figure/Steps-of-the-K-mean-clustering-algorithm_fig5_321051036)","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00023-c9f781a5-d754-485b-8af5-c806d29683ac","output_cleared":false}},{"cell_type":"markdown","source":"![](https://miro.medium.com/max/500/1*-Ox371N1-L4oyJBRhi1uZw.gif)","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00012-25a9a5c3-134e-4995-8c89-6af5ae55cbb2","output_cleared":false}},{"cell_type":"markdown","source":"### Text clustering using Tf-IDF as features\n","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00012-423ca494-9810-44e9-8a4f-32f8e8e2c097","output_cleared":false}},{"cell_type":"markdown","source":"Let's import some data to explore: it is a BBC dataset containing news texts in 5 categories: business, entertainment, politics, sport and tech.","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00014-058fc135-42fe-430f-8cec-8bec206bb58f","output_cleared":false}},{"cell_type":"code","metadata":{"deepnote_cell_type":"code","tags":[],"cell_id":"00020-01010001-b8ae-4842-8f5b-3c358378d1b1","output_cleared":false,"source_hash":"91653e68","execution_millis":173,"execution_start":1606219763873},"source":"import requests\nimport pandas as pd\nimport requests\nimport io\n\nurl=\"https://storage.googleapis.com/dataset-uploader/bbc/bbc-text.csv\"\ns=requests.get(url).content\ndata = pd.read_csv(io.StringIO(s.decode('utf-8')))\n\ndata.head()","execution_count":null,"outputs":[{"name":"stderr","text":"/opt/venv/lib/python3.7/site-packages/ipykernel/ipkernel.py:287: DeprecationWarning: `should_run_async` will not call `transform_cell` automatically in the future. Please pass the result to `transformed_cell` argument and any exception that happen during thetransform in `preprocessing_exc_tuple` in IPython 7.17 and above.\n  and should_run_async(code)\n","output_type":"stream"},{"output_type":"execute_result","execution_count":22,"data":{"application/vnd.deepnote.dataframe.v2+json":{"row_count":5,"column_count":2,"columns":[{"name":"category","dtype":"object","stats":{"unique_count":4,"nan_count":0,"categories":[{"name":"sport","count":2},{"name":"tech","count":1},{"name":"2 others","count":2}]}},{"name":"text","dtype":"object","stats":{"unique_count":5,"nan_count":0,"categories":[{"name":"tv future in the hands of viewers with home theatre systems  plasma high-definition tvs  and digital video recorders moving into the living room  the way people watch tv will be radically different in five years  time.  that is according to an expert panel which gathered at the annual consumer electronics show in las vegas to discuss how these new technologies will impact one of our favourite pastimes. with the us leading the trend  programmes and other content will be delivered to viewers via home networks  through cable  satellite  telecoms companies  and broadband service providers to front rooms and portable devices.  one of the most talked-about technologies of ces has been digital and personal video recorders (dvr and pvr). these set-top boxes  like the us s tivo and the uk s sky+ system  allow people to record  store  play  pause and forward wind tv programmes when they want.  essentially  the technology allows for much more personalised tv. they are also being built-in to high-definition tv sets  which are big business in japan and the us  but slower to take off in europe because of the lack of high-definition programming. not only can people forward wind through adverts  they can also forget about abiding by network and channel schedules  putting together their own a-la-carte entertainment. but some us networks and cable and satellite companies are worried about what it means for them in terms of advertising revenues as well as  brand identity  and viewer loyalty to channels. although the us leads in this technology at the moment  it is also a concern that is being raised in europe  particularly with the growing uptake of services like sky+.  what happens here today  we will see in nine months to a years  time in the uk   adam hume  the bbc broadcast s futurologist told the bbc news website. for the likes of the bbc  there are no issues of lost advertising revenue yet. it is a more pressing issue at the moment for commercial uk broadcasters  but brand loyalty is important for everyone.  we will be talking more about content brands rather than network brands   said tim hanlon  from brand communications firm starcom mediavest.  the reality is that with broadband connections  anybody can be the producer of content.  he added:  the challenge now is that it is hard to promote a programme with so much choice.   what this means  said stacey jolna  senior vice president of tv guide tv group  is that the way people find the content they want to watch has to be simplified for tv viewers. it means that networks  in us terms  or channels could take a leaf out of google s book and be the search engine of the future  instead of the scheduler to help people find what they want to watch. this kind of channel model might work for the younger ipod generation which is used to taking control of their gadgets and what they play on them. but it might not suit everyone  the panel recognised. older generations are more comfortable with familiar schedules and channel brands because they know what they are getting. they perhaps do not want so much of the choice put into their hands  mr hanlon suggested.  on the other end  you have the kids just out of diapers who are pushing buttons already - everything is possible and available to them   said mr hanlon.  ultimately  the consumer will tell the market they want.   of the 50 000 new gadgets and technologies being showcased at ces  many of them are about enhancing the tv-watching experience. high-definition tv sets are everywhere and many new models of lcd (liquid crystal display) tvs have been launched with dvr capability built into them  instead of being external boxes. one such example launched at the show is humax s 26-inch lcd tv with an 80-hour tivo dvr and dvd recorder. one of the us s biggest satellite tv companies  directtv  has even launched its own branded dvr at the show with 100-hours of recording capability  instant replay  and a search function. the set can pause and rewind tv for up to 90 hours. and microsoft chief bill gates announced in his pre-show keynote speech a partnership with tivo  called tivotogo  which means people can play recorded programmes on windows pcs and mobile devices. all these reflect the increasing trend of freeing up multimedia so that people can watch what they want  when they want.","count":1},{"name":"worldcom boss  left books alone  former worldcom boss bernie ebbers  who is accused of overseeing an $11bn (£5.8bn) fraud  never made accounting decisions  a witness has told jurors.  david myers made the comments under questioning by defence lawyers who have been arguing that mr ebbers was not responsible for worldcom s problems. the phone company collapsed in 2002 and prosecutors claim that losses were hidden to protect the firm s shares. mr myers has already pleaded guilty to fraud and is assisting prosecutors.  on monday  defence lawyer reid weingarten tried to distance his client from the allegations. during cross examination  he asked mr myers if he ever knew mr ebbers  make an accounting decision  .  not that i am aware of   mr myers replied.  did you ever know mr ebbers to make an accounting entry into worldcom books   mr weingarten pressed.  no   replied the witness. mr myers has admitted that he ordered false accounting entries at the request of former worldcom chief financial officer scott sullivan. defence lawyers have been trying to paint mr sullivan  who has admitted fraud and will testify later in the trial  as the mastermind behind worldcom s accounting house of cards.  mr ebbers  team  meanwhile  are looking to portray him as an affable boss  who by his own admission is more pe graduate than economist. whatever his abilities  mr ebbers transformed worldcom from a relative unknown into a $160bn telecoms giant and investor darling of the late 1990s. worldcom s problems mounted  however  as competition increased and the telecoms boom petered out. when the firm finally collapsed  shareholders lost about $180bn and 20 000 workers lost their jobs. mr ebbers  trial is expected to last two months and if found guilty the former ceo faces a substantial jail sentence. he has firmly declared his innocence.","count":1},{"name":"3 others","count":3}]}},{"name":"_deepnote_index_column","dtype":"int64"}],"rows_top":[{"category":"tech","text":"tv future in the hands of viewers with home theatre systems  plasma high-definition tvs  and digital video recorders moving into the living room  the way people watch tv will be radically different in five years  time.  that is according to an expert panel which gathered at the annual consumer electronics show in las vegas to discuss how these new technologies will impact one of our favourite pastimes. with the us leading the trend  programmes and other content will be delivered to viewers via home networks  through cable  satellite  telecoms companies  and broadband service providers to front rooms and portable devices.  one of the most talked-about technologies of ces has been digital and personal video recorders (dvr and pvr). these set-top boxes  like the us s tivo and the uk s sky+ system  allow people to record  store  play  pause and forward wind tv programmes when they want.  essentially  the technology allows for much more personalised tv. they are also being built-in to high-definition tv sets  which are big business in japan and the us  but slower to take off in europe because of the lack of high-definition programming. not only can people forward wind through adverts  they can also forget about abiding by network and channel schedules  putting together their own a-la-carte entertainment. but some us networks and cable and satellite companies are worried about what it means for them in terms of advertising revenues as well as  brand identity  and viewer loyalty to channels. although the us leads in this technology at the moment  it is also a concern that is being raised in europe  particularly with the growing uptake of services like sky+.  what happens here today  we will see in nine months to a years  time in the uk   adam hume  the bbc broadcast s futurologist told the bbc news website. for the likes of the bbc  there are no issues of lost advertising revenue yet. it is a more pressing issue at the moment for commercial uk broadcasters  but brand loyalty is important for everyone.  we will be talking more about content brands rather than network brands   said tim hanlon  from brand communications firm starcom mediavest.  the reality is that with broadband connections  anybody can be the producer of content.  he added:  the challenge now is that it is hard to promote a programme with so much choice.   what this means  said stacey jolna  senior vice president of tv guide tv group  is that the way people find the content they want to watch has to be simplified for tv viewers. it means that networks  in us terms  or channels could take a leaf out of google s book and be the search engine of the future  instead of the scheduler to help people find what they want to watch. this kind of channel model might work for the younger ipod generation which is used to taking control of their gadgets and what they play on them. but it might not suit everyone  the panel recognised. older generations are more comfortable with familiar schedules and channel brands because they know what they are getting. they perhaps do not want so much of the choice put into their hands  mr hanlon suggested.  on the other end  you have the kids just out of diapers who are pushing buttons already - everything is possible and available to them   said mr hanlon.  ultimately  the consumer will tell the market they want.   of the 50 000 new gadgets and technologies being showcased at ces  many of them are about enhancing the tv-watching experience. high-definition tv sets are everywhere and many new models of lcd (liquid crystal display) tvs have been launched with dvr capability built into them  instead of being external boxes. one such example launched at the show is humax s 26-inch lcd tv with an 80-hour tivo dvr and dvd recorder. one of the us s biggest satellite tv companies  directtv  has even launched its own branded dvr at the show with 100-hours of recording capability  instant replay  and a search function. the set can pause and rewind tv for up to 90 hours. and microsoft chief bill gates announced in his pre-show keynote speech a partnership with tivo  called tivotogo  which means people can play recorded programmes on windows pcs and mobile devices. all these reflect the increasing trend of freeing up multimedia so that people can watch what they want  when they want.","_deepnote_index_column":0},{"category":"business","text":"worldcom boss  left books alone  former worldcom boss bernie ebbers  who is accused of overseeing an $11bn (£5.8bn) fraud  never made accounting decisions  a witness has told jurors.  david myers made the comments under questioning by defence lawyers who have been arguing that mr ebbers was not responsible for worldcom s problems. the phone company collapsed in 2002 and prosecutors claim that losses were hidden to protect the firm s shares. mr myers has already pleaded guilty to fraud and is assisting prosecutors.  on monday  defence lawyer reid weingarten tried to distance his client from the allegations. during cross examination  he asked mr myers if he ever knew mr ebbers  make an accounting decision  .  not that i am aware of   mr myers replied.  did you ever know mr ebbers to make an accounting entry into worldcom books   mr weingarten pressed.  no   replied the witness. mr myers has admitted that he ordered false accounting entries at the request of former worldcom chief financial officer scott sullivan. defence lawyers have been trying to paint mr sullivan  who has admitted fraud and will testify later in the trial  as the mastermind behind worldcom s accounting house of cards.  mr ebbers  team  meanwhile  are looking to portray him as an affable boss  who by his own admission is more pe graduate than economist. whatever his abilities  mr ebbers transformed worldcom from a relative unknown into a $160bn telecoms giant and investor darling of the late 1990s. worldcom s problems mounted  however  as competition increased and the telecoms boom petered out. when the firm finally collapsed  shareholders lost about $180bn and 20 000 workers lost their jobs. mr ebbers  trial is expected to last two months and if found guilty the former ceo faces a substantial jail sentence. he has firmly declared his innocence.","_deepnote_index_column":1},{"category":"sport","text":"tigers wary of farrell  gamble  leicester say they will not be rushed into making a bid for andy farrell should the great britain rugby league captain decide to switch codes.   we and anybody else involved in the process are still some way away from going to the next stage   tigers boss john wells told bbc radio leicester.  at the moment  there are still a lot of unknowns about andy farrell  not least his medical situation.  whoever does take him on is going to take a big  big gamble.  farrell  who has had persistent knee problems  had an operation on his knee five weeks ago and is expected to be out for another three months. leicester and saracens are believed to head the list of rugby union clubs interested in signing farrell if he decides to move to the 15-man game.  if he does move across to union  wells believes he would better off playing in the backs  at least initially.  i m sure he could make the step between league and union by being involved in the centre   said wells.  i think england would prefer him to progress to a position in the back row where they can make use of some of his rugby league skills within the forwards.  the jury is out on whether he can cross that divide.  at this club  the balance will have to be struck between the cost of that gamble and the option of bringing in a ready-made replacement.","_deepnote_index_column":2},{"category":"sport","text":"yeading face newcastle in fa cup premiership side newcastle united face a trip to ryman premier league leaders yeading in the fa cup third round.  the game - arguably the highlight of the draw - is a potential money-spinner for non-league yeading  who beat slough in the second round. conference side exeter city  who knocked out doncaster on saturday  will travel to old trafford to meet holders manchester united in january. arsenal were drawn at home to stoke and chelsea will play host to scunthorpe. the only other non-league side in the draw are hinckley united  who held brentford to a goalless draw on sunday. they will meet league one leaders luton if they win their replay against martin allen s team at griffin park.  a number of premiership teams face difficult away games against championship sides on the weekend of 8/9 january. third-placed everton visit plymouth  liverpool travel to burnley  crystal palace go to sunderland  fulham face carling cup semi-finalists watford  bolton meet ipswich  while aston villa were drawn against sheffield united. premiership strugglers norwich  blackburn  west brom are away at west ham  cardiff and preston north end respectively. southampton visit northampton  having already beaten the league two side in the carling cup earlier this season. middlesbrough were drawn away against either swindon or notts county  while spurs entertain brighton at white hart lane.  arsenal v stoke  swindon/notts co v middlesbrough  man utd v exeter  plymouth v everton  leicester v blackpool  derby v wigan  sunderland v crystal palace  wolves v millwall  yeading v newcastle  hull v colchester  tottenham v brighton  reading v stockport/swansea  birmingham v leeds  hartlepool v boston  milton keynes dons v peterborough  oldham v man city  chelsea v scunthorpe  cardiff v blackburn  charlton v rochdale  west ham v norwich  sheff utd v aston villa  preston v west brom  rotherham v yeovil  burnley v liverpool  bournemouth v chester  coventry v crewe  watford v fulham  ipswich v bolton  portsmouth v gillingham  northampton v southampton  qpr v nottm forest  luton v hinckley/brentford  matches to be played on weekend of 8/9 january.","_deepnote_index_column":3},{"category":"entertainment","text":"ocean s twelve raids box office ocean s twelve  the crime caper sequel starring george clooney  brad pitt and julia roberts  has gone straight to number one in the us box office chart.  it took $40.8m (£21m) in weekend ticket sales  according to studio estimates. the sequel follows the master criminals as they try to pull off three major heists across europe. it knocked last week s number one  national treasure  into third place. wesley snipes  blade: trinity was in second  taking $16.1m (£8.4m). rounding out the top five was animated fable the polar express  starring tom hanks  and festive comedy christmas with the kranks.  ocean s twelve box office triumph marks the fourth-biggest opening for a december release in the us  after the three films in the lord of the rings trilogy. the sequel narrowly beat its 2001 predecessor  ocean s eleven which took $38.1m (£19.8m) on its opening weekend and $184m (£95.8m) in total. a remake of the 1960s film  starring frank sinatra and the rat pack  ocean s eleven was directed by oscar-winning director steven soderbergh. soderbergh returns to direct the hit sequel which reunites clooney  pitt and roberts with matt damon  andy garcia and elliott gould. catherine zeta-jones joins the all-star cast.  it s just a fun  good holiday movie   said dan fellman  president of distribution at warner bros. however  us critics were less complimentary about the $110m (£57.2m) project  with the los angeles times labelling it a  dispiriting vanity project . a milder review in the new york times dubbed the sequel  unabashedly trivial .","_deepnote_index_column":4}],"rows_bottom":null},"text/plain":"        category                                               text\n0           tech  tv future in the hands of viewers with home th...\n1       business  worldcom boss  left books alone  former worldc...\n2          sport  tigers wary of farrell  gamble  leicester say ...\n3          sport  yeading face newcastle in fa cup premiership s...\n4  entertainment  ocean s twelve raids box office ocean s twelve...","text/html":"<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>category</th>\n      <th>text</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>tech</td>\n      <td>tv future in the hands of viewers with home th...</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>business</td>\n      <td>worldcom boss  left books alone  former worldc...</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>sport</td>\n      <td>tigers wary of farrell  gamble  leicester say ...</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>sport</td>\n      <td>yeading face newcastle in fa cup premiership s...</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>entertainment</td>\n      <td>ocean s twelve raids box office ocean s twelve...</td>\n    </tr>\n  </tbody>\n</table>\n</div>"},"metadata":{}}]},{"cell_type":"markdown","source":"Let's check we indeed have there five categories:","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00017-bd5b43d9-8256-4918-b630-2b00b46c30b7"}},{"cell_type":"code","metadata":{"deepnote_cell_type":"code","tags":[],"cell_id":"00023-3fd2fec1-5b87-486c-9add-d0771474f7d9","output_cleared":false,"source_hash":"a23ebb8d","execution_millis":4,"execution_start":1606219861337},"source":"category_counts = data.category.value_counts()\ncategories = category_counts.index\nprint(categories)","execution_count":null,"outputs":[{"name":"stdout","text":"Index(['sport', 'business', 'politics', 'tech', 'entertainment'], dtype='object')\n/opt/venv/lib/python3.7/site-packages/ipykernel/ipkernel.py:287: DeprecationWarning: `should_run_async` will not call `transform_cell` automatically in the future. Please pass the result to `transformed_cell` argument and any exception that happen during thetransform in `preprocessing_exc_tuple` in IPython 7.17 and above.\n  and should_run_async(code)\n","output_type":"stream"}]},{"cell_type":"markdown","source":"Install the packages we will need:","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00019-59c0ab88-286a-491b-933b-e2ba2f80f2b3"}},{"cell_type":"code","metadata":{"deepnote_cell_type":"code","tags":[],"cell_id":"00024-f952d561-a58f-419e-9d73-1336573b5793","output_cleared":false,"source_hash":"34690b9","execution_millis":12823,"execution_start":1606219122649},"source":"!pip install seaborn\n!pip install  wordcloud\n!pip install gensim\n!pip install nltk\n\n  ","execution_count":null,"outputs":[{"name":"stdout","text":"Collecting seaborn\n  Downloading seaborn-0.11.0-py3-none-any.whl (283 kB)\n\u001b[K     |████████████████████████████████| 283 kB 7.0 MB/s \n\u001b[?25hRequirement already satisfied: numpy>=1.15 in /opt/venv/lib/python3.7/site-packages (from seaborn) (1.19.4)\nRequirement already satisfied: scipy>=1.0 in /opt/venv/lib/python3.7/site-packages (from seaborn) (1.5.4)\nRequirement already satisfied: pandas>=0.23 in /opt/venv/lib/python3.7/site-packages (from seaborn) (1.0.5)\nRequirement already satisfied: matplotlib>=2.2 in /opt/venv/lib/python3.7/site-packages (from seaborn) (3.3.3)\nRequirement already satisfied: python-dateutil>=2.6.1 in /opt/venv/lib/python3.7/site-packages (from pandas>=0.23->seaborn) (2.8.1)\nRequirement already satisfied: pytz>=2017.2 in /opt/venv/lib/python3.7/site-packages (from pandas>=0.23->seaborn) (2020.4)\nRequirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.3 in /opt/venv/lib/python3.7/site-packages (from matplotlib>=2.2->seaborn) (2.4.7)\nRequirement already satisfied: pillow>=6.2.0 in /opt/venv/lib/python3.7/site-packages (from matplotlib>=2.2->seaborn) (8.0.1)\nRequirement already satisfied: cycler>=0.10 in /opt/venv/lib/python3.7/site-packages (from matplotlib>=2.2->seaborn) (0.10.0)\nRequirement already satisfied: kiwisolver>=1.0.1 in /opt/venv/lib/python3.7/site-packages (from matplotlib>=2.2->seaborn) (1.3.1)\nRequirement already satisfied: six>=1.5 in /opt/venv/lib/python3.7/site-packages (from python-dateutil>=2.6.1->pandas>=0.23->seaborn) (1.15.0)\nInstalling collected packages: seaborn\nSuccessfully installed seaborn-0.11.0\nCollecting wordcloud\n  Downloading wordcloud-1.8.1-cp37-cp37m-manylinux1_x86_64.whl (366 kB)\n\u001b[K     |████████████████████████████████| 366 kB 6.5 MB/s \n\u001b[?25hRequirement already satisfied: matplotlib in /opt/venv/lib/python3.7/site-packages (from wordcloud) (3.3.3)\nRequirement already satisfied: pillow in /opt/venv/lib/python3.7/site-packages (from wordcloud) (8.0.1)\nRequirement already satisfied: numpy>=1.6.1 in /opt/venv/lib/python3.7/site-packages (from wordcloud) (1.19.4)\nRequirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.3 in /opt/venv/lib/python3.7/site-packages (from matplotlib->wordcloud) (2.4.7)\nRequirement already satisfied: cycler>=0.10 in /opt/venv/lib/python3.7/site-packages (from matplotlib->wordcloud) (0.10.0)\nRequirement already satisfied: python-dateutil>=2.1 in /opt/venv/lib/python3.7/site-packages (from matplotlib->wordcloud) (2.8.1)\nRequirement already satisfied: kiwisolver>=1.0.1 in /opt/venv/lib/python3.7/site-packages (from matplotlib->wordcloud) (1.3.1)\nRequirement already satisfied: six in /opt/venv/lib/python3.7/site-packages (from cycler>=0.10->matplotlib->wordcloud) (1.15.0)\nInstalling collected packages: wordcloud\nSuccessfully installed wordcloud-1.8.1\nCollecting gensim\n  Downloading gensim-3.8.3-cp37-cp37m-manylinux1_x86_64.whl (24.2 MB)\n\u001b[K     |████████████████████████████████| 24.2 MB 347 kB/s \n\u001b[?25hRequirement already satisfied: numpy>=1.11.3 in /opt/venv/lib/python3.7/site-packages (from gensim) (1.19.4)\nRequirement already satisfied: six>=1.5.0 in /opt/venv/lib/python3.7/site-packages (from gensim) (1.15.0)\nRequirement already satisfied: scipy>=0.18.1 in /opt/venv/lib/python3.7/site-packages (from gensim) (1.5.4)\nCollecting smart-open>=1.8.1\n  Downloading smart_open-4.0.0.tar.gz (106 kB)\n\u001b[K     |████████████████████████████████| 106 kB 43.4 MB/s \n\u001b[?25hRequirement already satisfied: requests in /opt/venv/lib/python3.7/site-packages (from smart-open>=1.8.1->gensim) (2.25.0)\nRequirement already satisfied: chardet<4,>=3.0.2 in /opt/venv/lib/python3.7/site-packages (from requests->smart-open>=1.8.1->gensim) (3.0.4)\nRequirement already satisfied: urllib3<1.27,>=1.21.1 in /opt/venv/lib/python3.7/site-packages (from requests->smart-open>=1.8.1->gensim) (1.26.2)\nRequirement already satisfied: certifi>=2017.4.17 in /opt/venv/lib/python3.7/site-packages (from requests->smart-open>=1.8.1->gensim) (2020.11.8)\nRequirement already satisfied: idna<3,>=2.5 in /opt/venv/lib/python3.7/site-packages (from requests->smart-open>=1.8.1->gensim) (2.10)\nBuilding wheels for collected packages: smart-open\n  Building wheel for smart-open (setup.py) ... \u001b[?25ldone\n\u001b[?25h  Created wheel for smart-open: filename=smart_open-4.0.0-py3-none-any.whl size=108254 sha256=5e1e0cd320bf3bb3cb44999f68fc403fc68e3026199cd23463659fbae4a6122e\n  Stored in directory: /home/jovyan/.cache/pip/wheels/97/ef/4b/832eae6b1e594ffe720e8a0d0a3c024d7e4956ce3d38a1747f\nSuccessfully built smart-open\nInstalling collected packages: smart-open, gensim\nSuccessfully installed gensim-3.8.3 smart-open-4.0.0\nRequirement already satisfied: nltk in /opt/venv/lib/python3.7/site-packages (3.5)\nRequirement already satisfied: click in /opt/venv/lib/python3.7/site-packages (from nltk) (7.1.2)\nRequirement already satisfied: regex in /opt/venv/lib/python3.7/site-packages (from nltk) (2020.11.13)\nRequirement already satisfied: joblib in /opt/venv/lib/python3.7/site-packages (from nltk) (0.17.0)\nRequirement already satisfied: tqdm in /opt/venv/lib/python3.7/site-packages (from nltk) (4.53.0)\n","output_type":"stream"}]},{"cell_type":"code","metadata":{"deepnote_cell_type":"code","tags":[],"cell_id":"00024-ca18a403-674f-41dd-a971-cf776ce8bfd7","output_cleared":false,"source_hash":"b2a6ed8a","execution_millis":57,"execution_start":1606229752213},"source":"import os \nimport pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport re\nimport nltk\nnltk.download('stopwords')\nfrom nltk.corpus import stopwords","execution_count":null,"outputs":[{"name":"stderr","text":"/opt/venv/lib/python3.7/site-packages/ipykernel/ipkernel.py:287: DeprecationWarning: `should_run_async` will not call `transform_cell` automatically in the future. Please pass the result to `transformed_cell` argument and any exception that happen during thetransform in `preprocessing_exc_tuple` in IPython 7.17 and above.\n  and should_run_async(code)\n[nltk_data] Downloading package stopwords to /home/jovyan/nltk_data...\n[nltk_data]   Package stopwords is already up-to-date!\n","output_type":"stream"}]},{"cell_type":"markdown","source":"Number of texts in each category:","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00019-4f63f5bb-67aa-4b32-b9f0-0dfafb3a4350","output_cleared":false}},{"cell_type":"code","metadata":{"deepnote_cell_type":"code","tags":[],"cell_id":"00024-93a4d92f-08b7-4736-90c1-2daa0300616e","output_cleared":false,"source_hash":"8bee7342","execution_millis":239,"execution_start":1606229755289},"source":"fig = plt.figure(figsize = (12,5))\nax = fig.add_subplot(111)\nsns.barplot(x = category_counts.index , y = category_counts)\nfor a, p in enumerate(ax.patches):\n    ax.annotate(f'{categories[a]}\\n' + format(p.get_height(), '.0f'), xy = (p.get_x() + p.get_width() / 2.0, p.get_height()), xytext = (0,-25), size = 13, color = 'white' , ha = 'center', va = 'center', textcoords = 'offset points', bbox = dict(boxstyle = 'round', facecolor='none',edgecolor='white', alpha = 0.5) )\nplt.xlabel('Categories', size = 15)\nplt.ylabel('Number of articles', size= 15)\nplt.xticks(size = 12)\n\nplt.title(\"The number of articles by categories\" , size = 18)\nplt.show()","execution_count":null,"outputs":[{"name":"stderr","text":"/opt/venv/lib/python3.7/site-packages/ipykernel/ipkernel.py:287: DeprecationWarning: `should_run_async` will not call `transform_cell` automatically in the future. Please pass the result to `transformed_cell` argument and any exception that happen during thetransform in `preprocessing_exc_tuple` in IPython 7.17 and above.\n  and should_run_async(code)\n","output_type":"stream"},{"data":{"text/plain":"<Figure size 864x360 with 1 Axes>","image/png":"iVBORw0KGgoAAAANSUhEUgAAAtgAAAFZCAYAAACi11jlAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAABWIklEQVR4nO3dd5wV1f3/8ddnK7tL7x1UQFERRWzB3ruoxBp7STQx0Z89JkYTk5ivib3Hrom9K7EhWFBAsIAgvXfYZdneP78/ZhYuly134e7eXXg/H4/72Dtnzsx85s7cu5977pkz5u6IiIiIiEh8JCU6ABERERGRbYkSbBERERGROFKCLSIiIiISR0qwRURERETiSAm2iIiIiEgcKcEWEREREYkjJdgizZiZHWpmbmYXJjqW5sLMxpnZwkTHsaXM7Eozm2lmpeGx7Z/gePqHcdzWlMvGg5ktNLNxidi2BHQMRGqmBFukCYXJSKyP/omOV+LLzA4DHgJmAr8CzgPWNMF2+5vZbWa2Z2NvSxomPC4jEx2HiMRXSqIDENnOnBc1fRBwOfA48EXUvDVA/yaISZrOUeHfi909pwm32x/4E7AQ+D5q3iIgA6hownhkoz8BzwJvJTiOLbUzoDvWiURRgi3ShNz9hchpM0shSLC/jp4Xzm+q0KQWZtbG3fPjtLruAE2VXMcSuwe38y1pinhk22BmqUCyu5e4e2mi4xFpjtRFRKSFMLOLzGx62Hd3kZndUEu94Wb2ppmtDevOMrNbwmQ+lu24mT1jZgeY2WdmVmhm2Wb2hJm1jqpbY3/omvrmRvYnD/shzzKzEjObZmYnhnWGmNkHZpYXbvP+8J95TXHuaGZvm9n6sP6bZrZjDfXMzK4wsylmVmRmBWY2NuyuUWPMZnZmWL8YeCCG12ykmY0PX6uC8Pkp0esGLop4jb2+vqtmdrSZvWxm882s2MxyzewjMzukhrrjwv6wO5rZa2aWA+RZ0H9/bFjt6eht19WP2sxOD9ebG752s8JjkhbDa3KmmX1pZvnhshPNbFQN9U4Iz7O14T4uNrM3zGxQfduIWMcwM/s0fO1zzOxZM+saMf/UcB8vq2X56WY216z+b7RmdpiZvR+enyXhsXnSzDpH1LkyPE7LzKzMzFaY2QsW0e0r4pwAuCDiuHjU9o4M15Ubbm+qmf2qltiuCI9RqZnNMbPfWPB+czM7NKpufzN73sxWhfXnmdnfzCwzqt5t4fK7mdndZraU4AvZ/uH8GvtgW4yfQ+F6Xw1fq1IzW2nB+/OE+o6FSHOmFmyRluFXQDfgSSAX+AXwDzNb6u7/ra4U/lN6A5gL/AvIAQ4A/gzsCfw8xu3tCbwHPA38FzgUuASoImhx3xq/BjoATxD8o/4t8KaZ/Rz4N/Aiwc/lRwNXAauBO6LWkQWMAyYCNwMDgSuB/c1sL3dfGVH3eeBs4LVwf9KBc4GPzew0d38nat0jw5geAR4F8uraGTO7ko39qv8cFl8IvGVmv3T3xwm6+5xH8NodxMauQqvqWne4no7Ac8BSoBdwKTDGzA5z9+huRa2Bz4DxwC1AV+Bz4G/A79m0K1Kd2zazv4bLzADuAVYAOwGnA7cCZXUse0e4/Q+APxKcN6cCr5rZb9z9obDeIcA7wI/A3wnO7Z7AkcAAYHZdMYZ6A2OA1wmO8TDgYmC4me3j7kXAu8DKsPzfUbHuD+wK3BK25tf1mvyS4LxYFv5dBPQFTgrjWBtWvQ6YANxP8B7cneC4HW5mQ9w9m43nxPMEx+TxGrZ3OcE5OAH4K1BI0M3oETPbyd2vj6h7I3An8C3BeyITuJ4a+vibWT9gEtAOeBiYQ/AevxkYYWZHuHt0l6H/AMUEnytOcD7U9jrF9DlkZp2AT8PFHiV4PTsDw4H9gPdr24ZIs+fueuihR4IeBAmUAxfWMv/QcP5yoF1EeSbBP86vI8paESQRnwMpUeu5JlzPoTHE5AQJ0X5R5e8D5UDriLJxwMIa1tE/XM9tNezLsqh92SNim6dFrWcKsCKqbFxY/96o8lPD8kdrKLs8qm4KMBlYAFhUzOXA4BiPXweggCCRaBtR3haYB+QD7SPKnyHslRHj+rNqKOtGkMiNruV1uaOO82iz86yWY7VvWPYp0CqqvtXwmkUuOyws+1sN23qL4AtLm3D67rBu1y18/ywMl7+6lvP9poiyv4Vlu0bV/TdB//Oe9WyrN1BK8IWjfQ3zk+o5bkeE27+hhvfbMzXU70HwBfS/Ncy7D6gEdgynOxIkv1MjjxdBl6T1RL33CZJlB46PWu9dYfklEWW3hWXjiPpciTgG4yKmY/4cAk4Op8/YkuOvhx7N+aEuIiItw9Puvr56woNWuQkELbfVjiJIvp4G2ptZ5+oHMDqsc3SM2/va3SdGlX1KkJj234L4Iz0TtS9TCZKu5e7+RlTdL4HuFtU1JXRn5IS7vwnMImiBrvYLgiT3rajXoz1Bq2Z/Nn0NAd53959i3JejCFrT73f3DS3d4fP7CVqUj4xxXZtx98Lq52bWOmzxqyRoud+vlsX+uaXbi3Bu+Pdmd9+kf7aH6lnWgWcjX/PwdX8HaEPQmglB8gdwenTXgQbII2iFjfRwWH5qRNm/w7guqS4wsyzgTOB/7r68nu38HEgDbnf33OiZ7l4V8bwwXH+SmbUL9/0Hgv2t7bhFG0Xwa8uTNbyO7xJ08aw+t44iSGwfiTxeHvyS85/IlZpZEkFi+527j2ZTf2fjrw3R7vXNW7Vr0pDPoerjf5yZtY1h3SIthrqIiLQM82soywY6RUwPDv8+Vcd6um3l9oja5paoad3rgCW1lFdvsyCiPNc37QZS7SdgpJllhUnOYIKErq7uEN3YtCtCLN0Squ0Q/p1ew7zqss36hcfKzHYi6BpwDMGXgkg1Jblrakr+tsDAcP0/bMGygwlauWfWUaf6PHwQOIUgIf6HmX1J0K3kRXePdfjC+e6+SXcVdy81s/lEvPbuvsDMPgHOM7Ob3L0cOIPg/Hgihu1UfxH7rr6KZnY4QTea/QgS30gdYtgWbHw/f1JHnerXsfo8nFVDneiyLgRf/DY7Z909x8xWUPM5G+v7IubPIXf/zMyeI/gl71wz+4Zgf1929xkxbk+kWVKCLdIyVMZQp/oCrevZfCi2avW10sWyvcgLwWpryazrs6W2dce6zYYwgq4059RR58eo6aIt3FZcha32nxO0kN8LTCNoja8i6Ct7eA2LxTN2Z8uGX7NwueOo/ZhOB3D3bDPbh6Bf+lHAwQT9vW83s+Pd/est2H5dHgdeJWjBfZ2gNXslcezrG+7PRwTdhm4i6IZUTPCavETsgwtUn/PnU3t/55q+rDaWWM+tBn0OufsFZnYXwflyEHAtcIuZXe3uD25hrCIJpwRbZNsxJ/xb6O51tXrFUw6wdw3lW9xqG6P2Zta9hlbswcDqiK4Vc4BBwAR3LyD+qhOc3QgutIu0a1SdhjqC4IK/i9396cgZ4UWEDdHQRHk2QcIzlOBiuIaYAxwLLI6lq427VxL07x0HYGZ7EPS9/wMQy0gSO5pZWmQrtpmlE5yD0a3obxNcNHuJmf0IjAD+EWPXh+oW3D2puzX3HCAZOM7dF0TElEXsrdew8f28Nob388Lw785svGiQiLJIawi+qO0WvRIz60DQ9/v7BsQZrcGfQ+7+I8EX3bvMrD1BF6g7zeyherojiTRb6oMtsu34kCB5uMnMOkbPNLMMM2sT523OBtqY2b4R20kiuJipsd0UOWFmpxIkE29FFD9H8Dn395pWYGaxdpmpzccEIztcFfnahs+vIujW8vEWrru69XeT1nszO5rY+/FWq/5ysdl5UYvqkWn+ZjUMyWdW53B2z0csm1zDst0inneOnk+QFBc3INa2BCPIRLoyLH8rsjDsFvIMQZebP4XFT8a4ndcIRk75U039hSNekxqPG8GILDX9zy2g5n19heCiytvNLKOG7bULv0hAcI6VAleYWauIOt3Z2J8e2NBX/F1gLzM7Nmq1N4UxvllDPLGK+XPIzDqGnxeR8eUStPpnsnn3GpEWQy3YItsIdy80s/MJkopZZvYUwc/U7YFdgNMILl4aF8fNPk7wk+6bZnYfQQIyisb/bFkLnGZmPQn2p3qYvlUEox4A4O6vmdnTwG/MbBjB0INrCUaEOIBgKLgtbm1391wLxiN/CJhoZs+Esy4M1/3LyAs6G+hLgu4L/7Jg/OSlBK2n5xF0FxnSgHXNIGi1vNLMigiGw1vt7tGtnQC4+yQz+wdwI/Ctmb0cxrIDwfHdN1xHTct+Y8GY2rcB35vZqwRdAnoQ/NpxPMHFggD/NrPeBF0qqu8oeSZBv+jnYty3eQRJ7+4ELd97EwzHN5PgQtNo/ybovnA28Jm7z6mhTk37tdTMriY41tPCvsOLCIZOPCXc5vcEyek1wGgze5zgPXEUwWg5azdfMxOAI8Nh9hYHm/KXwu1dQdA//Cczez7cXheCYz+S4FeShWFXm9sJRkoZb2YvECSolxN8CR7Opr9i/D6M6S0ze5jgc+Jggtf+c4I7S26RBn4OnQ9cY2ZvhnXKgUMIvgC94u7FWxqHSMIlehgTPfTYnh/EPkzfZvOpZcg3gjF3XyAYDq+MIOn8imA84o4xxFTbsGHVsR4aVX48QWJRSpBI/YONt0++LcZ9WUjEUF8R5beFy/SPKBsX1t+R4Cf/PILk8W1gQC37dB7BWMN5BEOfLSQYp/fMiDr9o2NuwHE8NXyNC8PHV8DIWI9ZHevdg+Civ3XhPo4j6Ke62XqoZcjEqOP0bbj/Xv1617XfBEno+HDbhQRJ671AWgzLnkDQmpkTnhtLgP8Bv4qocxrByCJLwzprCMbxPj3G12dhuN/DCLpGFIav1fNAtzqWGxPGfd4WHOujCVqM14ev5XyCpL1TRJ2RBMl+IUFS/RLBeNkLiTrPCb4cfhSem17DcR1BkLSvJng/Lye4cdC1bD6E4q8JEupSgq4avyH4JcWBfaPq7hC+TtXrnU+QoGfW9x6M8b1b7+cQwRfGZwmS68LwNfgh3Lf0hh4bPfRoTo/qsUxFRES2C2Y2muAXjJ6+jbeSmtkDBIl2D6955B0RaQTqgy0iItsNMxtA0AXhhW0puY7sex1R1oOgG8aPSq5Fmpb6YIuIyDbPzPYjGGXmtwRdFv6V2Iji7tBwuLs3CLrc9AcuIxjz+qY6lhORRqAEW0REtgdXELTmzgfOdfeFiQ0n7uYSXPB5GcGNmUqAycDfvemG7RSRkPpgi4iIiIjEkfpgi4iIiIjE0TbXRaRz587ev3//RIchIiIiItu4KVOmrHX3LtHl21yC3b9/fyZPnpzoMERERERkG2dmi2oqVxcREREREZE4UoItIiIiIhJHSrBFREREROJICbaIiIiISBwpwRYRERERiSMl2CIiIiIicaQEW0REREQkjpRgi4iIiIjEkRJsEREREZE4UoItIiIiIhJHSrBFREREROIoJdEBNAd7X/9cokOQBphy1/lNsp3Ffx7SJNuR+Oh767REhyAiIgKoBVtEREREJK6aPME2s4VmNs3MvjezyWFZRzP72MzmhH87hOVmZveb2Vwzm2pmw5o6XhERERGRhkhUC/Zh7r6nuw8Pp28Cxrj7QGBMOA1wHDAwfFwOPNLkkYqIiIiINEBz6SJyCvBs+PxZYGRE+XMemAC0N7MeCYhPRERERCQmiUiwHfjIzKaY2eVhWTd3XxE+Xwl0C5/3ApZELLs0LBMRERERaZYSMYrIge6+zMy6Ah+b2czIme7uZuYNWWGYqF8O0Ldv3/hFmgDts9Lp0aE1aSnN48eF8ooqVuYWklNQkuhQRERERFqEJk+w3X1Z+He1mb0J7AusMrMe7r4i7AKyOqy+DOgTsXjvsCx6nY8DjwMMHz68Qcl5c5GSZBw1tD+d2mSwJDuf0vKKRIcEQNuMZIb270pBSRkffr+AsoqqRIfU4llaJul99iSlY3+S0jLALNEhNS53qkoLKc9eQNmS7/GK0kRHJCIi0qiaNME2sywgyd3zw+dHA38G3gEuAO4M/74dLvIO8BszewnYD1gf0ZVkm/KzXXpRWlHJf76YgTfDrwgH7tKLQ3btw8dTFyU6lBbN0jLJ2nMkFdmLKJrxIVUl+QS9prZdZkkkZbQjrefuZA09mYLv34bKskSHJSIi0miaugW7G/CmBS12KcB/3f0DM/sGeMXMLgEWAWeE9UcDxwNzgSLgoiaOt0kkJxk7dmvPK1/NbJbJNcDEOSv4xcG7kpaSpFbsrZDWczcq1i2lZN74RIfSZByozF9N8axPydz1GNK6DaRs+fREhyUiItJomjTBdvf5wNAayrOBI2ood+DXTRBaQrVplUZJWQVFpc2jW0hNyiuryC0spX1WK1avL0p0OC1WSsd+lMz9MtFhJEzZ6jmkdR+sBFtERLZpzeNKuu1cUpJRWdVMm64jVFRVkZy0jfcXbmRJqRlUlRYmOoyE8dJCLLVVosMQERFpVEqwZYOThu/EWzeOTHQY24HNv0z1/O0HZA45Me5banvgpXQ564G4r3fL+bZ/UaeIiGz3EjFMnzRDKWqZ3iblfflEokMQERHZ7ijBbubOGrEL5xw8mPaZrSgsLee9yfN4Y+Js3vv96fzl1a84/9Dd6dS6FZPnr+KOV79mXWEwXnW7zHT+38nD2X9QTwAmzFrOv975hrziYPSGd28+jXe+mcvwnbqza59OPDVmGpcdNZTU5CS+uONsAK5+6lOmzF+VmB0XERERaaGUYDdjfTu34arjh3He/e8zf9V6WrdKpX/Xdhvmn7D3Tlz2yAeUlFVy+1kjuOOcA/n1vz8B4I5zDqSisopRdwUjHt5x9oH85ewD+d1Tn25YfuR+A/l/T49l1vIc0lOSWZtfzCVHDGHkP95q0v2UQEqH3nS98FnSuu9C+doFrBv9F8qWT6fjyXdAVQU57922oW7P335A7tgHKZr2HsntetLxxFtJ7zUEHCpyl7L2jRupyF5Iu0OuIL3PMFa/cNmG5Qq+fY1WO+xHWq89qMhdRs57t1O29IcN687a63Ta7HcuKW27U7FuKblj7qFk/tcApHbfhQ7H3kxa14G4V1GxdgGrX/w1XpJH5m7H0u7gK0hu2w0vL6F43pfkvP2HJn0NRUREmgMl2M1YZZWDwU7d2rNiXSEFJeX8uHgtPTpkAfD4xz+QnR+0WN/73hTevulUOrfNAOBnO/di5D/eJD9ssb773cm8ccNIOrfJYG1+MQBvTZzDrOU5AJRWVDb17kmU1nv/nDUvXUX5qtm0PeACupz9MMsfOL7e5dof/jsq169g6UtXQVUlqV0GUFWcV2v9rD1PZe3Lv6V87QLaH3UdnU75KyseCvp/Z+11Om1HXMzaV6+hfNUcWg04kM5n3MvKx0ZRsW4JHY+7heJ541n97EVgSaT12BUqy7GUVnQa+TdW/+dXlC6chKVmkNZjcNxeGxERkZZEFzk2Y8tyCvjDf79k5H4D+eCPo3jyymPZf1CPDfNXrCvY7Hm3dpl0bxck4MtzNs5fmp0fzG+fuaFsecTykniF371J+YoZUFVB3vgn8YpSMgYdUu9yXllOcuvOpHToDV5F+erZVBXl1Fq/YMqrlK+ZB15F4Xevk9qpH5beGoA2+51L3uePUr5qNuCUzP2C0oWTyNz9uA3bSmnbneS23aGqgrJlU/Hy4AubV1WQ2nkHklq1xcuLKV387da/KCIiIi2QWrCbubE/Lmbsj4tJSU5i1P6D+NeFh/GLe98HoEeH1izNLtjwHGBVxBjVwfwgse7VMZyfu3F+VdRdbaKnpWlV5C7bdHr9CpLbdqt3udxP/kW7g35Jl7MeJCk1g6KfPiZ3zL0bEt9olQVrNzyvKgvqJKVlUVlaQEr7XnQ47hY6HHvTxgWSUqjIC/riZ7/zR9od9Eu6XfQcVJZTOO191n/2CF5Rwpr/Xkmb/c+n/WG/pWLdUvImPEvRj6Mb+jKIiIi0eEqwm7F+XdrSs2Nrvpu/ipLySgpKysDBw0T40iP3YN7KXErLK/ntCcOYMHs5a/OChOnrWcu55qTh/OmlLzEzrjlpOF/+tHRD95CaZOcX06F1K7LSUyksLW+SfZSNUtr32nS6XQ8q81aR0rY7SZntN86wZJKyOm6YrCpax7oP74QP7yS5fW+6nHk/bUdczPpxDzU4hsr1K8gd9zDFP31U8/zcZeS8eysAqV0H0vXcx6jIXUrh929RumgypYsmgyWRMehQOp9xD2XLplKxbmmD4xAREWnJlGA3Y6nJSVx25B7s1K09AEuy87n+uXEb+kv/79v5PHHlsXRq3YpvF6zi1pc23iHwDy9+wbUn78MbN4wEYMLs5dz9zuQ6tzd57komzlnBOzefRnKS8f+eGcu3GkWkyWTtOZKimWMoXz2HNvufh6W2onjO51hyKu2PvIbk9r2ozF9N+0N/gyVtfOtm7noMpct/pDJ3GV6aD5XleNWW9anPm/A87Q65goqcRZSvmoWlpJPWY1cqi3KpyF5A1h4nUzL/ayoL1lBVkhdsp6qKpKxOpPfZi5IFE/DSAqpKg19OvKoqLq+NiIhIS6IEuxmbuzKXix/6YLPy6oscv5q1nLcmza1x2dzCUv74Yu235D7p729sVlZR5dzw3GdbGK1srYJvX6PDsTeR1n0XKtYuZM2Lv8ZLCyic9j7pffaix2WvUFVeTN6XT1CZv3rDcqk9BtP+qGtJymiHlxVRPHsc+V89s0UxFH73OlSW0+nkv5DSoRdeWUHZyp/I/fhfAKTvsC/tj7gaS8+iqiSfomnvUzj1XZJad6LNPmfR6aTbISmZyryVZL91C5Xrl8fjpREREWlRlGCLNAPL7z8WgLzPH918ZjhEX+QwfQWTX9rwfP2Ye1k/5t4a17v+s0dq3E61yvXLWfznIZuUFU59h8Kp79S4vtqG3asqWMvq5y+tcZ6IiMj2RqOIiIiIiIjEkVqwW6AV6wrZ+/rnEh2GiIiIiNRALdgiIiIiInGkBFtEREREJI7URaSFue3Mn3HcXjtSFnFr8/vf/5ZXv54FwMAeHbjq+GHs3KsjndtkcMlDH/D9wtWbrOOPow5gSL8u9OvSlvcmz+Mvr33dpPsgIiIisi1Tgt0C1ZUUl1dWMXbaYh758Hte+N0JNdaZs2Idn0xdxGn7D2zMMKUJdTz5DrKGHI9XbrxBUO4nd1Mw+WUAUrsNov0R15DWfReSW3dm1dPnU7rku03WkdKhDx1PuJW03ntQVZJP/oTnyJ+gvv4iIiINpQR7G7Nw9XoWrl5fZ52Xxs8E4Ji9+jdBRNJUCn94Z5Oh/CJ5ZTlFP33C+rEP0P2ylzevYEl0OetBShZMYM1LV5HSeQe6nvsIlXmrKJrxYeMGLiIiso1Rgt0CHT6kL4cN6UtuYSmfTV/C4x//QHFZRaLDkmasYu0CKtYuqHV+er+9SW7fg9wx9+IVJZSv/ImCKa/RevgZSrBFREQaSAl2C/PSlzO5//1vWVdYwg5d2/GnM0bwh1EHcMt/v0h0aJJgmYOPImPwkVQVraN41ljWf/YIXl4c07Jp3XamInvRJvXLVsyg9T5nNVa4IiIi2yyNItLCzFyWQ05BCe4wf9V67n73G47cox+pyTqU27P8b/7L8odPZtldB7H2latJ7zecjifdFvPylpZFVWnBJmVVpfkkpWfFOVIREZFtn7KyFq7KHQCzBAciCVW+YgZVhdmAU75mHus++j8yBx8FyakxLe9lhSSlt96kLCm9DVWlhY0QrYiIyLZNCXYLc/TQ/rRuFSRNfTq34ZoTh/PZjCWUVVRtqJOWkkRaSnBoU5OD50kRGXhKRFlSkpGWkkSKWsC3LeEXL4jtm1fZqlmkdOqHpWZsKEvrMZjyVbMaITgREZFtm/pgtzCnHzCIm07bj7SUJHIKShj74xIe/+iHDfN7dMjivd+fvmH60V8dDcBtL4/n3cnzAHjosiMZvlP3DXVO3mcAk+et5JePftREeyHxlrnbsRTPHY+X5pPSsS8djrqO4tnjoLJsY6XktIjnqcF0VQV4FaWLplCZu4L2h/+W3DH3ktKpP62H/Zx1H97Z5PsiIiLS0inBbmHqS4JXrCtk7+vrHrtYifS2p/XeZ9Dh+D9gyalUFeZQNOtT1o97eMP85HY96fW7jaOBdDv/SQCy3/4DhT+8DV7Fmpd+Q8cTb6XX9V9QVZJP3tdPUzT9gybfFxERkZZOCbbINmD1cxfXOb9y/XIW/3lInXUq1i1h9fOXxTMsERGR7ZI63oqIiIiIxJES7GagsspbxEWGqSlJVFRW1V9R6uBgzf9YNx6LuABTRERk27Q9/6dvNvKKS0lJTqJNRlr9lROkVWoy7TLSWVdYmuhQWrSqsiKSWrVNdBgJk5TRFi/T0H8iIrJtU4LdDLjD7OU5HLxr7w3D6zUnKUnGQbv2Yd6qXLVgb6WK7IWkdd8l0WEkiJHWfRfKsxcmOhAREZFGpYscm4lJc1dw4C69OPegXVmdV0RpeSWe4J/SzYzUlCS6t8ti0do8xs9cltB4tgWly36k9dBTyBh8NOWrZlJVkr/td5mwJJIy2pHWczfAKF89N9ERiYiINCol2M2EO3zx0zImzVlJ1/aZpCUnJzokAMorKxm7fgkl5RWJDmXbUFlGwQ9vk9ZjMOl99sLSMon1ZjAtl+OlBZSvXUDZypnB2NsiIiLbMCXYzUxpRSVL1uYnOgxpTJVllC39gbKlP9RfV0RERFqc5tfhV0RERESkBVOCLSIiIiISR0qwRURERETiSH2wRUQaYMQDIxIdgsRo/FXjEx2CiGyn1IItIiIiIhJHSrBFREREROJICbaIiIiISBwlJME2s2Qz+87M3gundzCziWY218xeNrO0sDw9nJ4bzu+fiHhFRERERGKVqBbs3wE/RUz/A7jH3QcA64BLwvJLgHVh+T1hPRERERGRZmurRhExs/buntvAZXoDJwB/Bf6fmRlwOHBOWOVZ4DbgEeCU8DnAa8CDZmbu7lsTt4hIS5CSlMKgLoPo26EvmamZBB+XLU9lVSV5JXnMz57PwnULEx2OiEijiynBNrMrgDbu/n/h9J7Ae0APM/seOMXdl8a4zXuBG4A24XQnINfdK8LppUCv8HkvYAmAu1eY2fqw/toYtyUi0iKlJKVwzM7HUFpRyoyVM8grzYMW2rSQlJREp8xO7NlrT7q16cbExRMTHZKISKOKtQX7KuD+iOn7geXAdcCNwJ3AL+pbiZmdCKx29ylmdmiDIq17vZcDlwP07ds3XqsVEUmYgV0GUlZZxidzPkl0KHGRU5TDonWLOHXIqcxaM4vc4txEhxRXnx18SKJDkBgd8vlniQ5BtgOxJth9gVkAZtYFGAEc4e7jzKwMeDDG9YwATjaz44FWQFvgPqC9maWErdi9gWVh/WVAH2CpmaUA7YDs6JW6++PA4wDDhw9voW08IiIb9evQjxmrZiQ6jLgqqyxjYc5C+nXot80l2CIikWK9yLEUSAufHwYUAV+E0zlA+1hW4u43u3tvd+8PnAV86u7nAmOBUWG1C4C3w+fvhNOE8z9V/2sR2R5kpGaQX5Kf6DDiLr80n8zUzESHISLSqGJNsCcBvzaz3YDfAh+4e2U4b0eC7iJb40aCCx7nEvSxfjIsfxLoFJb/P+CmrdyOiEiLYBjxbk+4eN+LuXfkvRum/3nyPzln2Dm1LxBjnYao8ipomddqiojELNYuItcC7wLTCC46vDhi3pnA+IZu2N3HAePC5/OBfWuoUwL8vKHrFhGR+l33znWbTI+/ajxXvHYFU1dMrbWOiIjUL6YE291nADuZWScgJ6qbxnXAysYITkRERESkpWnoONg5QG8z6wP84O6F7j6tEeISEZEavHbBa7w/43326bsPAzsPZPG6xdw17i5mrp5JsiVz/vDzOW7wcbRJb8PsNbO59/N7WZCzoMZ1PXDqA0xeMplnJz/LM2c/A8A9p9xDlVcxZs4Y7vz0zk3qAHRv051fj/g1e/Tcg/SUdBZkL+DG928krySPy/e/nBMGn0BmWibrS9bz0ncv8drU15rqpZEIlpJCZr9+ZPTuTXKrVk2+fa+qonz9egoXLKBszZom375IosWcYJvZlcAfgO4Eo7HuA3xrZm8An7v7vY0SoYiIbGLk7iO54b0bmJc9j7P2Oot/nvxPznj2DE7f43SO3eVYrnv3OlasX8F5w8/j3pH3cvbzZ1NUXlTnOi988ULGXzWea96+ZpMuIpHSU9K5/9T7mbBoAue8cA4l5SXs0nUXKior2LfPvhw3+Dgue/UyVhespn1Ge7pkdWmM3Zd6WFoa3Y85hsqSEgoXLKCyqAiaeHwAS04mrVMnuh52GHnTp5M3fXqTbr+5ePDadxMdgsToN/86Ka7ri/VGM9cDfyG4VflY4NOI2eOAswluICMiIo3svRnvMWvNLABemPICpw45lRE7jOD4wcfzwrcvsHjdYgCenvQ0J+12Ej/r/7O4jKc9ov8I0lPSue/z+6gMr3OfvipInMqryklLTmOHjjuQW5y74SFNr92QIZTl5pL9xRf1V25ExUuXUjB3Lj1PPpnCRYuoLChIaDwiTSnWUUR+Ddzq7n9i4/B81WYBg+IalYiI1GpF/opNplflr6JL6y50a9ONFXkb5znOirwVdG3TNS7b7d62O8vXL9+QXEf6btl3PPb1Y1ywzwW8d+l73HPKPezSdZe4bFcaJrNfP/JnNI8W48rCQooWLyJTN4GT7UysCXZ3YEot86oIbhojIiJNoEebHptMd2vTjTUFa1iVv2qTeYbRo20PVuevjmm9VV5V5/yVeSvp2bYnSVbzv453pr/Dla9fyUlPnsScNXP46/F/jWm7El8prbMoz12f6DA2KF+/npTWrRMdhkiTijXBngvUdh/Yg4Ft63ZjIiLN2Am7nsCgLoNITkrmnGHn0CqlFV8t/Ir/zfwf5ww7hz7t+5CSlMIF+1xAsiXz1cKvYlpvTmEOvdv3rnX+Vwu/oryqnN8e9Fuy0rJItmR267YbmamZDO42mKE9h5KalEp5ZTlF5UVUVdWdsEvjMEuK+xjq0bodeyz7/vc/MdX1yirMNPi5bF9ivcjxXuDh8Lbo1ZeEdzWzSwhuAHNZI8QmIiI1eHv621x98NXBKCK5i7n+3espLCvkP9/+h9TkVO455R6y0rKYu3Yu17x9Tb0XOFZ7bMJjXLrfpVx14FV8OvdT7hp71ybzSypK+O2bv+U3B/6Gl897mZTkFOZnz+fG924kIzWD34z4DX3a96HSK5mfPZ9bP7y1MXZfttDQ++5l3ZQpLH7u+USHIrLNi3Uc7CfMrANwK3B7WDya4Jbpt7n7fxspPhERibJs/TKenvT0ZuWVVZU8OfFJnpz4ZA1LwVOTntpk+qo3r9pkevRPoxn90+g66yzPW87vR/9+s3V/u/RbLn754s3KRWRTySlJpKQmJzqM7VZVlVNeVhGMh9eIYh6mz93vMrNHgQOAzgRjYn/t7s2no5eIiIhsZsDVv6PdHnvQdrfd6HvOOZSuXcs3vziP7ieeSO9Rp5PetSvFy1ew4LFHWffN5A3LdT74IPr+4hdk9OpFVVkZK94fzcInntgwv9fpp9Pn7LNIyshgzdixzLn7HlDXoBq17ZRJ7wGdychKo6K8srHzO6lFcnLQXWnt8nyWzlnTaCNYNuhGM+6eD3zUOKGIiIhIY5h7731k7bDDJl1Eup94In3POZvpf7yVwvnz6bjfvuz2l78w+ZJLKVm2jI777cfOv/89P93+Z3ImTSI5PZ2snXbasM707t1J7dCBiWefQ3rXrgx77FHW//ADqz/e+iEhtzVtOmSw05AeLJixivVrCpp6WHKJkp6RSr9durLj7j2YN21F/QtsgVoTbDM7viErcvfR9dcSEZGtMerZUYkOQbYRvUedzqJnn6Nw3jwAciZMJPe77+l6xOEsfu55ep5+Givefpucr78GoLKoiLxpG2/eXFVaysKnnoKqKkqWLSN3yre02XlnJdg16N6vA0vmrCF3tcYCbw5Ki8uZ88Ny9jxkR9JapVBWUhH3bdTVgv0eQQ+VWC79dUAdikRERFqIVj16MOCaqxnw24397C05mdLw1uatundn7ee136ymfN26TbqDVJYUk5yZ2XgBt2BtO2ayYMaqRIchEbzKycsuom3HTNYuz4v7+utKsHeI+9ZEREQkIaKH7itZtYqFTz3N2nHjaqxfsnIlmb1rH7ZRYpeckkRF2eY3aGrJ9j5iAN37deD9p75JdChbrKKskuSUWEesbphaE2x3X9QoWxQRkSZlGI+MeoQhPYYw8qmRrClcw4DOA/jVz37FoM6D6JTViSteu4KpK6ZuWGZoz6H886R/brKetJQ0FuYs5IIXL2jqXZA4KMvOIaNXrw3TS195lf4XXUjx0qUUzp1LUloarXfemfL16ylevJjlb7zJ4Nv+RO5335EzefKGPtiR3UQkcXbZpzfDjxzEC3//dKvXdcRZe1JVVcXYV6bWXzk0Zczcrd5uY4vna9RQMV3kaGZnAX3c/a4a5l0HLHb3V+IdnIiIbL0z9zqTkoqSTcrKK8v5bN5nPDHhCZ48c/Nh/X5Y/gNHPXbUhmnDeO2C1/hw1oeNHq80jmWvvsrON93IiPffo3TtWiZfcCFeUc7ON91ERo/uVFVWUjB7NvMffgSAnAkTmP2P/6P/ZZcx+LY/UVlSwsr33lOCvQ0xa/TR6rZbsY4icjPwRC3zisL5SrBFRJqZPu37cNqQ07hl9C08c/YzG8oXrVvEonWx/1B5QP8D6JjVkdEzdD17S5U/cyaTL7xok7JVH3zIqg9q/9K0ZuxY1owdu1n5qg8+YNUHH2xSNuvvd8Yn0O1ISmoy+x27MzsN6UFaRgqrFufy+RvTWJ9dxKlXHMDqpetp2zGTPoO6UFxQypfvTGfB9FV079eBQ0/fg+TkJC7/23EAvP/kJJbNy6Zj9zYcePKudOnVjoryKmZ/u5SJH8yiqspp0yGDC/5wJGNe/p69DtmJdp0zmfzJHAYNC37ZGLhn8Pfft/yPjt3bcvCpu9OxexvMYNWiXD57cxp52cGNq/Y9ehA9dujI249NAOD8W45g+oRF9B7YmW59O5CfU8TY16aycuE6IGglNzOqqqrYaUgPyssqGP/uDNatKuCwnw+lQ9fWrF6ay8f/+ZbCvNJ6Xx9gi1+jJjm2MdYbAPxYy7yfgIHxCUdEROLFMG4+4mYe/PJB8kvzt2pdI3cfybi548gtyY1PcCLCYWfsQXqrVF69/wtKi8sZfuRATrx0P168axwAu+zTh/efmsT/npvM0IN25Miz9+Lp2z9m5aJ1jHt96mbdHzJap3HalT/j69Ezee/JSWRkpXPCxftQUV7JNx/P2VBv0LBevPXo15QUleFVTttOWTV0EXEmfTiLFQvXkZKaxOFnDOXoc/bitQfG17o/g/fty+inJrFudQEjTtqNI8/akxfu3PgFbcDQHvzv2cl8+soP7LZ/Pw77+VCWzV3L6Ge+obSonBMv2Zd9j9mZsa9Orff1qaryLXqNmkqsPbuLgNqudOgDlMYnHBERiZcz9jyDnKIcPp//+Vatp1vrbuzfb3/e/vHtOEUmIq2y0th5WG/GvT6N4oIyqiqdSR/NJrNNOt36dQBgzvfLghZgh+kTFpGekUq7zlm1rnOX4b1ZuzyP6RMWUVXpFOaVMGXMXHYe3meTet98NJui/FKqKr3WMbmzV+SzbF42VZVVlJVUMOmj2XTv37HOu1BO/3oROauCcb5nTFxE+y6tSWu1sS136Zy1LPppNTjMmryUtPQUZk5eSuH6EirKK5k7dTld+7SP+fXZkteoqcTagv0J8Ecz+9DdV1cXmlkX4BZ08xkRkWalV7tenLXXWVzy8iVbva6TdjuJxbmL+X7591sfmIgA0LZjBgBnX3fIJuVJyUm0bt8KgKK8je2X1aOQRCas0dp0zKTHDh247I5jNym3pE1HXM7LKa4/vk6ZjDhxV7r1a09aesqGRDyjdRr562pevjBv47Ue5WG8qekbx5kuzN84v6I8mF+UX7pJWWp6sH+xvD7Q8NeoqcQawY3ABGCemX0ArAB6AMcAucANjRKdiIhskaE9htI+oz3PnxPctS/Jgh8snzvnOR6f8DhvTnszpvUkWzIn7noi//n2P40Wq8j2qDpJff7vn1JSWLbZ/N3371fn8l7DHenz1xWzZPZa3ntyUt3LRjVbR08DHDpqD4rWl/DSPz+jpKicjt3bcM71h9a53niq7/WJRU2vUVOJqYuIuy8GhgIPEnQJOS78+wAwzN2XNFqEIiLSYGPmjuGMZ8/gwhcv5MIXL+S6d64D4Jq3r+GDmcHFaWnJaaQlpwGQkpxCWnLahkS82ogdRtAmvQ3/++l/TbsDItu44oIyZn27lENPH0JW26BFNq1VCjvu3p3UtPrv3VeUX0pm67QNLb4QdLvo2qc9g/ftE4zvbMFNbvru3KXudeWV0q5j1ia3FkxLT6G8rJLS4nJaZaWx3zE7b9mObqGtfX2g5teoqcS8RXdfQzBaiIiINHOlFaWsqVizYTo5KfiHlF2YTXF5Md3bdOf1C1/fMP+BUx8A4K8f/5XRMzeOFHLK7qcwZs4YCsp0i+dtjhl7PvQg7Xbfna9PH0VZeAfHbsccQ78LLyCtUycK589nzt33UDB7NgBJaWnscsstZA0cQEbPnix86ikWP/d8IveiRRv7ylT2PmIAp155AJltWlFaUs6K+dksnr2m3mWXzV3LktlrueCWI7Ak4/2nvmH5/GzefOQrfnbCYPY/bhdSUpPJX1fEj1/XPWLQjImL6T2wM5f++RjMjCf++AFfvjOdw0btweV/PY783GK+GzePnfboEa9dj8nWvD5Q+2vUFKymnwVasuHDh/vkyZMbtMze1z/XSNFIY5hy1/lNsp3Ffx7SJNuR+Oh7a9OMzTvigRFNsp3ThpzGp3M+3eZG7dil6y50yOzA1wu/bvRtjb+q9tEO4u2zgw+pv1IT6XfBBSx6/vlNbmNek95nnkHH/fajw/DhGxLstkOGsMc/72L6H/5A7vc/0HvUKHqfeQaTzjmXyqIiLC2NnqecQuHcuexw+WVkf/11vQl2m8GDSW3blpyJE+O5m1vskM8/a7JtPXjtu0AwpN2kj2Y32XYlNv126UpJURmrFufym3+dtEXrMLMp7j48urzWLiJmNsnMdg2ffxNO1/rYoqhERESkyWX07k3PkSOZF95UplqPk05k7edfsO6byXh5OUtefJGq8nI6H3wQAF5WxrJXXyX3u++oKtuyfrEi24O6uohMB4ojnm9bTd0iIs2d1V+lpTHbBneqpTFj55tuZN7Dj1BRsGnXn9Y77cTKqBvIFMyZQ+udBrAK3cVTJFa1JtjuflHE8wubJBoREQGgpKKErLQscotzEx1KXGWlZVFarlsnJFKvUaMoy8kh+4svSO/efZN5yZmZVBQUblJWUVBAclZmU4Yo0uLFNIqImT1lZjvUMq+fmT0V37BERLZvS3OXslOnnRIdRlwlWzL9O/Rnce7iRIey3WrVqxd9zjyDOffeV+P8yqIiUlpvepOOlNatqSwsaorwRLYZsY4iciHwKLCghnmdgQuAi+MUk4jIdm/WmlmcuOuJHND/AGatnkV+ST5VJHBQ162QbMl0zurMHj33ILsom7WFaxMd0nar3ZAhpLZvzz7PPB0UJAXtbMOffoqFTzxJwbx5tB44aJNlWg8YwNrPt+5uoCLbm4YMDFhbH+zdgdjGSxERkZiUVpQy+qfRDOk+hMMHHE5mWsv9id7dyS3JZUH2AmasmpHocLZra8aOZd2UKRum07t0YdijjzD12usoWryYgnnz2OOu/2PVB8NYP3UqvUadTlJaGms//2LDMpaaCmaQlIQlJ2NpaVBZiVdWJmKXJE72P24XBu3Vi1ZZqVSUV7F8fjZfvjODgtxizGD/4wczaFgv0lulkr+uiEkfzWbe1BUblk9JS+aA43dhwNCepKanUJBbzEf/+Za1y/ISuFeJU2uCbWa/A34XTjrwlplFd5xrBXQDnmmU6EREtmPF5cVMWjKJSUs0UJPER1Vp6YbxrgEsORgfvSwnh6riYvKmTWPOPfcy6IbrN4yDPe2GG6ks2thFZN8XnqdVj2A85PZDh9L/ootY+b//MevvdzbtzkhczZyylG/HzqWspIKU1GT2P25njjlvGK8/MJ4hI3Zgl7178+YjX5G7ppAddu/OseftzYsrPyN3dXCh7PEX7UNleSWv3vcFBbkltO2YSXlZRYL3KnHqasGeAVTfheBaYCzBLdIjlQEzgVfiH5qIiIg0ptKVKzcbw3vVhx+y6sPaRwyZeOZZjR2WJEB1ogyAgTt06NIagHadM1k2by25a4ILYBf8uJKSojI6dW9D7uoC+u7chR79O/LMnz+mtLgcgLyc7bvffl2jiHwMfGxmSUAH4DZ3X9pkkYmIiIhIkxm0Vy8OOX0I6RmpVFZWMf6d6QDMmLCYo84dRodurcldXcCOQ3qQlGQb7orYa0An8nKK2O/YnRkwtCflpRXM+X45kz6cRVXV9jnKcyx9sJMILmJ8DVCCLSIiIrHT2OctxuzvljH7u2Vktkln8L59yV6RD8D6nCKWL8jmnOsOxd2prKji4xe/o7gguNlQq6w0OnVvw+KZq3n2jk9o3T6Dky7dl/KySqaMmZPIXUqYeofpc/cKYBHQcq+wERER2U5UlpaSnJGR6DA2SG7VSnd9bGGK8kuZMXERJ1yyL+kZqRx62hC69GrHc38bw8M3vs/bj0/g0NP3oM+gLgCUl1RSVVnF16NnUllRxfq1hUwbv5Add++W4D1JnJjGwQb+AdxiZp0bMxgRERHZOiUrlpPZr1+iwwiYkdmvL8XLliU6EmmgpCQjLT2FrHat6NK7HbMmLyV/XTE4rFy4jhULcug3uCsAa5evr3Edvn32DgFiT7CPBnoAi8zsczN71cxeiXi83IgxioiISIzyfpxOuz32oPWgQcGQegmS2q4dnQ85hMqiYkpXr05YHBIDgyEj+pPROg2ArHatOPi0IeRlF7FudQErFq5j0N69yWrbCoBufdvTa6dOrFkaJNbzpq2guLCM/Y4dRFKy0aZjBrv/rD/zp0WPjbH9iHUc7M7ArKhpERERaWbKsrNZ/ckntBs6lI7774dXVlH7rSwahyUlUVVaRuGC+WR/8UX9C0jC9RvclX2OGkRqWjKlJeUsm5fNW499jVc549+dwYgTB/Pzqw8kLT2VooJSvvtsHrOmBJfmlZdW8s7jEzjktCFc9pdjKS4sY+Y3S/hu3LwE71XixJRgu/thjR2IiIiIxEfZ2rWsGTMGzLDUVJr6MkOvqsIrtt8xkFsch/eeqH28/fLSCsa9Pg1en1ZrnewV+bzx0FeNEV2L1JA7OYqIiEhL4o6XlTVx+7WIxJxgm1kb4BRgEMEdHDfh7jfEMS4RERERkRYppgTbzHYCvgIygCxgDdAxXH4dsB5Qgi0iIiISxd0x275H1WiOgmPSOAcl1lFE7gG+AboBBhxPkGz/AigAzoxlJWbWyswmmdkPZjbdzG4Py3cws4lmNtfMXjaztLA8PZyeG87v37DdExEREUms0uJyMtukJzoMiZLRJp3SovJGWXesCfa+wKNAaTid5u6V7v5f4F/AfTGupxQ43N2HAnsCx5rZ/gTjbN/j7gMIWsQvCetfAqwLy+8J64mIiIi0GDmrCujev2Oiw5AIbTpk0CorjbycokZZf6x9sFsBee5eZWY5QM+IeT8CQ2NZiQft8AXhZGr4cOBw4Jyw/FngNuARgj7ft4XlrwEPmpl5Y7Xni4iIiMTZ8vnZ7DysN7vu15eclfmUlVaov0iCJCUn0aZDBu27tmbeDysa7TDEmmDPBqpvC/Ud8CszGw1UErQyL491g2aWDEwBBgAPAfOA3PCW7ABLgV7h817AEghu2W5m64FOwNpYtyciIiKSSFWVzszJS2jXOYt2nbPIarfZWBHSRNydorxSls1bRFlJ4w0lGWuC/RJBl47ngT8CHwJ5QBWQDFwY6wbdvRLY08zaA28Cu8QcbS3M7HLgcoC+fftu7epERERE4sodctcUkrumMNGhSBOI9UYzd0c8n2BmuwPHElzo+Km7/9jQDbt7rpmNBQ4A2ptZStiK3RtYFlZbBvQBlppZCtAOyK5hXY8DjwMMHz5cv7mIiIiISMLEepHjJtx9ibv/293vb0hybWZdwpZrzCwDOAr4CRgLjAqrXQC8HT5/J5wmnP+p+l+LiIiISHPW1Hdy7AE8G/bDTgJecff3zGwG8JKZ3UHQx/vJsP6TwPNmNhfIAc5q4nhFRERERBqkSRNsd58K7FVD+XyCoQCjy0uAnzdBaCIiIiIicbFFXURERERERKRmtSbYZtbXzFKbMhgRERERkZaurhbsBYTdOczsUzPb6uH0RERERES2dXUl2MVAZvj8UKBto0cjIiIiItLC1XWR43fAfWb2cTh9lZmtqKWuu/uN8Q1NRERERKTlqSvBvgy4CzgFcOAIoLSWug4owRYRERGR7V6tCba7zwROAjCzKmCku09qqsBERERERFqiWMfB3gGorXuIiIiIiIiEYkqw3X2RmaWY2ZnAgUBHgjsrfgG84e4VjRijiIiIiEiLEVOCbWZdgY+APYCFwCrgAODXwA9mdrS7r2msIEVEREREWopY7+R4N9AJ2N/dd3T3A9x9R2C/sPzuxgpQRERERKQliTXBPh64MfoiR3f/BrgZOCHegYmIiIiItESxJtjpQH4t8/KBtPiEIyIiIiLSssWaYE8AbjSzrMjCcPrGcL6IiIiIyHYv1mH6rgXGAkvM7COCixy7AscARnArdRERERGR7V5MLdju/j0wEHgc6AIcRZBgPwoMdPcfGitAEREREZGWJNYWbNx9LXBTI8YiIiIiItLixdoHW0REREREYqAEW0REREQkjpRgi4iIiIjEkRJsEREREZE4qjfBNrN0M7vFzIY2RUAiIiIiIi1ZvQm2u5cCtwDtGz0aEREREZEWLtYuIhOBYY0ZiIiIiIjItiDWcbBvAP5rZuXAaII7OXpkBXcvinNsIiIiIiItTqwJ9sTw7/3AfbXUSd76cEREREREWrZYE+yLiWqxFhERERGRzcWUYLv7M40ch4iIiIjINiHWFmwAzGxXYG+gD/CUu680swHAKnfPb4wARURERERakpgSbDNrDTwFjALKw+U+AFYCfwMWA9c1UowiIiIiIi1GrMP03Q38DDgCaANYxLzRwLFxjktEREREpEWKtYvIacDv3H2smUWPFrII6BffsEREREREWqZYW7AzgOxa5rUBKuMTjoiIiIhIyxZrgv0NcH4t80YBX8UnHBERERGRli3WLiJ/BD42s0+AVwnGxD7ezK4hSLAPbqT4RERERERalJhasN39C4ILHNOBBwkucrwd2BE40t2/abQIRURERERakJjHwXb38cBBZpYBdABy3b2o0SITEREREWmBYu2DHamEYCzs4jjHIiIiIiLS4sWcYJvZ8Wb2FUGCvRIoMbOvzOyERotORERERKSFiSnBNrNfAu8CBcDvgJ+HfwuAd8L5IiIiIiLbvVj7YP8eeMzdr4wqf9TMHgVuAR6La2QiIiIiIi1QrF1EOgFv1jLvdaBjfMIREREREWnZYk2wxwKH1DLvEODzWFZiZn3MbKyZzTCz6Wb2u7C8o5l9bGZzwr8dwnIzs/vNbK6ZTTWzYTHGKyIiIiKSELV2ETGzXSMm7weeMLNOwFvAaqArcCpwHHBpjNurAK5192/NrA0wxcw+Bi4Exrj7nWZ2E3ATcGO47oHhYz/gkfCviIiIiEizVFcf7B8J7thYzYBfhg8Pp6t9ACTXtzF3XwGsCJ/nm9lPQC/gFODQsNqzwDiCBPsU4Dl3d2CCmbU3sx7hekREREREmp26EuzDGnPDZtYf2AuYCHSLSJpXAt3C572AJRGLLQ3LNkmwzexy4HKAvn37Nl7QIiIiIiL1qDXBdvfPGmujZtaa4OLIq909z2xjY7i7u5l5rQvXwN0fBx4HGD58eIOWFRERERGJp5hvlV7NzFKAtOjyWG+bbmapBMn1f9z9jbB4VXXXDzPrQdDHG2AZ0Cdi8d5hmYiIiIhIsxTrjWbamdnDZraC4E6O+TU8YlmPAU8CP7n73RGz3gEuCJ9fALwdUX5+OJrI/sB69b8WERERkeYs1hbsZwiG4/s3MBco28LtjQDOA6aZ2fdh2e+BO4FXzOwSYBFwRjhvNHB8uM0i4KIt3K6IiIiISJOINcE+Avilu7+4NRtz9y/ZdPSR6G1E13fg11uzTRERERGRphTrjWYWE7Qgi4iIiIhIHWJNsG8A/mBmGgNPRERERKQOMXURcffRZnYkMNfMFgK5NdTZN76hiYiIiIi0PDEl2Gb2T+Bq4Bu27iJHEREREZFtWqwXOV4K3OLuf2/MYEREREREWrpY+2AXAVMaMxARERERkW1BrAn2fcDlFnlPcxERERER2UysXUQ6A/sBs8xsHJtf5OjufmMc4xIRERERaZFiTbBHARVAKnBUDfMdUIItIiIiItu9WIfp26GxAxERERER2RbE2gdbRERERERiEOs42FfWV8fdH976cEREREREWrZY+2A/WMc8D/8qwRYRERGR7V5MXUTcPSn6AXQEzgZ+AHZtzCBFRERERFqKWFuwN+PuucDLZtYOeAw4NE4xiYiIiIi0WPG4yHEBMDwO6xERERERafG2KsE2sx7AtQRJtoiIiIjIdi/WUUTWsPFixmppQBugBDgtznGJiIiIiLRIsfbBfojNE+wSYCnwgbtnxzUqEREREZEWKtY7Od7WyHGIiIiIiGwTdCdHEREREZE4qrUF28w+bcB63N2PiEM8IiIiIiItWl1dRGLpV90D+Bmb988WEREREdku1Zpgu/vPa5tnZn2BG4ETgbXAPfEPTURERESk5WnQnRzNbABwM/ALYHX4/DF3L26E2EREREREWpxYx8HeDbgF+DmwBPgd8JS7lzVibCIiIiIiLU6do4iY2d5m9gYwFRgGXAoMdPdHlVyLiIiIiGyurlFE/gccDUwDznL3V5ssKhERERGRFqquLiLHhH97Aw+Z2UN1rcjdu8YtKhERERGRFqquBPv2JotCRERERGQbUdcwfUqwRUREREQaSLdKFxERERGJIyXYIiIiIiJxpARbRERERCSOlGCLiIiIiMSREmwRERERkThSgi0iIiIiEkdKsEVERERE4kgJtoiIiIhIHCnBFhERERGJIyXYIiIiIiJx1KQJtpk9ZWarzezHiLKOZvaxmc0J/3YIy83M7jezuWY21cyGNWWsIiIiIiJboqlbsJ8Bjo0quwkY4+4DgTHhNMBxwMDwcTnwSBPFKCIiIiKyxZo0wXb3z4GcqOJTgGfD588CIyPKn/PABKC9mfVokkBFRERERLZQc+iD3c3dV4TPVwLdwue9gCUR9ZaGZSIiIiIizVZzSLA3cHcHvKHLmdnlZjbZzCavWbOmESITEREREYlNc0iwV1V3/Qj/rg7LlwF9Iur1Dss24+6Pu/twdx/epUuXRg1WRERERKQuzSHBfge4IHx+AfB2RPn54Wgi+wPrI7qSiIiIiIg0SylNuTEzexE4FOhsZkuBPwF3Aq+Y2SXAIuCMsPpo4HhgLlAEXNSUsYqIiIiIbIkmTbDd/exaZh1RQ10Hft24EYmIiIiIxFdz6CIiIiIiIrLNUIItIiIiIhJHSrBFREREROJICbaIiIiISBwpwRYRERERiSMl2CIiIiIicaQEW0REREQkjpRgi4iIiIjEkRJsEREREZE4UoItIiIiIhJHSrBFREREROJICbaIiIiISBwpwRYRERERiSMl2CIiIiIicaQEW0REREQkjpRgi4iIiIjEkRJsEREREZE4UoItIiIiIhJHSrBFREREROJICbaIiIiISBwpwRYRERERiSMl2CIiIiIicaQEW0REREQkjpRgi4iIiIjEkRJsEREREZE4UoItIiIiIhJHSrBFREREROJICbaIiIiISBwpwRYRERERiSMl2CIiIiIicaQEW0REREQkjpRgi4iIiIjEkRJsEREREZE4UoItIiIiIhJHSrBFREREROJICbaIiIiISBwpwRYRERERiSMl2CIiIiIicaQEW0REREQkjpRgi4iIiIjEkRJsEREREZE4UoItIiIiIhJHzT7BNrNjzWyWmc01s5sSHY+IiIiISF2adYJtZsnAQ8BxwK7A2Wa2a2KjEhERERGpXbNOsIF9gbnuPt/dy4CXgFMSHJOIiIiISK2ae4LdC1gSMb00LBMRERERaZbM3RMdQ63MbBRwrLtfGk6fB+zn7r+Jqnc5cHk4uTMwq0kDbb46A2sTHYQ0OzovpCY6L6QmOi+kJjovNurn7l2iC1MSEUkDLAP6REz3Dss24e6PA483VVAthZlNdvfhiY5DmhedF1ITnRdSE50XUhOdF/Vr7l1EvgEGmtkOZpYGnAW8k+CYRERERERq1axbsN29wsx+A3wIJANPufv0BIclIiIiIlKrZp1gA7j7aGB0ouNoodRtRmqi80JqovNCaqLzQmqi86IezfoiRxERERGRlqa598EWEREREWlRlGCLNCNmttDMjozj+g4yMw1bKRuYmZvZgPD5o2b2xzrq/t7Mnmi66KQlMLNDzWxpouOQxNLnQ92UYAsAZvaMmd2R6Dgkvtz9C3ffOdFxSPPk7r9y979AzUmTu/+t+j4Esm2I95d4aR7icVzNbJyZxfx+b66fDw3dj8bS7C9ylMZnZsmJjkFERESanpkZYImOY1ujFuwWwMxuNLNlZpZvZrPM7Agzu83MXjOzl8Pyb81saMQyg8NvcblmNt3MTo6Y94yZPWJmo82sELgEOBe4wcwKzOzdBOymbLSPmc0ws3Vm9rSZtTKzC83sy8hKUT/1Hx8ukx+eK9eF5Zu0SoatHNeZ2VQzWx+eP60i5p9oZt+H581XZrZHxLzNzsOwfF8zm2xmeWa2yszubuwXSDYcy5ujz5Vw3mVmNtfMcszsHTPrWcs6njGzO8wsC/gf0DP8DCgws57h58wLEfUPDM+LXDNbYmYXhuU1nn/SvJjZ80Bf4N3wGN9gZvtHHNMfzOzQiPodw/NqeXiOvRW1vmvNbLWZrTCzi5p0Z7ZR4fvudTNbY2YLzOy3YfltZvaKmT0Xvs+mm9nwcN5mxzUsr+vYjjOzv5rZeKAIeB44CHgwXMeDYb37wvd6nplNMbODItax4fPBzPqH/5MuMLPFZrbWzG6Jqvuqmb0Qxj/NzAaFn2Grw20cHVG/nZk9GZ5by8LPqeRw3oVm9qWZ/TM8LxeY2XHhvL/WtB8J4e56NOMHwa3flwA9w+n+wE7AbUA5MApIBa4DFoTPU4G5wO+BNOBwIB/YOVzHM8B6YATBl6xWYdkdid7f7f0BLAR+JLiDaUdgPHAHcCHwZVRdBwaEz1cAB4XPOwDDwueHAkuj1j8J6Bmu/yfgV+G8vYDVwH4E485fENZPr+08DJ9/DZwXPm8N7J/o13F7eNRxrhxOcAvjYeGxewD4vJbzZsP7PvpcCctuA14In/cLP0fODj9jOgF71nX+6dH8HuF5c2T4vBeQDRwf/i84KpzuEs5/H3g5PKapwCER50oF8Oew/HiCJK1DovevJT/CYzAFuJXgf/eOwHzgmPC9WBK+1snA34EJNR3XGI/tOGAxsBtBb4bUsOzSqJh+Eb7XU4BrgZVAq3Be5OdD//Cz5d9ABjAUKAUGR9QtCfclBXiOIGe5Jdz2ZcCCiO2+CTwGZAFdCf5v/TKcdyFB/nNZ+FpcASxn48h4m+1HIh5qwW7+Kgn+Se5qZqnuvtDd54Xzprj7a+5eDtxNkCjvHz5aA3e6e5m7fwq8R/CPsdrb7j7e3avcvaTpdkdi8KC7L3H3HOCvbHrcalNOcI60dfd17v5tHXXvd/fl4frfBfYMyy8HHnP3ie5e6e7PEnxA7k/d52E5MMDMOrt7gbtPaPAey5aq6Vw5l+CmXN+6eylwM3CAmfXfym2dA3zi7i+6e7m7Z7v79+G8hpx/0nz8Ahjt7qPD/wUfA5OB482sB3AcwRfwdeEx/yxi2XLgz2H5aKCA4Iu4bLl9CBLgP4f/u+cTJKxnhfO/DI9VJUGL89DaVkQdxzaizjPuPt3dK8I8YjPu/kL4Xq9w93+xscGlNre7e7G7/wD8EBXjF+7+obtXAK8CXQjylHLgJaC/mbU3s25hnFe7e6G7rwbuiXgdABa5+7/D1+JZoAfQrY64mpwS7GbO3ecCVxN8+1ttZi9F/Ny7JKJeFbCUoGWyJ7AkLKu2iOAbLdHLSrMTeWwWERzP+pxO8IG0yMw+M7MD6qi7MuJ5EcGXMQhaKK8Nf07MNbNcgtbRnvWch5cAg4CZZvaNmZ0YQ7wSHzWdKz3D5wC4ewFBy1Uvtk4fYF4t8xpy/knz0Q/4edR7/kCCZKUPkOPu62pZNjtMlKpFfpbIlulH0E0r8nj8no2JY/Rndyszq+1aurqObbV68wALuhT+ZEGXwlygHdC5jkVq+/8CsCrieTGwNkyQq6cJ6/cjaNVeERH7YwQt2Zttx92LIpZtNpRgtwDu/l93P5DgpHPgH+GsPtV1zCwJ6E3wM8lyoE9YVq0vsCxytdGbiXfcssX6RDzvS3A8C4HM6kIz6x65gLt/4+6nEHwAvQW8sgXbXQL81d3bRzwy3f3FcBs1nofuPsfdzw63/Q/gNQv69Erjq+lcWU5wjAAIj0UnNn3/16S+z4AlBN3TNl8wPuefNI3I47wEeD7qPZ/l7neG8zqaWfuERLl9WkLQTSLyeLRx9+PrXXLz929dx7a2ZTaZDvtb3wCcQdD9pz1B99LGviByCcGvp50jYm/r7rvFuHyzyGeUYDdzZrazmR1uZukE/ZeKgeqW6b3N7LTwG+zVBCfkBGAiwTfHG8wsNbyw4SSCn2Bqs4qgv5ck3q/NrLeZdSTon/YywU9tu5nZnhZcyHZbdWUzSzOzc82sXfhTWx4bz5GG+DfwKzPbzwJZZnaCmbWp6zw0s1+YWZfwF5PccF1bsn1puJrOlReBi8JzJR34GzDR3RfWs65VQCcza1fL/P8AR5rZGWaWYmadwm3E6/yTphH5Wf8CcJKZHWNmyRZcUH2omfV29xUEF74+bGYdwv8lBycs6u3DJCDfggvKM8JjsruZ7RPDstH/w2s9tg1YRxuCvvZrgBQzuxVo27Bdarjw3PsI+JeZtTWzJDPbycwOiXEVzSKfUYLd/KUDdxJctLSSoIXo5nDe28CZwDrgPOC0sD9cGUFCfVy43MPA+e4+s47tPEnQhzLXoq4Ulyb3X4IPl/kEP8nf4e6zCS4o+gSYA3wZtcx5wEIzywN+RdAPt0HcfTLBRSMPEpxTcwkuJoG6z8NjgelmVgDcB5zl7sVIU6jpXPkE+CPwOsHFhzuxad/FGoWfDy8C88PPgZ5R8xcTdAO5FsgBvmdj/8qtPv+kyfwd+EP4s/uZwCkE3RDWELQcXs/G3OA8gr7WMwkugL66iWPdroTdJU4kuC5mAcHn7RME3TLqs+G4mtl17r6Euo9tTe4DRlkwMsf9wIfAB8Bsgm5nJTRd99LzCS70nEHw/+g1Nu3eUpfo/UiI6isupYUxs9sIRgL4RaJjEZGmZ2YLCa6U/yTRsYiIyKbUgi0iIiIiEkdKsEVERERE4khdRERERERE4kgt2CIiIiIicaQEW0REREQkjpRgi4iIiIjEkRJsEZEmZmanm9mn4Zi1pWY228zujh57up513BDeRKpZMrNnzGxyouMQEUkEXeQoItKEzOxfBDfseJrgZlF5wK4EN2iZ7+6nxrietcCD7n5b40S6dcxsJyDD3X9MdCwiIk0tJdEBiIhsL8zsJOD/AZe4+1MRsz4zs8eBoxMTWfyYWYa7F7v7vETHIiKSKOoiIiLSdK4Bvo1KroHgNsnu/j8AM7vTzKaZWYGZLTWz/5hZ9+q64V0cOwF/MjMPH4eG85LM7CYzmxvR/eSCyG1Z4C9mttrM8szsKTM7K1xP/4h6nc3sWTPLNrMiMxtnZsOj1rXQzP5lZn80s6UELfI1dhExs75m9pKZ5YTr+9DMdo6qc3MYe4mZrTKzDyL3XUSkJVCCLSLSBMwsFfgZ8EEM1bsCfwNOIOhOsiPwqZlVf2afCqwHngQOCB/fhvMeAP4APB4u/ybwlJmdGLH+q4HfA48Co4Bi4P9qiOMt4BjgOuBMgv8ZY81sQFS9c4BDgCvDepsxs47Al8DOBN1hzgCygE/MLCOsc34Y193hdq8A5ob1RERaDHURERFpGp2AdGBxfRXd/eLq52aWDHwNLAUOBD539+/MrAJY6u4TIuoOIEhKL3L3Z8PiT8ysB/An4L1wfTcAj7r7rWGdj8xsB6BPxLqOBUYAh7r7Z2HZp8BC4Hrgl1Fhn+juJXXs1jUEifKe7p4Trm98uL6LgYeAfYGP3P3hiOXeqGOdIiLNklqwRUSaVr1XlpvZcWb2lZmtByoIkmuAQfUsegRQBbxpZinVD2AMsGeYXPcBugPvRC0bPb0vsLo6uQZw90LgPYJEP9KYepJrgCOBj4G8iLjygSlAdbeT74Hjzex2M9s3jFdEpMVRgi0i0jSygVKgb12VzGwfgmR3KXAeQfeP/cPZrerZRmcgmaD7SHnE4xmCXyx7ECTXAGuilo2e7gGsrmEbq4CONZTVpzNB95HyqMdhbGw5f4qgi8gZwERglZndoURbRFoadREREWkC7l4edok4hqCPdG1OJUh2z/RwHFUz6xfjZnIIWrxHELRkR1vNxs/9LlHzoqdXEPQFj9Yt3E6kWMZ7zSH44vCXGublA7h7FXAPcI+Z9QHOBf5K8GXj0Ri2ISLSLKgFW0Sk6dwLDI8e1QM2jP5xLJABlPumNyk4t4Z1lbF5i/anBC3Y7dx9cg2PMmAJsBI4JWrZk6OmJwJdzezgiBgzCS6c/LK+Ha3BGGA3YHoNcc2KruzuS9z9ToKLHHfdgu2JiCSMWrBFRJqIu79rZncDT5rZCIIbzRQAuxCMrLEQ+DdwtZndC7xLMPLIL2pY3UzgBDP7IFzHLHefZWaPAi+Z2f8BkwmS8N2AQe5+qbtXmtldwF1mtgYYT5BcDwnXWxXG+qGZfQW8bGY3EXRxuY7gC8BdW7D7d4f78amZPQAsI2gNPwT40t1fNLPHCFq6JxB0czkMGAjcuAXbExFJGLVgi4g0IXe/lqAv8kDgvwQX/l1L0MJ7hbuPJkgoTyfoUnEIcGINq7oeKATeB74B9g7Lf03QDeN8YDRB/+sTgM8jlr0H+DvBsHqvAx0IhgWEcBzr0MgwvnuBVwEDDnf3uVuw32sJ+pLPDLf/EcHQgO2AqWG1r4GDCe5yOZqgu8xl7v5WQ7cnIpJIulW6iIhgZk8AR7l7rP29RUSkFuoiIiKynTGz3Qla0b8i6BJyHHAR6oohIhIXasEWEdnOhDeVeQrYk+DmL4uAx4B/uf4piIhsNSXYIiIiIiJxpIscRURERETiSAm2iIiIiEgcKcEWEREREYkjJdgiIiIiInGkBFtEREREJI6UYIuIiIiIxNH/B/ZkAWxk5lEnAAAAAElFTkSuQmCC\n"},"metadata":{"needs_background":"light"},"output_type":"display_data"}]},{"cell_type":"markdown","source":"Extract Tf-iDF features from the news text dataset:","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00028-c50a0f8f-d78e-4022-847b-038c3f34b67e","output_cleared":false}},{"cell_type":"markdown","source":"**Code IT (1)** Using the TfidfVectorizer class of sklearn library(refer to Practical 9 notebook) extract the tf-idf features from the data.\n\n\n\n**Note** Set the maximum number of features to 1000 to avoid long processing times. You may refer to the [sklearn documentation on TFidfVectorizer](https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfVectorizer.html)\n\n","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00023-e2792743-9b8b-437f-bb65-4fe477a1c01a","output_cleared":false}},{"cell_type":"code","metadata":{"deepnote_cell_type":"code","tags":[],"cell_id":"00023-bdcfbac6-429c-4d57-afc6-fa4a52244784","output_cleared":false,"source_hash":"29c7819e","execution_millis":1,"execution_start":1606229760429},"source":"from sklearn.feature_extraction.text import TfidfVectorizer\n\ntexts = data.text\ndef get_tfidf_features(texts):\n    #insert your code here    \n    return features\nfeatures = get_tfidf_features(texts)\n\n","execution_count":null,"outputs":[{"name":"stderr","text":"/opt/venv/lib/python3.7/site-packages/ipykernel/ipkernel.py:287: DeprecationWarning: `should_run_async` will not call `transform_cell` automatically in the future. Please pass the result to `transformed_cell` argument and any exception that happen during thetransform in `preprocessing_exc_tuple` in IPython 7.17 and above.\n  and should_run_async(code)\n","output_type":"stream"}]},{"cell_type":"markdown","source":"Documentation for Sklearn KMeans implementation: https://scikit-learn.org/stable/modules/generated/sklearn.cluster.KMeans.html\n\n-  ** n_clusters** = number of clusters to form as well as the number of centroids to generate (default=8)\n\nNumber of clusters is the only parameter required. There are some optional one, too:\n\n- **random_state** = set it to an integer to make the process deterministic\n- **init** - there are different ways to initialise centroids, inc. passing an array (not random anymore!)\n\n","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00023-507c6863-e5da-4d86-b269-5aeb89059a94","output_cleared":false}},{"cell_type":"markdown","source":"In the following code we use KMeans clustering implementation of sklearn library to fit a model on the tf-idf features we extracted from text data.","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00029-31a44441-8db0-49ed-a5d9-a197f3436b6c","output_cleared":false}},{"cell_type":"code","metadata":{"deepnote_cell_type":"code","tags":[],"cell_id":"00028-09c3a18c-1672-45f1-b27e-6d5f64e03d92","output_cleared":true,"source_hash":null,"execution_millis":3222,"execution_start":1606220102809},"source":"from sklearn.cluster import KMeans\nimport numpy as np\n\nkmeans = KMeans(n_clusters=5, random_state=0).fit(features)","execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Then we use the model to put the samples in the clusters:","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00031-12807a46-0e73-4605-a4ea-71905c2438d5","output_cleared":false}},{"cell_type":"code","metadata":{"deepnote_cell_type":"code","tags":[],"cell_id":"00031-1612908a-ed77-4a50-b3c4-e2a833835fa7","output_cleared":true,"source_hash":null,"execution_millis":75,"execution_start":1606220110054},"source":"kmeans_results = kmeans.predict(features)","execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"In order to visualize the results, we plot the wordcloud of the samples clustered into one category for each of the 5 categories:\n\n","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00033-3093d61d-b0e6-47b8-8975-31790b945bce","output_cleared":false}},{"cell_type":"markdown","source":"** CODE IT (2)**  Use [this code piece](https://python-graph-gallery.com/260-basic-wordcloud/) to create wordclouds for each of the clusters inside the loop(in the for loop the text is saved in variable called **text**)","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00034-89f1ee0a-1d54-4f3c-976b-94c9e01b8ad3","output_cleared":false}},{"cell_type":"code","metadata":{"deepnote_cell_type":"code","tags":[],"cell_id":"00034-fa3a28b8-b22e-4c14-8c2d-da42bb09b1d3","output_cleared":true,"source_hash":null,"execution_millis":5762,"execution_start":1606220130510},"source":"from wordcloud import WordCloud\nresult={'cluster':kmeans_results,'text':data.text}\nresult=pd.DataFrame(result)\nfor k in range(0,5):\n   s=result[result.cluster==k]\n   text=s['text'].str.cat(sep=' ')\n   text=text.lower()\n   text=' '.join([word for word in text.split()])\n   print('Cluster: {}'.format(k)) \n\n   #insert your code here \n   ","execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Observe and Reflect (3)** Map clusters 1-5 illustrated in wordcloud to the categories : sport, business, politics, technology, entertainment\n\nGive a brief explanation how you chose each cluster's tag.\n","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00036-c36916ad-c89c-447f-9316-f8d4d50db387","output_cleared":false}},{"cell_type":"markdown","source":"### write your code here","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00025-83275989-6d6a-4643-877d-b0e4afff5331","output_cleared":false}},{"cell_type":"markdown","source":"### Visualize cluster samples in a 2-dimentional coordinate space","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00038-c701293d-34cc-4c7d-870a-3e2fba5c70b6","output_cleared":false}},{"cell_type":"code","metadata":{"deepnote_cell_type":"code","tags":[],"cell_id":"00033-24f69d46-67c6-4f99-b9a0-dd420d785a8f","output_cleared":true,"source_hash":null,"execution_millis":27666,"execution_start":1606220158873},"source":"from sklearn.manifold import TSNE\nimport seaborn as sns\ndef find_coords(X,assigned_clusterers,predicted_classes):\n    tsne = TSNE(n_components = 2)                      \n    X_tsne = tsne.fit_transform(X)\n    df_coords = pd.DataFrame(X_tsne , columns = ['x', 'y'] )\n\n    df_coords['clusters'] = assigned_clusterers\n    df_coords['pred_labels'] = predicted_classes\n    return df_coords\n        \n\nclasses  = data['category']\ndf_coords = find_coords(features,classes,kmeans_results)\n","execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"deepnote_cell_type":"code","tags":[],"cell_id":"00032-e754787f-da72-4fed-94f8-7db947638511","output_cleared":true,"source_hash":null,"execution_millis":832,"execution_start":1606219177825},"source":"sns.scatterplot(x =df_coords.x, y = df_coords.y, hue =df_coords.pred_labels ,palette=\"Set2\")","execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"----","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00029-bcbf712a-fa72-4276-abe2-3fd8f40ebbc5","output_cleared":false}},{"cell_type":"markdown","source":"### Topic modelling","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00036-52861d64-ff07-4a55-9aeb-3f8f89d80643","output_cleared":false}},{"cell_type":"markdown","source":"**Topic modelling** is an unsupervised machine learning technique (a family of algorithms) which provides a way to automatically organise, understand, search and summarise large amounts of text. ","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00036-55361b87-276f-402e-a75f-7b850c619047","output_cleared":false}},{"cell_type":"markdown","source":"What do we mean by **topics**?\nLet's imagine that we have a collection of 200 scientific papers, and we want to find out what themes this text collection covers. \n","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00036-bbf75314-3394-4cb1-9d95-ed3144d6da07","output_cleared":false}},{"cell_type":"markdown","source":"Let's also imaging that these 200 texts have (combined) **vocabulary** of 3000 (unique) words.","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00037-1e30a536-6299-4353-8b5c-2d8e458d554b","output_cleared":false}},{"cell_type":"markdown","source":"**Topic** is a probability disctibution over this vocabulary, which allows us to identify clusters of words that are more likely to occur together in **the same document**.","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00038-6fc6fa1c-a9d4-4f18-a097-7814a06969a2","output_cleared":false}},{"cell_type":"markdown","source":"Here we have our pile of papers. Each of them has words from the vocabulary. Topic modelling algorithsm are designed to take these documents as input (in the form of **bags of words**) and return as an output for each document two things: \n- all topics our corpus contains, each topic represented as a list of weighted words (e.g. \"yellow\" topic below: \"gene\" - probabilty 0.04, \"dna\" - probability 0.02....)\n- distribution of topics in each document (e.g. document below consists mostly of words from the \"yellow\" topic, has some words from \"pink\" and \"blue\" topics, and nothing from the \"green\" topic)\n","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00036-ca77b362-f0b1-486b-b6bb-51b5d99e4d83","output_cleared":false}},{"cell_type":"markdown","source":"![] (https://miro.medium.com/max/700/1*pZo_IcxW1GVuH2vQKdoIMQ.jpeg)","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00036-133cf0d8-a851-43e8-8db6-66c46496c7ea","output_cleared":false}},{"cell_type":"markdown","source":"It is an unsupervised model because the data is unlabelled. However, the algorithm does require setting up one parameter: number of topics. It is tricky because it is completely arbitrary. Usually it is set to 50-200 topics.  \n","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00036-c55df4a1-ef58-4594-bfe4-b3916e399037","output_cleared":false}},{"cell_type":"code","metadata":{"deepnote_cell_type":"code","tags":[],"cell_id":"00049-b88dcf4e-eea4-4b1c-a235-15b6f02143f0","output_cleared":true,"source_hash":null,"execution_millis":7860,"execution_start":1606219178661},"source":"!pip install pyLDAvis","execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Run the flllowing code applying topic modeling on the same data we used in the previous section (no need to change the code now).","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00050-67a60b81-bf63-4872-b690-d1490ebbd281"}},{"cell_type":"code","metadata":{"deepnote_cell_type":"code","tags":[],"cell_id":"00040-7f580057-40f1-4291-a373-664a6ef06d14","output_cleared":true,"source_hash":null,"execution_millis":61894,"execution_start":1606220199157},"source":"import nltk\nfrom gensim import corpora, models\nfrom nltk.corpus import stopwords\n\ntexts = data.text\n\ntokens = [text.split() for text in texts]\n\n\n#insert your code here\n\ndictionary_LDA = corpora.Dictionary(tokens)\nprint(dictionary_LDA)\ncorpus = [dictionary_LDA.doc2bow(list_of_tokens) for list_of_tokens in tokens]\n\nnum_topics = 5\nlda_model = models.LdaModel(corpus, num_topics=num_topics, \\\n                                  id2word=dictionary_LDA, \\\n                                  passes=4, alpha=[0.01]*num_topics, \\\n                                  eta=[0.01]*len(dictionary_LDA.keys()))\n\n\n%matplotlib inline\nimport pyLDAvis\nimport pyLDAvis.gensim\nvis = pyLDAvis.gensim.prepare(topic_model=lda_model, corpus=corpus, dictionary=dictionary_LDA)\npyLDAvis.enable_notebook()\npyLDAvis.display(vis)","execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Whaaaaat?!?!\n\nyes of course! We forgot to remove stop words!!","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00051-77dd44a2-7323-4441-a352-a97ad09390b2","output_cleared":false}},{"cell_type":"markdown","source":"**CODEIT (4)** fix the above code by removing stop words and see how the topics change.","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00052-c470d095-8e46-4178-a815-dbcde53346b2","output_cleared":false}},{"cell_type":"markdown","source":"**Observe and Reflect (5)** Can you still find a match between topics found by topic modeling method and the tagged categories? Explain which words related to each topic of the output you can associate with each cateogry.\n","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00053-6ee095d8-2173-4dce-baf5-37dd889ff160"}},{"cell_type":"markdown","source":"### write you answer here","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00054-3b8a8e70-b50a-40e9-a730-b364d0c106db"}},{"cell_type":"markdown","source":"## Topic modeling with Mallet","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00053-4e579424-9409-44ee-b24d-4f0d28afeb14","output_cleared":false}},{"cell_type":"markdown","source":"MALLET(MAchine Learning LangugE Toolkit) is a java-based tool intergating tools for some statistical machine learning applications.\n\n[This Demo](https://mimno.infosci.cornell.edu/jsLDA/jslda.html)  incorporates a web based MALLET topic model implementation.\n\n\n![](http://mallet.cs.umass.edu/logo3.png)","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00054-a751c00d-1968-4cc6-ab53-f1889272abf4","output_cleared":false}},{"cell_type":"markdown","source":"----","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00053-6e36addb-dcbd-4778-882b-4f7d9f730718","output_cleared":false}},{"cell_type":"markdown","source":"Download MALLET package [here](http://mallet.cs.umass.edu/download.php), compile it and use the follwoing wrapper code in python's gensim library to use it as a python library.\n\n**NOTE:** The following code will not run until you download the mallet package and compile it and put the executive file address in path_to_mallet variable.","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00058-5038cd6f-25eb-48b6-90d9-496172edad36"}},{"cell_type":"code","metadata":{"deepnote_cell_type":"code","tags":[],"cell_id":"00056-ac258d93-82dc-41a8-b2fe-df01fb6db318","output_cleared":true,"source_hash":null,"execution_millis":891,"execution_start":1606221165183},"source":"from gensim.models.wrappers import LdaMallet\nmodel = LdaMallet(path_to_mallet, corpus, num_topics=5, id2word=dictionary_LDA)","execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Use of topic modeling by historians:\n","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00058-1a1289a4-196e-44b5-9c59-739d20dbb15d"}},{"cell_type":"markdown","source":"https://dsl.richmond.edu/dispatch/\n\nhttp://www.cameronblevins.org/posts/topic-modeling-martha-ballards-diary/","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00060-cd1aecbe-9ca1-4e63-91c0-182be7198c71"}},{"cell_type":"markdown","source":"**Important note regarding Unsupervised Machine Learning: ** In the exercises above we used a dataset which was already labeled with categories and we could map the results of the method on the categories we already were aware of, however unsupervised machine learning approaches are useful when there is no labeled data. Thus, in such methods the number of topics/categories or category labels is set through trial and error.","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00063-d6d838e0-588f-454a-8f07-4192a03bba24"}},{"cell_type":"markdown","source":"----------\n","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00062-281c571e-c622-46c1-946b-0fd6b934dc31"}},{"cell_type":"markdown","source":"** Homework (6) ** Explain briefly what hierarchical clustering is. ","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00053-b4070e95-ef02-411f-a5e9-0b15abc32507","output_cleared":false}},{"cell_type":"markdown","source":"### Write your answer here","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00064-6af3ec41-bfa7-4c79-87fd-7c0176aab6b4"}},{"cell_type":"markdown","source":"#### References:\n\nhttps://www.kaggle.com/cempek/bbc-nltk-clustering-word2vec-vs-tfidfvectorizer\n\nhttps://towardsdatascience.com/a-friendly-introduction-to-text-clustering-fa996bcefd04\n\nhttp://mallet.cs.umass.edu/","metadata":{"deepnote_cell_type":"markdown","tags":[],"cell_id":"00041-feef7fe6-c41a-4537-88c9-c0040833a9e5","output_cleared":false}}],"nbformat":4,"nbformat_minor":2,"metadata":{"orig_nbformat":2,"deepnote_notebook_id":"18d43850-0d45-4c7f-b553-e7cb133d8164","deepnote_execution_queue":[]}}