Jekyll2022-01-06T14:25:55-06:00https://arjunsriva.com/feed.xmlArjun SrivastavaArjun Srivastava's home on the internet, I write about AI, Computer Science, Statistics, Investing.Experimentation Platforms2021-03-21T00:00:00-05:002021-03-21T00:00:00-05:00https://arjunsriva.com/experimentation/2021/03/21/exerimentations-platforms<h2 id="books">Books</h2>
<ul>
<li><a href="https://www.arjunsriva.com/library/book_1/book_1183.html">Trustworthy Online Controlled Experiments</a></li>
</ul>
<h2 id="resources">Resources</h2>
<ul>
<li>
<p><a href="https://research.fb.com/wp-content/uploads/2019/06/Top-Challenges-from-the-first-Practical-Online-Controlled-Experiments-Summit.pdf">Top Challenges from the first Practical Online Controlled Experiments Summit</a></p>
</li>
<li>
<p><a href="https://www.youtube.com/watch?v=HEGI5QN3fXE">A/B Testing Pitfalls: Getting Numbers You Can Trust is Hard</a></p>
</li>
<li>
<p><a href="https://www.youtube.com/watch?v=wT3dBSVGgjA">USF Business Analytics Forum - Ron Kohavi</a></p>
</li>
<li>
<p><a href="http://videolectures.net/kdd2017_tutorial15_software_innovation/">A/B Testing at Scale: Accelerating Software Innovation </a></p>
</li>
<li>
<p><a href="https://www.youtube.com/watch?v=kTAFOCynWIg">Trustworthy Online Controlled Experiments at Large Scale </a></p>
</li>
<li>
<p><a href="https://www.youtube.com/watch?v=BanBrr3Hzm8">Always Valid Inference: Continuous Monitoring of A/B Tests</a></p>
</li>
</ul>
<h2 id="companies">Companies</h2>
<h4 id="netflix">Netflix</h4>
<ul>
<li><a href="https://netflixtechblog.com/tagged/experimentation">Netflix Articles tagged Experimentation</a></li>
<li><a href="https://netflixtechblog.com/its-all-a-bout-testing-the-netflix-experimentation-platform-4e1ca458c15">It’s All A/Bout Testing: The Netflix Experimentation Platform</a></li>
<li><a href="https://netflixtechblog.com/reimagining-experimentation-analysis-at-netflix-71356393af21">Reimagining Experimentation Analysis at Netflix</a></li>
<li><a href="https://drive.google.com/file/d/1TjTkgSzGQXtFxro5vdInUUIpsmJPdCiO/view">Success stories from a democratized experimentation platform</a></li>
<li><a href="https://netflixtechblog.com/key-challenges-with-quasi-experiments-at-netflix-89b4f234b852">Key Challenges with Quasi Experiments at Netflix</a></li>
<li><a href="https://netflixtechblog.com/data-compression-for-large-scale-streaming-experimentation-c20bfab8b9ce">Data Compression for Large-Scale Streaming Experimentation</a></li>
<li>
<p><a href="https://netflixtechblog.com/page-simulator-fa02069fb269">Page Simulation for Better Offline Metrics at Netflix</a></p>
</li>
<li><a href="https://netflixtechblog.com/streaming-video-experimentation-at-netflix-visualizing-practical-and-statistical-significance-7117420f4e9a">Streaming Video Experimentation at Netflix: Visualizing Practical and Statistical Significance</a></li>
<li><a href="https://netflixtechblog.com/interleaving-in-online-experiments-at-netflix-a04ee392ec55">Innovating Faster on Personalization Algorithms at Netflix Using Interleaving</a></li>
</ul>
<h4 id="microsoft">Microsoft</h4>
<ul>
<li><a href="https://exp-platform.com/">ExP Experimentation Platform Accelerating software innovation through trustworthy experimentation</a></li>
<li><a href="https://ai.stanford.edu/~ronnyk/ExPThinkWeek2009Public.pdf">Online Experimentation at Microsoft</a></li>
<li><a href="https://www.microsoft.com/en-us/research/group/experimentation-platform-exp/">Experimentation Platform</a></li>
<li><a href="https://www.microsoft.com/en-us/research/group/experimentation-platform-exp/articles/a-b-testing-and-covid-19-data-driven-decisions-in-times-of-uncertainty/">A/B Testing and Covid-19: Data-Driven Decisions in Times of Uncertainty</a></li>
<li><a href="https://www.microsoft.com/en-us/research/group/experimentation-platform-exp/articles/patterns-of-trustworthy-experimentation-pre-experiment-stage/">Patterns of Trustworthy Experimentation: Pre-Experiment Stage</a></li>
</ul>
<h4 id="twitter">Twitter</h4>
<ul>
<li><a href="https://blog.twitter.com/engineering/en_us/a/2015/twitter-experimentation-technical-overview.html">Twitter experimentation: technical overview</a></li>
</ul>
<h4 id="google">Google</h4>
<ul>
<li><a href="https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/36500.pdf">Overlapping Experiment Infrastructure: More, Better, Faster Experimentation</a></li>
</ul>
<h4 id="facebook">Facebook</h4>
<ul>
<li><a href="http://facebook.github.io/planout/">PlanOut is a library and interpreter for designing online experiments.</a></li>
<li><a href="https://github.com/facebook/Ax">Adaptive Experimentation Platform</a></li>
</ul>
<h4 id="spotify">Spotify</h4>
<ul>
<li><a href="https://engineering.atspotify.com/2020/10/29/spotifys-new-experimentation-platform-part-1/">Spotify’s New Experimentation Platform part 1</a></li>
<li><a href="https://engineering.atspotify.com/2020/11/02/spotifys-new-experimentation-platform-part-2/">Spotify’s New Experimentation Platform part 2</a></li>
<li><a href="https://www.infoq.com/news/2016/12/large-experimentation-spotify/">Large Scale Experimentation at Spotify</a></li>
</ul>
<h4 id="tinder">Tinder</h4>
<ul>
<li><a href="https://medium.com/tinder-engineering/phoenix-tinders-testing-platform-part-i-8e51c19ca977">Phoenix — Tinder’s Testing Platform, Part — I</a></li>
<li><a href="https://medium.com/tinder-engineering/phoenix-tinders-testing-platform-part-ii-e99fbc72af55">Phoenix — Tinder’s Testing Platform — Part II</a></li>
<li><a href="https://medium.com/tinder-engineering/phoenix-tinders-testing-platform-part-iii-520728b9537">Phoenix — Tinder’s Testing Platform — Part III</a></li>
</ul>
<h4 id="linkedin">LinkedIn</h4>
<ul>
<li><a href="https://engineering.linkedin.com/blog/2020/our-evolution-towards-t-rex--the-prehistory-of-experimentation-i">Our evolution towards T-REX: The prehistory of experimentation infrastructure at LinkedIn</a></li>
<li><a href="https://engineering.linkedin.com/blog/2020/making-the-linkedin-experimentation-engine-20x-faster">Making the LinkedIn experimentation engine 20x faster</a></li>
</ul>
<h4 id="uber">Uber</h4>
<ul>
<li><a href="https://eng.uber.com/xp/">Under the Hood of Uber’s Experimentation Platform</a></li>
<li><a href="https://conferences.oreilly.com/strata/strata-ny-2018/public/schedule/detail/69126.html">A/B testing at Uber: How we built a BYOM (bring your own metrics) platform</a></li>
<li><a href="https://eng.uber.com/experimentation-platform/">Building an Intelligent Experimentation Platform with Uber Engineering</a></li>
</ul>
<h4 id="airbnb">AirBnB</h4>
<ul>
<li><a href="https://medium.com/airbnb-engineering/4-principles-for-making-experimentation-count-7a5f1a5268a">4 Principles for Making Experimentation Count</a></li>
<li><a href="https://medium.com/airbnb-engineering/https-medium-com-jonathan-parks-scaling-erf-23fd17c91166">Scaling Airbnb’s Experimentation Platform</a></li>
<li><a href="https://medium.com/airbnb-engineering/experiment-reporting-framework-4e3fcd29e6c0#.cbl72jip2">Experiment Reporting Framework</a></li>
</ul>
<h4 id="instagram">Instagram</h4>
<ul>
<li><a href="https://instagram-engineering.com/lessons-learned-at-instagram-stories-and-feed-machine-learning-54f3aaa09e56">Lessons Learned at Instagram Stories and Feed Machine Learning</a></li>
</ul>
<h4 id="go-jek">Go-Jek</h4>
<ul>
<li><a href="https://blog.gojekengineering.com/introducing-litmus-gojeks-own-experimentation-platform-3803467b6a53">Introducing Litmus: GOJEK’s Own Experimentation Platform</a></li>
</ul>
<h4 id="instacart">InstaCart</h4>
<ul>
<li><a href="https://tech.instacart.com/it-all-depends-4bb7b22e854b">Randomized, controlled experiments and multivariate regression are used to continuously improve the grocery delivery engine </a></li>
</ul>
<h4 id="pintrest">Pintrest</h4>
<ul>
<li><a href="https://medium.com/pinterest-engineering/building-pinterests-a-b-testing-platform-ab4934ace9f4">Building Pinterest’s A/B testing platform</a></li>
</ul>
<h2 id="conferences">Conferences</h2>
<ul>
<li><a href="https://experimentationcultureawards.com/#register">Experimentation Culture Awards</a></li>
</ul>
<h2 id="sass-solutions">SASS solutions</h2>
<ul>
<li><a href="https://www.split.io/">Split.io</a></li>
<li><a href="https://www.optimizely.com/">Optimizely</a></li>
</ul>
<h2 id="when-you-cant-run-ab-tests">When you can’t run A/B Tests</h2>
<ul>
<li><a href="https://netflixtechblog.com/quasi-experimentation-at-netflix-566b57d2e362">Quasi Experimentation at Netflix</a></li>
<li><a href="https://netflixtechblog.com/key-challenges-with-quasi-experiments-at-netflix-89b4f234b852">Key Challenges with Quasi Experiments at Netflix</a></li>
<li><a href="https://www.arjunsriva.com/library/book_1/book_1176.html">Mostly Harmless Econometrics: An Empiricist’s Companion</a></li>
</ul>
<h2 id="notes">Notes</h2>
<h3 id="2020-codemit-experimentation-platforms"><a href="https://youtu.be/8yLC5oBP9fI">2020 CODE@MIT Experimentation platforms</a></h3>
<ul>
<li>Gaussian processes</li>
<li>Multi touch attributions</li>
<li>Heterogenous treatment effect</li>
<li>Interaction effects</li>
<li>Overlapping experiments</li>
<li>What are potential over evaluation criteria ?</li>
<li>What are good guardrail metrics ?</li>
<li>run A/A tests.</li>
</ul>
<p>Canary Deploys by using Experimentation platform to tell when you break guard rail metrics</p>BooksCourse Schedule Generator2020-01-08T00:00:00-06:002020-01-08T00:00:00-06:00https://arjunsriva.com/iit%20indore/2020/01/08/course-schedule-generator<p>A website which allows students of IIT Indore to add courses they are interested in to their calendar.<br />
I was tired of doing this manually every time.<br />
Use it <a href="https://arjunbazinga.github.io/course-schedule-generator/">here</a></p>A website which allows students of IIT Indore to add courses they are interested in to their calendar. I was tired of doing this manually every time. Use it hereHarry Potter quiz2018-08-26T00:00:00-05:002018-08-26T00:00:00-05:00https://arjunsriva.com/iit%20indore/quiz/2018/08/26/harry-potter-quiz<h3 id="questions">Questions</h3>
<p>Get the <a href="/static/harry-potter-quiz/Questions.pdf">questions.</a></p>
<p>Get the <a href="https://youtu.be/gskB-IyGD6I">video for question number 14</a></p>
<p>Get the <a href="/static/harry-potter-quiz/music.mp3"> Music for question number 20</a></p>
<h3 id="answers">Answers</h3>
<p>Get the <a href="/static/harry-potter-quiz/PotterMania-Answers.pdf">answers.</a></p>QuestionsThe state of reading in 2018 and beyond.2018-08-05T00:00:00-05:002018-08-05T00:00:00-05:00https://arjunsriva.com/future/2018/08/05/state-of-reading<p>Audio books are seeing a resurgence, because of companies like Audible.<br />
Text to speech is good, and is getting <a href="https://deepmind.com/blog/article/wavenet-generative-model-raw-audio">better</a> .<br />
Wireless Bluetooth <a href="https://www.apple.com/airpods/">earphones</a> are becoming common place, the increase in convenience and battery life allows people to have them on longer.<br />
Unlike content that you have to watch/read, you can work on other stuff while you listen.<br />
I see a time where we can listen to most of the things we read.<br />
Creates a whole new kind of medium.<br />
News, blog posts and Stories most affected.<br />
Most of the speech generated by machines.<br />
Most of the classic texts available for free on the <a href="http://libgen.is/">internet</a>.</p>Audio books are seeing a resurgence, because of companies like Audible. Text to speech is good, and is getting better . Wireless Bluetooth earphones are becoming common place, the increase in convenience and battery life allows people to have them on longer. Unlike content that you have to watch/read, you can work on other stuff while you listen. I see a time where we can listen to most of the things we read. Creates a whole new kind of medium. News, blog posts and Stories most affected. Most of the speech generated by machines. Most of the classic texts available for free on the internet.Friend Or Foe2018-07-28T00:00:00-05:002018-07-28T00:00:00-05:00https://arjunsriva.com/ai/projects/2018/07/28/friend-or-foe<p>This is a mini interactive “game”, I made inspired by the paper
<a href="https://arxiv.org/abs/1711.09883">AI Safety Grid Worlds</a>,
without giving too much away, I urge you to <a href="/fof">play</a>.</p>This is a mini interactive “game”, I made inspired by the paper AI Safety Grid Worlds, without giving too much away, I urge you to play.Infant Mortality In India2018-06-15T00:00:00-05:002018-06-15T00:00:00-05:00https://arjunsriva.com/analysis/2018/06/15/infant-mortality<p>read the analysis <a href="/infantmortality">here</a>.</p>read the analysis here.Jargonizer2018-06-12T00:00:00-05:002018-06-12T00:00:00-05:00https://arjunsriva.com/jargonizer<p>This was a fun project, the goal was to create a website which lets you complicate your sentences, there are plenty of websites online trying to simplify but none that go the other way.</p>
<p>play with it <a href="https://arjunbazinga.github.io/jargonizer">here</a></p>This was a fun project, the goal was to create a website which lets you complicate your sentences, there are plenty of websites online trying to simplify but none that go the other way.Multi Armed Bandit Problem2017-11-12T00:00:00-06:002017-11-12T00:00:00-06:00https://arjunsriva.com/ai/2017/11/12/multi-armed-bandits<!--
#################################################
### THIS FILE WAS AUTOGENERATED! DO NOT EDIT! ###
#################################################
# file to edit: _notebooks/2017-11-12-multi-armed-bandits.ipynb
-->
<div class="container" id="notebook-container">
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="Problem-Description">Problem Description<a class="anchor-link" href="#Problem-Description"> </a></h2><p>Imagine you are at a casino, and you have N slot machines to play, each slot machine gives rewards according to a fixed probability distribution. What strategy should you play with to maximise your total reward ?</p>
<p>This problem is known as <a href="https://en.wikipedia.org/wiki/Multi-armed_bandit">Multi Armed Bandit</a> problem.</p>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span>
<span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="o">%</span><span class="k">matplotlib</span> inline
</pre></div>
</div>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="Arms">Arms<a class="anchor-link" href="#Arms"> </a></h2><p>An arm when pulled, gives a random number from a normal distribution with fixed mean(mu) and deviation(sigma).
When pulled many times the frequency of the rewards look like this:</p>
<p><img src="https://upload.wikimedia.org/wikipedia/commons/thumb/7/74/Normal_Distribution_PDF.svg/350px-Normal_Distribution_PDF.svg.png" alt="normal distribution" />
X axis is the magnitude of reward<br />
Y axis is it's frequency.</p>
<p>The Arm class provides an arm with these properties.</p>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="k">class</span> <span class="nc">Arm</span><span class="p">:</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">mu</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span> <span class="n">sigma</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
<span class="k">if</span> <span class="n">mu</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">mu</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">absolute</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">uniform</span><span class="p">())</span>
<span class="k">else</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">mu</span> <span class="o">=</span> <span class="n">mu</span>
<span class="k">if</span> <span class="n">sigma</span> <span class="ow">is</span> <span class="kc">None</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">sigma</span><span class="o">=</span><span class="n">np</span><span class="o">.</span><span class="n">absolute</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">uniform</span><span class="p">())</span>
<span class="k">else</span><span class="p">:</span>
<span class="bp">self</span><span class="o">.</span><span class="n">sigma</span> <span class="o">=</span> <span class="n">sigma</span>
<span class="k">def</span> <span class="nf">pull</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">reward</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">normal</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">mu</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">sigma</span><span class="p">,</span> <span class="mi">1</span><span class="p">)</span>
<span class="k">return</span> <span class="n">reward</span>
<span class="k">def</span> <span class="nf">get_arms</span><span class="p">(</span><span class="n">k</span><span class="p">):</span>
<span class="c1"># returns a list of arms</span>
<span class="n">arms</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">k</span><span class="p">):</span>
<span class="n">arms</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">Arm</span><span class="p">())</span>
<span class="k">return</span> <span class="n">arms</span>
</pre></div>
</div>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="Agents">Agents<a class="anchor-link" href="#Agents"> </a></h2><p>An agent here is a player who pulls arms to play.
It has a policy, which is a list of probabilities associated with each arm.</p>
<p>The agent class makes designing agents <strong>fast</strong>. The object is initialised with arms and whether it should play all arms once as part of the initialisation.</p>
<p>Features provided by this class:<br />
Attributes:</p>
<ul>
<li>expectations[i]: gives the expected reward on playing arm[i]</li>
<li>times_played[i]: gives the number of times the agent has played arm[i]</li>
<li>N = Total number of times agent has played</li>
<li>reward_history : list of rewards earned by the agent</li>
<li>choice_history : list of choices made by the agent </li>
</ul>
<p>Methods:</p>
<ul>
<li>gamble(i): Plays for i iterations while updating it's policy.</li>
<li>play(i): Pulls arm[i] and updates reward_history, N , times_played</li>
<li>select_arm(): returns index of an arm by sampling probability distribution given by the policy</li>
</ul>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="k">class</span> <span class="nc">agent</span><span class="p">:</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">arms</span><span class="p">,</span> <span class="n">play_once</span><span class="o">=</span><span class="mi">1</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">expectations</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">zeros</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">arms</span><span class="p">))</span>
<span class="bp">self</span><span class="o">.</span><span class="n">times_played</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">zeros</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">arms</span><span class="p">))</span>
<span class="bp">self</span><span class="o">.</span><span class="n">arms</span> <span class="o">=</span> <span class="n">arms</span>
<span class="bp">self</span><span class="o">.</span><span class="n">number_of_arms</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">arms</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">N</span> <span class="o">=</span> <span class="mi">0</span>
<span class="bp">self</span><span class="o">.</span><span class="n">reward_history</span> <span class="o">=</span> <span class="p">[]</span>
<span class="bp">self</span><span class="o">.</span><span class="n">choice_history</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">if</span> <span class="n">play_once</span> <span class="o">==</span> <span class="mi">1</span><span class="p">:</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">number_of_arms</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">expectations</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">play</span><span class="p">(</span><span class="n">i</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">play</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">index</span><span class="p">):</span>
<span class="n">reward</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">arms</span><span class="p">[</span><span class="n">index</span><span class="p">]</span><span class="o">.</span><span class="n">pull</span><span class="p">()</span>
<span class="bp">self</span><span class="o">.</span><span class="n">times_played</span><span class="p">[</span><span class="n">index</span><span class="p">]</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="bp">self</span><span class="o">.</span><span class="n">N</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="bp">self</span><span class="o">.</span><span class="n">choice_history</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">index</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">reward_history</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">reward</span><span class="p">)</span>
<span class="k">return</span> <span class="n">reward</span>
<span class="k">def</span> <span class="nf">policy</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">pass</span>
<span class="k">def</span> <span class="nf">update_expectations</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">reward</span><span class="p">,</span> <span class="n">index</span><span class="p">):</span>
<span class="bp">self</span><span class="o">.</span><span class="n">expectations</span><span class="p">[</span><span class="n">index</span><span class="p">]</span> <span class="o">+=</span> <span class="p">(</span><span class="n">reward</span> <span class="o">-</span> <span class="bp">self</span><span class="o">.</span><span class="n">expectations</span><span class="p">[</span><span class="n">index</span><span class="p">])</span><span class="o">/</span><span class="bp">self</span><span class="o">.</span><span class="n">N</span>
<span class="k">def</span> <span class="nf">select_arm</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">options</span> <span class="o">=</span> <span class="nb">range</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">number_of_arms</span><span class="p">)</span>
<span class="n">i</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">choice</span><span class="p">(</span><span class="n">options</span><span class="p">,</span> <span class="n">p</span><span class="o">=</span><span class="bp">self</span><span class="o">.</span><span class="n">policy</span><span class="p">(),</span> <span class="n">replace</span><span class="o">=</span><span class="kc">False</span><span class="p">)</span>
<span class="k">return</span> <span class="n">i</span>
<span class="k">def</span> <span class="nf">gamble</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">iterations</span><span class="p">):</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">iterations</span><span class="p">):</span>
<span class="n">index</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">select_arm</span><span class="p">()</span>
<span class="n">reward</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">play</span><span class="p">(</span><span class="n">index</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">update_expectations</span><span class="p">(</span><span class="n">reward</span><span class="p">,</span> <span class="n">index</span><span class="p">)</span>
</pre></div>
</div>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h3 id="Example-agents">Example agents<a class="anchor-link" href="#Example-agents"> </a></h3><p>To make a new agent we <a href="">inherit</a> the agent class.</p>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Time to make some agents!</p>
<h3 id="First-up:-epsilon-greedy">First up: epsilon-greedy<a class="anchor-link" href="#First-up:-epsilon-greedy"> </a></h3><p>This agent plays the arm with the highest expected reward with <em>1 - epsilon</em> probability, and plays a random arm with <em>epsilon</em> probability</p>
<p>So<br />
epsilon = 1 => random choices<br />
epsilon = 0 => greedy choices</p>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="k">class</span> <span class="nc">epsilon_greedy</span><span class="p">(</span><span class="n">agent</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">arms</span><span class="p">,</span> <span class="n">play_once</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">epsilon</span><span class="o">=</span><span class="mf">0.1</span><span class="p">):</span>
<span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="n">arms</span><span class="p">,</span> <span class="n">play_once</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">epsilon</span> <span class="o">=</span> <span class="n">epsilon</span>
<span class="k">def</span> <span class="fm">__str__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="s2">"Epsilon-Greedy Agent, epsilon= "</span><span class="o">+</span><span class="nb">str</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">epsilon</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">policy</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">temp</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">zeros_like</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">expectations</span><span class="p">)</span>
<span class="n">temp</span><span class="p">[</span><span class="n">np</span><span class="o">.</span><span class="n">argmax</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">expectations</span><span class="p">)]</span> <span class="o">=</span> <span class="mi">1</span><span class="o">-</span><span class="bp">self</span><span class="o">.</span><span class="n">epsilon</span>
<span class="n">ans</span> <span class="o">=</span> <span class="n">temp</span> <span class="o">+</span> <span class="bp">self</span><span class="o">.</span><span class="n">epsilon</span><span class="o">/</span><span class="bp">self</span><span class="o">.</span><span class="n">number_of_arms</span>
<span class="k">return</span> <span class="n">ans</span>
</pre></div>
</div>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h3 id="Beta-Softmax">Beta-Softmax<a class="anchor-link" href="#Beta-Softmax"> </a></h3><p>This agent plays an arm[i] with probability proportional to: e^(expected_reward(arm[i])/beta)<br />
We normalise the whole thing by the sum over all the arms.</p>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="k">class</span> <span class="nc">softmax</span><span class="p">(</span><span class="n">agent</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">arms</span><span class="p">,</span> <span class="n">play_once</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">beta</span><span class="o">=</span><span class="mi">1</span><span class="p">):</span>
<span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="n">arms</span><span class="p">,</span> <span class="n">play_once</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">beta</span> <span class="o">=</span> <span class="n">beta</span>
<span class="k">def</span> <span class="fm">__str__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="s2">"Softmax agent, beta= "</span><span class="o">+</span> <span class="nb">str</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">beta</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">policy</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">temp</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">exp</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">expectations</span><span class="o">/</span><span class="bp">self</span><span class="o">.</span><span class="n">beta</span><span class="p">)</span>
<span class="n">ans</span> <span class="o">=</span> <span class="n">temp</span> <span class="o">/</span> <span class="n">np</span><span class="o">.</span><span class="n">sum</span><span class="p">(</span><span class="n">temp</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span>
<span class="k">return</span> <span class="n">ans</span>
</pre></div>
</div>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h3 id="Upper-Confidence-Bound-(UCB1)">Upper Confidence Bound (UCB1)<a class="anchor-link" href="#Upper-Confidence-Bound-(UCB1)"> </a></h3><p>UCB1 agent plays the arm with the highest metric, where metric of arm i is :
metric[i] = expected_reward[i] + sqrt(2*log(N)/times_played[i])</p>
<p><strong>Note</strong> Best peformance when rewards are between 0 and 1</p>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="k">class</span> <span class="nc">ucb</span><span class="p">(</span><span class="n">agent</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">arms</span><span class="p">,</span> <span class="n">play_once</span><span class="o">=</span><span class="mi">1</span><span class="p">):</span>
<span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="n">arms</span><span class="p">,</span> <span class="n">play_once</span><span class="p">)</span>
<span class="k">def</span> <span class="fm">__str__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="s2">"UCB1 agent"</span>
<span class="k">def</span> <span class="nf">policy</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">temp</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">expectations</span> <span class="o">+</span> <span class="n">np</span><span class="o">.</span><span class="n">sqrt</span><span class="p">(</span><span class="mi">2</span><span class="o">*</span><span class="n">np</span><span class="o">.</span><span class="n">log</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">N</span><span class="p">)</span><span class="o">/</span><span class="bp">self</span><span class="o">.</span><span class="n">times_played</span><span class="p">)</span>
<span class="n">ans</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">zeros_like</span><span class="p">(</span><span class="n">temp</span><span class="p">)</span>
<span class="n">ans</span><span class="p">[</span><span class="n">np</span><span class="o">.</span><span class="n">argmax</span><span class="p">(</span><span class="n">temp</span><span class="p">)]</span> <span class="o">=</span> <span class="mi">1</span>
<span class="k">return</span> <span class="n">ans</span>
</pre></div>
</div>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="Metrics">Metrics<a class="anchor-link" href="#Metrics"> </a></h2><p>Metric : A scalar number, makes comparison easier.<br />
To compare the performance of our agents we can use these metrics</p>
<ul>
<li>avg_reward[i] : this gives the average reward till i+1 iteration.</li>
<li><p>max_reward : this tells us the maximum expected reward</p>
</li>
<li><p>euclid_distance : we can think of as learnt policy and optimal policy as vectors and compute the distance between them , smaller is better</p>
</li>
<li>cosine_simmilarity : compute the cos(q) between the policies. larger is better</li>
</ul>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="k">def</span> <span class="nf">maxreward</span><span class="p">(</span><span class="n">arms</span><span class="p">):</span>
<span class="c1">#Max rewards</span>
<span class="n">a</span><span class="o">=</span> <span class="p">[</span><span class="n">arm</span><span class="o">.</span><span class="n">mu</span> <span class="k">for</span> <span class="n">arm</span> <span class="ow">in</span> <span class="n">arms</span><span class="p">]</span>
<span class="k">return</span> <span class="nb">max</span><span class="p">(</span><span class="n">a</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">avg_reward</span><span class="p">(</span><span class="n">rewards</span><span class="p">):</span>
<span class="n">ans</span> <span class="o">=</span> <span class="p">[]</span>
<span class="n">ans</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">rewards</span><span class="p">[</span><span class="mi">0</span><span class="p">])</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span><span class="nb">len</span><span class="p">(</span><span class="n">rewards</span><span class="p">)):</span>
<span class="n">ans</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">ans</span><span class="p">[</span><span class="n">i</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span><span class="o">+</span><span class="n">rewards</span><span class="p">[</span><span class="n">i</span><span class="p">])</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">ans</span><span class="p">)):</span>
<span class="n">ans</span><span class="p">[</span><span class="n">i</span><span class="p">]</span><span class="o">/=</span><span class="n">i</span><span class="o">+</span><span class="mi">1</span>
<span class="k">return</span> <span class="n">ans</span>
<span class="k">def</span> <span class="nf">cosine_similarity</span><span class="p">(</span><span class="n">a</span><span class="p">,</span><span class="n">b</span><span class="p">):</span>
<span class="n">temp</span> <span class="o">=</span> <span class="n">a</span><span class="o">*</span><span class="n">b</span>
<span class="n">temp</span><span class="o">/=</span><span class="p">(</span><span class="n">euclid_distance</span><span class="p">(</span><span class="n">a</span><span class="p">)</span><span class="o">*</span> <span class="n">euclid_distance</span><span class="p">(</span><span class="n">b</span><span class="p">))</span>
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">sum</span><span class="p">(</span><span class="n">temp</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">euclid_distance</span><span class="p">(</span><span class="n">a</span><span class="p">):</span>
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="n">sqrt</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">sum</span><span class="p">(</span><span class="n">a</span><span class="o">*</span><span class="n">a</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">0</span><span class="p">))</span>
</pre></div>
</div>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h3 id="Test">Test<a class="anchor-link" href="#Test"> </a></h3><p>This function takes a list of agents and the number of iterations. Makes each agent play, and prints its metrics.</p>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="k">def</span> <span class="nf">test</span><span class="p">(</span><span class="n">agents</span><span class="p">,</span> <span class="n">iterations</span><span class="p">):</span>
<span class="k">for</span> <span class="n">agent</span> <span class="ow">in</span> <span class="n">agents</span><span class="p">:</span>
<span class="n">agent</span><span class="o">.</span><span class="n">gamble</span><span class="p">(</span><span class="n">iterations</span><span class="p">)</span>
<span class="n">temp</span> <span class="o">=</span> <span class="p">[</span> <span class="n">arm</span><span class="o">.</span><span class="n">mu</span> <span class="k">for</span> <span class="n">arm</span> <span class="ow">in</span> <span class="n">levers</span><span class="p">]</span>
<span class="n">optimal_policy</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">zeros_like</span><span class="p">(</span><span class="n">agent</span><span class="o">.</span><span class="n">expectations</span><span class="p">)</span>
<span class="n">optimal_policy</span><span class="p">[</span><span class="n">temp</span><span class="o">.</span><span class="n">index</span><span class="p">(</span><span class="nb">max</span><span class="p">(</span><span class="n">temp</span><span class="p">))]</span> <span class="o">=</span> <span class="mi">1</span>
<span class="n">avg_rewards_earned</span> <span class="o">=</span> <span class="n">avg_reward</span><span class="p">(</span><span class="n">agent</span><span class="o">.</span><span class="n">reward_history</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="n">agent</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">"maximum possible reward:"</span><span class="p">,</span> <span class="n">maxreward</span><span class="p">(</span><span class="n">levers</span><span class="p">))</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">"average reward:"</span><span class="p">,</span> <span class="n">avg_rewards_earned</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">])</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">"cosine similarity"</span> <span class="p">,</span><span class="n">cosine_similarity</span><span class="p">(</span><span class="n">agent</span><span class="o">.</span><span class="n">policy</span><span class="p">(),</span> <span class="n">optimal_policy</span><span class="p">))</span>
<span class="n">euclid_norm</span> <span class="o">=</span> <span class="n">euclid_distance</span><span class="p">(</span><span class="n">agent</span><span class="o">.</span><span class="n">policy</span><span class="p">()</span><span class="o">-</span><span class="n">optimal_policy</span><span class="p">)</span><span class="o">/</span><span class="nb">len</span><span class="p">(</span><span class="n">optimal_policy</span><span class="p">)</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">"euclidian norm "</span><span class="p">,</span><span class="n">euclid_norm</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">avg_rewards_earned</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">ylabel</span><span class="p">(</span><span class="s1">'Average Reward'</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">xlabel</span><span class="p">(</span><span class="s1">'Iteration'</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">show</span><span class="p">()</span>
<span class="nb">print</span><span class="p">(</span><span class="s2">"</span><span class="se">\n</span><span class="s2">"</span><span class="p">)</span>
<span class="c1"># print("optimal policy:" , optimal)</span>
<span class="c1"># print("learnt policy:" ,agent.policy())</span>
<span class="c1"># plt.scatter(range(len(agent.choice_history)),y=agent.choice_history)</span>
<span class="c1"># plt.title("Choices")</span>
<span class="c1"># plt.xlabel("time")</span>
<span class="c1"># plt.ylabel("arm")</span>
<span class="c1"># plt.show()</span>
<span class="c1"># print("\n")</span>
</pre></div>
</div>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">levers</span> <span class="o">=</span> <span class="n">get_arms</span><span class="p">(</span><span class="mi">10</span><span class="p">)</span>
<span class="n">agents</span> <span class="o">=</span> <span class="p">[</span>
<span class="n">epsilon_greedy</span><span class="p">(</span><span class="n">levers</span><span class="p">,</span> <span class="n">epsilon</span><span class="o">=</span><span class="mi">1</span><span class="p">),</span>
<span class="n">epsilon_greedy</span><span class="p">(</span><span class="n">levers</span><span class="p">,</span> <span class="n">epsilon</span><span class="o">=</span><span class="mi">0</span><span class="p">),</span>
<span class="n">softmax</span><span class="p">(</span><span class="n">levers</span><span class="p">,</span> <span class="n">beta</span><span class="o">=</span><span class="mf">0.1</span><span class="p">),</span>
<span class="n">ucb</span><span class="p">(</span><span class="n">levers</span><span class="p">)</span>
<span class="p">]</span>
</pre></div>
</div>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">plt</span><span class="o">.</span><span class="n">plot</span><span class="p">([</span> <span class="n">arm</span><span class="o">.</span><span class="n">mu</span> <span class="k">for</span> <span class="n">arm</span> <span class="ow">in</span> <span class="n">levers</span><span class="p">]</span> <span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">title</span><span class="p">(</span><span class="s2">"distribution of expected value of arms"</span><span class="p">)</span>
</pre></div>
</div>
</div>
</div>
<div class="output_wrapper">
<div class="output">
<div class="output_area">
<div class="output_text output_subarea output_execute_result">
<pre>Text(0.5, 1.0, 'distribution of expected value of arms')</pre>
</div>
</div>
<div class="output_area">
<div class="output_png output_subarea ">
<img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAXQAAAEICAYAAABPgw/pAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dd3xU15nw8d+j3gVCBUn0IgECFNsY3C3ibiM7iVNspzmbTfZ9symbYidO8nrTnLbZZLNJNmU3ZVOM45Zd3OLYMTIOLoAxRQIkusoIFUAzkkD9vH/cO2IYq4ykGd0pz/fz4YNm7p17H12NHp0557nniDEGpZRSkS/O6QCUUkoFhyZ0pZSKEprQlVIqSmhCV0qpKKEJXSmlooQmdKWUihKa0ENIRH4jIt+wv75SRGqDeOxnROSD9td3i8jfgnjs94rIX4J1vAmc93IROSgiXSLytuk+/3QTka+IyO9DcNzh9910EpG3i0iD/fO7YLrPrzShTxtjzEvGmNLx9gv0l9wYc5Mx5r+nGpeILBARIyIJPsf+gzHm+qkeexK+BvzYGJNhjPkfB84/ISJyTESudTqOMPI94OP2z+8Np4OJRZrQI4xYovXnNh+ocToINWlB+/n5NjBU4KI1MThCRC4QkZ0i0ikifwRSfLZViEijz+PPi0iTvW+tiFwjIjcCXwTeY39s3W3vWyUiD4jIVuAMsMh+7u/PP738WETcInJARK7x2XBeS9LvU8AW+/8O+5yX+nfhiMhlIrLdPvZ2EbnMZ1uViHxdRLba38tfRCR3jGv0ERE5JCKnRGSTiBTZzx8GFgFP2HEkj/DaIhF5TETaROSoiHzSZ9vTIvKvPo8fEpFf2V/fbcc32vXJFpFfikiz/TP5hojE+8W83/7+9onIhSLyO2CeT7z32vteIiIvi0iHiOwWkQqf4ywUkRft4zwHjHWd9ovIBp/HCfb3faH9+BEROWF/P1tEpGyU47ypO87+RLbE/jpZRL4nIvUi0iIiPxOR1FGOFSciXxaR4yLSKiK/ta9dsoh0AfHAbvtnOdLrfyhWl4xHRF4XkSt9tn1FRB4Vkd+LiAe4237uEfu5ThHZKyIlInKfff4GEbne5xh3i8gRe9+jIvLe0a5v1DLG6L8g/AOSgOPAp4FE4J1AP/ANe3sF0Gh/XQo0AEX24wXAYvvrrwC/9zt2FVAPlAEJ9vGrgL+3t98NDPic+z2AG8ixtx8DrvU53vA57HMbIMFn+93A3+yvc4DTwPvtc99pP57lE9thoARItR9/e5Rr9FagHbgQSAZ+BGzx2X5enH6vjQNeB+63r/Ui4Ahwg719NtBqn+O99rbMAK/Pn4CfA+lAPrAN+Ad727uAJuBiQIAlwPxRrmsxcBK42Y73Ovtxnr39FeD79vd+FdDp/7P2Odb9wB98Ht8C7Pd5/HdApn2sfwN2+Wz7Defed8M/S5/tBlhif/0DYJP9c84EngC+NUpMfwccsq99BvA48LuRjjvK698HzLLfR58FTgApPu/JfuBt9rVLtZ/rAW6wX/Nb4CjwJfvn+BHgqP36dMADlNqPC4Eyp/PCdP9zPIBo+Wf/groA8XnuZUZO6Euwks+1QKLfcb7i/0uOlSS/NsJzvgnd/9zbgPfbX/snnuFzMH5Cfz+wze/crwB3+8TxZZ9tHwP+PMo1+iXwXZ/HGfYv8YKR4vR77Tqg3u+5+4Bf+zy+HesPZTtwhd/3M+L1AQqAXiDVZ9udwGb762eBT40Sk/91/Tw+Cc7n9R/Eas0PAOk+2x70/1n7bFuClfDT7Md/AO4fZd8Z9s8w2378GwJI6Fh/oLqxGxP2tkuxk+QI5/kr8DGfx6X2zy/B97gT+J05DZT7vCe3+G3/CvCcz+NKoAuItx9n2uecgZXQO+z3QGqgMUTbP+1yCZ4ioMnY7zTb8ZF2NMYcAv4J6w3bancPFI1z/IZxto907vGOGYgi3vx9HMdqjXqd8Pn6DFaiHvdYxpgurBZs8Sj7+5oPFNldGR0i0oHVPVXgs88TWB/7a40x/lU/o12f+VitvWaf4/4cq6UOMBfrE0gg5gPv8ovxCqzWYhFw2hjT7RfDiOz3yH6gUkTSgFux/gAgIvEi8m0ROWx3TxyzXzZqF84o8oA04HWfeP9sPz8S//fCcayWc8HIu59PRD5ndyW57XNl+8U80nu8xefrs0C7MWbQ5zFAhn1d3wP8H6yf5VMisiyQuKKJJvTgaQaKRUR8nps32s7GmAeNMVdgJQEDfMe7abSXjHP+kc7tsr/uxvrF9Zo9geO67Bh9zcPqhpio844lIulYH8EDOVYDVstxhs+/TGPMzT77PICVBAtF5E6/1492fRqwWui5PsfNMsZ4+6QbgMWjxOR/7RqwWui+MaYbY76N9f6YaX/PvjGMZSPWp4XbgH12kge4y37uWqykuMB+XvwPgN/PXkR8f/btWEmxzCfebGPMaH+Q/d8L3k8dLSPvfo7dX34v8G5gpjFmBla3l2/MU5r61RjzrDHmOqw/oAeA/5zK8SKRJvTgeQXrzf1JEUkUkXcAa0faUURKReStYg389WD9Ug3Zm1uABTLxSpZ8n3O/C1gOPG1v2wXcYW9bg9W/79Vmn3vRKMd9GigRkbvsgbn3ACuAJycYH1gJ6kMi8hb7e/8m8Jox5lgAr90GdIo1mJxqt1JXisjFACJyFfAh4ANYXRw/EhHflv+I18cY0wz8BfhXEcmyB/4Wi8jV9uv+C/iciFwkliUi4k1qLZx/3X6P1aK+wY4vRazB8DnGmOPADuCrIpIkIldgdSGM5SHgeuD/YrfObZlYf4ROYiXrb45xjN1AmX3NU7A+FQJgjBnCSno/EJF8+zoWi8gNoxxrI/BpsQZ3M+zz/tEYMzDO9+GNeQDr/ZYgIvcDWQG8LiAiUiAit9l/MHuxumaGxnlZ1NGEHiTGmD7gHVh9lqewPv49PsruycC3sVpIJ7CSzX32tkfs/0+KyM4JhPAasNQ+5gPAO40xJ+1t/w+rlXka+Co+ycEYc8bef6v9sfsSv+/rJLABaxDrJFYra4Mxpn0CsXmP9bwdy2NYLdbFwB0BvnbQjuMtWANj7VjJNltEsrAGzD5ujGkyxryE1V//a59W+VjX5wNYA637sK7Ro1itPIwxj9j7P4jVp/0/WAOIAN8Cvmxft88ZYxqwWs5fxEpcDcA9nPs9uwtrLOAU8M92zGN9z81YDYXLgD/6bPotVndHkx3zq2Mcow6rvv954CDg3xX1eayBzlft7pvnsfrGR/Ir4HdYlVFHsRojnxjre/DxLFZ3Tp0dew/jdyNORBzwGaxPEaeAq7H+EMYUOb9bUanoIyJ3Yw0gX+F0LEqFkrbQlVIqSmhCV0qpKKFdLkopFSW0ha6UUlHCsQlwcnNzzYIFC5w6vVJKRaTXX3+93Rgz4s1fjiX0BQsWsGPHDqdOr5RSEUlERr3DWLtclFIqSmhCV0qpKDFuQheRX9lzD1ePsl1E5N/FmuN6j9jzNSullJpegbTQfwPcOMb2m7BuqV4KfBT46dTDUkopNVHjJnRjzBasuRFGcxvwW2N5FZghIoXBClAppVRggtGHXsz5k+w0Msr81iLyURHZISI72tragnBqpZRSXtM6KGqM+YUxZo0xZk1e3mhz6CullJqMYCT0JqxVXbzmMLnFD5SKCq6Oszz2eiM6rYaabsG4sWgT8HEReQhrrme3PY+zUjHnmb3NfP6xPXh6Bpibk8bahTnjv0ipIBk3oYvIRqwFjnNFpBFrYv5EAGPMz7BWtLkZa5L8M1irxigVU872DfL1p/bx4Gv1rCrOZn+zh6raVk3oalqNm9CNMf5rM/pvN8A/Bi0ipSLM/mYPn9z4Bgdbu/iHqxfx2etKed8vX6Oqto17b4y5dYqVg/ROUaUmyRjDf798jNt+spWOs/387sNrue+m5SQlxFFRmse+Zg8tnh6nw1QxRBO6UpNwqruPj/x2B/+8qYbLF8/imU9dyZVLz1VuVZTkA/BirZbnqunj2GyLSkWqlw+18+mHd3G6u5/7N6zgQ5cv4Nxa1JblhZkUZCVTVdfKuy+eO8qRlAouTehKBah/cIgfPFfHT188zMLcdH5198WUFWWPuK+IUFGSz9PVzQwMDpEQrx+GVejpu0ypANSfPMO7fvYK/1F1mHdfNJcnP3HFqMncq6I0j86eAXbWd0xTlCrWaQtdqXH8764mvvSnakTgx3ddwIbVRQG97vKluSTEiZYvqmmjLXSlRtHVO8BnH97Npx7aRensTJ751JUBJ3OArJRELpw/k806MKqmiSZ0pUawt9HNhn9/iT+90cgnr1nKHz96CXNmpk34OOtL89mv5YtqmmhCV8rH0JDhP7cc4R0/3UrvwBAPfuQSPnNdyaQHNStKrVJGLV9U00H70JWytXb28NmHd/PSwXZuKCvgO7evZkZa0pSOuWx2JrOzUrR8UU0LTehKAVW1rXzukd109gzwwNtXctfaeW+qLZ8MEeHqkjye3ttM/+AQiVq+qEJI310qpvUODPKNJ/dx96+3Mys9mSc+cQXvXTc/KMnca/2yPDp7B9h5/HTQjqnUSLSFrmLWkbYuPrHxDWpcHj5w6Xy+ePNyUhLjg36ey5fY5Yt1baxbNCvox1fKS1voKuYYY3h4RwMbfvQ3mjrO8ov3X8TXblsZkmQOkJmSyEXzZ1KlA6MqxDShq5ji6ennkw/t4t5H97B6TjbPfOpKri+bHfLzVtjliyfcWr6oQkcTuooZrx8/zc0/fImn9zZzzw2l/OHvL6EwO3Vazj1cvljXOi3nU7FJE7qKeoNDhh+/cJB3//wVAB7+h0v5x/VLiI8L3sDneIbLF7XbRYWQDoqqqHbC3cM//fENXj1yig2rC/nmO1aRlZI47XGICBWleTy1R8sXVejou0pFrb/UnODGH25hT6Ob775zNT+68wJHkrlXRalVvvi6li+qENEWuoo6Pf2DPPDUfn736nHKirL40Z0XsCgvw+mwzpUv1rZxiZYvqhDQFrqKKnUtndz246387tXj/P0VC3n8Y5eFRTIHq3xxzYKZVNXqwKgKDU3oKmo0u89y24+3crK7l9986GK+vGEFyQmhqS2frIrSfA6c6NTyRRUSmtBV1Nh29BRn+wf55QcvpqI03+lwRqTliyqUNKGrqFHj8pAUH8eKoiynQxlVaYFVvrj5gJYvquDThK6iRnWTm2WFmWFdEigirF+Wx9ZD7fQPDjkdjooy4fvOV2oCjDFUN7nHXbg5HFxdkq/liyokNKGrqNB4+iyengFWFodvd4vX5UtmDZcvKhVMmtBVVKhucgOwMgJa6Fq+qEJFE7qKCtUuN/FxQunsTKdDCYi3fLHZfdbpUFQU0YSuokJ1k4el+Rkhm9M82NbbZZW6eLQKJk3oKuJ5B0RXFod/d4tXSUEGhdk6+6IKLk3oKuK1eHo52d3HyjCuP/fnnX3xb4fa6RvQ8kUVHAEldBG5UURqReSQiHxhhO3zRGSziLwhIntE5Obgh6rUyIYHRCOohQ5W+WKXli+qIBo3oYtIPPAT4CZgBXCniKzw2+3LwMPGmAuAO4D/CHagSo2m2uVGBJYXRk4LHazyxcR4oUqnAVBBEkgLfS1wyBhzxBjTBzwE3Oa3jwG8v03ZgCt4ISo1tuomD4ty00lPjqzZoDNTElkzP0cHRlXQBJLQi4EGn8eN9nO+vgK8T0QagaeBT4x0IBH5qIjsEJEdbW36JlbBUeOKrAFRXxWleVq+qIImWIOidwK/McbMAW4Gficibzq2MeYXxpg1xpg1eXl5QTq1imXtXb00u3si4oaikXhnhdRqFxUMgST0JmCuz+M59nO+Pgw8DGCMeQVIAXKDEaBSY6lxeQAoi4Bb/kdSUpBBUXaK3jWqgiKQhL4dWCoiC0UkCWvQc5PfPvXANQAishwroWuTQ4Wct8IlEiblGomIcHVpPlsPndTyRTVl4yZ0Y8wA8HHgWWA/VjVLjYh8TURutXf7LPAREdkNbATuNsaYUAWtlFeNy828nDSyU51b/HmqKkrztHxRBUVAZQHGmKexBjt9n7vf5+t9wOXBDU2p8VU3eSJihsWxXL4k1ypfrG3l0sW6eLSaPL1TVEUs95l+6k+didjuFq+M5ATWzM/RgVE1ZZrQVcSqaY7MO0RHsn5ZHrUtnbg6tHxRTZ4mdBWxaprsCpcImsNlNN7yxRfrtJWuJk8TuopY1S43hdkp5GYkOx3KlC3Nt8oXNx/Q8kU1eZrQVcSKlDVEA3GufFFnX1STpwldRaTu3gGOtHdHfIWLr/WleXT3DbLj+CmnQ1ERShO6ikj7mz0YExlriAbqMrt8USfrUpOlCV1FpEidA30sGckJXLxAyxfV5GlCVxGp2uUhNyOJgqzIHxD1VVGq5Ytq8jShq4jkHRAVEadDCar1OvuimgJN6Cri9PQPcrC1K6oGRL2W5GdQPCNVZ19Uk6IJXUWc2hOdDA6ZqBoQ9bLKF/O0fFFNiiZ0FXGqXdE3IOqrosQuXzym5YtqYjShq4hT3eQhKyWBOTNTnQ4lJLzli1U6DYCaIE3oKuJ41xCNtgFRr4zkBNYuzNF+dDVhmtBVROkfHOJAc2fUdrd4VZTkU9fSRZOWL6oJ0ISuIsrBli76BoeiYobFsVSUWouoaytdTYQmdBVRon1A1Otc+aL2o6vAaUJXEaWmyU16UjwLZ6U7HUpIiQgVpXm8rOWLagI0oauIUu3ysKIoi7i46BwQ9VVRmq/li0G0pa6NP73RSF1LJwOD0flHMqBFopUKB4NDhn0uD++5eK7ToUyLyxbPIik+js21rVy2JNfpcCLa2b5BPvLbHfTan3ZSEuMonZ1FWZH3XzbLZmeSkhjvcKRTowldRYyj7V2c7R+M+v5zr/TkBC5eOJOq2ja+dIvT0US2V4600zswxDffvoqUxDhqXB5qXG6e2O3iwdfqAYiPExbnpVNWlE1ZURYrCq1En52W6HD0gdOEriJGtb2GaDTO4TKa9aX5fOOp/TR1nKV4RnTeSDUdqmrbSE2M5/aLiklOiOcdF1rPG2NoPH2WGpfbTvIeXj7czp/eaBp+bfGM1OFWfFlRFmXFWczOSgnL+yA0oauIUd3kJjkhjiV5GU6HMm0qSvP4xlP7qapt5b3r5jsdTkQyxrC5tpXLl8wiOeH8LhURYW5OGnNz0rhxZeHw8+1dvcOt+H0uD/tcHp7b34Ix1vac9CSrFW8n+hWFWSzMTSfe4bEdTegqYlS73CwrzCIhPnbG8hfnnStf1IQ+OUfau2k4dZaPXrU44NfkZiRzdUkeV5fkDT/X1TvAgWbPcKKvcXn41d+O0j9oZfm0pHiWzc4815IvyqZkdsab/oiEkiZ0FRGGhgw1TR5ufUuR06FMK2/54p/eaKJ3YHBak0O08NbyV/gk58nISE5gzYIc1izIGX6ub2CIg62d1Nit+BqXm8d3NvK7VwcBSIgTluRn+CT5LJYXZZGVEpp+eU3oKiI0nD5DZ+9AzAyI+qoozecPr9Wz49hpLtdqlwmrqm1lSX4Gc3PSgn7spIQ4O1mfe18ODRmOnzpzXr/8i3WtPLazcXifr91WxgcuXRD0eDShq4gwPCAahXOgj8dbvlhV26oJfYLO9A3w2pFTfODS6euuiosTFuamszA3nQ2rz32ibPX0DHfXXDhvZkjOrQldRYRql5uEOKFkduwMiHqlD8++qOWLE/XK4ZP0DQ5RYS/t56T8rBTys1JYvyx0scTO6JKKaNVNbkoKMmO2D7miNI+DrV00nj7jdCgRZXNtK2lJ8Vy8MDQt4nCjCV2FPWMMNS5PTNWf+zs3+6JO1hUoYwxVtW1ctjg3ZhoCASV0EblRRGpF5JCIfGGUfd4tIvtEpEZEHgxumCqWNbt7ONXdF5MDol6L8zKYM1NnX5yIw23dNJ4+O/zHMBaM24cuIvHAT4DrgEZgu4hsMsbs89lnKXAfcLkx5rSION9hpaJGdZM1ZW5ZDA6IennLFx/fqeWLgfLOJR9LCT2QFvpa4JAx5ogxpg94CLjNb5+PAD8xxpwGMMborPwqaKpdHuIElhdmOh2KoypK8jnTN8j2o6edDiUiVNW2sTQ/gzkzg1+uGK4CSejFQIPP40b7OV8lQImIbBWRV0XkxpEOJCIfFZEdIrKjrU0/OqrA1DS5WZyXQVpSbBdlXbbkXPmiGlt37wDbjp6KqdY5BG9QNAFYClQAdwL/KSIz/HcyxvzCGLPGGLMmLy+2LrSavGp7UehYl5aUwLpFOVTVaWNoPC/b5Yrrw6BccToFktCbAN8JqOfYz/lqBDYZY/qNMUeBOqwEr9SUtHb20OLpjfo1RAN1dUkeh7R8cVxVta2kJ8Wfd5t+LAgkoW8HlorIQhFJAu4ANvnt8z9YrXNEJBerC+ZIEONUMarG5Z0yV1vowPANMlrtMrrhcsUluSQlxFZl9rjfrTFmAPg48CywH3jYGFMjIl8TkVvt3Z4FTorIPmAzcI8x5mSoglaxo8aucFmhLXQAFuel2+WL2o8+mkOtXTR1nI257hYI8NZ/Y8zTwNN+z93v87UBPmP/Uypoqps8LJiVFrLZ6SKNli+Ob3h2xRgbEAW9U1SFuWqXmzLtbjnP+lItXxxLVV0rJQUZFMXgCk+a0FXY6jjTR+PpszE5w+JYLl2s5Yuj6bLLFWOxuwU0oaswdm5AVPvPfXnLFzdrQn+Tlw+10z9ouDoGu1tAE7oKY3rL/+iuLsnjcFs3Dae0fNFXVV2bVa44P7bKFb00oauwVe3yUDwjlZz0JKdDCTveObX1JqNzjDFUHWjliqWxV67oFZvftYoINU1uvaFoFIty05mbk8qL2u0y7GBrFy53T1gsZuEUTegqLHX29HOkvVtvKBqFiFBRks/WQyfp6R90OpywEIuzK/rThK7C0v7mTkAHRMdSUZrH2f5Bth875XQoYWHzgTZKCzIpzI69ckUvTegqLHkHRLVkcXSXLp5FUkKcTgOAVa644/gpKpbFbuscNKGrMFXtcpOXmUx+VorToYSttKQE1i3M0Xp0YKtdrlhRErv956AJXYWpmiYPK3VAdFwVpflavojVf56RnMCaBbGxGPRoNKGrsHO2b5CDrZ06IBqAc4tHx24r3Tu74hVLckmMj+2UFtvfvQpLB054GDJ6Q1EgvOWLsdyPXtfSRbO7J6arW7w0oauwU623/AdMRFhfms/Lh2O3fNE7BUKs3u7vSxN6hBscMgwNGafDCKqaJjcz0hIpjsHZ8iYj1ssXq2pbWTY7tssVvTShRzBjDO/++Svc+9gep0MJqmqXm5VF2YiI06FEhEsXWbe6bz4Qe90unT397Dh2OqbvDvWlCT2C7W508/rx0zyx20VX74DT4QRF38AQtSc6KdPuloClJsVb5Yt1sTcwuvVQOwNDRvvPbZrQI9jG1+qJE+gdGOL5fS1OhxMUdS2d9A8avaFogtaX5nMkBssXq2rbyExO4KL5sV2u6KUJPUJ19vSzabeLd140h6LsFJ7Y7XI6pKCocdl3iGrJ4oTEYvnicLniUi1X9NKrEKH+d5eLs/2DvHfdfDaUF7HlYBsdZ/qcDmvKqps8ZCQnMD8nzelQIsrC3HTm5aSxOYbKFw+c6OSER8sVfWlCj0DGGB58rZ4VhVmsnpNN5eoi+gcNz9accDq0Kat2uVlRlEVcnA6IToR38eiXD7fHTPniucWgdUDUSxN6BNrb5GZfs4c7181DRFhZnMWCWWk8sbvZ6dCmZGBwiP3NHu0/n6T1pfn09A+x7WhslC9W1bayvDCLAp3vZ5gm9Aj04Gv1pCbGc9tbigCrdVZZXsTLh9tp7exxOLrJO9LeTU//kN5QNEmXLIqd2Rc9Pf3sOH5au1v8aEKPMN7B0FvLi8hKSRx+vrK8iCEDz+yN3G6X4SlzdUB0UlKT4rlk0ayYKF/cerCdwSHDeu1uOY8m9AizabeLM32D3Llu3nnPlxRkUlqQGdHVLjUuDymJcSzKTXc6lIhVUZLHkbZu6k9Gd/liVW0bmSkJXDhvhtOhhBVN6BFm47Z6lhdmUT7nza3YyvJCdhw/TVPHWQcim7rqJjfLC7NI0BK0SRsuX4ziVroxhqq6Vq5cmqvvFT96NSLI3kY31U0e7lo7d8Tb4ivLrT71p/ZEXit9aMiwz6UDolO1MDed+bPSoroffX9zJy2eXq1uGYEm9Ajy4LZ6UhLjuO2C4hG3z5+VTvmc7Iisdqk/dYbO3gEdEJ0ia/Ho6C5f9H76qCjRAVF/mtAjRFfvAJt2NVG5+vzBUH+V5UXsbXJztL17GqObumr7DlGdA33qKuzyxdeitHyx6kAbKwqzdHnCEWhCjxBP7HbRPcJgqL9bVhcC8GSEDY5WN3lIjBdKCjKdDiXinStfjL5+dPfZfl6vP836GF8MejSa0CPExm31LJudyQVzxx7VL8xOZe2CHJ6IsH70Gpeb0tmZJCXoW3KqUpPiuXTRLF6Mwn70rYesckXtPx+Z/vZEgOomN3sa3dy5dl5Ac4RXlhdS19JF7YnOaYhu6owxVDe5dUA0iCpK8zjS3s3xk5HV9TaezQdayUpJGLdhE6sCSugicqOI1IrIIRH5whj73S4iRkTWBC9EtXFbPckJcbxtlMFQfzetKiROYNPuphBHFhwudw+nz/RTpjcUBY23BRtN1S7GGF6sa+PKkjwtVxzFuFdFROKBnwA3ASuAO0VkxQj7ZQKfAl4LdpCxrLt3gP/d5WLD6iKyU0cfDPWVm5HM5UtyeWJ3M8aE//J0w3eIFmmFS7CcK1+Mnn70fc0eWjt7tbplDIH8mVsLHDLGHDHG9AEPAbeNsN/Xge8AkTuZSBh6co+1GtFd6+ZO6HWVq4uoP3WGPY3uEEUWPDVNbuLjhOWFmtCD6a3L8tl6+CQn3NHxK+n9tKGLQY8ukIReDDT4PG60nxsmIhcCc40xT411IBH5qIjsEJEdbW3R81EwlB7c1kBJQQYXzpvYiiw3rJxNYrxExFQA1S4PS/IySEmMdzqUqPJ3ly/EGMMP/3rQ6VCCoqq2lZXFWeRnarniaKbcESUiccD3gc+Ot68x5hfGmDXGmDV5efpXdjw1Lje7GzoCHgz1lZ2ayNUl+S0u6LEAABgASURBVDy5p5mhofDudqlucusaoiEwNyeNu9bO4+EdDRxp63I6nClxn+lnZ30HFSVa3TKWQBJ6E+D7eX+O/ZxXJrASqBKRY8AlwCYdGJ0672Do2wMcDPVXWV7ICU8PO46fDnJkwdPq6aG1s1crXELk429dSnJCHN9/rs7pUKbkpUNtdrmiNgTHEkhC3w4sFZGFIpIE3AFs8m40xriNMbnGmAXGmAXAq8CtxpgdIYk4RpzpG+B/3nBxy6pCZqQlTeoY1y4vICUxLqy7XWpcHkCnzA2VvMxkPnzFQp7c0zw8+ByJqmrbyE5N5C1arjimcRO6MWYA+DjwLLAfeNgYUyMiXxORW0MdYKx6cnczXb0D494ZOpb05ASuWV7A03ubGRgcCmJ0weNNMiu0wiVkPnLVImakJfIvz9Y6HcqkDA3Z5Yo6u+K4Aro6xpinjTElxpjFxpgH7OfuN8ZsGmHfCm2dT92D2+pZmp/BmvkTGwz1V7m6iJPdfbxy5GSQIguuapebRbnpZCQnOB1K1MpKSeRjFYt5sa6NV8P0fTCWfc0e2jp1dsVA6J+7MLTP5WHXJAdD/VWU5pGRnMCmXeHZ7VLd5NEbiqbBBy5dwOysFL775wMRcW+CL28t/dVafz4uTehh6KHt9SQlxPGOCyc3GOorJTGe68sK+HPNCXoHwms61dPdfTR1nNUbiqZBSmI8n7p2KTvrO3h+f2TdbFRV28aq4mzyMpOdDiXsaUIPM2f7BvnTzqYpDYb6qywvorNngC117UE5XrDogOj0etdFc1iYm873nq1lMMxLWb06zvSxs14Xgw6UJvQw8+QeF529A9y5dvKDof6uWJLLzLTEsKt2OTcHurbQp0NCfByfvb6E2pZO/ndXZMzz89LBdoYM2n8eIE3oYWbjtnoW56Vz8YKpDYb6SoyP46ZVhTy3r4UzfQNBO+5UVTe5mTMzNWifRNT4bl5ZSFlRFt9/ro6+gfCsfPJVVdvGjDQtVwyUJvQwcuCEh531wRkM9Ve5uoiz/YO8cCB8+k9rdA3RaRcXJ9x74zIaT59l47Z6p8MZk1Wu2MqVS/OIjwvu70O00oQeRh7a1kBSfBy3Xzgn6MdeuzCH/MzksOl26ezp52h7t64h6oCrluaybmEOP3rhEN294fOJzV+Ny0N7Vx/rtf88YBGX0Lt7B3hmb+Qtgjyes32DPL6zkZtWzWZmevC7IOLjhFtWF7K5tg1PT3/Qjz9R++wBUS1ZnH4iViu9vauXX2896nQ4o/KWK16l5YoBi7iE/tOqw3zswZ3sOBZdC+A+vbcZT09wB0P9VZYX0TcwxHM1LSE7R6CqvRUu2uXiiIvmz+Ta5QX8/MUjnO7uczqcEW2ubWX1nGxyM7RcMVARl9D/b8Viimekcs+je+jpD6+66qnYuK2eRbnprFuYE7JzXDB3BsUzUsNivdGaJjcFWclaW+yge24opatvgJ+9eNjpUN6k40wfuxo6tLplgiIuoacnJ/Dd21dztL2bf/1LZM5N4a+upZMdx0+HZDDUl4hQWV7E3w62c8rhVlm1S9cQdVrp7EzefkExv3n5WNgtgrFluFxRu1smIuISOsBlS3K5a908fvm3o7wexlPDBmrjtnprMPSi4A+G+qssL2RgyPBMtXPjEGf7BjnU2qX952Hg09eWMBSGi2BUHWhlZloi5XO0XHEiIjKhA9x30zIKs1O599HdEd310tM/yGOvN3LDytnkhGAw1N+KwiwW56U7Wu2y/4SHIaNriIaDuTlpvHfd/LBaBMM7u+JVJVquOFERm9AzUxL59u2rONzWzb89H16ti4k4Nxg6sTVDJ8vb7fLa0VO0eJz5mF3jXRRaW+hh4R/XLwmrRTCqXW5Odvdpd8skRGxCB7hyaR53XDyXX2w5zK6GDqfDmZSN2+pZMCuNSxfNmrZzblhdhDHw1B5nul2qmzzkpCdRmK1rQ4aDcFsEo6q2DRG4aqkm9ImK6IQO8MVbllOQlcI9j+wOu9kEx3OwpZPtx0I/GOpvSX4GKwqzHKt2qXa5KSvKmtbvWY0tnBbBsMoVZzBLyxUnLOITelZKIt96xyoOtnbx72E2sDOejdsaSIyXaRkM9VdZXsQb9R00nDozreftHRikrqVTu1vCTLgsgnG62y5X1JuJJiXiEzpYM7G966I5/OzFI+xpjIyul57+QR7b2cgNZbMduXFiw+pCAJ6c5m6Xgy1d9A8aLVkMQ+GwCMaWg20YLVectKhI6ABf3rCC3Iwk7nlkT0R0vfy5+gTus/3cFcI7Q8cyNyeNC+bNmPZql+rhAVGtcAk34bAIRlVtGznpSazWcsVJiZqEnp1qdb3UtnTykxcOOR3OuB60B0MvmcbBUH+Vq4vY1+zhUOv0latVu9xkpiQwLydt2s6pAufkIhhDQ4YtdW1ctTRXyxUnKWoSOsBblxXwjguL+UnV4bAYrR/NodYuth09xR1r5xHn4Bv3ltWFiDCtrfTqJo8OiIYxJxfB2NvkLVfU2/0nK6oSOsA/byhjVnoSn3tkd9hO4P/QtnoS44V3OjAY6qsgK4VLFs7iiT2uaekzHRgcYn+zzoEe7pxaBGNzbatVrqgDopMWdQk9Oy2RB96+igMnOvmPqvDrevEOhl6/wpnBUH+V5UUcaetmX7Mn5Oc63NZN78CQVriEOacWwaiqbaN8zoxpuWM6WkVdQge4bkUBb3tLET9+4dDwvNvh4tmaE5w+0x/SaXIn4saVs0mIE57YHfpqFx0QjRxXLc3lkkXTtwjGqe4+djd2aHXLFEVlQgf458oyZqRZXS/9g+HT9bJxWz3zctK4bLFzg6G+ctKTuGJpLk/sDn23S7XLTWpiPAtzM0J6HjV1070IxpY6q1xxvfafT0nUJvSZ6Ul8420r2dfs4WdV4THf85G2Ll49coo71s51dDDUX+XqIpo6zvJGiKdPqGnysKIoSysYIsSF82Zy3YrpWQSjqraVWelJrNLuuCmJ2oQOVndCZXkR//7CQQ6ccL7r5aHtDSTEOT8Y6u+6sgKSEuJCWu0yNGSocbl1hsUI87nrQ78IxuCQYcvBdq4qyQurhk4kiuqEDvDVW8vISknknkf2MOBg10vvwCCPvt7IdSsKyM8Mr0mpslISWV+ax1N7mkNWe3zsZDfdfYM6B3qEmY5FMPY0dnBKZ1cMiqhP6DnpSXz9bSvZ2+Tm51uOOBbHszUtnOruC5vBUH+V5UW0dvay7Who1mrVNUQjV6gXwaiqbSNOZ1cMiqhP6AA3ryrkllWF/PD5g9S1dDoSw8bX6pkzM5UrluQ6cv7xXLOsgLSkeDaFqNulpslNUnwcSwt0QDTShHoRjKq6NsrnzmCmlitOWUwkdICv3lZGRkoC9zw6/V0vR9q6eOXISe50+M7QsaQmxXPdigKeqW4OSVVQtcvNssJMEuNj5i0XVUK1CMbJrl72NHZodUuQBPTbJSI3ikitiBwSkS+MsP0zIrJPRPaIyF9FZH7wQ52a3IxkvnprGbsbOvivv4W+DMvXH7c3EB8nvCvMBkP9Va4uouNMP3871B7U4xpj7Fv+tbslUoVqEQydXTG4xk3oIhIP/AS4CVgB3CkiK/x2ewNYY4xZDTwKfDfYgQbDhtWF3FBWwPefq5u2Cal6BwZ55PVGrl2eT35WeA2G+ruyJJeslISgV7s0nj6L+2y/3lAU4UKxCEZVbRu5GUk6thIkgbTQ1wKHjDFHjDF9wEPAbb47GGM2G2O8KyW8CoRlU1RE+PrbVpKWFM89j+6eltnkntsX3oOhvpIT4rlx5Wz+UtMS1IW3a1z2HaL6SxvRgr0IxqDPYtDh2hUZaQJJ6MVAg8/jRvu50XwYeGYqQYVSfmYKX721jDfqO/jVNHS9bNxWT/GM1IgZwa8sL6Krd4Cq2ragHbO6yUN8nFA6OzNox1TOCOYiGLsbO+g406+zKwZRUEeoROR9wBrgX0bZ/lER2SEiO9ragpcwJurW8iKuXV7A9/5SG5JRe69j7d1sPXSSO8PsztCxXLpoFrPSk4K63mi1y83S/AxSEuODdkzljGAugnGuXDE8K78iUSAJvQmY6/N4jv3ceUTkWuBLwK3GmN6RDmSM+YUxZo0xZk1ennMtVhHhm29fSXJCHPc+uidkXS8PeQdD18wdf+cwkRAfx82rCvnr/pagTMpkDYi6dYbFKPKui+awKAiLYFTVtnLBvJnMSNNyxWAJJKFvB5aKyEIRSQLuADb57iAiFwA/x0rmzqxdNUH5WSl85dYydhw/zW9ePhb04/cNDPHo6w1csyyfgjAfDPVXWV5ET/8Qz+9vmfKxWjt7ae/q01v+o0hCfByfmeIiGO1dvexpdOti0EE2bkI3xgwAHweeBfYDDxtjakTkayJyq73bvwAZwCMisktENo1yuLDy9guKeeuyfP7l2QMca+8O6rGf399Ce1cfd64L/8FQf2vmz6QwOyUoU+qemzJXW+jR5OaVhawsnvwiGFvqrC5X7T8ProD60I0xTxtjSowxi40xD9jP3W+M2WR/fa0xpsAY8xb7361jHzE8WF0vq0iMt7pehoLY9RJpg6G+4uKEDasLebGuFfeZ/ikdq7rJgwgsL9QWejSJixPuuWHyi2Bsrm0jNyOZMv3kFlQxf9ve7OwU/t+GFWw7dorfvnIsKMesP3mGlw62856L50bsVLGV5UX0DxqerTkxpeNUu9wsyk0nPTkhSJGpcDHZRTAGhwwvHWzjai1XDLqYT+hgDfJcXZLHd/5cS/3JM+O/YBwPba8nTuDdETQY6m9VcTbzZ6VNudqlRgdEo9ZkF8HY1eAtV4y8T6/hThM61hvzW+9YRUKccO9ju6fU9dI/OMTDOxp567ICZmdH1mCoLxGhcnURWw+10941YtHSuE529eJy9+gNRVFsMotgVNW26uyKIaIJ3VY0I5Uv3bKcV4+c4g+vHZ/0cZ7f10J7Vy93rYvc1rlXZXkRQwae2Tu5wdEae8rcMr3lP6pNdBGMqto2Lpw3k+y0xBBHFns0oft4z8VzuXJpLt965gANpybX9fLgtnoKs1O4uiTyR+9LZ2dSUpAx6WqXavuWf52UK7pNZBGMts5e9ja5tbslRDSh+xARvn37auJE+PxjeyZ8a3PDqcgfDPVXubqIbcdO0ew+O+HX1jR5mJeTRnaqtsSiXaCLYLyo5YohpQndT/GMVO67eRkvHz7JgxMsx4qGwVB/G8qLAHhqz8Rb6dUut86wGCMCXQSjqraVvMxkVmgZa0hoQh/BXWvncdniWXzzqf00ng6s68U7GLq+NJ+iGakhjnD6LMxNZ1Vx9oSn1HWf7ef4yTPa3RJDxlsEY2BwiJcOtmu5YghpQh+BiPCd21djgPse3xtQ18tf97fS1tkbEdPkTtSt5UXsbnRP6G7afd41RLVkMWaMtwjGroYO3Ge1XDGUNKGPYm5OGvfdtIyXDrbzx+0N4+6/cVs9s7NSovLNesvqQgCenEBNes3wgKh+tI4lYy2CUVXbRnyccOWS6PsdCRea0Mfw3nXzuWRRDg88tR9Xx+iDgg2nzrDlYBvvvnguCVG4ZmbRjFQuXjBzQtUu1U1uCrNTyM1IDmFkKtyMtQhGVV0rF86boeWKIRR92SeI4uKE795ezsCQGbPr5eEdDQhW2WO0qiwvoralk9oTnQHtX+3SNURj1UiLYLR29lDd5NHqlhDThD6OebPS+PyNpbxY18Yjrze+afvA4BB/3N5ARWk+xVE0GOrvppWFxElg3S5n+gY43NalFS4xaqRFMF6s9ZYrandLKGlCD8AHLl3A2gU5fP3JfW+6ceKFA620RulgqK+8zGQuW5zLE7td4w4S72/2YIyuIRrL/BfBqKprI1/LFUNOE3oA4uKE775zNf2DQ3zxT+d3vWzcVk9BVjLrY6DlUVleyLGTZ6hu8oy5n3e7VrjELt9FMB7b2chLddbsiiJarhhKmtADtCA3nXtuWMYLB1r50xvWKi1NHWepqmvjPWuiczDU3w1ls0mMl3FnYKxucpObkURBlg6IxjLvIhhf2VSDp2eA9cu0/zzUoj8LBdHdly1gzfyZfGVTDa2enuFyxndH8WCorxlpSVy1NI8nd7vGnJHSOyCqrbHY5l0E40zfIPFxwuVLdDHoUNOEPgHxdtdL78AQ9z2+l4e3N3B1SR5zZqY5Hdq0ufUtRbjcPeysPz3i9p7+QQ62dOqAqAKsRTCuLsnjqqW5OqfPNNBlZCZoUV4Gn7u+lAee3g/AV28rczii6XXt8gJSEuPYtNvFmgU5b9pe19LJwJDRAVEFWHdd//KDa4jTT2vTQlvok/B3VyzkwnkzKMxO4a0x1i+YnpzANcsKeHpvMwODb14cWAdElb+E+Didu2WaaAt9EuLjhN9+eB1n+gZIjIHBUH+V5YU8tbeZV4+c4oql5/eLVrvcZKUkMGdm9NbkKxWuYi8bBUlGcgL5mZG7xNxUVJTmk5GcMOIMjN41RHVAVKnppwldTVhKYjzXryjgmepm+gbOdbv0Dw6x/0Sndrco5RBN6GpSKsuL8PQM8NLBtuHnDrV20TcwpDMsKuUQTehqUi5fksuMtMTzul28c2BrC10pZ2hCV5OSlBDHTStn89y+Fs72DQJQ4/KQnhTPwlnpDkenVGzShK4mrbK8iO6+QTbXWjPqVTe5WVGUpSVqSjlEE7qatHULZ5GXmcymXS4Ghwz7mnUOdKWcpAldTVp8nHDLqkJeqG1lb5ObM32D2n+ulIM0oaspqSwvom9giB/YK73rHC5KOUcTupqSC+fNoHhGKi/WtZGcEMeSvAynQ1IqZmlCV1MiImwoLwRgWWFWTMwLr1S4Cui3T0RuFJFaETkkIl8YYXuyiPzR3v6aiCwIdqAqfFWuLgJgpd5QpJSjxk3oIhIP/AS4CVgB3CkiK/x2+zBw2hizBPgB8J1gB6rCV1lRFp+5roT3rpvvdChKxbRAWuhrgUPGmCPGmD7gIeA2v31uA/7b/vpR4BrR2ZlihojwyWuWskJb6Eo5KpCEXgw0+DxutJ8bcR9jzADgBmb5H0hEPioiO0RkR1tbm/9mpZRSUzCtI1jGmF8YY9YYY9bk5eVN56mVUirqBZLQmwDfVZDn2M+NuI+IJADZwMlgBKiUUiowgST07cBSEVkoIknAHcAmv302AR+0v34n8IIxZvRl4ZVSSgXduEvQGWMGROTjwLNAPPArY0yNiHwN2GGM2QT8EvidiBwCTmElfaWUUtMooDVFjTFPA0/7PXe/z9c9wLuCG5pSSqmJ0Nv6lFIqSmhCV0qpKCFOjV2KSBtwfJIvzwXagxhOpNPrcT69HufotThfNFyP+caYEeu+HUvoUyEiO4wxa5yOI1zo9TifXo9z9FqcL9qvh3a5KKVUlNCErpRSUSJSE/ovnA4gzOj1OJ9ej3P0Wpwvqq9HRPahK6WUerNIbaErpZTyowldKaWiRMQl9PGWw4sVIjJXRDaLyD4RqRGRTzkdUzgQkXgReUNEnnQ6FqeJyAwReVREDojIfhG51OmYnCIin7Z/T6pFZKOIpDgdUyhEVEIPcDm8WDEAfNYYswK4BPjHGL4Wvj4F7Hc6iDDxQ+DPxphlQDkxel1EpBj4JLDGGLMSa5LBqJxAMKISOoEthxcTjDHNxpid9tedWL+s/itJxRQRmQPcAvyX07E4TUSygauwZkLFGNNnjOlwNipHJQCp9noNaYDL4XhCItISeiDL4cUcEVkAXAC85mwkjvs34F5gyOlAwsBCoA34td0F9V8iku50UE4wxjQB3wPqgWbAbYz5i7NRhUakJXTlR0QygMeAfzLGeJyOxykisgFoNca87nQsYSIBuBD4qTHmAqAbiMkxJxGZifVJfiFQBKSLyPucjSo0Ii2hB7IcXswQkUSsZP4HY8zjTsfjsMuBW0XkGFZX3FtF5PfOhuSoRqDRGOP91PYoVoKPRdcCR40xbcaYfuBx4DKHYwqJSEvogSyHFxNERLD6R/cbY77vdDxOM8bcZ4yZY4xZgPW+eMEYE5WtsEAYY04ADSJSaj91DbDPwZCcVA9cIiJp9u/NNUTpAHFAKxaFi9GWw3M4LKdcDrwf2Csiu+znvmivLqUUwCeAP9iNnyPAhxyOxxHGmNdE5FFgJ1Z12BtE6RQAeuu/UkpFiUjrclFKKTUKTehKKRUlNKErpVSU0ISulFJRQhO6UkpFCU3oSikVJTShK6VUlPj/XoiGGgd7pq4AAAAASUVORK5CYII=
" />
</div>
</div>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="n">test</span><span class="p">(</span><span class="n">agents</span><span class="p">,</span> <span class="mi">5000</span><span class="p">)</span>
</pre></div>
</div>
</div>
</div>
<div class="output_wrapper">
<div class="output">
<div class="output_area">
<div class="output_subarea output_stream output_stdout output_text">
<pre>Epsilon-Greedy Agent, epsilon= 1
maximum possible reward: 0.9851042878107023
average reward: [0.47962497]
cosine similarity 0.3162277660168379
euclidian norm 0.09486832980505139
</pre>
</div>
</div>
<div class="output_area">
<div class="output_png output_subarea ">
<img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de5xdVX338c/3nDlzySSThGTCJSEkaDCCysVAUVBRBAMqVKsVtFqvtNZbtRextkK17cvqo1VbFaP1Qfso1nqNlQqoCFRFCAJCuCVAgHDL5J65zznn9/yx94SZ5MzkJJl9Jpn9fb9e88rea++z92+dnJnfWXvtvZYiAjMzy6/CZAdgZmaTy4nAzCznnAjMzHLOicDMLOecCMzMcq5psgPYW3Pnzo1FixZNdhhmZgeVW265ZWNEdNbadtAlgkWLFrFq1arJDsPM7KAi6aGxtvnSkJlZzjkRmJnlnBOBmVnOORGYmeWcE4GZWc45EZiZ5ZwTgZlZzuUmEdz35A4+ffW9bOwemOxQzMwOKJklAklflbRB0p1jbJ8p6UeSbpe0WtJbsooFYM2T3Xzu52vZ3DOY5WnMzA46WbYILgeWj7P9XcBdEXE8cAbwKUnNGcZjZmY1ZJYIIuJ6YPN4uwAzJAmYnu5bzioeMzOrbTL7CP4NeCbwGHAH8L6IqNbaUdJFklZJWtXV1dXIGM3MprzJTAQvA24DjgBOAP5NUketHSNiRUQsi4hlnZ01B8+rm6doNjMbbTITwVuA70ViLfAgsDSrk0lZHdnM7OA2mYngYeBMAEmHAs8AHpjEeMzMcimz+QgkXUFyN9BcSeuBS4ASQERcBnwMuFzSHYCAD0bExqziMTOz2jJLBBFx4R62PwacndX5zcysPrl5snhY4N5iM7ORcpMI3FdsZlZbbhKBmZnV5kRgZpZzTgRmZjnnRGBmlnO5SwQeYsLMbLTcJAIPMWFmVltuEoGZmdXmRGBmlnNOBGZmOZe7RODOYjOz0XKUCNxbbGZWS44SgZmZ1eJEYGaWc5klAklflbRB0p3j7HOGpNskrZZ0XVaxmJnZ2LJsEVwOLB9ro6RZwBeA8yLiOOC1Gcayk+cjMDMbLbNEEBHXA5vH2eX1JJPXP5zuvyGrWMBPFpuZjWUy+wiOAWZL+oWkWyS9aRJjMTPLrczmLK7z3M8FzgTagF9LujEi7tt1R0kXARcBLFy4sKFBmplNdZPZIlgPXBURPRGxEbgeOL7WjhGxIiKWRcSyzs7OhgZpZjbVTWYi+CFwuqQmSdOA3wPuzvqkfrLYzGy0zC4NSboCOAOYK2k9cAlQAoiIyyLibkk/AX4HVIGvRMSYt5rudzxZHdjM7CCXWSKIiAvr2OeTwCezisHMzPbMTxabmeWcE4GZWc45EZiZ5VxuEoH8aLGZWU25SQRmZlabE4GZWc45EZiZ5VzuEoGfLDYzGy03icBdxWZmteUmEZiZWW1OBGZmOedEYGaWc04EZmY5l7tE4MnrzcxGy00i8AgTZma15SYRmJlZbZklAklflbRB0rizjkk6WVJZ0muyisXMzMaWZYvgcmD5eDtIKgL/DFydYRxmZjaOzBJBRFwPbN7Dbu8BvgtsyCqOXXmICTOz0Satj0DSfOBVwBfr2PciSaskrerq6trH8+3Ty8zMprzJ7Cz+DPDBiKjuaceIWBERyyJiWWdnZwNCMzPLj6ZJPPcy4FvpzGFzgXMllSPiB5MYk5lZ7kxaIoiIxcPLki4H/ttJwMys8TJLBJKuAM4A5kpaD1wClAAi4rKszrsn7is2Mxsts0QQERfuxb5vziqOYfKMBGZmNfnJYjOznHMiMDPLuTEvDUn6V8a5pB4R780kIjMza6jxWgSrgFuAVuAkYE36cwLQnH1o2Qg/WmxmNsqYLYKI+BqApHcCp0dEOV2/DLihMeFNIPcVm5nVVE8fwWygY8T69LTMzMymgHpuH/04cKuka0m+V78QuDTLoMzMrHHGTQRKxn/4KfA/wO+lxR+MiCeyDszMzBpj3EQQESHpyoh4NvDDBsWUKXcVm5mNVk8fwW8lnZx5JBlzX7GZWW319BH8HvAGSQ8BPSR/UyMinpNpZGZm1hD1JIKXZR6FmZlNmj0mgoh4CEDSPJKHy8zMbArZYx+BpPMkrQEeBK4D1pHcRXRQ8oPFZmaj1dNZ/DHgVOC+dDKZM4EbM40qA/KkxWZmNdWTCIYiYhNQkFSIiGtJppk0M7MpoJ5EsFXSdOB64BuSPkty99C4JH1V0gZJd46x/Q2SfifpDkm/knT83oVuZmYToZ5EcD7QC7wf+AlwP/DKOl53ObB8nO0PAi9KH1b7GLCijmOamdkEq+f20QuA6yNiDfC1eg8cEddLWjTO9l+NWL0RWFDvsc3MbOLUkwgWAl+StJhkjoLrgRsi4rYJjONtjHMnkqSLgIsAFi5cuJ+n8m1DZmYj7fHSUERcEhEvAY4lmYfgr0gmrJkQkl5Mkgg+OE4MKyJiWUQs6+zs3Lfz7GN8ZmZT3R5bBJL+FjiNZB6CW4G/ZIImppH0HOArwDnpnUlmZtZg9VwaejVQBn5M8kDZryNiYH9PLGkh8D3gjRFx3/4ez8zM9k09Q0ycJKmDpFVwFrBC0oaIOH2810m6AjgDmCtpPXAJUEqPeRnwEWAO8IX0Ya9yRPj5BDOzBqvn0tCzgBcALyJ5kOwR6rg0FBEX7mH724G31xfmxPEQE2Zmo9U7VeUNwOeAmyNiKNuQsuERJszMaqvn0tArJLUBCw/WJGBmZmOrZ/TRVwK3kTxVjKQTJK3MOjAzM2uMeoaYuBQ4BdgKkD5ItjjDmMzMrIHqHX102y5lB22X60EbuJlZRurpLF4t6fVAUdIS4L3Ar/bwmgOO/GyxmVlN9bQI3gMcBwwA3wS2A3+eZVBmZtY49dw11At8OP0Bdj4V/HCGcZmZWYOM2yKQ9DxJr0knrkfScyR9E/hlQ6IzM7PMjZkIJH0S+CrwB8CPJf0DcDXwG2BJY8KbeH6y2MxstPEuDb0cODEi+iXNJhla4lkRsa4hkU0wP1lsZlbbeJeG+iOiHyAitgBrDtYkYGZmYxuvRXD0Lk8QLx65HhHnZReWmZk1yniJ4Pxd1j+VZSBmZjY5xkwEEXFdIwNplHBvsZnZKPU8UDYluK/YzKy2zBKBpK9K2iDpzjG2S9LnJK2V9DtJJ2UVi5mZja3uRCBp2l4e+3Jg+TjbzyF5HmEJcBHwxb08vpmZTYB65iN4vqS7gHvS9eMlfWFPr4uI64HN4+xyPvD1SNwIzJJ0eJ1xm5nZBKmnRfAvwMuATQARcTvwwgk493ySh9SGrU/LdiPpIkmrJK3q6urar5O6q9jMbLS6Lg1FxCO7FFUyiGW886+IiGURsayzs3PfDuLeYjOzmuqZj+ARSc8HQlIJeB9w9wSc+1HgyBHrC9IyMzNroHpaBH8KvIvkss2jwAnp+v5aCbwpvXvoVGBbRDw+Acc1M7O9UM98BBuBN+ztgSVdAZwBzJW0HrgEKKXHvAy4EjgXWAv0Am/Z23OYmdn+22MikPS5GsXbgFUR8cOxXhcRF4533Ege8Z2IloWZme2Hei4NtZJcDlqT/jyH5Hr+2yR9JsPYMuERJszMRquns/g5wGkRUQGQ9EXgBuB04I4MY5tQnrzezKy2eloEs4HpI9bbgUPSxDCQSVRmZtYw9bQIPgHcJukXJHfjvxD4J0ntwE8zjM3MzBqgnruG/l3SlcApadHfRMRj6fJfZRaZmZk1RL2DzvUDjwNbgKdLmoghJiZFeJAJM7NR6rl99O0kTxMvAG4DTgV+Dbwk29AmlievNzOrrZ4WwfuAk4GHIuLFwInA1kyjMjOzhqknEfRHRD+ApJaIuAd4RrZhmZlZo9Rz19B6SbOAHwDXSNoCPJRtWGZm1ij13DX0qnTxUknXAjOBn2QaVZbcV2xmNsq4iUBSEVgdEUsBIuK6hkSVAfcVm5nVNm4fQfr08L2SFjYoHjMza7B6+ghmA6sl3QT0DBdGxHmZRWVmZg1TTyL4u8yjMDOzSVNPZ/F1ko4ClkTETyVNA4rZh5YN9xWbmY22x+cIJL0D+A7wpbRoPsmtpHskabmkeyWtlXRxje0LJV0r6VZJv5N07t4EvzfkR4vNzGqq54GydwGnAdsBImINMG9PL0rvOPo8cA5wLHChpGN32e1vgW9HxInABcAX6g/dzMwmQj2JYCAiBodXJDVR3xWWU4C1EfFA+vpvAefvsk8AHenyTOAxzMysoepJBNdJ+hugTdJZwH8BP6rjdfOBR0asr0/LRroU+KN0cvsrgffUOpCkiyStkrSqq6urjlObmVm96kkEFwNdJNNS/gnJH+y/naDzXwhcHhELgHOB/5C0W0wRsSIilkXEss7Ozv06oecsNjMbrZ7bR38f+HpEfHkvj/0ocOSI9QVp2UhvA5YDRMSvJbUCc4ENe3muPXJfsZlZbfW0CF4J3CfpPyS9Iu0jqMfNwBJJiyU1k3QGr9xln4eBMwEkPRNoJWl9mJlZg+wxEUTEW4Cnk/QNXAjcL+krdbyuDLwbuAq4m+TuoNWSPipp+KnkvwDeIel24ArgzRG+eGNm1kh1fbuPiCFJ/0Nyl08byeWit9fxuitJ+hRGln1kxPJdJLemmpnZJKnngbJzJF0OrAH+APgKcFjGcWXGcxabmY1WT4vgTcB/An8SEQMZx5MZ9xWbmdVWz1hDF45cl3Q6cGFEvCuzqMzMrGHq6iOQdCLweuC1wIPA97IMyszMGmfMRCDpGJK7hC4ENpJcHlJEvLhBsZmZWQOM1yK4B7gBeEVErAWQ9P6GRGVmZg0z3l1DrwYeB66V9GVJZzIF+lz9lIKZ2WhjJoKI+EFEXAAsBa4F/hyYJ+mLks5uVIATxUNMmJnVVs+TxT0R8c2IeCXJeEG3Ah/MPDIzM2uIesYa2ikitqQjgZ6ZVUBmZtZYe5UIzMxs6sldInBfsZnZaDlKBO4tNjOrJUeJwMzManEiMDPLOScCM7OcyzQRSFou6V5JayVdPMY+fyjpLkmrJX0zy3gAPAGamdlo9c4/vNckFYHPA2cB64GbJa1MZyUb3mcJ8CHgtIjYImledvFkdWQzs4Nbli2CU4C1EfFARAwC3wLO32WfdwCfj4gtABGxIcN4zMyshiwTwXzgkRHr69OykY4BjpH0S0k3Slpe60CSLpK0StKqrq6ujMI1M8unye4sbgKWAGeQzHvwZUmzdt0pHdZiWUQs6+zsbHCIZmZTW5aJ4FHgyBHrC9KykdYDKyNiKCIeBO4jSQyZcVexmdloWSaCm4ElkhZLagYuAFbuss8PSFoDSJpLcqnogSyCcV+xmVltmSWCiCgD7wauAu4Gvh0RqyV9VNJ56W5XAZsk3UUy58FfRcSmrGIyM7PdZXb7KEBEXAlcuUvZR0YsB/CB9MfMzCbBZHcWm5nZJMtfInBvsZnZKLlJBIX00eJwJjAzGyU3iWB4iIlqdXLjMDM70OQmETzVIqjt/q5u1m7Y0biAzMwOEJneNXQg2dkiGGP00TM/dR0A6z7+8kaFZGZ2QMhfi6BGItjRP7Rz+dKVqxsWk5nZgSCHLYLdtz33H366c/nyX63j0vOOG7V9Y/cAzU0FOlpLWYY4YYYqVVZc/wD3PLGD0542h9se2cqGHQMc2tHKj25/jO6BMkvmTee9Zy7h8W197OgvM39WGycvPoSndU5nW+8Q01qKlIq7f0+4ed1mbnpwM3+47Eh6Bsqs29TDkYdM44iZbezoH+KRLX2ctHAWkqhUg219Q8yeVuKJ7f30DVa478kdnLRwNvM6WgGoVoOhapVN3YO0tzQxs62x7/GGHf3MbCshxGClypaeQTpntFCQaCqIru4BdvQPcdSc9prvhx2YBstVHtnSS1upSOeMFnoHK2zvG2KgXN25bUd/mektRdpbmpje0kSpWEBKvjRWqkE1gpamIsUCdLSWkERE0FQsMKO1aZ8/D8NfRjVibPyIIAKGqlU29wwyVA5mTisxVKlSKhZoby7SlOHnLzeJ4KkWwe7bBsuje5AXXfxjbv27s2gtFWktFViWJopPvOY5vOrE+TU/AD0DZdpbmvjZ3U/ytq+tAuDq97+Qp3VOp1gY/R/+0KZeChKHzWylVBSS6BkoI0FrU5FCQQxVkpjq+bCt29jDOZ+9gb6hym7bfnT7YzVfs2ZDN++54tbdyp/W2c79XT2jyp55eAdrN+ygGlBJM+knr7p33JhaSwX6h8bume9oTT56/ekv5rD5s9qY1lykrblIQeJZ8ztoLhbpHhjixIWz2d43RFtzkaPnTmfdph7ufHQbdzy6jY3dA7zomE6O7pzOULlKz2CFtlKR5qYCbaUCPYMVigUxb0YLG7sHuPGBzdzz+HYe29Y/ZozFgnbWtyCYP7uN2dOaKRULPOOwGUQES+bNoLmpwM3rNvPs+TN5eHMvpWKBSjWoVIMFs9uY19HCMw7toFyt0jdYoVINtvYNceTsabQ1F+loa2JT9yB9QxX6Byt0zmjh6fOmj/pDMZ7ewTIPb+4F4PCONlDyZWCoUqWtVGTWtGYGyhW29Q6xuXeQHf1lHt7Uy9Gd7fQMVLjr8W3c+vBWKtVgZluJ447o4Lj5MxkYqtLV3U/vYIVyJdjSO0hTQTQ3FWhpKtLSVGBeRwuSaGkqMDBUpW+ogoDmpuRzO1QJugfKlIqipSn5f21Nt5WrweaeQR7b2kf/UJWZbU1MT79sdfcPsWD2NObOaGEw/YwMVioMDFXZ2JPEUakGW3oG2dg9wBPb+9neV2Zb3xD9QxXWb+ljsJLtnSHtzUVmtpUoFET/UJWWpgItpeT/fnpLE3OmtzCzrcRjW/vY3DPIYLlKqSge29ZPRCCJappwan1B3VWpKP70RU/jL85+xoTXRQfbjF3Lli2LVatW7fXr1m7YwUs/fT3/euGJvPL4I3aW/+r+jbz+y7/hsI5WNuzor+s/5HlHz+GKi04FoG+wwjM/8hMA3vuSp/O5n6/dbf87Lj2bGa0lHt3ax2kf//lexf3TD7yQp8+bAcAjm3tZv6WPpYfNYFv67ebq1U/wqWvu2+11Jy6cxTMP72DpYTM4/elz6Rmo8Olr7uVVJy3gpc+cx/+78SFueWgLbz1tMTv6y7z968l7Omtaie19Q+O+DzPbShzd2c7iOe08bd501m/p44ltffQOVmhuKrBg9jRuenATEXDErDbamos0FcQxh87g0I5WunYMsLlnYOcfuqaCOGrONO7v6mFzzyA7+ocYqgRPbu9nzYZuWpoKBLsn7GFHd7YzWK6yfktf3e9r54wWFs9p57j5HbSWikxLk8aW3iFWP7aNE4+cRbkaHD6rjaLE49v6WLeply09g2ztG+ShTb00Fwts6hkcddy2UpGBcoWWpmLNxFyv5mKB1lKBw2e2MXNaiWnNRTZsH2Ddpp6d33KHKlW295fp2jEw7rGmNRfpHRw/lpltJbb3DzGnvZmN3YPj7nugmdHaxLwZLcye1szMthLFgjhiVhvPnj+TgXKVDTv6md7SREdbidZSkVIh+RI2p72FnsEy3QPJz1C5SjX9Zl4siIJEf7lCNWBb3xDValBQksC295XZ3j/E5p5BKtWgvaWJgXKFgXKVcqXKUCXYsKOfHf1lDu1o5dCOViKSLwdHzGqjqZh89osSxULyZTBZhkPaW2gqiu19Q5SKBQbLSYLtG6pwyqJDePHSfZu/S9ItEbGs5rb8JIJuXvrp6/jsBSdw/glPTYuw/DPXc88TOzhiZis/ePdpnPKPP6vreB86ZylvOPUonnXJVXvc90XHdPJHpx7FO76+93EDXP6Wk3nz/715zO1zp7fwmdedgATPPWo2raXiPp2nlk3dA9zf1cNjW/t48dJ5Db90M1Cu0FQoUI3gtw9t4c7HttNaKlAqFJgzvZmTFx+y85JdpRr0DpYJkj+k2/uHWPNkN0/rnL7zFwugvSX5w1HvN+7xPLK5l4Fyld7BMofPbKNzRsvOpn81kv6nGx/YzEC5QjWC2dOaaU2TzuNb+xmqVHl4cy8LZrfR3tJEe3MTD2zs5qFNvQyWq9zf1c2m7kGam5L6Hjl7Gjv6h9jWN8S05ibaW4ocNaedhYdMA2D9lj5KRTFQrjKjtYnugTJPbutn7vQWZrU3M3taifaWJpqLBdZv6WVmWzMnLpzFoemluojg4c29PLAxSThzp7fQ0dpEsSBmpO/zQLnCYLlKz0CFJ7b37/wD19ZcpLVUTN6PgTItpQKtpSLtzU1UIugfqtA/VKU/TZBNBdHRVuKIWW20NxfZ1jdEz0CFIDnWw5t62d4/RHOxmLZCCsn70N7MUDUoSsxuL9HSNHGf96nMiQB4oKubl3zqOv7ldcfzqhMX7CxfdPGPATjuiA5+/N4X8OxLr2JHf3m316/5x3O46cHNvOErvxn3PDNamrjj718GJL9Uiz80aqglOlqb+M47n8/c6S0MlCscPrNt5+WHXS8hvf1rq/jZPWNP2nbq0Ydw7rMP57zjj2DWtOY9vANmlmfjJYLc9BEM/7H9xx/fMyoRDBv+Fn3Hpckf8eEEcd7xR/Chc5dSKhY47elzWffxl3PqP/2MJ7Y/dW35i284iaWHd/D9367n/Wcds7Nc0qjLRbsmoWEjE8DI137pjc/lU9fcxxd/cT83ffhM5s1o3dfqm5mNKTeJYCC9vryxu/b11GMOnTFq/bvvfD6zp5U4unP6bvuufPdp/N0P76RULPB/Xnv8ziTygRqdOO85cwmvPmkBi+a273XMTcUCH1y+lA8uX7rXrzUzq1duEsFYZk0r0VYqcskrjx1V/tyjZo/5mnkdrXzpjTVbWLspFQv7lATMzBol1zdGRwS9AxXOO+GICe1gNTM7mGSaCCQtl3SvpLWSLh5nvz+QFJLq+5q9Dwo17hC57ZGtDFaqTG/OfcPIzHIss0QgqQh8HjgHOBa4UNKxNfabAbwPGP92nP2O56nlJ9OO3tde9msAbli7MctTm5kd0LJsEZwCrI2IByJiEPgWcH6N/T4G/DMw9iOeE2DkXbJ96cM1zzgs6SB+xwuOzvLUZmYHtCwTwXzgkRHr69OynSSdBBwZET8e70CSLpK0StKqrq6ufQqmveWpPoCCxBd+sZZlaYfwWcceuk/HNDObCibt4rikAvBp4M172jciVgArIHmgbF/Od9Scp+7cufquJ/jET8YfK8fMLC+ybBE8Chw5Yn1BWjZsBvAs4BeS1gGnAiuz7DAe1j2w+5PDZmZ5lWUiuBlYImmxpGbgAmDl8MaI2BYRcyNiUUQsAm4EzouIfRuQZy/87xp3DpuZDcssEUREGXg3cBVwN/DtiFgt6aOSzsvqvPVY9dCWncsjRyI1M8ujTPsIIuJK4Mpdyj4yxr5nZBnLWJYeNmPPO5mZTWG5frIYkrHazczyLPeJYOQoomZmeZT7RNBUYwhoM7M8yX0iaC760pCZ5VvuE8GbT1s02SGYmU2q3A67+aU3PpfjF8xq+By8ZmYHmty2CNqbmzhspqd+NDPLbSKoNU+wmVkeORGYmeVcbhPBvBktkx2CmdkBIVeJ4Pt/9vydy55Q3swskatEcOLC2ZMdgpnZASdXicDMzHbnRGBmlnNOBGZmOZfpk8WSlgOfBYrAVyLi47ts/wDwdqAMdAFvjYiHsoxpxRufi+RbR83MhmWWCCQVgc8DZwHrgZslrYyIu0bsdiuwLCJ6Jb0T+ATwuqxiAjj7uMOyPLyZ2UEny0tDpwBrI+KBiBgEvgWcP3KHiLg2InrT1RtJJrg3M7MGyjIRzAceGbG+Pi0by9uA/8kwHjMzq+GAGH1U0h8By4AXjbH9IuAigIULFzYwMjOzqS/LFsGjwJEj1hekZaNIeinwYeC8iBiodaCIWBERyyJiWWdnZybBmpnlVZaJ4GZgiaTFkpqBC4CVI3eQdCLwJZIksCHDWMzMbAyZJYKIKAPvBq4C7ga+HRGrJX1U0nnpbp8EpgP/Jek2SSvHOJyZmWUk0z6CiLgSuHKXso+MWH5pluc3M7M985PFZmY5p4iY7Bj2iqQuYF+fPp4LbJzAcA4GrnM+5K3Oeasv7H+dj4qImnfbHHSJYH9IWhURyyY7jkZynfMhb3XOW30h2zr70pCZWc45EZiZ5VzeEsGKyQ5gErjO+ZC3OuetvpBhnXPVR2BmZrvLW4vAzMx24URgZpZzuUkEkpZLulfSWkkXT3Y8+0PSVyVtkHTniLJDJF0jaU367+y0XJI+l9b7d5JOGvGaP073XyPpjyejLvWQdKSkayXdJWm1pPel5VO5zq2SbpJ0e1rnv0/LF0v6TVq3/0zH8UJSS7q+Nt2+aMSxPpSW3yvpZZNTo/pIKkq6VdJ/p+tTvb7rJN2RDrGzKi1r/Oc6Iqb8D8lUmfcDRwPNwO3AsZMd137U54XAScCdI8o+AVycLl8M/HO6fC7JPA8CTgV+k5YfAjyQ/js7XZ492XUbo76HAyelyzOA+4Bjp3idBUxPl0vAb9K6fBu4IC2/DHhnuvxnwGXp8gXAf6bLx6af9xZgcfp7UJzs+o1T7w8A3wT+O12f6vVdB8zdpazhn+u8tAj2OFvawSQirgc271J8PvC1dPlrwO+PKP96JG4EZkk6HHgZcE1EbI6ILcA1wPLso997EfF4RPw2Xd5BMojhfKZ2nSMiutPVUvoTwEuA76Tlu9Z5+L34DnCmJKXl34qIgYh4EFhL8vtwwJG0AHg58JV0XUzh+o6j4Z/rvCSCvZ0t7WB0aEQ8ni4/ARyaLo9V94PyPUkvAZxI8g15Stc5vUxyG7CB5Jf7fmBrJCP7wuj4d9Yt3b4NmMPBVefPAH8NVNP1OUzt+kKS3K+WdIuSCbhgEj7XB8QMZTaxIiIkTbn7giVNB74L/HlEbE++ACamYp0jogKcIGkW8H1g6SSHlBlJrwA2RMQtks6Y7Hga6PSIeFTSPOAaSfeM3Nioz3VeWgR1zZZ2kHsybSaS/js80c9YdT+o3hNJJZIk8I2I+F5aPKXrPCwitgLXAs8juRww/AVuZPw765Zunwls4uCp82nAeZLWkVy6fQnwWaZufQGIiEfTfzeQJPtTmITPdV4SwR5nS5sCVgLDd8uYrXcAAAMdSURBVAv8MfDDEeVvSu84OBXYljY7rwLOljQ7vSvh7LTsgJNe+/134O6I+PSITVO5zp1pSwBJbcBZJH0j1wKvSXfbtc7D78VrgJ9H0pO4ErggvctmMbAEuKkxtahfRHwoIhZExCKS38+fR8QbmKL1BZDULmnG8DLJ5/FOJuNzPdm95o36Ielxv4/kOuuHJzue/azLFcDjwBDJ9cC3kVwf/RmwBvgpcEi6r4DPp/W+A1g24jhvJelMWwu8ZbLrNU59Tye5lvo74Lb059wpXufnALemdb4T+EhafjTJH7a1wH8BLWl5a7q+Nt1+9IhjfTh9L+4FzpnsutVR9zN46q6hKVvftG63pz+rh/8uTcbn2kNMmJnlXF4uDZmZ2RicCMzMcs6JwMws55wIzMxyzonAzCznnAgstyR1p/8ukvT6CT723+yy/quJPL7ZRHIiMINFwF4lghFPu45lVCKIiOfvZUxmDeNEYAYfB16Qjgn//nSwt09Kujkd9/1PACSdIekGSSuBu9KyH6QDhq0eHjRM0seBtvR430jLhlsfSo99ZzoO/etGHPsXkr4j6R5J39DIwZTMMuRB58ySMd//MiJeAZD+Qd8WESdLagF+KenqdN+TgGdFMsQxwFsjYnM6DMTNkr4bERdLendEnFDjXK8GTgCOB+amr7k+3XYicBzwGPBLkvF3/nfiq2s2mlsEZrs7m2RMl9tIhrueQzJmDcBNI5IAwHsl3Q7cSDLw1xLGdzpwRURUIuJJ4Drg5BHHXh8RVZJhNBZNSG3M9sAtArPdCXhPRIwauCsdHrlnl/WXAs+LiF5JvyAZA2dfDYxYruDfT2sQtwjMYAfJFJjDrgLemQ59jaRj0tEhdzUT2JImgaUk0wcOGxp+/S5uAF6X9kN0kkw7ekCOjmn54W8cZskIn5X0Es/lJOPgLwJ+m3bYdvHUdIEj/QT4U0l3k4x0eeOIbSuA30n6bSTDKQ/7Psm8AreTjKj61xHxRJpIzCaFRx81M8s5XxoyM8s5JwIzs5xzIjAzyzknAjOznHMiMDPLOScCM7OccyIwM8u5/w/Vb49l+Dtm/QAAAABJRU5ErkJggg==
" />
</div>
</div>
<div class="output_area">
<div class="output_subarea output_stream output_stdout output_text">
<pre>
Epsilon-Greedy Agent, epsilon= 0
maximum possible reward: 0.9851042878107023
average reward: [0.98686237]
cosine similarity 1.0
euclidian norm 0.0
</pre>
</div>
</div>
<div class="output_area">
<div class="output_png output_subarea ">
<img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAYoAAAEGCAYAAAB7DNKzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de5zcdX3v8dd7Zmf2kjtJuIaQoLEUFIGzAgpFVFSwCD2tbaH2SKuVakVr77S01tqec2w9vRx7aC21HNGHgpZWTCuKqCjUCiRc5Z4QbgmBhEBue53Lp3/8frOZHXYnk+zO/Dab9/Px2Ed+t53f5zuZ/X3me/l9f4oIzMzMJpPLOgAzM5vZnCjMzKwpJwozM2vKicLMzJpyojAzs6a6sg5gui1ZsiRWrFiRdRhmZgeUu+6664WIWDrRvlmXKFasWMHatWuzDsPM7IAi6anJ9rnpyczMmnKiMDOzppwozMysKScKMzNrKtNEIelqSVskPTDJfkn6tKT1ku6XdEqnYzQzO9hlXaP4HHBuk/3nAavSn0uBv+9ATGZmVifTRBERtwIvNjnkQuDzkbgdWCjpiM5EZ2ZmMPPvozgKeKZufWO6bXP9QZIuJalxsHz58o4Fl5VypUpXPkdEIKkt55jstZ98YYDbN2zj0PndSCIiKFeC3SNlAOb1FFjUV2DZoj4Wzy3ywKYdVCM4cdlCCvn9/14SEWzaPkRfsYuFvQUAcrn2lN1sMrXHMrTr726mmumJoiURcRVwFUB/f/+secBG7WIdEdz99Etc/R9PsnnHEHc/vX3cccV8jqMW9fLm4w7l7ScczryeLgZHK2zdNcIdT2zjgU07yEkMlSosnlOkt5hntBwctbCHf7t/M6etPISeQp5li3oZKVe5Y8M27tu4gx87bB5HLuxh6+4RNmwdQMDAaGW/y/OaoxZwzOI+ivkcyxf38cjmXQyMlpnbncS7c7hEqVJlTrGLfE5IEAGVarDhhQG27hoZV+bDFnQzp9hFIZ+jr5inu5CnUq1yyJxuunJK4y2zddcIhy/o4dntw+wcLnHskjkcs3gORy7sZfGcIgv6CmzePszgaJneYp5iPkc1gmpATyFHPpdj13CJvmKekVKV3mKevmIX3V05tuwaoa+YHzuuuytHuRIMjJbJSXTlRFde5HNi6dxuDl/QQyGfQ4JCLkepWiUCegr5vb5/g6NlhktVBkbKlKtBbyHP9qFRdgyW2DVcJp8XxXyOud1dBNCVS86bU/JelipVypWgu5Abe49LlerY65cqtT+doFxNvgBUIxguVXl+5zClSpVdw2UGRyvp9go9hXx6DshJlNLfqVSTn4hgXk+BcjXYMVRiuFRJ9kWyLyL5TO0cKpHPJe/XUKnC7pEyu4bLVKpBXzGPBEKUq8FIucJoucpIuUqpUqWYz1HsylGtJvEXunLkc4KASP+OaiXL50Qhl/z/BlC73FfTWPI5jf3/DJcqjJSrDJeqjJST5dFy8n4V8qKnK09PMc/8ni7m9hToSc/bV8yzdF4PEcFo+p4X8jl6CjlGy1Uq6fszOFpJPx858kq++AyOVNg1UmLnUJmdwyX6il30FfOUq8FoOfm/37Z7hJFylXJa3q405q68OOnohXzhfaft99/oZJT1g4skrQD+PSJePcG+fwC+FxHXpuuPAmdHxObGY2v6+/tjJtyZ/d1Hnue9n1vL3158MuefeMTLvoEMjJT5wfoXkJIL2qbtQwDc+cSLDI6WKeRzfOuh51/2un3FPH3FPC/sHuWQOUVWLO5j6+4RKpXg2R3Dk8Zz7JI5zO3pYsdQCQFbdo1QqlQ5YkEvlfSPuFYr+PEj5nPInAJbd41QrgTzewu8Yulc5vV0sWxRL6cfu5iBkTK59AIxMFJhQW+BfE6MVqrsHCrx1LZBtg2McswhfewaLrHhhQGeSH+GRitsGxhlUV+BIxb0MlyqkM+Jw+b3UMiLgdEK1WpywQHoyiWJ8DVHLaAawc7hMqPlKpt3DDE0Whm7gJWqkZRtZ/I+SGJOdz69oJY4fH4PC/sKPL51gI0vDTJcqk7ybnVeTyHH0nnddOWSWtdIqcL83gIvDowylF6Yp5KkZ4JCXvQU8uMSGEBvMc/C3gKVtHbaU8gzr6eLeT1dSGIw/VwmyS9Jxt1duT0X30qVkVKVfE4ESY279lmo/X3V/vzKlaBUqY77IiIlx+UkypU9yaC3mKe7K093oXbOPMWuHAJGK1WGSxWGSxV2DJUYGKkwlCbBgZEyL+weJZ9L4i3kNZbYar9f6MrRW8hTqSZJuZZY+4p55vcWmNvdxYLeAoOjFQZHy3Tlkxj6inkWz+mmp5CjK61Rl6tJmUqV4MiFPVx61iv26/9H0l0R0T/Rvpleo1gNXCbpOuA0YEezJJGliOC1f/Itdg6XueC1R7L6vmcB+PC19/Dha+/hmMV9/O3FJ3PLI1v5628/tk+vfezSOWzYOsAvnr6cP73w1ZNWe7927yae3jZIJYKdQ2V2j5S45A0rOOHIBRPGC+Or0LUENZUmolYNjpbpLeQzq8JXq8HO4RIv7B5l53CJud1Jk1YAo+UqubRGMlyqUK4G83sKDJUqFLtyDI1Wkp9ShUPmFChVgsHRMqX0ItRTSJJTBJSrVSrVoFQJNu8YYmuaoCtVGCpVmNudp1KFncMlXhwYHTt3d1eO7YMlTjhyQXrBhEPn9VDsylHMi7lprXFBb4GFvUXm9XRRieRb5+7hMlJyAYm0ZlRLur2F/FitpBox9n9diaC7K0dE8s22djGvfVutJfEFvQX6isllo6eQY6RcHVfO7q7k238+J/JpTWbHUInurqTWdbA12cwWmdYoJF0LnA0sAZ4H/hgoAETEZ5R8qv4fycioQeCXI6JpdSGLGsXT2wY561O3vGz7/J4udg6XW3qNvmKeS96wgmOXzOGEIxewcskceot7b44wM5sOM7ZGEREX72V/AB/qUDj75ZkXxyeJ4w6fxyPP7aKnkOP+j78dSL4tfuyGB7jh3qSWce37T+f1r1icSbxmZvsq8z6K6dbJGsUZn/zuWN8CwDXvPZU3vmrCWXrNzGa0GVujOJANjVbGksS87i7u/eO3JSMtzMxmGSeKfTRcqnDcH31zbP2IBT3c9rtvcpIws1kr6yk8Djj/8+sPj1u/8SM/QVcHRgmZmWXFV7h9sGOwxBduTx4C9ZOvOYLD5/ewaE4x46jMzNrLTU/74H9/I6lNHHf4PK58tyeyNbODg2sU++C6Ncm0U//4ngkHBpiZzUpOFC0q182Jc/QhfRlGYmbWWU4ULXrlFd8A4JwfPyzjSMzMOsuJYh99+uKTsg7BzKyjnChacN8zybTePYXc2IRoZmYHCyeKFvzS/78TYEZNS21m1ilOFHtxwz2beGmwBMCjf9bs8d5mZrOTE8VefPTL944td3d52m8zO/g4UezFKcsXAnD9B16fcSRmZtlwomjiiRcGuPvp7fy3YxbRv+KQrMMxM8uEE0UTb/o/3wPguSbPojYzm+2cKFpw/Qfd7GRmBy8nihYcsaA36xDMzDLjRDGJZ9On1/2P04/JOBIzs2w5UUzi0ed3AXD+iUdkHImZWbacKCbx9LZBAFYunZNxJGZm2XKimMS2gVEkWDynO+tQzMwylWmikHSupEclrZd0+QT7l0u6RdI9ku6X9I5OxbZjcJT5PQXyOXXqlGZmM1JmiUJSHrgSOA84HrhY0vENh/0h8JWIOBm4CPi7TsV37Zpn2DFU6tTpzMxmrCxrFKcC6yNiQ0SMAtcBFzYcE8D8dHkB8GyngusteF4nMzOALB+ucBTwTN36RuC0hmM+DnxL0oeBOcA5nQktSRRvPd5PszMzm+md2RcDn4uIZcA7gC9IelnMki6VtFbS2q1bt075pBHBtoERlsx1R7aZWZaJYhNwdN36snRbvfcBXwGIiB8CPcCSxheKiKsioj8i+pcuXTrlwHYMlShVgiVzi1N+LTOzA12WiWINsErSSklFks7q1Q3HPA28BUDSj5MkiqlXGfZi664RAJbOc43CzCyzRBERZeAy4CbgYZLRTQ9K+oSkC9LDfgt4v6T7gGuBX4qIaHds29PRTov6XKMwM8uyM5uIuBG4sWHbx+qWHwLO6HRcI+mzsXuLHvlkZjbTO7MzMVKuANDd5bfHzMxXwgmMlJMahZ+RbWbmRDEh1yjMzPbwlXACtT6K7oLfHjMzXwknUGt6Kub99piZ+Uo4gbGmJ8/3ZGbmRDGRsaYn91GYmTlRTOQvb34MgC4/i8LMzImiGcmJwszMicLMzJpyojAzs6Yynetppjp8fg9vfNXUpys3M5sNXKOYwEi54pvtzMxSvhpOYKRc9dBYM7OUr4YTSBKFb7YzMwMnipcpV6pUquEahZlZylfDBmNTjLuPwswMcKJ4GT+LwsxsPCeKBn4WhZnZeL4aNtg5VAagr9u3mJiZgRPFywyOJoliXo8ThZkZOFG8TLkagGeONTOrcaJoUK4kiSLvRGFmBjSZ60nS3wIx2f6I+EhbIspYuZqMeir4MahmZkDzGsVa4C6gBzgFWJf+nAQUp+Pkks6V9Kik9ZIun+SYn5P0kKQHJX1pOs7bTK3pyTUKM7PEpDWKiLgGQNIHgTMjopyufwa4baonlpQHrgTeCmwE1khaHREP1R2zCvh94IyIeEnSoVM9795U0qanQs41CjMzaK2PYhEwv259brptqk4F1kfEhogYBa4DLmw45v3AlRHxEkBEbJmG8zZVa3pyjcLMLNHKGNBPAvdIugUQcBbw8Wk491HAM3XrG4HTGo55FYCkHwB54OMR8c3GF5J0KXApwPLly6cU1Niop7wThZkZ7CVRKHlo9LeBb7DnIv57EfFcuwNLdQGrgLOBZcCtkl4TEdvrD4qIq4CrAPr7+yftgG9FbdSTh8eamSWaJoqICEk3RsRrgK9N87k3AUfXrS9Lt9XbCNwRESXgCUmPkSSONdMcy5g991G4j8LMDFrro7hb0uvacO41wCpJKyUVgYuA1Q3H3EBSm0DSEpKmqA1tiGVMpdZH4aYnMzOgtT6K04B3S3oKGCDpp4iIOHEqJ46IsqTLgJtI+h+ujogHJX0CWBsRq9N9b5P0EFABficitk3lvHtTGhv15ERhZgatJYq3t+vkEXEjcGPDto/VLQfwm+lPR1R8H4WZ2Th7TRQR8RRAeg9DT9sjytieUU/uozAzgxb6KCRdIGkd8ATwfeBJklFQs1K5kvRReNSTmVmila/NfwqcDjwWESuBtwC3tzWqDHkKDzOz8VpJFKW0AzknKRcRtwD9bY4rM7X7KDwpoJlZopXO7O2S5gK3Al+UtIVk9NOsVBse6wqFmVmila/NFwKDwG8A3wQeB97ZzqCyVK4GhbxIbko3M7NWahQXAbdGxDrgmjbHk7lyNdw/YWZWp5VEsRz4B0krSZ5RcStwW0Tc29bIMlKuhKfvMDOrs9crYkT8cUS8GTie5DkUv0PyQKNZqVyteuZYM7M6e61RSPpD4AyS51DcA/w20/DgopmqXA3fQ2FmVqeVpqefBsrA10luuPthRIy0NaoMVdz0ZGY2TitNT6cA5wB3kjy29EeS/qPdgWWlVK26M9vMrE4rTU+vBn4CeCPJjXbPMIubnirVcB+FmVmdVh+FehvwaWBN+hChWct9FGZm47Uye+z5knqB5bM9SUAyKaD7KMzM9mhl9th3AveS3JWNpJMkNT6JbtZw05OZ2XitfHX+OHAqsB0gvdFuZRtjylSp4qYnM7N6rc4eu6NhW7QjmJmg4ik8zMzGaaUz+0FJvwDkJa0CPgL8Z3vDyk5yZ7b7KMzMalq5In4YOAEYAb4E7AQ+2s6gslR205OZ2TitjHoaBK5IfwCQtBx4uo1xZaZcDfpcozAzG9P0iijp9ZLeJenQdP1ESV8CftCR6DJQrlZdozAzqzNpopD0KeBq4GeAr0v6M+BbwB3Aqs6E13lDoxV6Cq5RmJnVNGt6+kng5IgYlrSIZOqOV0fEkx2JLCPDpSo9hXzWYZiZzRjNvjoPR8QwQES8BKyb7iQh6VxJj0paL+nyJsf9jKSQ1D+d559IpRoUfGe2mdmYZjWKYxvuwF5Zvx4RF0zlxJLywJUkM9JuBNZIWh0RDzUcNw/4dZImr7YrV4O878w2MxvTLFFc2LD+l9N87lOB9RGxAUDSdek5H2o47k+BPyd5sl7bVdyZbWY2zqSJIiK+3+ZzH0XS71GzETit/gBJpwBHR8TXJU2aKCRdClwKsHz58ikFVfad2WZm48zYxnhJOeCvgN/a27ERcVVE9EdE/9KlS6d03oqnGTczGyfLRLEJOLpufVm6rWYe8Grge5KeBE4HVre7QzupUczY/Glm1nEtXxEl9U3zudcAqyStlFQELgLqO8t3RMSSiFgRESuA24ELImLtNMcxTjIpYDvPYGZ2YGnleRRvkPQQ8Ei6/lpJfzfVE0dEGbgMuAl4GPhKRDwo6ROSpjSiagoxpYnCmcLMrKaV2WP/Gng76bf9iLhP0lnTcfKIuBG4sWHbxyY59uzpOGcz1XTydPdRmJnt0dJX54h4pmFTpQ2xZK5crQJ41JOZWZ1WahTPSHoDEJIKJDe/PdzesLJRSasUrlGYme3RSo3iA8CHSO572ASclK7POuU0UbhGYWa2RyvPo3gBeHcHYslcpeIahZlZo70mCkmfnmDzDmBtRHxt+kPKzliNwuNjzczGtHJF7CFpblqX/pxIcnPc+yT9TRtj6zj3UZiZvVwrndknAmdERAVA0t8DtwFnAj9qY2wdNzbqSU4UZmY1rdQoFgFz69bnAIekiWOkLVFlpOLObDOzl2mlRvEXwL2SvgcIOAv4X5LmAN9uY2wdN9b05OdRmJmN2WuNIiL+CXgDcAPwVeDMiPhsRAxEREeeEdEp19+1EYC7n3op40jMzGaOVof3DAObgZeAV07XFB4zzZonXwRg3ZbdGUdiZjZztDI89ldI7sZeBtxLMt33D4E3tze0zqvN9eS+bDOzPVqpUfw68DrgqYh4E3AysL2tUWWkGkmmyDlTmJmNaSVRDEfEMICk7oh4BPix9oaVjVqCKPqGOzOzMa1cETdKWkjSmX2zpK8BT7U3rGy8+7Tkedu/9qZXZhyJmdnM0cpcT/89Xfy4pFuABcA32xpVRq66dQMA83paGTVsZnZwaHpFlJQHHoyI4wAi4vsdiSojjzy3C0huFjEzs0TTpqf07utHJS3vUDwzgtyZbWY2ppU2lkXAg5LuBAZqGyMik+dad4Jn8DAz26OVRPFHbY9ihnGNwsxsj1Y6s78v6RhgVUR8W1IfkG9/aNlxjcLMbI+9Do+V9H7geuAf0k1HkQyVnbXk7mwzszGt3EfxIeAMYCdARKwDDm1nUFlzy5OZ2R6tJIqRiBitrUjqAqJ9IWUv57YnM7MxrSSK70v6A6BX0luBfwb+bTpOLulcSY9KWi/p8gn2/6akhyTdL+k7aV9J2zlNmJnt0UqiuBzYSvLY018FbgT+cKonTm/muxI4DzgeuFjS8Q2H3QP0R8SJJP0kfzHV87bCkwKame3RyvDYnwI+HxH/OM3nPhVYHxEbACRdB1wIPFQ7ICJuqTv+duAXpzmGCTlPmJnt0UqN4p3AY5K+IOn8tI9iOhwFPFO3vjHdNpn3Ad+YaIekSyWtlbR269atUw7MicLMbI9WHoX6y8ArSfomLgYel/TZdgdWT9IvAv3ApybaHxFXRUR/RPQvXbp06udzL4WZ2ZiWagcRUZL0DZLRTr0kzVG/MsVzbwKOrltflm4bR9I5wBXAGyNiZIrnbEneo57MzMa0csPdeZI+B6wDfgb4LHD4NJx7DbBK0kpJReAiYHXDuU8mudHvgojYMg3nbElfcVbfeG5mtk9aqVG8B/gy8KvT+Y0+IsqSLgNuIpkS5OqIeFDSJ4C1EbGapKlpLvDP6fxLT7dzMsL+YxbxxAsD9BScKMzMalqZ6+ni+nVJZwIXR8SHpnryiLiRZLht/baP1S2fM9Vz7ItiV46VS+Z08pRmZjNeS30UaRPQLwA/CzwB/Gs7g8pKhO+hMDNrNGmikPQqklFOFwMvkDQ/KSLe1KHYOq4a4duyzcwaNKtRPALcBpwfEesBJP1GR6LKSOA8YWbWqNmop58GNgO3SPpHSW9htl9H3fRkZvYykyaKiLghIi4CjgNuAT4KHCrp7yW9rVMBdlI1wndlm5k1aOXO7IGI+FJEvJPkprh7gN9re2QZCDx9h5lZo1bmehoTES+l02W8pV0BZSki3PRkZtZgnxLFbFed1Y9jMjPbP04UdZKmJ9cozMzqOVGkypUqj2/Zzc6hUtahmJnNKE4UqR1DJXaPlLn3me1Zh2JmNqM4UZiZWVNOFGZm1pQThZmZNeVEkfJoJzOziTlRmJlZU04UZmbWlBNFyg1PZmYTc6IwM7OmnCjMzKwpJ4qUBz2ZmU3MicLMzJpyojAzs6acKFLyuCczswllmigknSvpUUnrJV0+wf5uSV9O998haUXnozQzO7hlligk5YErgfOA44GLJR3fcNj7gJci4pXAXwN/3tkozcwsyxrFqcD6iNgQEaPAdcCFDcdcCFyTLl8PvEWelMnMrKOyTBRHAc/UrW9Mt014TESUgR3A4sYXknSppLWS1m7dunX/onH6MTOb0KzozI6IqyKiPyL6ly5dmnU4ZmazSpaJYhNwdN36snTbhMdI6gIWANs6Ep2ZmQHZJoo1wCpJKyUVgYuA1Q3HrAYuSZffBXw3IqIdwbjnw8xsYl1ZnTgiypIuA24C8sDVEfGgpE8AayNiNfBPwBckrQdeJEkmZmbWQZklCoCIuBG4sWHbx+qWh4Gf7XRcZma2x6zozJ4OtZannoLfEjOzer4qmplZU04UZmbWlBNFyjd8m5lNzInCzMyacqJo4OnGzczGc6JoELTlfj4zswOWE0XK9Qgzs4k5UTRw05OZ2XhOFA3c9GRmNp4TRcqjY83MJuZE0cBNT2Zm4zlRNHDTk5nZeE4UZmbWlBNFAzc9mZmN50TRwE1PZmbjOVGYmVlTThQN3PRkZjaeE4WZmTXlRGFmZk05UZiZWVNOFGZm1pQThZmZNeVEYWZmTWWSKCQdIulmSevSfxdNcMxJkn4o6UFJ90v6+SxiNTM72GVVo7gc+E5ErAK+k643GgTeExEnAOcCfyNpYQdjNDMzsksUFwLXpMvXAD/VeEBEPBYR69LlZ4EtwNKORWhmZkB2ieKwiNicLj8HHNbsYEmnAkXg8Un2XyppraS1W7dund5IzcwOcl3temFJ3wYOn2DXFfUrERGSJp2JT9IRwBeASyKiOtExEXEVcBVAf3//fs3qV5u6o7eY359fNzObtdqWKCLinMn2SXpe0hERsTlNBFsmOW4+8HXgioi4vU2hAkmCuPy843jr8U0rN2ZmB52smp5WA5eky5cAX2s8QFIR+Crw+Yi4vhNBfeCNr+AVS+d24lRmZgeMrBLFJ4G3SloHnJOuI6lf0mfTY34OOAv4JUn3pj8nZROumdnBSxGz60E9/f39sXbt2qzDMDM7oEi6KyL6J9rnO7PNzKwpJwozM2vKicLMzJpyojAzs6acKMzMrCknCjMza2rWDY+VtBV4agovsQR4YZrCORAcbOUFl/lgcbCVearlPSYiJpx4ddYliqmStHayscSz0cFWXnCZDxYHW5nbWV43PZmZWVNOFGZm1pQTxctdlXUAHXawlRdc5oPFwVbmtpXXfRRmZtaUaxRmZtaUE4WZmTXlRJGSdK6kRyWtl3R51vFMhaSrJW2R9EDdtkMk3SxpXfrvonS7JH06Lff9kk6p+51L0uPXSbpkonPNBJKOlnSLpIckPSjp19Pts7nMPZLulHRfWuY/SbevlHRHWrYvpw8AQ1J3ur4+3b+i7rV+P93+qKS3Z1Oi1knKS7pH0r+n67O6zJKelPSj9Jk8a9Ntnf1sR8RB/wPkgceBY4EicB9wfNZxTaE8ZwGnAA/UbfsL4PJ0+XLgz9PldwDfAAScDtyRbj8E2JD+uyhdXpR12SYp7xHAKenyPOAx4PhZXmYBc9PlAnBHWpavABel2z8DfDBd/jXgM+nyRcCX0+Xj0897N7Ay/TvIZ12+vZT9N4EvAf+ers/qMgNPAksatnX0s+0aReJUYH1EbIiIUeA64MKMY9pvEXEr8GLD5guBa9Lla4Cfqtv++UjcDixMn2P+duDmiHgxIl4CbgbObX/0+y4iNkfE3enyLuBh4Chmd5kjInanq4X0J4A3A7VHBzeWufZeXA+8RZLS7ddFxEhEPAGsJ/l7mJEkLQN+Evhsui5meZkn0dHPthNF4ijgmbr1jem22eSwiNicLj8HHJYuT1b2A/I9SZsXTib5hj2ry5w2wdwLbCH5w38c2B4R5fSQ+vjHypbu3wEs5gArM/A3wO8C1XR9MbO/zAF8S9Jdki5Nt3X0s921P1HbgS0iQtKsGxctaS7wL8BHI2Jn8uUxMRvLHBEV4CRJC4GvAsdlHFJbSTof2BIRd0k6O+t4OujMiNgk6VDgZkmP1O/sxGfbNYrEJuDouvVl6bbZ5Pm0Ckr675Z0+2RlP6DeE0kFkiTxxYj413TzrC5zTURsB24BXk/S1FD7Algf/1jZ0v0LgG0cWGU+A7hA0pMkzcNvBv4vs7vMRMSm9N8tJF8ITqXDn20nisQaYFU6eqJI0vG1OuOYpttqoDbS4RLga3Xb35OOljgd2JFWaW8C3iZpUTqi4m3pthknbXf+J+DhiPirul2zucxL05oEknqBt5L0zdwCvCs9rLHMtffiXcB3I+nlXA1clI4QWgmsAu7sTCn2TUT8fkQsi4gVJH+j342IdzOLyyxpjqR5tWWSz+QDdPqznXWP/kz5IRkt8BhJO+8VWcczxbJcC2wGSiRtke8jaZv9DrAO+DZwSHqsgCvTcv8I6K97nfeSdPStB34563I1Ke+ZJO249wP3pj/vmOVlPhG4Jy3zA8DH0u3Hklz01gP/DHSn23vS9fXp/mPrXuuK9L14FDgv67K1WP6z2TPqadaWOS3bfenPg7VrU6c/257Cw8zMmnLTk5mZNeVEYWZmTTlRmJlZU04UZmbWlBOFmZk15URh1oSk3avOcUQAAAHvSURBVOm/KyT9wjS/9h80rP/ndL6+2XRxojBrzQpgnxJF3d3CkxmXKCLiDfsYk1lHOFGYteaTwE+kzwT4jXRCvk9JWpPO+/+rAJLOlnSbpNXAQ+m2G9IJ3R6sTeom6ZNAb/p6X0y31WovSl/7gfQ5BD9f99rfk3S9pEckfVH1E1qZtYknBTRrzeXAb0fE+QDpBX9HRLxOUjfwA0nfSo89BXh1JFNYA7w3Il5Mp9pYI+lfIuJySZdFxEkTnOungZOA1wJL0t+5Nd13MnAC8CzwA5L5j/5j+otrtodrFGb7520kc+rcSzKl+WKSOYMA7qxLEgAfkXQfcDvJxGyraO5M4NqIqETE88D3gdfVvfbGiKiSTFWyYlpKY9aEaxRm+0fAhyNi3MRq6fTXAw3r5wCvj4hBSd8jmYNof43ULVfw37B1gGsUZq3ZRfKY1ZqbgA+m05sj6VXp7J6NFgAvpUniOJLHU9aUar/f4Dbg59N+kKUkj7adkbOb2sHB30bMWnM/UEmbkD5H8hyEFcDdaYfyVvY8jrLeN4EPSHqYZKbS2+v2XQXcL+nuSKbLrvkqybMl7iOZFfd3I+K5NNGYdZxnjzUzs6bc9GRmZk05UZiZWVNOFGZm1pQThZmZNeVEYWZmTTlRmJlZU04UZmbW1H8Bf4Pb8dOIuDMAAAAASUVORK5CYII=
" />
</div>
</div>
<div class="output_area">
<div class="output_subarea output_stream output_stdout output_text">
<pre>
Softmax agent, beta= 0.1
maximum possible reward: 0.9851042878107023
average reward: [0.91348264]
cosine similarity 0.9992727823574249
euclidian norm 0.008915931500017809
</pre>
</div>
</div>
<div class="output_area">
<div class="output_png output_subarea ">
<img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAYoAAAEGCAYAAAB7DNKzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deZxcdZnv8c9TVV29ZukknYTJQkcIYkAEbIIIMsjOqOC4jIAz4oiDOuLM6OhMBhzkMndBvbPoDNcRlSt6ZRBXogYRBQSUJQ2EJUDIAiGdBNIknfTetT33j3O6qW66K0W6q06l6/t+verV5/zO6TrPL69OPXV+57eYuyMiIjKRWNQBiIhIZVOiEBGRgpQoRESkICUKEREpSIlCREQKSkQdwFSbN2+et7a2Rh2GiMhB5eGHH37Z3VvGOzbtEkVrayvt7e1RhyEiclAxs60THVPTk4iIFKREISIiBSlRiIhIQUoUIiJSkBKFiIgUpEQhIiIFKVGIiEhB024chYhIOWRzwRINmVyOTNbJ5Jx0NsdAKksyEaOuJj5ynrvjgDskYsas+hpiMcPdSWVz9A9l6R3K0D2YJptzUpkcZgZAKpMj5042fP99A2n29qeJGcHvprLUxGPUJmLMa6rl3cctmvK6KlGISMVyd3IefBgPpnMMpbMMhK/BdPChPJjJMpjKK0tnGUxn6ewZoqOrn739abr6U8xuSJKMx4jFIGZG71CG3sEMC2bWMbM+QTYXXCsZjxGPGYm4MZjO0jOYoXsgTc9QhkzWGUhn6R5IM5TJHXC9EjEjHjMyOR9JOFPhTYtnKVGIyMGjP5XhqR3dvNw7xFAmRzbn9A1lGMrkqE3EiMdidHT109kzxN6BNN0DafpTWV7uHWJPX2rSH6JNtQkWN9fT3JDksJYm9g0E39YHMzlyDjNqE8yqr2FX9xA79w0QjxnukM7mSGeD69Yn48yoSzC7IcniOQ3UxIz6ZJyZdTXUJ+MYQUIZ/uCviceoT8bJZJ3+VAaAeMyImRELbhBIZZ1dPYPgwbHG2gQNyTiNyQQz6xPEY8HdQXAH4iQTMeI2fJ0Ys+prmFVfgwM1caMxmSCdyzGUyZGMl+ZpghKFyDTk7nQPZkiGH1weNl30DGboGcyQymaZ3ZBkdn0NOYfuweAb8p7eFH2pDP2pDNkc7Ng7AEAibmzp7GNPX4rOniGee7mPRNyoTcQY/izP5TxoInGnbyhLV3+K/S2gmYgZ85pqmd1Qw8z6GuY2JTliwQzmNNYEH5CxVz4kaxNBXepr4tTVvPKzriYor0vER37WJWMk47GR5pvprjYWpzYRL9n7K1GIlEgmm2Pjrl66+lKkwrbrnqEMqUyOhTPrmD+zloFUlv50lv6hLM+82M2L+wZJZ4MmjblNtXQPpJnTmCSddXqH0uzuTdE7lCFmRtDqDbkcbN87wEA6SyJmmMGevtTIt+JEzMiFTTiTUZsIvs3W1cR586HNeFjH4c/i4Ftz8M26PhlnwYw6jvqDmRwyuw7DSCaMhmSCxmSCoWyWbM6Z21hLMqE+NZVOiULkAHX1pdjW1c+2PQO81D3I9r0DPL2zmz19KV7sHqQ/lSX1GtqxzaC5IUljbZy+oSz9qQyGMZDO0lSboLE2aJ6oq4nTkIwTG27LMDihtZmmugSZrOMOzY1JZjfU0DeUGWm+SSZiNNUmSMSMupp48PB0IE0iPpwAYsTMWDirjoZkgpjBIbPqyboTN2NeU5LElDVt1EzR+0g5KFFI1RpuKhlum05lc3QPpukeyLCtq59NL/Wy4aUecu7Mrk8ykM7SN5QhmYjx5PZ9PPNiz6j3q03EeP3CGSyZ00BbazMNyQRHLpzBotn1JOIx6muC9m53WL9jHzXxGA3JOHVhc8ri5npm1L36A9Tdq6YJRSqTEoVMW3v6UnR09fNS9xC9Q2mWzWuibyjDum17eei5Pdy/eTepbI6auI0004w1f0Yt6WzQ46Y+GSebc2riMZbPb+Jz57yew+c3saS5gYWz6mhuqCn6A33p3Iai66EkIVFTogj1DmW45mfrefexi3jr4fOiDkeK5B50V6xNxHn0hS6+v3Ybz77Uw5aX++gZzEz4e0vm1PNnJx060n7fn8rS3JBkwcxamuoSLG5uYNm8RuY11ZaxNiKVSYkiNJTOckt7B0cvmqVEUSH29qd4YMseduwdoKk2wbMv9XDfppdZOqeBTZ29zKiroWNPP7v7UqN+77ilsznnqIUsm9dI69xG/mB2HXv703QPpunoGuDCE5YwVwlApGhKFFJxXtw3yHcfeJ4bf7+V3qFX3xVs3d1PImYsml3Paa+fz9ymJO3P7+HiEw/l7KMWMHOcdn4ROXBKFBIJd2fTrl46e4Z45sUebvjdczSFA6Dat3aRc+fsFQu45KRWkmG3zM6eIY4/tHlkagQRKQ8lCimJzp4hOnuG6B5Mk4gZ/aks63d0c/eGXXT2DrFz7yAD6eyrfm/+jFouPWUZf3rioa964Lt8wYxyhS8ieZQoZNI2vtTDTQ+9wBMd+9jTn2JLZ1/B8085fB5vPWwuKw6ZRevcBpa1NLJgRh2ZnFMTN/XyEakwShRVpnswzY8f7uCW9g6e2tnNkjn1NNXW8P43Lyabcx7fvo+fPbaD017fwgmtc3j0hb3cs7GThmScXM5ZuWwOPYMZVvzBTO7fvJt4zHhqZzcGNCYT9AxlqIkbHzl5GUcvmsXcxuTIgK35M+s4rKVxwkSQjClBiFQiJYpprKsvxSMvdPG9B1/gzmd2ETNeNY3Dtj0DwADX/PypUeV3b+jk7g2dI/upTI5DZtXx66d3AfDgc3sAeNvyeXzq9OV8+K2tzGlMlrQ+IhINJYpp6Mnt+7h13XZufmgbPXm9hnIORy6cwd+d+3retHg2O/cNMpTJsW8gRV0iztK5DSTjMdbv6Gb5giYe3trFikNmjno2kM7mWL0uuONQF1OR6qBEcRBzd/b2p2luTDKYzvK/b99AJuf810MvMJTJ8cZFs7js1Ndx5hsWUJ+Mk8rkRk3ANtEH/fyZdQAsbn716OGaeIz3vnlxaSokIhUp0kRhZucCXwHiwDfd/doxx5cCNwKzw3NWufuasgdaYQbTWd549e0j006cd/RCbnvyxZHjcxqT/L+PvpkTWueM+j3N0ikiByKyRGFmceA64CygA1hrZqvdPb+x/PPALe7+NTNbAawBWssebAXIZHN85Tcb+fc7N73q2HCS+Ptzj+SMN8yndW6jkoKITJko7yhWApvcfQuAmd0MXADkJwoHZobbs4AdpQ5qfwutRGFX9yCf+N4jPLy1a6TsA21LuPKdb2Dtc3voHcrw9iPna0SyiJRElIliEbAtb78DOHHMOVcDvzKzTwGNwJnjvZGZXQZcBrB06dIDCqYS++73DmU4/z/uGxmX0JiMc/8VZ1ATC1b0AjjjDQuiDFFEqkClt09cBHzb3RcDfwR818xeFbO7X+/ube7e1tLSUvYgS+ULt64fSRJnvmEBv1t1+shavSIi5RLlHcV2YEne/uKwLN+lwLkA7n6/mdUB84BdZYkwIv2pDFfdup4fPdLBJ99+GJ8758ioQxKRKhZlolgLLDezZQQJ4kLg4jHnvACcAXzbzN4A1AGdTGPfuu85/ikc/LZ0TgOfOn15xBGJSLWLLFG4e8bMLgduJ+j6eoO7rzeza4B2d18N/C3wDTP7NMGD7Q+7V+Lj5qmx5omdI0nif/zx0Vy8cmlFPjsRkeoS6TiKcEzEmjFlV+VtPwWcXO64ovDjRzr4zC2PAXD3Z0+jdV5jxBGJiAQ0MrsCvOOr97J+RzdHLpzBDz5+EjPUzVVEKkil93qa9v75VxtYv6MbgO9/TElCRCqPEkWEbl23fWSk9YNXnMGseiUJEak8anoqs+17Bzj52jtHld3ysZNYEE7EJyJSaZQoyugDX79/ZB2HYTd99ERWLpszwW+IiERPiWKMUvW+fWF3/6gk8Z7jF42sAiciUsmUKEKlHK3wRMc+3vUf9wHwtQ8eT10yzmlHtGiMhIgcFJQoymA4ScQMznvjIRFHIyLy2qjXU4n94vGdI9tb/tc7IoxEROTAKFGU0O83v8wnb3oEgJ9/6pSIoxEROTBKFCX0xdueAeCcoxboobWIHLSUKEokk83xWMc+AL7+Z20RRyMicuCUKEpkTbiO9TUXHBVxJCIik6NeTyXwxV8+w9fu3kwiZvzpiYdGHY6IyKTojmKKuTtfu3szAJmcE4tprISIHNx0RzGFjrn6droHMyP73710ZYTRiIhMDSWKKZSfJB684gxN9Cci04KansY40JmeNu3qHdm+49OnKkmIyLShRBGa7LRL/3HnRgD+74dPYPmCGVMQkYhIZVCimAKZbI6frtsBwKlHtEQcjYjI1FKimALDA+sA4urlJCLTjBLFFHh4a7DOxJ1/+4cRRyIiMvWUKKbA+h3dLJpdz+tamqIORURkyilRTJK7c+u6HaSyuahDEREpCSWKSdrcGXSLfdvyeRFHIiJSGkoUk7Slsw+AS05qjTYQEZESiTRRmNm5ZrbBzDaZ2aoJzvkTM3vKzNab2U3ljnF/vnXfcwAcOrch4khEREojsik8zCwOXAecBXQAa81stbs/lXfOcuAfgJPdvcvM5kcT7fgG01kefC7o8TS7IRlxNCIipRHlHcVKYJO7b3H3FHAzcMGYc/4CuM7duwDcfVeZYyzolvZtACTjasETkekrykkBFwHb8vY7gBPHnHMEgJn9DogDV7v7L8e+kZldBlwGsHTp0kkF5UVM9pTK5LjiJ0/ww4c7AHjsC2dP6poiIpWs0mePTQDLgdOAxcA9ZvZGd9+bf5K7Xw9cD9DW1nZA8/oZxY+oPuLzt43ar0/GD+SSIiIHhSjbTLYDS/L2F4dl+TqA1e6edvfngGcJEkdkegbTo/bfc9yiiCIRESmPKBPFWmC5mS0zsyRwIbB6zDk/JbibwMzmETRFbSlnkGN994Gto/Y/c/YREUUiIlIekTU9uXvGzC4Hbid4/nCDu683s2uAdndfHR4728yeArLA59x9d1QxwysPrn92+Sl09g6yuFndYkVkepswUZjZv1NgHR93/6vJXtzd1wBrxpRdlbftwGfCV0XY3NnL7IYajl40E7NZUYcjIlJyhZqe2oGHgTrgeGBj+DoWqNpBA49t28cR82dgk13pSETkIDHhHYW73whgZp8ATnH3TLj/n8C95Qmvsgxlsjy1szvqMEREyqqYh9nNwMy8/aawrOrs3DsIwJ+f3BptICIiZVTMw+xrgUfN7C7AgFOBq0sZVKXasXcAgLNWLIg4EhGR8imYKCxoiP81cBuvjJr+e3d/sdSBVaIXu4M7ioUz6yKORESkfAomCnd3M1vj7m8Ebi1TTJEqNKx7T18KgLlNteUJRkSkAhTzjOIRMzuh5JFErYhOTLv7UtTEjZl1lT7ziYjI1CnmE+9E4INmthXoI/hIdXc/pqSRVaDbnthJOuvqGisiVaWYRHFOyaM4SDy/uz/qEEREym6/icLdtwKEiwbpKa6ISJXZ7zMKMzvfzDYCzwG/BZ4n6AU1LW18qYfWVb/gtid2jirf2x88yNYiRSJSbYr51Psn4C3As+6+DDgDeKCkUUXo5rXBWkpfun3DqPLdYY+nL7+/6h7NiEiVKyZRpMMZW2NmFnP3u4C2EscVuUwuN2q/K0wUzVobW0SqTDEPs/eaWRNwD/A9M9tF0PtpWhuTJ7hrQ7Bc95xGJQoRqS7F3FFcAPQDnwZ+CWwG3lXKoCrB2DuK6+7aDChRiEj1KeaO4kLgHnffCNxY4ngqRjY3frmankSk2hSTKJYCXzezZQRrVNwD3Ovu60oaWcRe7h0im3PiMaM/lRkpr0/GI4xKRKT89tv05O5fcPfTgRUE61B8jmBBo2nv6tXrAdjbn444EhGR6BQzjuLzZnYb8CvgcOCzwOJSB1Zu483K8aNHOgB4YU8wIvtL71XXWBGpPsU0Pb0HyAC/IBhwd7+7D5U0qgoRD7PHhdcHw0YOm98YZTgiIpEopunpeOBM4CHgLOAJM7uv1IFVgrF3GYtmN0QTiIhIhPZ7R2FmRwNvA/6QYKDdNqpkzezuwcyo/fkztA6FiFSfYpdCvRf4KrDW3avqye5AKgvAYS2NxGKaXlxEqk8xs8e+08zqgaXVliQA2rfuAWBz57QfjC4iMq5iej29C1hHMCobMzvWzFaXOrBKkc0Fi6M21WpVOxGpTsVM4XE1sBLYCxAOtFtWwpgqSiYbJIpvfGjaz4MoIjKuYmeP3TemzKfi4mZ2rpltMLNNZraqwHnvNTM3s7J/Wn/0O+0AzG3S1B0iUp2KSRTrzexiIG5my83s34HfT/bCZhYHrgPOIxj1fZGZrRjnvBnAXwMPTvaak6GmJxGpVsUkik8BRwFDwE1AN/A3U3DtlcAmd9/i7ingZoKZasf6J+CLwOAUXPOANdUpUYhIdSpmwF2/u1/p7ieEryuB+VNw7UUEYzKGdYRlI8zseGCJu/+i0BuZ2WVm1m5m7Z2dnVMQ2qs1JpUoRKQ6FUwUZnaSmb3PzOaH+8eY2U3A70odmJnFgH8B/nZ/57r79e7e5u5tLS0tB3a9vO2/Ov1w/uVP3jTqeFxjKESkSk2YKMzsy8ANwHuBX5jZfyeYGPBBYPkUXHs7sCRvf3FYNmwGcDRwt5k9T7Bu9+pyPNBumVHL0jmarkNEBAoPuHsHcJy7D5pZM0Ez0dHu/vwUXXstsDxc52I7wQJJFw8fDHtazRveN7O7gc+6e/sUXX9iZrz50OaSX0ZE5GBQqOlp0N0HAdy9C9g4hUkCd88AlwO3A08Dt7j7ejO7xszOn6rrHCgz40MnHRp1GCIikSt0R/G6MSOwl+Xvu/ukP8zdfQ2wZkzZVROce9pkr/da/eVph/Od+7fy6TOPKPelRUQqRqFEMbar6j+XMpBKtHBWHY/+41nMqq+JOhQRkchMmCjc/bflDKRSNTdqRLaIVLdiBtyJiEgVU6IQEZGCik4UZqaBBSIiVaiY9SjeamZPAc+E+28ys/9T8shERKQiFHNH8a/AOcBuAHd/DDi1lEGJiEjlKKrpyd23jSnKliCWSJlpLicRkfEUMyXqNjN7K+BmVkOwNsTTpQ1LREQqRTF3FB8HPkkwBfh24NhwX0REqsB+7yjc/WXgg2WIRUREKtB+E4WZfXWc4n1Au7vfOvUhiYhIJSmm6amOoLlpY/g6hmDtiEvN7N9KGJuIiFSAYh5mHwOc7O5ZADP7GnAvcArwRAljExGRClDMHUUz0JS33wjMCRPHUEmiEhGRilHMHcWXgHXhCnNGMNjuf5pZI/DrEsYmIiIVoJheT98yszXAyrDoCnffEW5/rmSRiYhIRSh2UsBBYCfQBRxuZprCQ0SkShTTPfajBKOxFwPrgLcA9wOnlza06Jx39MKoQxARqRjF3FH8NXACsNXd3w4cB+wtaVQRyJ/paV5TbWRxiIhUmmISxaC7DwKYWa27PwO8vrRhiYhIpSim11OHmc0GfgrcYWZdwNbShiUiIpWimF5PfxxuXm1mdwGzgF+WNCoREakYBROFmcWB9e5+JIC7/7YsUYmISMUo+IwiHH29wcyWlikeERGpMMU8o2gG1pvZQ0DfcKG7n1+yqEREpGIUkyj+sVQXN7Nzga8AceCb7n7tmOOfAT4KZIBO4CPurgfpIiJltN/useFzieeBmnB7LfDIZC8cPv+4DjgPWAFcZGYrxpz2KNDm7scAPySYd0pERMpov4nCzP6C4EP662HRIoKuspO1Etjk7lvcPQXcDFyQf4K73+Xu/eHuAwSjw0VEpIyKGXD3SeBkoBvA3TcC86fg2ouAbXn7HWHZRC4FbhvvgJldZmbtZtbe2dk5BaGJiMiwYhLFUPiNHwAzSwBeupBezcz+FGgDvjzecXe/3t3b3L2tpaWlnKGJiEx7xSSK35rZFUC9mZ0F/AD42RRcezuwJG9/cVg2ipmdCVwJnO/uJVsoyWz/54iIVKNiEsUqgh5HTwAfA9YAn5+Ca68FlpvZMjNLAhcCq/NPMLPjCJ6NnO/uu6bgmiIi8hoV0z323cB33P0bU3lhd8+Y2eXA7QTdY29w9/Vmdg3Q7u6rCZqamoAfWPCV/wWN3xARKa9iEsW7gH81s3uA7wO/dPfMVFzc3dcQ3KHkl12Vt33mVFxHREQOXDHjKP4cOJzg2cRFwGYz+2apAxMRkcpQzB0F7p42s9sIejvVEzRHfbSUgYmISGUoZsDdeWb2bWAj8F7gm4DWChURqRLF3FF8iODZxMdK2T1VREQqUzELF12Uv29mpwAXufsnSxaViIhUjKKeUYTjGS4G3g88B/y4lEGJiEjlmDBRmNkRBL2cLgJeJmh+Mnd/e5liExGRClDojuIZ4F7gne6+CcDMPl2WqEREpGIU6vX0HmAncJeZfcPMzgCm7YxINn2rJiIyKRMmCnf/qbtfCBwJ3AX8DTDfzL5mZmeXK0AREYlWMSOz+9z9Jnd/F8EMr48Cf1/yyEREpCIUM3vsCHfvCtd+OKNUAYmISGV5TYlCRESqjxKFiIgUpEQhIiIFKVGIiEhBShQiIlKQEoWIiBSkRCEiIgUpUYiISEFKFCHTVE8iIuNSohARkYKUKEREpCAlChERKUiJQkREClKiEBGRgiJNFGZ2rpltMLNNZrZqnOO1Zvb98PiDZtZa/ihFRKpbZInCzOLAdcB5wArgIjNbMea0S4Eudz8c+Ffgi+WNUkREoryjWAlscvct7p4CbgYuGHPOBcCN4fYPgTPMNOJBRKScokwUi4BtefsdYdm457h7BtgHzB37RmZ2mZm1m1l7Z2dnicIVEalO0+Jhdrg8a5u7t7W0tEQdjojItBJlotgOLMnbXxyWjXuOmSWAWcDuskQnIiJAtIliLbDczJaZWRK4EFg95pzVwCXh9vuAO93dyxijiEjVS0R1YXfPmNnlwO1AHLjB3deb2TVAu7uvBr4FfNfMNgF7CJKJiIiUUWSJAsDd1wBrxpRdlbc9CLy/3HGJiMgrpsXDbBERKR0lChERKUiJQkREClKiEBGRgpQoRESkICWKMTSTlIjIaEoUIiJSkBLFGLqhEBEZTYlCREQKUqII6dmEiMj4lChERKQgJQoRESlIiWIMrbQqIjKaEoWIiBSkRDGG7idEREZTohARkYKUKMbQIwoRkdGUKEREpCAlChERKUiJYgzT42wRkVGUKEREpCAlChERKUiJIuQedQQiIpVJiWIsPaIQERlFiUJERApSohARkYIiSRRmNsfM7jCzjeHP5nHOOdbM7jez9Wb2uJl9oCyxleMiIiIHkajuKFYBv3H35cBvwv2x+oEPuftRwLnAv5nZ7FIFpIfZIiLjiypRXADcGG7fCLx77Anu/qy7bwy3dwC7gJZSB6a5nkRERosqUSxw953h9ovAgkInm9lKIAlsnuD4ZWbWbmbtnZ2dBxSQo1sKEZHxJEr1xmb2a2DhOIeuzN9xdzezCT+lzewQ4LvAJe6eG+8cd78euB6gra1tUp/4msJDRGS0kiUKdz9zomNm9pKZHeLuO8NEsGuC82YCvwCudPcHShRqcK0wQdQn46W8jIjIQSeqpqfVwCXh9iXArWNPMLMk8BPgO+7+w1IHVJ+Ms+q8I/nBx08q9aVERA4qUSWKa4GzzGwjcGa4j5m1mdk3w3P+BDgV+LCZrQtfx5YyqI//4WEc1tJUykuIiBx0zKdZv9C2tjZvb2+POgwRkYOKmT3s7m3jHdPIbBERKUiJQkREClKiEBGRgpQoRESkICUKEREpSIlCREQKUqIQEZGCpt04CjPrBLZO4i3mAS9PUTgHg2qrL6jO1aLa6jzZ+h7q7uPO0D3tEsVkmVn7RINOpqNqqy+oztWi2upcyvqq6UlERApSohARkYKUKF7t+qgDKLNqqy+oztWi2upcsvrqGYWIiBSkOwoRESlIiUJERApSogiZ2blmtsHMNpnZqqjjmQwzu8HMdpnZk3llc8zsDjPbGP5sDsvNzL4a1vtxMzs+73cuCc/faGaXjHetSmBmS8zsLjN7yszWm9lfh+XTuc51ZvaQmT0W1vm/heXLzOzBsG7fD1eKxMxqw/1N4fHWvPf6h7B8g5mdE02NimdmcTN71Mx+Hu5P6zqb2fNm9kS4eFt7WFbev213r/oXEAc2A68DksBjwIqo45pEfU4FjgeezCv7ErAq3F4FfDHc/iPgNsCAtwAPhuVzgC3hz+Zwuznquk1Q30OA48PtGcCzwIppXmcDmsLtGuDBsC63ABeG5f8JfCLc/kvgP8PtC4Hvh9srwr/3WmBZ+P8gHnX99lP3zwA3AT8P96d1nYHngXljysr6t607isBKYJO7b3H3FHAzcEHEMR0wd78H2DOm+ALgxnD7RuDdeeXf8cADwGwzOwQ4B7jD3fe4exdwB3Bu6aN/7dx9p7s/Em73AE8Di5jedXZ37w13a8KXA6cDw2vMj63z8L/FD4EzzMzC8pvdfcjdnwM2Efx/qEhmthh4B/DNcN+Y5nWeQFn/tpUoAouAbXn7HWHZdLLA3XeG2y8CC8Ltiep+UP6bhM0LxxF8w57WdQ6bYNYBuwj+428G9rp7JjwlP/6RuoXH9wFzOcjqDPwb8HdALtyfy/SvswO/MrOHzeyysKysf9uJA4laDm7u7mY27fpFm1kT8CPgb9y9O/jyGJiOdXb3LHCsmc0GfgIcGXFIJWVm7wR2ufvDZnZa1PGU0Snuvt3M5gN3mNkz+QfL8betO4rAdmBJ3v7isGw6eSm8BSX8uSssn6juB9W/iZnVECSJ77n7j8PiaV3nYe6+F7gLOImgqWH4C2B+/CN1C4/PAnZzcNX5ZOB8M3ueoHn4dOArTO864+7bw5+7CL4QrKTMf9tKFIG1wPKw90SS4MHX6ohjmmqrgeGeDpcAt+aVfyjsLfEWYF94S3s7cLaZNYc9Ks4OyypO2O78LeBpd/+XvEPTuc4t4Z0EZlYPnEXwbOYu4H3haWPrPPxv8T7gTg+ecq4GLgx7CC0DlgMPlacWr427/4O7L3b3VoL/o3e6+weZxnU2s0YzmzG8TfA3+STl/tuO+ol+pbwIegs8SwY+Mp8AAAKgSURBVNDOe2XU8UyyLv8F7ATSBG2RlxK0zf4G2Aj8GpgTnmvAdWG9nwDa8t7nIwQP+jYBfx51vQrU9xSCdtzHgXXh64+meZ2PAR4N6/wkcFVY/jqCD71NwA+A2rC8LtzfFB5/Xd57XRn+W2wAzou6bkXW/zRe6fU0besc1u2x8LV++LOp3H/bmsJDREQKUtOTiIgUpEQhIiIFKVGIiEhBShQiIlKQEoWIiBSkRCFSgJn1hj9bzeziKX7vK8bs/34q319kqihRiBSnFXhNiSJvtPBERiUKd3/ra4xJpCyUKESKcy3wtnBNgE+HE/J92czWhvP+fwzAzE4zs3vNbDXwVFj203BCt/XDk7qZ2bVAffh+3wvLhu9eLHzvJ8N1CD6Q9953m9kPzewZM/ue5U9oJVIimhRQpDirgM+6+zsBwg/8fe5+gpnVAr8zs1+F5x4PHO3BFNYAH3H3PeFUG2vN7EfuvsrMLnf3Y8e51nuAY4E3AfPC37knPHYccBSwA/gdwfxH9019dUVeoTsKkQNzNsGcOusIpjSfSzBnEMBDeUkC4K/M7DHgAYKJ2ZZT2CnAf7l71t1fAn4LnJD33h3uniOYqqR1SmojUoDuKEQOjAGfcvdRE6uF01/3jdk/EzjJ3fvN7G6COYgO1FDedhb9H5Yy0B2FSHF6CJZZHXY78IlwenPM7Ihwds+xZgFdYZI4kmB5ymHp4d8f417gA+FzkBaCpW0rcnZTqQ76NiJSnMeBbNiE9G2CdRBagUfCB8qdvLIcZb5fAh83s6cJZip9IO/Y9cDjZvaIB9NlD/sJwdoSjxHMivt37v5imGhEyk6zx4qISEFqehIRkYKUKEREpCAlChERKUiJQkREClKiEBGRgpQoRESkICUKEREp6P8DRIuN1tzJDCgAAAAASUVORK5CYII=
" />
</div>
</div>
<div class="output_area">
<div class="output_subarea output_stream output_stdout output_text">
<pre>
UCB1 agent
maximum possible reward: 0.9851042878107023
average reward: [0.89258379]
cosine similarity 0.0
euclidian norm 0.1414213562373095
</pre>
</div>
</div>
<div class="output_area">
<div class="output_png output_subarea ">
<img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXxddZ3/8dcn+94kTVpKt7RSKEsp1BZa2WEERJZRXEBnxAVh3HAZFdAZRZ2fgyujDgNTGQb9/QRxt6PI9pNVKLZACy0UWkohLaXpljR7cnM/88c5SW/aJL2hOfcmOe/n43EfOVvO/XzzuDmfe77f7/l+zd0REZH4ysl2ACIikl1KBCIiMadEICISc0oEIiIxp0QgIhJzedkOYLhqamq8rq4u22GIiIwpTz755A53rx1o35hLBHV1daxcuTLbYYiIjClm9spg+1Q1JCISc0oEIiIxp0QgIhJzSgQiIjGnRCAiEnNKBCIiMadEICISc7FJBImeJL9YWU8yqWG3RURSjbkHyt6oWx59mev/tA53572LZmQ7HBGRUSM2dwS7WrsAaGzrznIkIiKjS2wSgYiIDCw2iUBTcoqIDCw2iUBERAYWm0RgZtkOQURkVIpNIlDVkIjIwGKTCHrpxkBEpL/YJQLdGIiI9Be7RCAiIv3FLhGoakhEpL/YJQIREelPiUBEJOaUCEREYk6JQEQk5mKTCNRtVERkYLFJBL0MdRsSEUkVWSIws1vNrMHM1gyy//1m9oyZPWtmj5nZ/KhiSeXo1kBEJFWUdwS3AecOsf9l4DR3nwd8A1gaYSwiIjKIyKaqdPeHzaxuiP2PpawuB6ZFFUsqVQ2JiPQ3WtoIPgL8KdtBiIjEUdYnrzezMwgSwclDHHMFcAXAjBmaeF5EZCRl9Y7AzI4FbgEucvedgx3n7kvdfaG7L6ytrc1cgCIiMZC1RGBmM4DfAH/v7i9mKw4RkbiLrGrIzO4ATgdqzGwz8FUgH8Ddbwa+AkwE/iOcRjLh7gujikedRkVEBhZlr6FLD7D/cuDyqN5fRETSM1p6DUVOnUZFRAYWm0SgqiERkYHFJhH00gxlIiL9xS4RaBRSEZH+YpcIRESkv9glAlUNiYj0F7tEICIi/SkRiIjEnBKBiEjMKRGIiMRcbBKBuo2KiAwsNolAREQGpkQgIhJzSgQiIjGnRCAiEnNKBCIiMadEICISc0oEIiIxp0QgIhJzsUkErjnKREQGFJtEICIiA4tNIjBNXy8iMqDYJIIFMysBmDKhOMuRiIiMLrFJBFMmFAFQXpSX5UhEREaX2CSCXmoyFhHpL0aJQG0EIiIDiVEiEBGRgSgRiIjEXOwSgWuqMhGRfiJLBGZ2q5k1mNmaQfabmf3QzDaY2TNmtiCqWIL3i/LsIiJjV5R3BLcB5w6x/23AnPB1BXBThLGIiMggIksE7v4wsGuIQy4CfuqB5UClmU2JKh4RERlYNtsIpgL1Keubw237MbMrzGylma3cvn17RoITEYmLMdFY7O5L3X2huy+sra09uHONUEwiIuPFoOMtmNmPGOK66e5XHeR7bwGmp6xPC7dFQm3FIiIDG+qOYCXwJFAELADWh6/jgIIReO9lwAfC3kOLgSZ33zoC5xURkWEY9I7A3X8CYGYfA05290S4fjPwyIFObGZ3AKcDNWa2GfgqkB+e+2bgLuA8YAPQBnzoYAoiIiJvTDpDcVYBFeztAVQWbhuSu196gP0OfCKN9x9ZaiQQEeknnURwPfC0mT1AUNV+KnBdlEFFwfREmYjIgIZMBBZcPe8H/gScGG6+2t1fjzowERHJjCETgbu7md3l7vOA32coJhERyaB0niN4yswWRR5JhrgaCURE+kmnjeBE4P1m9grQStBO4O5+bKSRjTC1EIiIDCydRHBO5FGIiEjWHDARuPsrAGY2ieDhMhERGUcO2EZgZhea2XrgZeAhYBNBL6IxSfPSiIj0l05j8TeAxcCL7j4LOAtYHmlUEdBjBCIiA0snEXS7+04gx8xy3P0BYGHEcYmISIak01jcaGZlwMPAz8ysgaD3kIiIjAPp3BFcRDAo3GeBu4GXgAuiDCpKaiMQEekvnTuCS4CH3X098JOI44mM6UkCEZEBpZMIZgD/aWazCOYoeBh4xN1XRRqZiEjMJXqSdCSS5Bjk5hh5OTnk5oz8l9p0niP4KoCZFQMfBb4A/BuQO+LRiIiMM8mk09yZoKO7h/auHl7d1cbuti72tHfTNOAr0bevpTPR71xXnjaba9925IjHeMBEYGb/BJxEMA/B08DnSWNiGhGRsSqZdFq7EjhQWpDHnvZuOhI9JHqcpvZudrZ20djWxa7WLna3drGrrYvdrd3sau0CIJFMsqOli50tnTR3JoZsmyzOz2VCcT4TivOpKM5jamURR04p79tWUpBL0qEn6Rw/vTKS8qZTNfROIAH8keCBssfdvTOSaDJAbcUi8eHubG/p5PWmDna0dLKjuYu2rgStXT00tXf3ffPe0xH+bE/Q1N5Nc0c3yfBikWP0LQ/EDKpKCqgqyae6tICepJOXm8MxUydQXRJczMuL8ikqyKUkP5fp1SVUlxb0XfgL87JfuZJO1dACM6sguCt4K7DUzBrc/eTIoxtBeqBMZHRr60qwbU9w0a4qzQegvauHwrxcWrsStHQmaOlI0NoZLLd29tDS2U1LZw+tncH25vBnZyJJrhlbGtv3q17pVZSfQ0VR7zfxfCaVF3FYbR4Vvd/Oi/KD6pzuHmrKCinKDy7Y1aUFTCwroLq0gOqSAiqK8yOpt8+kdKqGjgFOAU4jeJCsHlUNicgQOhM97GrtorkjQf2uNprau9m0s43XGtvpTCTp7O6hqydJVyLJ9uZONu9up727Z9jvU5yfS2lhHmWFwc/SwjwOqSiiIC+HRNJZ8qaJzJxYwrSqEiaWFVBbVhgelzsqvomPFulOVfkI8ENghbt3RxuSiIxmHd09NLZ1s7O1k12tXazb2sz6hmZe3dVGT9J5cVsLTe37XyZyDCaVF1FckEthXg4FeTnk5+Ywq6aUU+bUMrGsgEMqiphcUcSuti5yzSjKz6GjO8mE4nxKC3MpCy/2pYV5lBbkkpebzqNQciDpVA2dH/YYmjEekoDriTIRIOiaWL+7na2N7f3qyZvau2nrCqpbNm5vZUdLJ2ZGVyLJtj0dJAaoMK8pK2DmxFJyc4zz5h3C1MpiKksKKC3MpW5iKROK8zm0srivekVGl3Sqhi4AvgsUALPM7Djg6+5+YdTBiUj6mtq7WV3fyI6WTjbtaGVrUwcOHFpZTP2uNjZubwkONGPL7jZ2tXYN2AiamxM8fllVWsDM6hLmTask0ZOkKD+XyRVFlBXmUl1aSHVp0Ag695ByJpYVZrKoMsLSqRq6DjgBeBDA3VeFD5eJSBb0JJ3nt+5hVX0ja1/bw+tN7dTvbmdDQ0vfMTkGteWFdPc4u1q7qCjKY/70SnLMSLpz5JGTqS0vZHpVCdOrS4KuimEPl9KC4Fu7qYdFbKSTCLrdvWmfD4XqV0RGkLuzo6WLzbvbeHVXG6vrm2hs7+JNtWV0dvewYXsLtWWFrHltD8+9tqevYbW8MI9p1SXMrC7hwvmHctz0SiZXFFFXU9LXGNqVSJKXY+SM8Z4tEp10EsFaM3sfkGtmc4CrgMeiDSs6ymCSSe5OT9Jp7exhe0sHPcmgR83qzU28vL2Vl7a30NqZoH53G9v27H08pzAvh5KCXH7z1JZ+5ztueiXvXTSd42dUsmBGFVMriw94gS/IU4OqDC2dRPAp4MtAJ3A7cC/BZDVjiu5yZaS1dCZ4qaGFTTtb2dPeTWNbN6/uaiPHjF1tXdTvaqN+VxutXQN3iyzOz2VaVTHFBbksmT2RedMqqSjK45ipE5hdW0p+Tg7bmjuoKilQI6tEKp1eQ20EieDLvdvMbAbwaoRxiYw6Hd09rKpvZPnGnSzfuJOnXm2kK5Hsd0xteSGd3T3UlBcyu6aMxbMnUhX2nqktL6QzkaSkIJfjplcytbL4gPXwUyYUR1kkEeAAicDMlgBTCYahbjCzY4FrCB4wm56B+ESypjPRw9Ov7n/hN4NjDp3AZUtmsqiumpkTS6ksye/r4y4y1gz6qTWz7wDnA6uAq83sHuBy4F+BD2cmvJGnxwiko7uHna1d/HldAxu2NdPRnaSiOI/JFUWs2dLElsZ2kg4v72hlV2sXZnD0oRV8YPFMFs+eyKJZ1Uwozs92MURGzFBfX94OHO/uHWZWRTC0xDHuvindk5vZucAPCIasvsXdr99n/wyCyW4qw2Oucfe7hleENGPRxDSx1NTWzZrXmtjT3k1DcyfrXm/mD6tfozkcfyY/1+ju2fvtYFJ5IQ3NnRxSUcTJh9Vw3rwpLJk9kQkluvDL+DVUIuhw9w4Ad99tZuuHmQRygRsJBqrbDKwws2Xu/lzKYf8E/MLdbzKzo4C7gLphlkGkT3dPkuUbd7K7rZs/PvMa9z/fQE/KU1NlhXmcOXcSM6pLqCkr4OI3T6MoP5dV9Y1MmVCUVr29yHgzVCKYbWbLUtZnpa6n8WTxCcAGd98IYGY/J5j/ODUROFARLk8AXks3cBEIumc+v7WZO1e8ypbGDu5/flvfvomlBXzk5Fksnl1NXk4ONWWFHDapbMDulIvqqjMZtsioMlQiuGif9e8N89xTCaqTem0GTtznmOuAe83sU0Ap8DcDncjMrgCuAJgxY8Yww5Dxwt1xh7buHp6pb+Sh9du5c0U9jW3BEFhlhXksmT2RyRWFXH7KbA6fXK4+9CJpGDQRuPtDGXj/S4Hb3P17YQ+l/2tmx7h7vz557r4UWAqwcOHCg2zuVWvxWJFMOtuaO/j1k5v55ZObea2xncK83H7jy58yp4YzjpjEm2dWMWdyGSUF6rUjMlxR/tdsoX8X02nhtlQfAc4FcPfHzawIqAEaRjoYVfuODY1tXfz08Ve48YENdKb00T9sUhmHTy7nyCkVzKopZUZ1CSfMqmZyRVEWoxUZH6JMBCuAOeEAdVuAS4D37XPMq8BZwG1mdiRQBGyPMCYZhZJJ5/ert/Cvd62joTkYZqEgL4dFdVXMn1bJuxZOY+4hFQc4i4i8UWknAjMrCZ8yTou7J8zsk8A9BF1Db3X3tWb2dWCluy8D/hH4sZl9lqDO5oOuCQPGveaOblo6Eyx9eCPPbm5i5Su7+/adNXcS/3j2ERx1qC78IpmSznwEbwFuAcqAGWY2H7jS3T9+oN8Nnwm4a59tX0lZfo5gLuSMUZrJntebOrjtsU3c/NBL/bafOKuav1s8k7fPm6IRMkWyIJ07ghuAc4BlAO6+2sxOjTSqCKiNIPOee20PV/38aXIMmjsSbG3qAGBGdQnHTa/k75fMZMGMqjE/8bfIWJdW1ZC71+/zkM3wZ5mWWGho7uD+5xq45dGNbNze2rfdDBbVVfGJMw7j9CMmZTFCEdlXOomgPqwecjPLBz4NPB9tWDLatXUl+P2q17jhvhcpLcyjYU8Hs2pLWbNlDxA09r7rzdO44tTZzKoppbUzQWVJQZajFpGBpJMI/oFgvKCpBL1/7gU+EWVQUVITwRvXlUjyzbue57bHNvXfEfb0WbNlD0tmT+Szbz2cRXVV/YZqUBIQGb3SmY9gB/D+DMQSKQ0698bs6ejm2OvuBYKx9reHF/2plcVc9paZnHv0FKZXF/c1wquxV2TsSafX0A8H2NxE0AX09yMfkmTbjpZOEj3Opp2tXLJ0ed/25o5urjxtNl88Z+5+DbxqjBcZu9KpGioC5gK/DNcvBl4G5pvZGe7+maiCk8z75cp6vvCrZ/pt++Y75nH+/CmUF+ZpZE6RcSidRHAscJK79wCY2U3AI8DJwLMRxhYJPUcwsBdeb+Y9//k4Te3BAG5zJpUxq6aUfz7/KKZXl2Q5OhGJUjqJoIrgYbKmcL0UqHb3HjPrjCyyEaYvsoO7e83r/MP/exKA9504g8+ffQTVpWrcFYmLdBLBt4FVZvYgYMCpwDfNrBS4P8LYJELJpPOrJzfzxV/vrQZ64POnM6umNItRiUg2pNNr6L/M7C6CiWYAvuTuvRPIfCGyyCQSLZ0JvnvPC/zsiVf6pmicUJzPrz/2FiUBkZhKd9C5DmArQcPxYWZ2mLs/HF1Y0fGYPkmwfONO7l7zer9nAE6oq+amv1vAxLLC7AUmIlmXTvfRywmeJp4GrAIWA48DZ0Yb2siKYxOBu3P1r5/hFys399v+9YuO5gNL6rITlIiMOuncEXwaWAQsd/czzGwu8M1ow5KD0ZVIcvlPV/Lwi3undphdU8qZcyfxubMP1yxeItJPOleEDnfvMDPMrNDd15nZEZFHNsJ668P//c8bOP/YQ7McTXTqd7Vxwb8/2jeP75lzJ/GFc47g8MnlGuVTRAaUTiLYbGaVwO+A+8xsN/BKtGGNvPbuYJ7bda83ZzmS6Kzf1sxbbwiabqpLC7j706cwSVM5isgBpNNr6B3h4nVm9gAwAbg70qgiMN4fJGvtTPQlgXOOnsx33z2f8qL8LEclImPBkInAzHKBte4+F8DdH8pIVBEYr3ngxW3NnH3D3g5c5807hP94/5uzGJGIjDVDJoLw6eEXzGyGu7+aqaCiMNbvCF7e0coLr+/hpoc2smhmFYdMKOJf/th/Wojywjy+dfGxWYpQRMaqdIeYWGtmfwX6ppxy9wsjiyoCPkYzgbtz/Dfu62v8BVhd39jvmAvmH8oN75lPXm5OpsMTkXEgnUTwz5FHIQPq6O5h4b/cT0tn0NBdmJfDB5bM5MePvAzAHz51MkcfWqERQUXkoKTTWPyQmc0E5rj7/WZWAuRGH9rISo7BG4Kr7ni6Lwn87PITOemwGgC+/PajshmWiIwz6TxZ/FHgCqAaeBPBlJU3A2dFG9rIGgtVQ9+6ex03PfgSNWWFXH3uEdz73DYAHr/2TKZMKM5ydCIyXqVTNfQJggHnngBw9/VmNinSqCIwWtNA6hDQvXa0dPZNDrP82rM4ZIKeBRCR6KTTutjp7l29K2aWx+i9rg4qOQrvCB5dv2O/JHDZkplMKA76/1997lwlARGJXDp3BA+Z2ZeAYjN7K/Bx4H+iDWvkjYY2gld2tnLeDx6htauH6985j9v/urdHbmlBLlee9iauOmsOX7voGFo7E5QWakwgEYleOleaa4CPEExLeSVwF3BLlEFFIdt3BMmkc9p3Huxbv+Y3wSyf5YV5PPu1c/Y7XklARDIlnavN3wI/dfcfRx1MlLLdWPzc1j0Dbv/vDy3KcCQiIv2lkwguAG4ws4eBO4G73T0RbVgjryKL4+60dib4zJ2rKMjL4Ylrz6JK8wGLyChywMZid/8QcBjwS+BS4CUzS6tqyMzODYeo2GBm1wxyzHvM7DkzW2tmtw8n+OFYWFcNwEdOnhXVWwyoo7uHo796DxsaWphdU6okICKjTloV0e7ebWZ/IugtVExQXXT5UL8TDlh3I/BWYDOwwsyWuftzKcfMAa4FTnL33VF3Sy3IzSE/w8Mw/O7pLX3LN7z3uIy+t4hIOg54VTSzt5nZbcB64GKChuJD0jj3CcAGd98Ydj/9OXDRPsd8FLjR3XcDuHvDMGIfPsvsnMW/fXpzX6Pw2q+dw5FTKjL23iIi6UrnjuADBG0DV7p75zDOPRWoT1nfDJy4zzGHA5jZXwiGrbjO3feb68DMriB4upkZM2YMI4R9zgMZewLitr+8zHX/E9z8HD+jUr2ARGTUSmesoUtT183sZOBSd//ECL3/HOB0YBrwsJnNc/d+w2u6+1JgKcDChQvf8KXcLDN54KzvPchL24OBWi9eMI3vvltDQ4vI6JVWhbmZHW9m3zGzTcA3gHVp/NoWYHrK+rRwW6rNwDJ373b3l4EXCRJDJAzLSDfS3iQA8O13HavRQUVkVBs0EZjZ4Wb2VTNbB/wIeBUwdz/D3X+UxrlXAHPMbJaZFQCXAMv2OeZ3BHcDmFkNQVXRxuEXIz1m0U9Q05nowSwYKmLT9W/XhPEiMuoNVTW0DngEON/dNwCY2WfTPbG7J8zsk8A9BPX/t7r7WjP7OrDS3ZeF+842s+eAHuAL7r7zDZblgIzoqoY2NDTzN9/fO2XkolnVEb2TiMjIGioRvJPgW/wDZnY3Qa+fYX29dfe7CIakSN32lZRlBz4XviJnZpHdEaQmAYDjpldG80YiIiNs0ETg7r8DfmdmpQTdPj8DTDKzm4Dfuvu9GYpxxAR3BOllgh0tnbyys43q0gIu/8kKjpxSwYdOmsWbZ1b1HbNmSxM/fmQjzR39H7SuLS9kWlXJSIYuIhKZdHoNtQK3A7ebWRXwbuBqYMwlAobRRvCemx9n4469jb4vbW/lD89s5ScfPoHTDq/lt09v5rN3ru73O18+70jesWAqNWWFIxm1iEikhvWYrbvvdvel7j6mZifrNZx6rdQkkOqyW//Klsb2/ZLA2UdN5vJTZikJiMiYE6unnII2goNvJDjp+j/3W7/jo4tZ8qaJB31eEZFsiFkieGO9hi6Yfyhnzq3lovlTmf2lvW3f71wwlY+ffhiHTSobuSBFRDIsXomA9NoIXt3Z1rdcVpjHjy49vm99amUxWxrbAfj+ezSInIiMfZkdijPLzCytXkMrNu3qW9534ph/fec8AH5wiZKAiIwPsbsjSEdOmB4f+PzpzKop7bfv1MNrWfeNcynKzx3Z4EREsiRWdwSQXtVQb4+gskFGDFUSEJHxJFaJYLiNxROKsze9pYhIpsQqEcCBh5jYuL2lb7kgL2Z/HhGJpVhd6SyNmWn+8MzWjMQiIjJaxCsRcOA2gj+GiWD2Po3EIiLjVbwSQRpjDb2wrRmA2z+6OAMRiYhkX7wSAUM/R5A6/MQhE4oyEZKISNbFKxEMckfQk3Qu+NGjPPTi9swHJSKSZfFKBAzcVPzHZ7fy7JYmPvjfKwD47w8uGuAoEZHxKV6JYJAZyp56ZXe/9dJBHiQTERmPYpUIYOAZym57bFO/9erSggxFIyKSfbFKBC2dCVo7Ewc8TsNKi0icxCoRNLV3c8/abUMeo0nnRSRuYlkZvr25k9ry/aeU/M+/fzNnHzU5CxGJiGRPrO4Iei36P/f3LSeTjhlcddYczjn6EMyGM7OxiMjYF8tEkGpLYzvuUFEUy5sjERElglO+/QCAHiYTkdiKfSLoVZCrP4WIxFOsrn7Hz+jfI+h7977Qt3zj+xdkOhwRkVEhVolgXz/684a+ZU0/KSJxFatEkNofqCc5nEkrRUTGr0gTgZmda2YvmNkGM7tmiOMuNjM3s4VRxpOqqb07U28lIjKqRZYIzCwXuBF4G3AUcKmZHTXAceXAp4Enooql11OvNvYtP/hCQ9/yo1efEfVbi4iMWlHeEZwAbHD3je7eBfwcuGiA474BfAvoiDCW/XzuF6sBqCkrZGplcSbfWkRkVIkyEUwF6lPWN4fb+pjZAmC6u/9xqBOZ2RVmttLMVm7fPrL9/b918Tw9TSwisZa1xmIzywG+D/zjgY5196XuvtDdF9bW1o5oHIV56i0kIvEWZSLYAkxPWZ8WbutVDhwDPGhmm4DFwLJMNhgDFOTFquOUiMh+orwKrgDmmNksMysALgGW9e509yZ3r3H3OnevA5YDF7r7yghj2k+hEoGIxFxkV0F3TwCfBO4Bngd+4e5rzezrZnZhVO87XLojEJG4i3TITXe/C7hrn21fGeTY06OMZTC6IxCRuIv9VTA50Gz2IiIxEvtEMKO6NNshiIhkVewTQX6uniEQkXiLfSLQw2QiEnexTASahEZEZK9YXhGPOrQi2yGIiIwasUwEq+qDUUjrJpZkORIRkeyLZSK446OLAbj7M6dmORIRkeyL9IGy0WrJmyay6fq3ZzsMEZFRIZZ3BCIispcSgYhIzCkRiIjEnBKBiEjMKRGIiMScEoGISMwpEYiIxJwSgYhIzCkRiIjEnBKBiEjMKRGIiMRcrMYauvWDC+lKJLMdhojIqBKrRHDm3MnZDkFEZNRR1ZCISMwpEYiIxJwSgYhIzCkRiIjEnBKBiEjMKRGIiMScEoGISMwpEYiIxJy5e7ZjGBYz2w688gZ/vQbYMYLhjAUqczzErcxxKy8cfJlnunvtQDvGXCI4GGa20t0XZjuOTFKZ4yFuZY5beSHaMqtqSEQk5pQIRERiLm6JYGm2A8gClTke4lbmuJUXIixzrNoIRERkf3G7IxARkX0oEYiIxFxsEoGZnWtmL5jZBjO7JtvxHAwzu9XMGsxsTcq2ajO7z8zWhz+rwu1mZj8My/2MmS1I+Z3LwuPXm9ll2ShLOsxsupk9YGbPmdlaM/t0uH08l7nIzP5qZqvDMn8t3D7LzJ4Iy3anmRWE2wvD9Q3h/rqUc10bbn/BzM7JTonSY2a5Zva0mf0hXB/v5d1kZs+a2SozWxluy/zn2t3H/QvIBV4CZgMFwGrgqGzHdRDlORVYAKxJ2fZt4Jpw+RrgW+HyecCfAAMWA0+E26uBjeHPqnC5KttlG6S8U4AF4XI58CJw1DgvswFl4XI+8ERYll8Al4TbbwY+Fi5/HLg5XL4EuDNcPir8vBcCs8L/g9xsl2+Icn8OuB34Q7g+3su7CajZZ1vGP9dxuSM4Adjg7hvdvQv4OXBRlmN6w9z9YWDXPpsvAn4SLv8E+NuU7T/1wHKg0symAOcA97n7LnffDdwHnBt99MPn7lvd/alwuRl4HpjK+C6zu3tLuJofvhw4E/hVuH3fMvf+LX4FnGVmFm7/ubt3uvvLwAaC/4dRx8ymAW8HbgnXjXFc3iFk/HMdl0QwFahPWd8cbhtPJrv71nD5daB3gubByj4m/yZhFcDxBN+Qx3WZw2qSVUADwT/3S0CjuyfCQ1Lj7ytbuL8JmMjYKvO/AV8EkuH6RMZ3eSFI7vea2ZNmdkW4LeOf61hNXh8X7u5mNu76BZtZGfBr4DPuvif4AhgYj2V29x7gODOrBH4LzM1ySJExs/OBBnd/0sxOz3Y8GXSyu28xs0nAfWa2LnVnpj7Xcbkj2AJMT1mfFm4bT7aFt4mEPxvC7YOVfUz9TcwsnyAJ/MzdfxNuHtdl7uXujcADwBKC6oDeL3Cp8feVLdw/AdjJ2CnzScCFZraJoOr2TMxx9HMAAANFSURBVOAHjN/yAuDuW8KfDQTJ/gSy8LmOSyJYAcwJeyAUEDQuLctyTCNtGdDbW+Ay4Pcp2z8Q9jhYDDSFt533AGebWVXYK+HscNuoE9b9/hfwvLt/P2XXeC5zbXgngJkVA28laBt5AHhXeNi+Ze79W7wL+LMHLYnLgEvCXjazgDnAXzNTivS5+7XuPs3d6wj+P//s7u9nnJYXwMxKzay8d5ng87iGbHyus91qnqkXQYv7iwT1rF/OdjwHWZY7gK1AN0F94EcI6kf/P7AeuB+oDo814Maw3M8CC1PO82GCxrQNwIeyXa4hynsyQV3qM8Cq8HXeOC/zscDTYZnXAF8Jt88muLBtAH4JFIbbi8L1DeH+2Snn+nL4t3gBeFu2y5ZG2U9nb6+hcVvesGyrw9fa3utSNj7XGmJCRCTm4lI1JCIig1AiEBGJOSUCEZGYUyIQEYk5JQIRkZhTIpDYMrOW8Gedmb1vhM/9pX3WHxvJ84uMJCUCEagDhpUIUp52HUy/RODubxlmTCIZo0QgAtcDp4Rjwn82HOztO2a2Ihz3/UoAMzvdzB4xs2XAc+G234UDhq3tHTTMzK4HisPz/Szc1nv3YeG514Tj0L835dwPmtmvzGydmf3MUgdTEomQBp0TCcZ8/7y7nw8QXtCb3H2RmRUCfzGze8NjFwDHeDDEMcCH3X1XOAzECjP7tbtfY2afdPfjBnivdwLHAfOBmvB3Hg73HQ8cDbwG/IVg/J1HR764Iv3pjkBkf2cTjOmyimC464kEY9YA/DUlCQBcZWargeUEA3/NYWgnA3e4e4+7bwMeAhalnHuzuycJhtGoG5HSiByA7ghE9mfAp9y938Bd4fDIrfus/w2wxN3bzOxBgjFw3qjOlOUe9P8pGaI7AhFoJpgCs9c9wMfCoa8xs8PD0SH3NQHYHSaBuQTTB/bq7v39fTwCvDdsh6glmHZ0VI6OKfGhbxwiwQifPWEVz20E4+DXAU+FDbbb2TtdYKq7gX8ws+cJRrpcnrJvKfCMmT3lwXDKvX5LMK/AaoIRVb/o7q+HiUQkKzT6qIhIzKlqSEQk5pQIRERiTolARCTmlAhERGJOiUBEJOaUCEREYk6JQEQk5v4Xp8BVdb9iPfQAAAAASUVORK5CYII=
" />
</div>
</div>
<div class="output_area">
<div class="output_subarea output_stream output_stdout output_text">
<pre>
</pre>
</div>
</div>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h3 id="Experimental-stuff:">Experimental stuff:<a class="anchor-link" href="#Experimental-stuff:"> </a></h3><p>Below are a few agents I wrote for fun.</p>
</div>
</div>
</div>
<div class="cell border-box-sizing code_cell rendered">
<div class="input">
<div class="inner_cell">
<div class="input_area">
<div class=" highlight hl-ipython3"><pre><span></span><span class="k">class</span> <span class="nc">softmax_with_exponentiation</span><span class="p">(</span><span class="n">agent</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">arms</span><span class="p">,</span> <span class="n">play_once</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">beta</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">exp</span><span class="o">=</span><span class="mi">1</span><span class="p">):</span>
<span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="n">arms</span><span class="p">,</span> <span class="n">play_once</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">beta</span> <span class="o">=</span> <span class="n">beta</span>
<span class="bp">self</span><span class="o">.</span><span class="n">exp</span> <span class="o">=</span> <span class="n">exp</span>
<span class="k">def</span> <span class="nf">policy</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">temp</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">exp</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">expectations</span><span class="o">/</span><span class="bp">self</span><span class="o">.</span><span class="n">beta</span><span class="p">)</span>
<span class="n">ans</span> <span class="o">=</span> <span class="n">temp</span> <span class="o">/</span> <span class="n">np</span><span class="o">.</span><span class="n">sum</span><span class="p">(</span><span class="n">temp</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span>
<span class="n">ans</span> <span class="o">=</span> <span class="n">ans</span><span class="o">**</span><span class="bp">self</span><span class="o">.</span><span class="n">exp</span>
<span class="n">ans</span> <span class="o">/=</span> <span class="n">np</span><span class="o">.</span><span class="n">sum</span><span class="p">(</span><span class="n">ans</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span>
<span class="k">return</span> <span class="n">ans</span>
<span class="k">class</span> <span class="nc">softmax_with_reccurence</span><span class="p">(</span><span class="n">agent</span><span class="p">):</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">arms</span><span class="p">,</span> <span class="n">play_once</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">beta</span><span class="o">=</span><span class="mi">1</span><span class="p">):</span>
<span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="n">arms</span><span class="p">,</span> <span class="n">play_once</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">old_policy</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">ones_like</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">expectations</span><span class="p">)</span><span class="o">/</span><span class="bp">self</span><span class="o">.</span><span class="n">l</span>
<span class="bp">self</span><span class="o">.</span><span class="n">beta</span> <span class="o">=</span> <span class="n">beta</span>
<span class="k">def</span> <span class="nf">policy</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">temp</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">exp</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">expectations</span><span class="o">/</span><span class="bp">self</span><span class="o">.</span><span class="n">beta</span><span class="p">)</span>
<span class="n">new_policy</span> <span class="o">=</span> <span class="n">temp</span> <span class="o">/</span> <span class="n">np</span><span class="o">.</span><span class="n">sum</span><span class="p">(</span><span class="n">temp</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span>
<span class="n">result</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">multiply</span><span class="p">(</span><span class="n">new_policy</span><span class="p">,</span> <span class="bp">self</span><span class="o">.</span><span class="n">old_policy</span><span class="p">)</span>
<span class="n">result</span> <span class="o">/=</span> <span class="n">np</span><span class="o">.</span><span class="n">sum</span><span class="p">(</span><span class="n">result</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">old_policy</span> <span class="o">=</span> <span class="n">result</span>
<span class="k">return</span> <span class="n">result</span>
<span class="k">class</span> <span class="nc">greedy_with_reccurence</span><span class="p">(</span><span class="n">agent</span><span class="p">):</span>
<span class="c1"># alpha = number < 1; will sum over a number of observations and will keep</span>
<span class="c1"># osiclating.</span>
<span class="c1"># alpha = N will allow the algo to converge to an arm, greedy doesn't</span>
<span class="c1"># really need this, kind of always give one answer.</span>
<span class="k">def</span> <span class="fm">__init__</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">arms</span><span class="p">,</span> <span class="n">play_once</span><span class="o">=</span><span class="mi">1</span><span class="p">,</span> <span class="n">alpha</span><span class="o">=</span><span class="mi">1</span><span class="p">):</span>
<span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="fm">__init__</span><span class="p">(</span><span class="n">arms</span><span class="p">,</span> <span class="n">play_once</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">old_policy</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">ones_like</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">expectations</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">alpha</span> <span class="o">=</span> <span class="n">alpha</span>
<span class="k">def</span> <span class="nf">policy</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="n">new_policy</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">zeros_like</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">expectations</span><span class="p">)</span>
<span class="n">new_policy</span><span class="p">[</span><span class="n">np</span><span class="o">.</span><span class="n">argmax</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">expectations</span><span class="p">)]</span> <span class="o">=</span> <span class="mi">1</span>
<span class="n">new_policy</span> <span class="o">=</span> <span class="p">(</span><span class="mi">1</span><span class="o">-</span><span class="bp">self</span><span class="o">.</span><span class="n">alpha</span><span class="p">)</span><span class="o">*</span><span class="n">new_policy</span> <span class="o">+</span> <span class="bp">self</span><span class="o">.</span><span class="n">alpha</span><span class="o">*</span><span class="bp">self</span><span class="o">.</span><span class="n">old_policy</span>
<span class="n">new_policy</span> <span class="o">/=</span> <span class="n">np</span><span class="o">.</span><span class="n">sum</span><span class="p">(</span><span class="n">new_policy</span><span class="p">,</span> <span class="n">axis</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span>
<span class="bp">self</span><span class="o">.</span><span class="n">old_policy</span> <span class="o">=</span> <span class="n">new_policy</span>
<span class="k">return</span> <span class="n">new_policy</span>
<span class="c1"># class magic(agent):</span>
<span class="c1"># def __init__(self, arms, play_once=1, exp=1):</span>
<span class="c1"># super().__init__(arms, play_once)</span>
<span class="c1"># self.old_policy = np.ones_like(self.expectations)/self.l</span>
<span class="c1"># self.exp = exp</span>
<span class="c1">#</span>
<span class="c1"># def policy(self):</span>
<span class="c1"># new_policy = f(old_policy, g(expectations))</span>
</pre></div>
</div>
</div>
</div>
</div>
</div>Learning Machine Learning2017-11-12T00:00:00-06:002017-11-12T00:00:00-06:00https://arjunsriva.com/ai/2017/11/12/learning-machine-learning<p>This post will talk about how I’m going about learning machine learning.</p>
<p>Mostly online content.</p>
<h3 id="podcasts">Podcasts</h3>
<p>These podcasts are amazing,and what got me interested in the first place. Get a podcast app, I love podcast addict (android).
Some awesome podcasts:</p>
<ul>
<li><a href="http://partiallyderivative.com">Partially Derivative</a></li>
<li><a href="http://lineardigressions.com/">Linear Digressions</a></li>
<li><a href="http://dataskeptic.com">Data Skeptic</a></li>
</ul>
<p>Some that are supposed to be good but never tried:</p>
<ul>
<li>Not so standard deviations</li>
<li>Data science at home</li>
<li>Talking machines</li>
</ul>
<h3 id="online-courses">Online Courses</h3>
<p>A few awesome courses.</p>
<ul>
<li>
<p><a href="http://coursera.org/learn/machine-learning">Andrew Ng Coursera</a><br />
A good first course, which teaches you bottom up, from basics to advanced techniques. Matlab/Octave.</p>
</li>
<li>
<p><a href="http://fast.ai">Fast.ai</a><br />
A course which aims to teach by coding, and takes a top down approach.</p>
</li>
</ul>This post will talk about how I’m going about learning machine learning.Game Of Thrones quiz2017-09-04T00:00:00-05:002017-09-04T00:00:00-05:00https://arjunsriva.com/iit%20indore/quiz/2017/09/04/game-of-thrones-quiz<p>Consists of three rounds, first round acts as elimination, followed by two rounds of interactive play.</p>
<h2 id="first-round">First round</h2>
<p><a href="/static/game-of-thrones-quiz/prelims.pdf">Questions</a><br />
<a href="/static/game-of-thrones-quiz/prelims_answers.pdf">Answers</a></p>
<h2 id="second-round">Second round</h2>
<p><a href="/static/game-of-thrones-quiz/round2.pdf">Slides</a></p>
<p><a href="/static/game-of-thrones-quiz/1.mp4">video for question 1</a><br />
<a href="/static/game-of-thrones-quiz/2.mp3">music for question 2</a><br />
<a href="/static/game-of-thrones-quiz/3.mp4">video for question 3</a></p>
<h2 id="third-round">Third round</h2>
<p><a href="/static/game-of-thrones-quiz/Connect.pdf">Connect</a></p>Consists of three rounds, first round acts as elimination, followed by two rounds of interactive play.Newbie quiz2017-08-20T00:00:00-05:002017-08-20T00:00:00-05:00https://arjunsriva.com/iit%20indore/quiz/2017/08/20/newbie-quiz<p>This was the first quiz conducted by the quiz club this year.</p>
<p>Indraneel and I had prepared this quiz as a way of introducting quizzing to new people and narrowing down potential recruits.
Lots of fun!</p>
<h3 id="questions">Questions</h3>
<p>Get the <a href="/static/newbie-quiz/Questions.pdf">questions.</a></p>
<h3 id="answers">Answers</h3>
<p>Get the <a href="/static/newbie-quiz/Answers.pdf">answers.</a></p>This was the first quiz conducted by the quiz club this year.Mental Health Checklists2017-01-08T00:00:00-06:002017-01-08T00:00:00-06:00https://arjunsriva.com/projects/2017/01/08/mental-health-checklist<p>Some quizes I made so it’s easy to keep track of mental health.</p>
<ul>
<li>Burn’s depression Checklist use it <a href="https://arjunsriva.com/bdc/">here</a></li>
<li>Novaco’s Anger Scale use it <a href="https://arjunsriva.com/nas/">here</a></li>
</ul>Some quizes I made so it’s easy to keep track of mental health.