Naga_Akkineni_p1_submitted.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Project 1: Digit Classification with KNN and Naive Bayes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this project, you'll implement your own image recognition system for classifying digits. Read through the code and the instructions carefully and add your own code where indicated. Each problem can be addressed succinctly with the included packages -- please don't add any more. Grading will be based on writing clean, commented code, along with a few short answers.\n",
    "\n",
    "As always, you're welcome to work on the project in groups and discuss ideas on the course wall, but <b> please prepare your own write-up (with your own code). </b>\n",
    "\n",
    "If you're interested, check out these links related to digit recognition:\n",
    "\n",
    "Yann Lecun's MNIST benchmarks: http://yann.lecun.com/exdb/mnist/\n",
    "\n",
    "Stanford Streetview research and data: http://ufldl.stanford.edu/housenumbers/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This tells matplotlib not to try opening a new window for each plot.\n",
    "%matplotlib inline\n",
    "\n",
    "# Import a bunch of libraries.\n",
    "import time\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from matplotlib.ticker import MultipleLocator\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.datasets import fetch_openml\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.metrics import confusion_matrix\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.naive_bayes import BernoulliNB\n",
    "from sklearn.naive_bayes import MultinomialNB\n",
    "from sklearn.naive_bayes import GaussianNB\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.metrics import classification_report\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "# Set the randomizer seed so results are the same each time.\n",
    "np.random.seed(0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load the data. Notice that we are splitting the data into training, development, and test. We also have a small subset of the training data called mini_train_data and mini_train_labels that you should use in all the experiments below, unless otherwise noted."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "data shape:  (70000L, 784L)\n",
      "label shape: (70000L,)\n"
     ]
    }
   ],
   "source": [
    "# Load the digit data from https://www.openml.org/d/554 or from default local location `~/scikit_learn_data/...`\n",
    "X, Y = fetch_openml(name='mnist_784', return_X_y=True, cache=False)\n",
    "\n",
    "# Rescale grayscale values to [0,1].\n",
    "X = X / 255.0\n",
    "\n",
    "# Shuffle the input: create a random permutation of the integers between 0 and the number of data points and apply this\n",
    "# permutation to X and Y.\n",
    "# NOTE: Each time you run this cell, you'll re-shuffle the data, resulting in a different ordering.\n",
    "shuffle = np.random.permutation(np.arange(X.shape[0]))\n",
    "X, Y = X[shuffle], Y[shuffle]\n",
    "\n",
    "print 'data shape: ', X.shape\n",
    "print 'label shape:', Y.shape\n",
    "\n",
    "# Set some variables to hold test, dev, and training data.\n",
    "test_data, test_labels = X[61000:], Y[61000:]\n",
    "dev_data, dev_labels = X[60000:61000], Y[60000:61000]\n",
    "train_data, train_labels = X[:60000], Y[:60000] \n",
    "mini_train_data, mini_train_labels = X[:1000], Y[:1000]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(1) Create a 10x10 grid to visualize 10 examples of each digit. Python hints:\n",
    "\n",
    "- plt.rc() for setting the colormap, for example to black and white\n",
    "- plt.subplot() for creating subplots\n",
    "- plt.imshow() for rendering a matrix\n",
    "- np.array.reshape() for reshaping a 1D feature vector into a 2D matrix (for rendering)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlIAAAJCCAYAAAALCSnoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzsnXm8TdX7x9/XPN2QhHQRkqJIidLXUPpeykwy9lWpKEOXKKIQoiRDGqWihBAaCCGKUESSKUOGG6KikOGe3x/n96yz7z3n3HvOvvuMPe/Xywtn77P3WmftYa1n+DwJLpcLRVEURVEUJXhyRLoBiqIoiqIosYpOpBRFURRFUWyiEylFURRFURSb6ERKURRFURTFJjqRUhRFURRFsYlOpBRFURRFUWyiEylFURRFURSb6ERKURRFURTFJjqRUhRFURRFsUmucJ4sISEhZmXUXS5XQiD7xXsf471/oH2MdrSPbuK9f6B9jHa0j27UIqUoiqIoimITnUgpiqIoiqLYRCdSiqIoiqIoNtGJlKIoiqIoik10IqUoiqIoimKTsGbthYr27dtzyy23pPvsoYceAuDYsWPcfffdAKxZswaAtLS08DYwxLRv357+/fsDcN111wGQI0cONm7cCMDzzz8PwIwZMyLTQCVuueaaawB46qmnAOjQoQN79uwB4O233wZg+vTpAPz8888RaKGSHW699VYAhg4dCkDu3LnNc/SDDz4A4Pvvv49M4xyiUKFCADzxxBMADBo0iJ07dwLw4YcfAjBhwgQOHz4cmQYqtkhJSTHv/ptvvhmA/fv307ZtWwC++eYbx86V4HKFLyvRqRTIwoULAzB+/HjAPZHInTt3lt+Th/1zzz0X9DkjneZZu3ZtHn300XSfTZo0CXD3q1SpUgBUr15d2oGM7dmzZwG4++67+fTTT/2eI5wp1/KA7tevHwBNmzb1dS5pF1988QUAS5YsAeCNN97gjz/+COqckR5DX8iYjh071nz20UcfAbB69WomTJgQ1PHC3Ud5KGU2ST9//jzgHkfZb/To0QBs3bo16HNG4zhecsklgHsiee7cOQCuvPJKAJo1a0aFChUA+OSTTwD3M+uvv/7ye7xIyh/kz5/fTCp69+4NeJ654BnPVq1aAZ4+BUM0jaHcYz169PC7z08//cRtt90GEPCEKlJ9HDNmDABJSUlmIvHCCy8A8MorrwCwb98+R84VTeOYlJQEwIsvvghg+g7uCZTsI/+uU6dOum3+UPkDRVEURVGUUOJyucL2B3Bl90++fPlcS5cudS1dutSVlpbm9efs2bOus2fPuiZOnOiaOHGia/r06a4LFy64Lly44Dp37pzr3LlzrieffDLo84azj9Y/iYmJrsTERNeSJUtc58+fd50/f970Z9SoUa5Ro0a5EhMTXaVKlXKVKlXKVatWLVetWrVcFy5cMPvLnzZt2mS7j9npy0UXXeS66KKLXAsWLHCdPHnSdfLkSdOXYP8sX77c1bhxY1fjxo2jfgzlT1JSkmvs2LGusWPHmv7LNenrWk5LS3NNnTrVNXXq1Kjt4+bNm12bN2/2235/fw4dOuQ6dOiQq0qVKjFzL/r607VrV1fXrl1du3btcu3atSvg63fu3LmuQoUKuQoVKhSRe9HXn/z587vy58/vmjx5cqZjl5qa6kpNTXVVrVrVVbVqVVvnivQYVqpUybxH5PmY1TU7ZMgQ15AhQ6K2j+XKlXOVK1fOdfz4cdfx48fTXW/SB3nupKSkZHr9RWsfff1JSkpyJSUluVavXu1avXq1y8qLL77oevHFF82+1u0Zt2Wnj2qRUhRFURRFsUnMBJtfdNFFgDt+pEGDBum2bdq0iddee81sBzhy5IjZvm3bNgCeeeYZAFq0aMG4ceMAOHPmTGgbbpMCBQoA8NJLLwF49Rngyy+/BODkyZOcPHkSgNTUVL/H7Ny5M7Nnz3a6qVlSrVo1AHPu8uXLZ/uYdevWpVixYgCcPn0agBUrVmT7uE5Srlw5ABYsWABA2bJlSUxMDOoYnTp1Svf/e++915G2OUWePHlsfa9kyZKAO+Bz2rRpADzyyCOOtSuUJCQkcP/99wMwceJEIPjfoXnz5uY32LVrl7MNtMntt98OYPrmjxIlSgDuGD6Arl27MmvWrNA2ziFuvPFGAF5//XWuv/76dNv27t0LwI4dO3jzzTcBTwzg3XffzU033RS+htpAnjMSz/bHH3+wfv16AK644grAE8s3ZswYLr74YgAGDx4c7qY6Rtu2bdPFhIEnQaBv375e8U/jxo0zgefytxPETLC5PGw7duxoPjt48CAAVatW5c8//8zyGH///TfgDqZs0aIF4Ln4ssIV5qA6CZSTzBgrK1euBNxBrIDPoNULFy6QcWz3799vbihfBNJHO/175513APdELiPr1q0DYMOGDV59LVKkCAADBw6kdOnSAFx++eVex5AHYO3atTl69KjfdoRzDMuXL2+CcCtXrmw+X7hwIeA94f3tt99M8oTQtm1bM5EW3nvvvUwnU+HsY8WKFVm1ahXgebn64rfffgPg1KlTlClTxmv7sWPHAChevHhA5w33vSjIC2rkyJF069Yty/3lJfb222+be1ESRHLkyMFVV10F+J5Ihepe9IUkfDz55JMAFC1a1Gufn376CYCrr77aa9vy5cvNJCxQwj2G1gkU4DWJAmjXrh1AukmhjPMrr7xiFmwFCxYM6Jzh7qNcbzVq1ADcwfOvvvpqun1uuOEGwB2QXbZsWQAefvhhABYvXhz0OcPdR5kszZw5E/A9GZLP/GXlyQJA9itTpkymAeeB9FFde4qiKIqiKHYJNFjMiT/YCCTr2LGjq2PHjq4zZ864zpw540pLS3Pt37/ftX//fleVKlWCClb9448/XH/88YcrLS3N9c0337i++eYbV86cOV05c+aMqqC6G264wWfA4LJly1zLli0L6BgrVqzwCnDdt29ftvsYbF8KFizoWrt2rWvt2rXp2jJt2jTXtGnTXDly5HDlyJEjy+MMHjzYNXjw4EwDeLt06RLxMZSA1CNHjpgAT7le69at68qXL58rX758AR0rMTHR9cEHH7g++OCDdEGvke6j/Nm4cWOmwbnr1693rV+/3iRCXHfdda69e/e69u7dm24/CfYdN26ca9y4cVF1LwKuIkWKuIoUKeJavHixa/HixZleg0eOHHENGzbMNWzYMJ/Pln/++cf1zz//uC5cuOBq06aN3wSQcPUvT548rhMnTrhOnDiRbkx2797t2r17t6tbt26ubt26md/g7NmzXuP8xRdfBH3ecI5h+fLlXatWrXKtWrXKtHnPnj3mj3zWqFEjV6NGjdJ9V67dtLQ0M8bJycmu5OTkqOpj6dKlzThKO2+66Sa/+5csWdJ1+PBh1+HDh10rV650rVy50tZ5w30vpqSkuFJSUly+mDVrlmvWrFlZHkP2E9q2bZvtPqpFSlEURVEUxSZRHWyeK1cuEwhnDeYU5eS+ffsCMHz4cHbv3p3l8SQgvX///iZwMEcO91zywoULzjXcJiKmuWTJEhNc//+zeU6dOmVE1QLhuuuuM98VROk8nDRu3NjEJwh//PGHif0JVGX+n3/+ATy/h4h1RgtDhgwB3PFc4L52RUT0vvvuA+DAgQNBHfPkyZMm8FeuDYmriSRt2rQBoEqVKj63v/XWWwA8++yzgCceLDU11cQ4SmwVeO7BBx98EIBXX32V7du3h6DlwVOgQAGTwFK3bl2/+504cQJwxy36is2Q8ZO+QnQogs+ePdsoewsul8tcs5LQkhn16tUzsTffffed8420iTxD3333XSO++MsvvwDua02eIRJYLoLN69evN3F78twBzzPHboJFKOnWrZsZx3fffRfwxJ/64tdffzXvURFHjnZSUlLSCRdnRBT3I0FUT6Ry5MhBpUqVvD6XgZe/9+/fbzLyYhGZ1M2ZMwdwB7VmnAQ988wzJlA5EHwd47333stmSwNHAh5FSRfg+PHjACQnJ7Nhw4agjieBlKIaHU0Ps1WrVlGrVi3APYECt+q+ZIYGO4GyIgGuH3/8MRAdEynrZDEjR44cMcrC8tKy8u233wIYhf277rrLbMufPz/gnowEs2gIJfPmzct0AvX7778DcOeddwL+X15NmjQBPBOpkydPmooDkUCenY0aNfLatmTJkoAmUEKOHDmibmEDHoVvmUQBdO/eHYClS5eaz6SslkxEZBIFvgPvo5Fy5cqZ573cY1khE6maNWuGrF1OIJmT/iZR8nnGxJxAcSLjVF17iqIoiqIoNolqi1RWuh1iig20HpnVTBst5M2bl0GDBgGYenlWZIUr2hixgtQ+tLoyZMVkZyUuFqhoWvmKFbR27drkzJkT8FyLAwYMcESjTKw+mclWhAtxVfpy6Ylu25tvvml023wh92CwdRLDTdeuXQH4z3/+43O7WFdbtmwJZO5GadeunaldJ0ydOtWnxS7UiISDXLtWq6Lcl6NGjQp7u5xECtaLexLg/fffB9JbogRxy8rfVu64445QNNFxrPp0n3/+eZb7FytWzFjqNm/eHLJ2ZQeROhDLIqSvmQdud56E+ARLVjX2gkEtUoqiKIqiKDaJaouUxJ1kRCpXS1CrCG1mhYhvRpOS65AhQ0x8hRUJRG3dujWQuWJ5NLJ27VrAXSld4gxEibxOnTps2bIlqOPJKuvpp58G4LnnnnOqqUEjauNyHeXIkcNYokTU0AlrVJ48eRg9ejTgCfAOJKkiVIhCvVgbrUiAdVb3logAJicnO9w6Z5A+vvzyy4Dvvp45c4amTZsC/kX/wBPz1aFDB1OpQCxxIswZbkR80ZeApsRFZVUhQKzCGWMwo4GKFSuaJAexEm/ZsoUBAwYAnhhLO4jVdfny5dlspfMkJyfz66+/AgQkTl20aFEjchyIBSsSPPbYY4DH+jR27Fgjoimf3XPPPUEdMykpyYhdO+nlieqJlD/ErG4tAxMI8gNGAxJA179/f69tOXLkMJPEYCdQ9erVM8cINCMulpgyZQrgVu2VB0G4kfOK23LixInGdeOk+/jaa6+ld+/egCe7cdiwYY4dPxjatGljsqB8IeOSFRIsL6UqoonSpUubwFNfEyjhzTffzHQCJXz11VeAJ2MPPCVlIpGVWLt2bZ8TXXFL+qo84ItonEAJ3bt3Nwr5ssDu3bu37YQPq8tMkn18VZKINN9++62ZOGZW3UG47bbbzL8lySmaqF27Nn369AE82Xjjxo3z+ixY95yTZWGsqGtPURRFURTFJjFpkRLze+3atYHMzevRiqzqfK3uHnnkEd54442gjieuAwm8S0tLM8cWbRdJOQ8H4sbLly+fo8eVMY9kWnLGWnfDhg1z1BIl17foFwFMnjwZ8GjEhJtatWr5lDuQ9P+tW7dmeYyKFSv6tMAKouUmLopw8/TTT1OxYkW/2yVQWWrT+eLSSy81VkRJqwfPWA4dOtSJptqib9++PuvESd2yYC380YQ8/8TSD54AczuuOKkdKa5Q8NQbjEZuvfXWgN6DUr908ODBxs0ptTCjCavlSKxP1s/sWhitXgwNNlcURVEURYkCotoidejQIZ+fS2yK+IRjkZ49e/rdlrFid1YkJiYaMTJfgeuS1izijuGgWbNmgFsoTpAVr6TRB4OkmUswt6xAAQ4ePAjAypUrbbU1GEqUKMGVV14JeEQyT548me3j5suXzyRXTJ06FXAHRp46dQog4gKV1pW5FVn179q1y+93y5cvD7hjiySGzxeiMD1t2jS7zbSFxG35i6GUdkl8hq+AZQnefv7559PFRIHbWieBs5GILypZsiSAV7uCoVu3boBvEdb9+/dH3Jol16fV4jB79mzbx+vSpQvguXY3bNjA66+/br+BIebo0aNGYDMz5Fq/7LLLTKxxoAKe4SQlJcX8W4SNrRapYK1JEpzu67hOoBYpRVEURVEUm0S1RWrOnDlmxeuLa665BoCvv/46oOO1a9fOkXZlBykVcf3113ttCzT+RTJJZHXfu3dvGjRo4LWfWDMkAzDSSLZLZtYLXyQkJJi4IWvdPslkk98tHNIADz30kFmVy/myU+rj8ssvB9ypvmLxEH799VdjbQv2N3MKiXXzJ4QqMgG+kDIkkmYscScZEYteZnW0QkmPHj0Aj1illbS0NObPnw947icrl156KeC2REF6q8+SJUsAt7hndsoEZZcyZcoAUKFCBa9t586dC0jQ2NdvI7z99tsRERe1YrUm7tixA7AfO1unTh1TBkn4+uuvA5IViBSff/65z/HNiDUL0QlLutNIjFtSUpJ5boj1KTsxUlK2ytdxnSCqJ1Lnzp0zujMSMCimVvCkcIrp3R8yybjsssvMZ6JzFG6JAHELicaMlR9++MHv94oUKWIU0KVAqD/VZUGCy9955x07TY0a7rzzTp9p25JeHk5dMNFzsouo18tD74MPPgDSuyTERJ+cnByxCZQg/bW6UjOjQIEC5joVZWl/EyghEPdgKJDJQf369f3u079/f68al3nz5jWTXlHRlskKYCZe8ttFuwxJZgXbJblD9NvAM6mWBUQkdZVksm6tgiFhDsHKFEioyKBBg8yEQ2qCSlHyaCYzSQ0JsZCklQsXLkSlgr31OZixCPGaNWu8JlWZ1ddLSkoyEyiZaO/fv9+2EnpmqGtPURRFURTFJlFtkXK5XGYGKsF/kyZN4tprrwU8q5HChQv7NbsWKVLESAmISyYhIYGRI0cCma/GQonVVSJqxxIsnZKSYlKnM6bagyfYPrOV7saNG02AaLQggo7XXHNNQOnysjrxJRi3e/fuiPTviiuuMGPXsGFDwJ3WLxXjRQ6gRIkS5noTK2Tnzp1p3Lgx4AkAlmOdO3fOrChFzT7S1ijABNi+8MILFCpUyGu73Edy/11++eVB1Sd7++23A66V6TRSb05CBKyIMr1YDAFuueUWAJ599lkvK5bciyNHjjS/SbRbosB9/YmApS8hx7p16wLpLegSMC/3sCiiR4KLL74YSF/TM1hq1KgBwIMPPgi4LcEydiNGjAA893U0IzI3vpDnjgRd79y5M2iJnXBgdd9JWIOwf/9+4/oTa1Xbtm2NiK4gfZw5c6Y5nswjHn/8cUddeoJapBRFURRFUWwS1RYpKxIP88EHHxiLlFgsxowZY1YTgqygZs6c6RWEt3btWhMIGimsadASqyFtKlKkSKaCnbJasm6TGboE4XXr1i2iwZEZq92DJ1jXX6CgWHjEh3311VcD6ct1TJ8+HXCvLA4fPuxcgwNk2LBhJrC4SpUqgDu4ddu2bYBndd6uXTuvAN2EhAQzZufPnwc8cQ0jRoxgxowZoe+ATU6dOuXTItWqVaugjiMWYIlrnDBhgvntwo1IdPhC2pc3b16zCpb6eGIFAW+x20gKbvrjxIkTgNvyLYKMQq5cuUztSGu8idQw8xV/KJarRx99NCTtDQaRopD7yl9SREYKFSrE448/DrhrIQJGjPXs2bNGbsQqihvtiMXUmgAisUGvvPIK4LGs3X///WFuXWDIe+zuu+82kiFWi5MkEFgtV2KBkphEa9KKHE+u51BYowASwqlrkpCQkO2T5c2bl08++QRIX3hTHmSSsSFBoFYlX3mIDxo0yBSDDRSXyxXQHZpVHyXYeO7cuQDUrFnT1zEy1ZuR+nt79+4F3PW7RJtJXEx2CKSPgY6hmMtnz55tEgZE/XvdunXGrSJUr16d4cOHA5kHNouOlB1dJafGUJSgA63dKDW/du3aZQL/ZRLo9OTJqT5mpHv37uYBHejLKiOnT5826tDW7MtgcaqPco9ZXXCirSPuuUcffZQrrrjC67vLli0DPO4guRedwsl7UXj11Vd96oHJy0Vcdb///rvJLvY1eZb71BqAHixOX6eSgVawYEFTEF0mvtaqA6Kz17hxYy+V902bNgFujT9ZuGeHUN2LvmjevLmZLEnm6K233mqCrSXYXO7hXr16ZfeUgPN9lOQGa6C5ZNmNHTvWTKSskyfRhpLPhD59+mQajB4ogfRRXXuKoiiKoig2iTmLFHhcYbKaT05OznR/CVqTyuvimgkGp2feoh+0ZMkSE4xsOYZZLYti8JEjR8xKcOfOnYBnBeUUoVgFL1++3ASs2sXlcpmVlJjjxTUW5HEcHUMxPVtdj74Qpe5w1JAL5SpYrGgSnBxAWwCP+6VNmzbGmpwdnOqjJHfUqVMnoPNKpYVly5YZaYdQBZSH4l4sUKCA0ZQTV0ewvPbaawwYMAAgW6EDTl+nIk8wePDggC2mMnbiChIrpCT/ZJdwWqTq1atnQkMkdKJEiRKm1unGjRsBj0XOKSX6UPWxbdu2jBkzBvC2NPlDLKtyPzvlxlOLlKIoiqIoSgiJSYuUIKJpzZo1MyKAUktI6soNHz6cKVOmAGQrODlUM+927dpRuXJlANOHlStXsmDBAsATjyNxUaEkFKvgpKQks3qX+Ka8efMG9F1ZWQ0aNMiR+mvhXCFGilD2UYRgn3rqKQBatGjhtc/OnTtNrUAZv0AV+wPFqT7269cPIEthwhUrVgCemDiJowolobgXAfLkyQN4+l6gQAFjrZDxFWs5eMZQ4o0OHjzoSL3AUF2nw4YNo3///oDHAlymTBljvZfEhh9//NHEqQYixWKHcD9vVq1aBXiCzsGTzCJegd9++82JUxlC2UerjAH4rrX34Ycfmhgqu0r2WRHQvRjLE6lwoi9hN9npX/PmzQF3Rp8U6LUiLktR3xU3kFNZXTqGHrSPnvIu69atA9wPbpkkSXbohg0bmDdvHpA9V1awhPpejDR6nXpwqo+iqyjPzUOHDhkF/kWLFjlxCi90HN2oa09RFEVRFMUmapEKEJ15u4n3/oH2MdrRPrqJ9/6B9jHa0T66UYuUoiiKoiiKTXQipSiKoiiKYhOdSCmKoiiKothEJ1KKoiiKoig2CWuwuaIoiqIoSjyhFilFURRFURSb6ERKURRFURTFJjqRUhRFURRFsYlOpBRFURRFUWyiEylFURRFURSb6ERKURRFURTFJjqRUhRFURRFsYlOpBRFURRFUWySK5wni/cK0BD/fYz3/oH2MdrRPrqJ9/6B9jHa0T66UYuUoiiKoiiKTXQipSiKoiiKYhOdSCmKoiiKothEJ1KKoiiKoig20YmUoiiKoiiKTXQipSiKoiiKYpOwyh8o2WPIkCEAPPPMM+k+X7FiBQ0aNIhAi8LD3LlzqVChAgDVqlWLcGsUxcMNN9wAwFdffQXAmjVruO222yLZJEdYuXIlAC6XO2u9Xr16kWyO43z22Wc0btwYgKVLlwJwxx13RLJJIWXevHkAnDlzBoB27dpFsjlxR0xPpFJSUgAYOnQow4YNA2DixIkA/PPPPxFrVygYMmSI1wRKqF+/fngbEyZq164NQIsWLfjtt98i3JrQMH/+fACaNWvG1q1bAc+E+cMPP4xUs0JO27ZtAciTJw/vvfdeRNty2WWXAXDo0KGgv3vy5Ml0f3/33XfONSyClC1bFvBMpOINl8tl+havfbTStGlTAE6dOgVAjRo12LBhQySb5DgyjmlpaQCMGjWKsWPHAnDs2LGQnltde4qiKIqiKDaJaYuUULBgQUaNGgV4TOzffPNNJJsUdpYvXw4QVy6+Sy+91Pz7yy+/jGBLnOfyyy8HoGLFioB7FVW5cmUAJk2aBLiv5dTU1Mg0MEQUKFAAgMmTJwOQmJgYcYtUdiwSF198cbq/Z86c6UibIs1HH30EQI8ePQC48cYbAfj2228j1iYnSEpKAuDqq682ny1evDhSzQk7YpGKJ4+NWPDFEiX38xNPPEG5cuUA6NixY0jboBYpRVEURVEUm8SFRcpKpUqVgH+fRSreWb16daSb4Ci9evUCMFYoK+fPnwfgzz//DGubwsGYMWMAtyUqWsiO1e/BBx9M93+Jt4p1Tp8+DUDOnDkBdyxbPFC8eHEAY6mA+H5XFClSJN3/ly1bBsCPP/4YieY4Tt26dXnsscf8bs+bN29Y2hF3E6nevXsDHtO0BIHGOv4CzYV4cukJ1157LeA21a5ZsybCrXGWYsWK+d0mAZJiho8XPvroI26++eZ0nx09ejRCrckeuXK5H53ifk5IcNc1jbcAXnl+2gnEVyLPoEGDIt2EkNKmTRsKFSrkd/u4cePC0g517SmKoiiKotgkLixSOXLkMIFm1atXB6Bw4cJA7FukJJDu34S4u5544gnArSMVL+b3kiVLAvjUGhL5g9mzZ4etPS1btjTW21Ah1tL69et7uRrEghxr1KhRA8BoEcVrCv3x48cB2Lt3b2QbEgLEfSnaSvGIWEpz5MiR7v+xTr58+QBP0o4/wmXxVouUoiiKoiiKTeLCIpWWlua1IrzrrrsAeP311yPRpLAydOjQSDfBUWrWrAlgfN/xtGIUi6msEIUDBw7w/PPPA/DLL7+ErT01a9YMmUWqYcOGAMyYMQNIH/j67rvvAvDpp5+G5Nyh5p577kn3/wMHDgAeK4cS/axfvx6IfUkHf1x22WU88MADgLc0QKwjMabNmjXzu8+SJUvMfRlqYnoitXPnTr/bRJ8n1gmkNMOKFStC35Aw0qZNm3T/F7X6WKdgwYJ0794d8DZJL1iwgGnTpoW9TQMHDgzZsV944QXAd2C9uPROnDgRsvOHE3E9h1pBWckeoqj/byBnzpyZBmLHMp06dcpyn9TUVP7+++8wtEZde4qiKIqiKLaJaYtUvKk++yKzOnpiiYoXi5S4vZKTkwGM5IGY4GOdKlWq0KRJk3SficKw6LvEA1KnzZdW1IABA4DYTwLp06cP4HGZxJvyviCK7eXLlwdg9+7dkWyObURRX+p3/hu49957I90ExxGdNnFZZsacOXNC3RyDWqQURVEURVFsEtMWKam0Pm7cOC9103hJ88yMeFsFi3icqCi/8847QOwHSEp/Bg8e7LVt6dKlACGXIAgXRYoU4cMPPwSgQoUK6bb9/PPPzJs3D/BYcmKRJk2aeAXvbty4MZJNchxJDBB1/YMHD0ayOdlGLGp169aNcEvCx3XXXRfpJjjORRddBHjG0xcrV64EwuupiemJlJWML9tSpUpFqCWKHW688Ubj9lq3bh0AU6ZMiWSTHKNbt24A3HnnneY5Sx9VAAAgAElEQVQzCUqOlwmU8OGHH5qsy4w0atSIXbt2hblFzmMdxy1btgDxU3JDkD5euHABiP0it3/99Rfg0RWSUjHxTrzqSGXGDz/8ABC2QHNQ156iKIqiKIpt4sIi5UsG4frrr49AS5wjsyDzeEJkKmbPnm1WvRKQLKvhWOfKK6/0+mzPnj0ATJ8+PdzNcRRZ4TZq1AjwJAxYkSLM586dC1/DQoho1IEnaD7Wg+fjHalbKZbgf4tFSjw14ooeNmxYJJvjGL4sa5G0uqlFSlEURVEUxSZxYZH69NNPefnll9N9JmmSsUq8W6Ry584NwEsvvQRAmTJl+PjjjwFYvnx5xNrlJHIN+lLfnTp1KhD7sSdiifrss8+8tklg+eTJkwHYt29f+BoWAsTaVqRIEbP6DWcchmKfI0eOALB9+3bAU8/z38KhQ4eA8NWeCzW+EpAiqd6uFilFURRFURSbxIVFCrz9or7EAJXoQdJYJd7kxIkTPPvss5FskuP873//A3xXKI8HkdH8+fPTs2dPr88lQ0rkHiSzLdaRWLeCBQua1a9Y2+KV/fv3R7oJSpBUqlQJgFq1apnPZs6cCcSPRSozRo8eHfZzxs1EKta1hjISSI29WCUhIYHHH3883Wfvv/9+XBUPzZ07N40bN0732dmzZxk7diwQH5OLV155xauPf/31F+3btwfio4//dlatWhXpJjjK5s2bAWjevHmEWxI6RI3e1wIu1qlSpUqW+0Qi8UNde4qiKIqiKDaJG4tUvBHPweY1atTgiSeeADxuoFGjRkWySY6TJ08e6tSpk+6z8+fPmyBzSceORYYOHQp4XJdWevbsySeffBLuJoUFq6SKBO9++umnkWpOyChXrhxFixaNdDNCwvz58wG321lcXzfddBPgEQKOdeTdYQ13yegBiFU6dOjgd9vTTz8NRObZqhYpRVEURVEUm6hFSgk71lX8N998A8RfUOtDDz3k9dnu3btN+nUsIrXzWrRoAaRf8Urdy3DWtwo31pW+xLqdPn06gi0KDaVLl/5XJOvkzZsXwFiO48Ui1bRpUyD+4oazQmJsIyHkHBcTqZMnT5oaXr5UpOOVIUOGRLoJQSHaLcWKFTOfzZ49O1LNCSl9+vTx+kx0lWIVaX/VqlXNZ1JHcM6cOQD89ttv4W9YmJAX07/tBRVPSLD50qVLadiwIQAPP/wwADNmzAAgNTU1Mo1TssXnn38esXOra09RFEVRFMUmcWGRSkxMNDXbMq4WL7vsMhMYqkSW2rVrA5AzZ05Tf23t2rWRbJLj3HvvvUB6ZX1Rv544cWJE2uQUvuqTiWJ0PFuifFGjRo1IN0GxgTx3vvzyS2OREt2lgQMHAjB8+HAOHz4cmQY6gCS0WHWkRO5Bgu0VZ1GLlKIoiqIoik3iwiKVGTVq1IhJi1SDBg0AT925FStW8OWXX5p/xyK33HIL4A7WlaDrTZs2RbJJjmON/xIkuD7WrTZffPEF4K41B2519niVOvDFhg0bAPdKX/4djxw9etSkkMd6fUR/PPfcc6xevRrwXNePPvooACVLluTuu++OWNuyy4IFCwB3wku1atWA+LFERWtt0oRwBk4mJCSE5GS5cuVi0KBBAObvN998E4Du3bs7cg6Xy5WQ9V6h62M4CKSP8d4/yF4fO3fuDMCkSZMoWLAg4ClaHA7NIb1OPcR7H+O9f6B9jHbC3UdZqC5atAhwa7tJkLmUG3OaQPqorj1FURRFURSbxIVFKhzo6sJNvPcPtI/RjvbRTbz3D7SP0Y720Y1apBRFURRFUWyiEylFURRFURSb6ERKURRFURTFJjqRUhRFURRFsUlYg80VRVEURVHiCbVIKYqiKIqi2EQnUoqiKIqiKDbRiZSiKIqiKIpNdCKlKIqiKIpiE51IKYqiKIqi2EQnUoqiKIqiKDbRiZSiKIqiKIpNdCKlKIqiKIpik1zhPFm8V4CG+O9jvPcPtI/RjvbRTbz3D7SP0Y720Y1apBRFURRFUWyiEylFURRFURSb6ERKURRFURTFJmGNkVIURYk1Hn74YQAmTJgAQO7cuUlIcIdNSNH3n3/+GYCRI0fy7rvvApCWlhbupqYjMTERgIIFCwLw+++/888//0SySRGnb9++ANx///2UKlUKgDNnzgCwcOFCAHbu3Mk777wDwK+//hr+RgaAXH958uQB4MKFC+TMmTPdPufOnYv4Neg0zZs3B+Cxxx4DoH79+gwdOhSAt99+G4B9+/aFvV1qkVIURVEURbFJgqyownIyhyP3ZaU1depUWrZsCXhWiNYV41dffQVAnz59APjuu++CPle4sxPy5s0LwKhRowBo1aoVl19+OQCbNm0C4JdffgFgz549fPbZZwAsWbLE9jk1U8hNpPt42WWXAfC///3PfCar5pdeeinT70Z7H+fNmwdA06ZNAffqsVGjRgDs2LEjoGOEo49irRg/fjxNmjQBYO/evQDMmjWLb7/9FoA2bdoA0Lp1awAKFCjAuHHjAOjfvz/gthYEixP34g8//ADA1VdfDcCyZctITU1Nt8+pU6d47bXX0n22Y8cOTp8+HVyDgyRS1+kTTzwBwHPPPZfpfrt27QKgR48eACxevDjoc4Wyj23btgVgxowZAKxZs4abb7453T6vvfYaGzduBGD58uWAp19OEY5xLFeuHOC+x5555hnAfZ/9/3HNO3/NmjUA/Oc//7F7Kp8EdC/G8kRqzpw5gNvcl9HUbv2//Pvo0aMATJw4kREjRgR1rnDf+PIQHjlypK9zSJvMZ2LWHDRoEACHDx8O+pxOTqQqVqwIuC/q7t27A3DFFVcE1Z4333wTgIEDBwb1PX9E4ySjcuXKANx+++1UqFABgK5duwKehwV4xrp///6ZTqaisY/Sj9dff50OHToA6a9ducaffvrpgI4Xjj6+//77ALRr146UlBQAXnnlFQDOnz/vtb9c759//rl58Ms4Pfnkkz6/kxlO3IsygcvsGW99EQlr1qzh77//BmDs2LGAZ1KWcSJml0hdpyVLlgTc7w5x88lzSbZdccUV5jotUqQIAHfccYeZjARKqPrYrFkzhgwZAkD16tUD+s5HH30EeN4r4orOLqEcR3nPbd68GfAsCDLuI9evXLOdO3cGYMGCBcGe0icqf6AoiqIoihJCYtIiNW3aNAA6duwIuC1N9erVA2Dbtm1e+4vlqkWLFgBs376dG2+8EXCbtgMhnCuofv36MXz4cACvAML/P4e0yWubuPiaNWsW9HmdWAXLylwCN8XKYgdZxW/dujXglVdmRIO1pmrVqgDGSnfPPfcAnpVvVpw4cYKLL77Y7/Zo6KMgfZXA0C5duvi8dvfv3w8EbrEMt0XqrrvuAmDRokVZfq9ixYpmP+nPSy+9xLBhwwD3+AVCJC1SGbeDJ4xg06ZN3HfffYA7eN0u0XSd+kKeNxs2bADciQONGzcGAg+fcLqPuXK5c8PWrVsX0PPw999/N8HmxYoVA+Cpp54CsnZtBkqoxjEhIcG4y8V96W+/jNfvypUrAbjtttuCOaVf1CKlKIqiKIoSQmLSIrV+/XrAk+bYp08fs2LyhcRoTJ06FYCWLVtSs2ZNwLPiyIpQrqBy5HDPZ9977z3AHcAqn0mA61tvvUW1atUAT7C50LNnTy699FIA1q5dC0CdOnWCbYYjq+CJEycC8OijjwZ9fn+4XC6zohdLXagCeCG0q+ABAwYA8Oyzz/rd58cffwSgSpUqPrfJdeCLSPcxKSmJwYMHAx4LsNWCFisWKXlmvPnmm5QvXx6Ahg0bAp5YDH/ccMMNAKxatQpwJ45IjIoEp2eFE/eiBCTLeGQVY+LnHNIe89kbb7wBwCOPPJJVE/0S6es0K+T5KzGnQ4YM4eWXXwagV69eAR3D6T5KApK/RACx4B88eBBwy3Z8//33gCe5Q54pL7/8Mrt37w7ktJkSqnGsWLGiT++Sj+P6vX5vv/12vvzyy2BO65NA+hhzOlLFixfnkksuATzByJlNosDjvpPMi9atW1O3bl0g8IlUKClevDjgefCBR4NGAl0//vhjv99v166dmUhFGrmoJbC/ePHiRrtGggYlE9Efkp0oCQGFChUy2Rpnz54FYMyYMZw7d87h1oeWihUrmpdyZhQuXNjvtunTpzvZpGwjLkl54aSkpGT6Yo4V5Jkh4QPBIFnBt956K+B+7vz3v/8FPJOazCbSTjFr1qx0fwMmpEGyDrt3724WaxLsX7ZsWbO9QYMGQPrEB8mKEp2qkydPhqoLEUOev6tXrzafybUgAfjyu4ULcan6Y8+ePQBcddVVXtveeustADOxqFGjhnEVikEiljTGtm7dCriNIvIul/mAkJKSYowugYbw2EVde4qiKIqiKDaJOYvU0aNHjR6UWD2CJS0tzeesPVKIRpBY1sqUKWPMt5lZonwhptsqVaoYF1E4EbO3pIrXq1ePI0eOAJ4U3EARXZR27dqZz5KTkwGYNGlSzFikZDVvTY3PzGojFjkr4sp0Kv3cCdq3b2/Sqa+77jrA7RLJqKYsFovJkycbt8OYMWPMdnGBxRti7R48eLBxC4llasSIERFRnRZLk/Dqq6+af0uCiBWxklstoeIizJ8/PxCfFilB3GkARYsWBdxq2oBRPw8XMlb+nh3ynA0Eq5Xy2muvBYjI+8If3bp18/m5hLW0atUKcFvT5HkplrWyZcsCbnem6ME5JffgD7VIKYqiKIqi2CTmLFIQvGUjIxJAGS38+eefgCcQuVOnTsa3mxkSv1C5cmWzSpEZe6Aq0aFCAgUDCRgMhp9++gkIPI08GhDL2aJFi8xqVmIvPvnkE7OfWHUkHsyKBPhKwkQkqFSpEuARvOvbt6+p9SXXX1pamrGszpw5E8Aoff/6668m5s+6qnZKOC9aeeutt0z80S233AK4a6SJJTqa8RXPKPE2f/zxR7ibEzYuuugiIL3FTrjpppuA8FukJHDcV7LJpEmTeP75520dV4RIo8ki5SsxAjzizNZ6ehL3Jd4K6ztHkjtGjx4dknYKMTmRyi4ul8vxF7wTyItH/vaHBKdPmTIFcGtNyQRD9LRiGQmclxd3rCMTqawyGaVUihXR6hHXUCQRPSWZ8FuRwNuZM2calX2ny1HEKmfPno2p4rFNmjQxYyxuEpn4pqammsxZSfyIN3LkyGFcz1Y3u/wGmekahRLJBn3hhReMjpS0c8uWLSb7NVhkcVazZk0OHTrkQEuzT+/evdm+fXu6z/bt25dpQWIxSIgKf/Xq1enUqRPgGbNQFTRW156iKIqiKIpN/lUWKUnb3bBhA+PHj49wa+wjLqCkpCTzmawS4wEp2lujRg2vbZ9++mm4mxMUElieK1cu7r77bsBTT+/ZZ581q/iMbp1q1arRvHlzr+OJRSrj6iwS/PXXX4BH92nfvn3GUiZuh6ysUPfee2+6/+/YsYNvvvnG6aYqNpBVe4sWLUxqfEbefvtt25aPaEcCy3v06GHqXVoRN6coZ4cbsWpKjUCnEKX2aHLVnjp1ylTpkGSUadOmZZpsI8ln4uq77rrrjItQnsFqkVIURVEURYky/hUWKakvJErL0RgfFQjiF589e3a6z3/88cdsB+BHmlKlSpk03J49e6bb5nK5WLp0KeAJNo82pHbe448/DnhiS6ykpKQY5XlZ3Yq8Rfny5b1EVY8ePWrSfKMBEbzLKHwXKM2bNzfWVIk3eeeddzhw4IAzDYxiJMEl2hJdEhMTzbUrFlQrMk4vvPAC4BHtjEd69OgBePpqZfDgwbz22mvhblJYkP6GWrQyGA4dOmTU2GOBuJ9IFS9e3KgIi+lPso5iiYIFCzJ06FDAoyYtpt4XXnghZgM/ZfIwa9Ysv2Vt9u7dazIyogkpytu/f/+AFbAla2vevHmAZ0LVtWtXr5fs3Llz2bJli1PNDSmS5ODLHStK7YMHDzalN+TajZSbJJyUKFGCfPnyAR6tODsljpxE7rvRo0eb56E1k1KelbKoybh4iyd69+4N+A6PkHJA06ZN49ixY2Ftl+IcUgD5888/D8nx1bWnKIqiKIpik7i3SE2dOtWstEaOHAnElmuvUKFCgDvIU9LPZTUv/ZJA31hCrBTi2qpZs6ZXgVQJZJ40aVIEWpg14hLp0KGDl9rwZ599ZmpfidbSgw8+6DV2TzzxhPlOxn6LmzBaEDkK0bHp2bOncWHKeCYmJvpVXna5XKb/Mu6RrHUpiQFiRatWrZp5NkjAqhSCzQ7dunUzv8/DDz8MEDFVfknkkN+/evXqPsdLioTHsyVKCqyLRc6qYi4K8BJ0nlU912hCNLAKFSpkKilIILnUqY1FypUr5+XukzG0Iu8Ra5WFUMsCqUVKURRFURTFJnFrkRIF5Ro1aphV7/vvvx/JJgVF7ty5AY9YWrNmzSLZHEe55557TAqvVKMHj0VGLFDydzSk/lsR6QxfKdIiktqrVy8jcSBxQevWrWPy5Ml+jyupvaJi7oQ1JLvIKn3IkCHcc889gDteD2Dx4sWMGDECgEGDBgFQunRpypQpk+VxRXQ2UrF9hQsXNqryErdmZcWKFYA7hmjZsmVA8ONRokQJwB3ELJYuq5J9JHj33XcB3+rYcv29/PLLthMKIk3RokWN9e/6668H3ErkUq9T6uWBx3KTkZ9++smI4x4/fjyUzbVFtWrVTFxXzZo1vbZLPF6ePHlMf+VZFIsWKZFMGTFihKmdJ/Ts2dPLoip9TEtLS1dxIZSoRUpRFEVRFMUmUW+RknIoLVu29LtP5cqVzX4yA5UsKpfLZSL2f/vtt1A21TGqVatm6ibdfvvtXtsl40dqlG3dujV8jcsGkl7dt2/fdJYoQWKKpDZbqCt22+Gqq67i/vvvB9yleQSpP9anTx8gveCmrIbWrVvn97iHDh2ibdu2QHSNp8Rp9evXzwhyimXKmgEj/65Ro4YR+BPLgC+ee+45ANavXx+RUjJNmjQxligRPZ0yZQqHDx8GPPFvCxcu5L333gM8v4VktPlDrHhyD1988cXmujh9+rST3QgaXxYJud7q1q0LZE+YUe7xsmXLmjp1kvkWCiRz9o477gDgkUceoUKFCtk65rx586LSEiXPm+HDh5t42UCRWFtfiMix1Tos1/8XX3wRbDMdQ97p/fr1A/CyRgFUqFDBb0ymFYlTrVixYkieN1E9kXrooYeMdof8WAkJCen+LdsyBipbU8kl4E4eYldffTVz584Foks7o2LFioA7TdzXBCqj60dkHawkJiYCUKVKFQ4ePAgQUSXicuXKMX/+fACuueYaIP0EROjbty+vv/46EF1jkpElS5aQP3/+dJ+99dZb5kVpfWmIiV2KS8u4+SJ//vzmwRENiOtAalUBRnldArF9Ua9evUwnUILUMCtWrFjEa/INHjwYSF+gVq7ZRYsWmd+gSpUqALRr185vm6tXr27ct+JaWrlyZdTovGXUs8qRI4eRQpA2WgumZ3yutmnTxksjzRrUa0WuZ2tChZPky5fPuP+laoWvNgeKLH5atWplnkXRFGQuavPBTqKywpcrXq6F48ePG6PE119/7eh5s0L0H/0VMA4GuWa3bdtm3LxOJlKoa09RFEVRFMUmCcHO2rN1soSEgE4mdXEWLlxoZstiQZo3b56XunXLli0ZOHAg4G2R8mWt2r59uwnSC9T64XK5ApIkDrSPVsRyIS6Rm2++2WufgwcPmpWIpE5LMOsdd9xhLFhiwr3mmmuMEnqgoo6B9DHQ/okVcP78+UaxPDP279/vSHC1mKl79erltc2JMbxw4YK5jmS82rRpYyxR8vvXq1eP/v37A6QTGhWrolgJZYzy5MljLAEiPnrixIlAmpsOJ/pYqVIlFi1aBLjdUuC2RvmyRIkFVALvx44da6wTUrtNXFybNm0yQdxiQdi3b59x8wUa4OxEHzt27GgSOSQxZcKECV77XXzxxbzyyiuAx2116tQpIx+wevVqwG1ZA7f1RVx74iLr2LEjV1xxBeBWcoesXXxO3otWNm7cCGDuSauF3885pD2Z7uNru/y+4gq3kp0xFDder169grbOnDx5EnC72cVCLNerhBIMGzbMXJ9XXXUVAEeOHAnqPOD8O0OuKyfcw3PmzDF98hWIb0WebfK7W2tjhuq9WLFixYCSjPxZQzPbJvIyYnXPikD6qBYpRVEURVEUm0SlRUpifwYOHGhmpRKcbLUgyUryySefNP54CQQVa03Lli3NNgmq/Oqrr4JueygtUnv37gU8cSO+OHv2LH/++SfgERIUiQQJpMuIWPMOHToUUDucXAXfd999gCcIO1yIQJuv1YbTFimxKo0aNYr//ve/gGdVJ4G7Vo4ePWosh3ItSgyGCCWCRxqgW7duZgUdKE70cfPmzSaeTdL1pU6llfvuu88IN0ogaEJCgrHWSHC2NZ5ILHay4r/33nuNlSbQ8g1O9LFy5cqmPI1YXYYPH27kK6y/u1gCGjZsCMBrr71G6dKlpS1ZtuP111830itr1qwBsk7HDpVFqnbt2oBHWsSfIKflHNKeTPfJbLvE9ljJzhiKALHUbcyKtLQ0Y+2VWoFiTfZFwYIFTeKHyHv85z//CTr5JZosUuLRkTI4W7ZsMR4AiWns1auXuS/lWrciv5m1XFeo3ou5c+c2FsPMSrpldu0lJCQYC7g8z4oXL24s/eINyCoOLqB7MRonUjfccAMAa9euNTeyuOIKFSpkHurysnS5XF4FVZ1WTA7lREoGViZIAZxD2uS1TW6yY8eOGb2jl156KaDjOvnwTkpKAtxBi77qrzmBmJytN4K4fdu0aeO1vxNjOGPGDJ/H9oVkV+7btw9wB7FmdLNKRunAgQPNy1kyq1asWMEzzzwDBB7o6UQf9+3bZ1xADzzwAOC+nmSyJ8HUt9xyi5kYyWJg5syZpiZkqDSinLoXJXhcNM2aNWtm7h9xbVoRVeXChQubl5rcg1KMeu7cuaYmmxTaPnjwYNA6NqGaSAmyuBwzZoyp3enLTZadiZRoVsk1ZCU7YyjtHDhwILVq1QI8Wm3gcanKs2D69OlBV0eQcAtZrDdv3tyMf1ZZm4LT7wwZi2bNmhndNnlXWpFs7m3btpmEI3FPZ1XjURbncn3XqlXLuPRWrVoFeMInILTvRQnxWb58OYDPZJzMJlI///yzabtMpN577z3zO0oyiBMTKXXtKYqiKIqi2CQqLVLCwoULjcKsrOisM1BxraSkpIQ8vTiUM29JK58zZ05A+4tbSEzN9evXZ+fOnYBH/+PAgQNGHydQy0AoVsFJSUnGzde+fXsAfvjhBz777LNgDuMT0fxZuHBhQPs7MYY33HCDWZGJwnf+/PmNdUxWfO+//76p1yUr86wQ166sfJOTk03qeKDq7k5ZpHr06AF43CjNmjUzAeUSqHzkyBETsCpWunBIGYTqXqxSpYqRW7GqnYsFQlbk4AkyF8kEucecUlAOtUXKilghfAUbZ8ciJWEYoj9mxakxFHeTPP/+/vtvY30SK6kTdOzY0ci2SBB9VoTynSFtsVriBHkGhVrNG0LbR0FU+Lt27Wreb5bjel17ImsgQfRWrCEX4trPCrVIKYqiKIqihJCotkjdeuutJuVa2vnVV18Z65MEcIZDsTwcq4uHHnoIcNctEz+3xGzMnTuXAwcOAJ76TxJbVbp0aWMRyY4qcThXwZHA6TEUVeW77rrLrFJF3iBSOGWRknitDN+RcwBuWQeJDQon4VgFRxq9F91oH6Mb7aObqJ5IRRN6wbiJ9/6B9jHa0T66iff+gfYx2tE+ulHXnqIoiqIoik10IqUoiqIoimITnUgpiqIoiqLYRCdSiqIoiqIoNtGJlKIoiqIoik10IqUoiqIoimKTsMofKIqiKIqixBNqkVIURVEURbGJTqQURVEURVFsohMpRVEURVEUm+hESlEURVEUxSY6kVIURVEURbGJTqQURVEURVFsohMpRVEURVEUm+hESlEURVEUxSa5wnmyhISEmFX/dLlcCYHsF+99jPf+gfYx2tE+uon3/oH2MdrRPrpRi5SiKIqiKIpNdCKlKIqiKIpiE51IKYqiKIqi2EQnUoqiKIqiKDbRiZSiKIqiKIpNwpq1p/w7ufXWWwGoWLGi333uv/9+/vOf/wBw5MgRAJ544gkAfv31VxYtWhTiViqBIuO4YMECrrrqKq/tOXK412cDBw4E4Lnnngtf47LBqlWruOWWW/xunzt3LgAHDhxg+fLlgPs3UKKL2rVrA9CiRQsA1qxZw6pVqwC46667AKhSpYq5jlu3bg3A+vXrAVi0aBHjxo0D4O+//wbgn3/+CVPrFaFq1aoAPP300wC0adPGbEtIcCfS/fzzz5w+fRqAMWPGAPDuu++Gs5nu9rhc4ctKDDQFsmjRogA8+OCDNG7cGID69esD8Mknn/DTTz95fWflypXp/v/1118D8Pvvv9turxVN83QTbP8aNmzI22+/DcBll10GQFpams995QWccfuff/5J7969AXj//feDOX06QjmG5cqVA6B9+/aZ7levXj0A7rjjjiyPOX78ePr06RNUO0LZx0suuQTw3GtXXnmlz/1kHE+dOgXAoEGDAJg0aRLnz58P9rRehKqPJ06coECBAgHtKw/vd955B4ABAwYA8NdffwVzSr+ES/7g2Wef5ZFHHgHg4osvBmDr1q2MHz8egJkzZwLue9BJQjWGVatWNZNc6U926Nu3LwBvvPGGuZ4DRd8ZHuy8N+bPnw9Avnz5vLZv3rwZgFKlSlG8ePF022TR3aVLF44ePRrMaX2i8geKoiiKoighJCotUjKjbNiwofW7APhrb8btqampgHsVPHXqVAAOHTpkp9lyXF1dEHz/xo4dS8+ePQH36jcQChcuDECvXr28tt19990AzJs3L5hmAKEbw2effZbHHnsMgPz582d1bGlLlsf96aefuPbaa4NpSkiv0ylTpgDwv//9D4Djx4+zadMmwGNpA5BcO/0AACAASURBVP+Wxb59+zJhwoRgT+tFqPq4Zs0aatasGdC+Gcdx2LBhgNvV9/333wdzWp+EyyL1/PPP8/jjj/vd/uOPPwKe/n300UdRbVWsXbs2S5YsAQjYuhgIjRs3ZvHixUF9JxreGZ07dwY8FvNLL70UgB49eph95P0o93UwON3HYsWKAW6XndxbYh0V1zqkt0glJSUB0K9fPwDuvPNOAHbv3s1tt90GkC3LlFqkFEVRFEVRQkhUWqRGjhwJuGeWiYmJAOzbtw9w+73Pnj0LeGavZcuWzXSl/+KLLwKe4GU7hGN1kSuXO/Y/Z86cJvB6586dAHTt2pVOnToBsH//fgB27doFwLJly/jwww8BuHDhgrTXWAQCHeNQrILz5ctHiRIlAM8YZoX8DqVLlwagVq1aJjZq2rRpgDs4PVhCNYY//vijz6BrP8eWtmS57/Tp07n33nuDaUpYLFLJyckAdOrUiY0bNwIeK6J1v2rVqqXbdubMGbp06QLAnDlzgj29IVR9LFy4sIkXyoxq1aoZy2jGcVy4cKHZdubMmWBOn45wWaSSk5O5/vrrAfjhhx/M5xLYKzF/efLkAdzWgZSUlOyeNqTX6aRJkwDMtZY3b95MrUnr1q0DoG3btgA+7+Vos0hdd911ADRq1Mh89uijjwKY5y14nqXy3MmM7t2788YbbwTVDqf7KM+KLVu2cNNNNwEe71KgiGVtypQptGrVCsDEW9khoHsxGidSQtGiRU2g2W+//QZAoUKFjGm5UKFCgHtCJUHpvjKE5LslS5a02XLnLxi5sFu1amUuGLkpgnXnWBFT/IEDB3jooYcAeOCBBwD3hPLkyZN+vxut9b3q169vzPV79uwBoGnTpmzfvj2o48TKREoSJO666y7zkA+UUD6869SpA3iup6wms8uWLQMwiwLrd8WdYIdocJmI23ns2LFe28RVJplfdoiWe1GeTZ9++ingdttKksHx48dtHzccY9ixY0cALrroIl599dUs95cJomR/WYnkREraNWTIEPNZ7ty5AfckMbtIZmLjxo1NklagRMO9mBH5nVJSUswiYffu3baPp649RVEURVGUEBLVOlK+pAusn4l1JTU11Ugi3HjjjYBHGwQ8advRhKRsikvODpKCbQ1wrlKlivlbji2rynz58vHkk0/aPl+oEV2X5s2bA+5A2IzIiilYa1Qo+eabbzhw4IDX5ydOnABg9OjRADzyyCMBBXRKskWw1qhQI799oKtWsb5J8Ln1s1jnk08+AXxbpMS1lx2LVLQg16C4a/v16+eIFSQcBCqVIu5LX1Ij4s345ZdfnGtYkIj1STww/ti7dy9AOs09abe4tsSlXrlyZbOPSHcEa42KNsQDI0Hn27dvz5YlKhjUIqUoiqIoimKTqLZIBcqll17KM888A6RXPxUCTbsPJyL05g+JA7OmbcoKS+KEvvvuOwBuuOEGs4+kt6akpBhLlPjAM4uPCgeyorJaCGVl1L9/fxPDJjEYvoQ7nRBYcxqJ+8mIBOjKOFglAnwhKckighhLSCzjU089ZZTsJdZPxvHAgQPGShet5MyZE0hv5ZWklg4dOpjPJK06I6dOnQpaRDUWkN8jNTXVPE9iHbH0DB48GPAIBluZNWsWANu2bQtfwzLwyiuvAG5JAHDH0m7YsAFILwMj6uvW2DUJPBfpAKslSr4rgsmxSrdu3QAYNWoU4LlWJa4vHMTcRKpMmTJUr14d8LjvWrVqZTRDMgbxzps3z/zA0cQff/wB+M5kmz9/Pl9++SXg1m3JinXr1lGwYEHAne0A7gBLoVmzZgBG8TdSSIBx06ZNfW73pz8EHtN0//79Q9Q6Z8mbN68pVfHFF1/43U8Uo5cuXWomUE6p8YcDqTgg2YWdO3f2O47169cPOHMzEuTKlYsRI0YAWS90MiYNyJhNmzaNtWvXhrCV4UUy2bp27Qq4E1aifTIsyDvh9OnTZpwkK6xTp04mzMHXBEqezy+//HI4mpopopYvbrlAM16Tk5NNVQF5FgkfffSRyWoMVrE9mhg5cqTJtJUMf3FRZqcKRrCoa09RFEVRFMUmMWORkgDHyZMn+zWr+6JEiRLGnOmE4rBTiEyDEwVdk5OTeeqppwC3ppYgxRvFDBxpApUI8EU0BZdnhrjznn766YB0y8SSE04zdHYRF+yUKVOM29KqIyWI6rlchwcPHgxTC+3x4IMPZmmJ8ocoSMdDce2iRYuawF0JmRALnBT/jVbKlStnLBQiu7Fr1y6jrycSHuXLl8/0OKIB9uuvv4aqqY4j+m6ScDVw4ECvOnVizXrggQccqwsZTsR6KM+U22+/3bg0RQOre/fuYW+XWqQURVEURVFsEjMWqaJFiwL+gzz9cfPNN5tV4u233w54RCtjFQnaHjp0KAD33XcfRYoUATxK6GvXrjUrM5FJiDQia3DLLbf43C6rXqnabpU/kNgoCbSfPHlyyNppB1EQFjG4QGO5RHk3OTnZBJWKYr0TNc1CgQjH3nHHHT63f/XVV4Anvubw4cPhaVg2EdFXO0SjxEoglClTxijQSxxNly5dKFWqVLr9RDU7WFHKcCHxMbNnzzYijIJYTYNBrK5vvvkm4LmWowXxPIicSuvWrY18TEYrlBWxVvXo0cNUifAl3RKtyDtc/na5XObZm5U8RCiJamVzK5JN07lzZ5NlIdkjCxcu9NpfLjRrFp/obMgFFwzhVnCVzIOrr74agLp165rijPJgEDM1eF5eUrDRjtk2WtSU5aUkE6mSJUsas/Xq1asBt1ZPsGb3UI6h3Niff/55VseWtvjdR5SV7Wh+heM6lay8WbNmpStHIVjHCJx3j4Syj5LIIgka7du3Z8eOHYBHo61w4cLmGs0YUP/FF1+YyXF2XCehuhflmSGT9vLly5tElcyQxec999zD1q1bgz2tF06PoTwT16xZk41WeSPvmIcffpgPPvggqO+G8jqVkBBfCzbRvtqzZ0+mRbhlUSqq71JKLRjC/V4Ug4oUP7/99ttNUoFMpqXU0auvvsrrr7+e7XOqsrmiKIqiKEoIiRmLlF2+//57o2cjadndunULeqYajpm3BNQPGTLEuL+sdcoyo2XLlkAYijNGYAyLFCliUlnFrbRo0SLuuuuuoI4TyjFs2LAhkLVFKjOJh4yMHz8+aE2icK4Qq1WrZoLlxfVRsmRJ00dJ7hDZi0mTJjnirox0fa/y5cub+me+Als/++wzwBOALhIXwRCqe1HaK4V9AVMEXnSF1qxZYxIDevfuDWAsG+fOnaNJkyYARqLFDk6PoVjwJ0yY4LMG5Pr16wGMNe3gwYNe6fHiJvNl5Vm+fLm5xwMllNepBP2LC/Lo0aNGV1AsTAcPHjRWVCkA//DDDwNQtWpVU11Dxn/gwIG89NJLQbUj0vcieLSxJNhcLObff/89NWrUyPbx1SKlKIqiKIoSQuLeIjVixAiThi7xKdu3b6dWrVpA4Grf4Zh5lytXDgi8UrUo2ObNm5dly5YBnnRlO1IP0WqRAk+tL1k17tmzJ+hYt1iJkRLS0tKMxWfGjBkBtSNSK0S5dq3BvhmtbhMmTDABrtmRIomGVbCs5kV8s0yZMl77yH1cu3btdGrTgRCqe1FW6BJzCJ74pwULFvj9nsirDB482ATl+xPWDYRQjWHevHlN4o0Vec5nJj4plQeWLl2arj4kRJ9FSpIBxItx5MiRoIQ169evb2JQpTLGsWPHTGWMQImGezEjEps5bdo0I3vxwgsvAJ5koGBQi5SiKIqiKEoIiXuLVOfOnU0tIas1oHHjxkDgKc/RYJGSsgXffvst4MmyuPrqq00FbykpI/WHgiGWLFLnzp0zlsbx48cHdIxQjqGsDCWV3N/vn9EiJVmlknmSkfvuuw/AWHKyItIrxGLFipk4NhkXq1jnsWPHAGjXrh3gtkzJdR0oke6jFbGKyn1XuXJlM8ZC69atg45djNZ78eOPPzbW18svvxwgaGsbRNcYZuSff/4xKfXCvn37TLxqoMKy0dxH8LxvpIbfuXPnjCCtNYYuM6K5j+3atTOWKBHyTE5OZunSpUEdJ5A+RrWOVPny5c0LRILrgk0JF9eIld9//53U1NTsNzBE7Nq1ywQRSkDnwYMHTcpnxvTjAgUKGFeDPNjz588fNfpR2aVy5crppB7APYYrVqyITIN8IOq669atS/d3Vkhtt0WLFnHFFVeEpnFh5NixY15BvO+88475txQAlvqDDRo0YOXKlWFrn9OI5pcktDzwwANeiSyDBg3KVhJINHH8+HFzrQeSMBFuSpQoYRY1hw4dAgLXY7PWJ81I2bJlTVBztCv0B8Lll19ugrOF3Llz06tXLyDwiVQ0M2PGDFPVQ0IJWrduHfREKhDUtacoiqIoimKTqLZI1alTxwSFi0sgUKQqvS/5gLlz57Jly5Zst89pZAVVr1494w7Ztm1blt+TlRLAlVdeCbhVXmPdIiXiiG3btjWWNgkCXbx4sanlFsuIhINYIOMJsUzJNSlCuuAZx4xusFilQoUKgCdw10qlSpXC3RzHERdzhw4djBszWJdsKBG5mDlz5piA6UCtKxnT5zO69QSREIglJGRAkiE6dOgAuBXOxUUr7NmzJ2hJmWhn//79gKfiR6is/mqRUhRFURRFsUlUW6RuvPFGs2Jt0KAB4E7fFb+nL6RelMRgWJESMSJKFm2IMFpqampAMVxS6+zBBx80n0nph6NHj4aghYHTpUsXIH2Mmi+hPClnYK1VJkHk//3vfwH3akriMaRuoggFRhuyuvUXbyGWUkmlltgvieuwcvr0afP7RJp8+fJ5rdQvueQSY5WQOle5cuUygngy3lKuyVdMTTiTXQJBAnDbt29vPpNgamvsV8+ePQFM2SZZ6fuq9yVikLGElNuQMRT5g5MnT6argRktiECqNX2/X79+AOzYscNnUpFYD2fPng34lrAQNm3aFDNxbmI5e+SRR0wffdXFlNixiRMnAvDaa6+ZmL94oH79+ua6lRhGKb/lNFE9kVqyZImZ9IiJ8qmnnjI3iDW7TaLyBw4cCKR/QMukIju6J9GA1DUTM7YE3p88edLUXgpWmTZUyMvT6lqV7BDrC1W0a5o1axaQ6reY36PJrZCYmGg0aKTIqQT/ZyQQHSkxQ/fo0cPngiAS1K5d20wepA9NmzY1yRDihi1cuHBA4ygFt/ft2xeyNttBgv/vuecer21WF1EwemCffvqpQ60LDeLOkeSNhx56yCjqZ3RndenSxWQNRzsyyV24cKFJaJD6clWrVjWLnsyK3UrSyNChQ20p1IcaeS/eeuutprqA1ImUxA4rsig4cOAAI0eOBODDDz8MR1NtI0r6devWNZ9ldv9JRYFLLrnEPIvk3SPq706jrj1FURRFURSbRL2OlKT8P/roo4B7BiqzaquLTwI6M5pnU1NTzep38uTJNlqNnDciehni8nrxxRdNH8X8OmrUKMCdjitVys+dO2f7XE5q14jeU+fOnbnqqqsAj9XQn6UioyVDXLGLFi0yJtnsWDCcHkNZDU6ZMsXoQQVwbGmL1zaxnIpV5OWXXw7omFZCdZ3Wr1/fuEeysjjJ9o0bNwIe6+FPP/0UleNoRcYxKyV5f+P42WefmfT4WbNmAfD1118bt32ghEtHqmXLlsycORPwSHgULFjQbBcrjFyTY8eOdUT2wOkxlDCAhQsXZqNVbs6fP2+0B+WdYccKF6rrNH/+/MY68/jjjwNw2223ee33559/GkvMW2+9BXhCI+TZml3C8V4UC9PkyZNNeEFmz1HrtgMHDgAe1+aOHTuCPr8qmyuKoiiKooSQqLdISayJrM5bt27t79iAZ4YqsSUDBgzINDg9UMJpkWrYsKGJfxKl7GLFipmVrgTeOx0YGKpVsARUi4SBy+WiefPmgMef/9Zbb7FmzRqzHTDyBk7JHDg9hhJj8cUXX5j4tQCOLW0BPBbEb7/91lzbR44cCehYvgjVdVqyZEmz6pfVui/LhHUcZfV7+PDhYE6VJaG8FyXYXAJwDxw4wCeffAJ46pt169bN1BOUcRTrU6dOncJmsXHiWfP8888bq4Zw8uRJ5s6dC3hiLjdv3pzdU6XD6TEUS0ViYqIJhveV3GJFnp/SV4mB++uvvxyJwXS6j5IAMH36dO68806v7eKVkISj8ePHh1x4OpzvxQceeMDUypPYPV/zl9WrVwPuJAKpiBFoTV1fBHQvRvtESsiZMyfgNvOJHo0ENANGM0kCyuXHDNak7o9QXjAS7ChZIXXq1CFPnjzp9pk/f76ZHNpx+QRCtJalcIpQjeHYsWONZo2VEydOADB69GjA7X4Q95jcd6Jn5lRAcjSXbHAK7aMbJ/q3atUqkygwfPhwwJ3ZJMVeQ4WOoYes+iiZvrIwyZ07t9kmk/bvv//eTK7CmbGt4+hGXXuKoiiKoig2iRmLVKQJ5cxbXAa+ajiJW6RBgwYBqZxnB7VIudE+RjfaRzfx3j/QPoJH7kY8Enny5DHhKmJFjJTGlY6jG7VIKYqiKIqi2CSqBTn/LUgqqwQiz5s3j2XLlgFutVnInqyBoiiKEptIvG/+/Pkj3BLFH+raCxA1YbqJ9/6B9jHa0T66iff+gfYx2tE+ulHXnqIoiqIoik3CapFSFEVRFEWJJ9QipSiKoiiKYhOdSCmKoiiKothEJ1KKoiiKoig20YmUoiiKoiiKTXQipSiKoiiKYhOdSCmKoiiKothEJ1KKoiiKoig20YmUoiiKoiiKTcJaay/eZeIh/vsY7/0D7WO0o310E+/9A+1jtKN9dKMWKUVRFEVRFJvoREpRFEVRFMUmOpFSFEVRFEWxiU6kFEVRFMUHdevWpW7durhcLqpXr0716tUj3SQlCtGJlKIoiqIoik3CmrWnKErgjB8/nkaNGgFw1VVXRbg1ivLvIXfu3AA0adIEAJcrZpPO4pJrrrkGgKVLl5rPbrvtNgC2bdsW9vbEzESqXLlyAPTs2ZO+ffsG9d2cOXMCMHfuXAD++usvOnbs6Gj7opl58+YB0KxZMwYMGADA6NGjI9kkJROKFCkCQLt27fj9998j3JrgqFSpEn/++ScAhw8fznL//2PvzOOtmt4//r7Nc2nSLNJERROlokhJJdGkVCgNRCQVGhQSUhHJGFF0mxAZQvMgDYgmSUppRKVJw/n9sX/P2vvce+695+x7hn3O93m/Xl73OufcfdZq7732Wp/neT5ryJAhHDhwAIA333wzom1T0qdIkSIA9OjRg3vuuQeA6tWrA3D06NGgjiHjdNGiRVmzZk34G+mSwoULA/DXX38F9fmrr74agIEDBwJw7NgxTp48GZnGeYhy5coB9jPjzz//pGPHjoD13PQKPXv2BKBEiRLmtZo1awKxmUhpaE9RFEVRFMUlcaNI9ejRA4AaNWqQI0cOAP7777+g/rZo0aIAtGzZEoD3338/Ai2MDAUKFACs1cC5c+dcHUNWUj6fj4svvjhsbQuW1157DYC7774bgCeffJJhw4ZFvR3xQu/evQHrul2yZEmMWxMc1113HQCjRo2iQYMGGX4+V65cADz00EMmfOklSpUqBVjte+CBBwBblRHFMFiKFi1qrn0JFc2bN48xY8aEq7lh4eWXXwagQ4cOjBgxAgheiRKFYPLkyQAsW7bMqP67d+8Od1NDokCBAiQnJwMwe/ZsAF555ZV0/0buQWHJkiUxUTqCZfLkyWacf/DBB4HQw5GVK1c2obIyZcoAlsozfPhwAAYNGhSu5rrmoosuAqBbt27mNTkvn376aUzaBKpIKYqiKIqiuMbzipTE7SVmX7BgQQoWLAhgcisyYvTo0X7/P3369DC2MPwUKVKEcePGAdC0aVPAUuIOHToU0nFy5swJQFKS5XB/9OhRZs6cGcaWBoesjOSnG1WsePHiAOzfvz98DfMYVapUAfDLAVy7dm2smhMSbdq0AeC8884L6vOiVmzevNmTfXz11VcBaNGihbluH3/8cVfHmjhxIu3bt/d77eOPP85U+8JFkSJFjDp8/fXXA1Z7n3rqqQz/VlTF+++/n1GjRgF2bs1tt93G6dOnI9HkoJHnRK9evUwi8u+//57h3+XOnZtixYr5vbZr167wNzAMyFjRvXt3cz4WLVoE2OciI+Se/eabb4wS62T9+vVhaGl4aN26NWDnvIEdcTly5Ahg5+mdPXs2aufN8xOpbNmsJspNcerUqZAlSxnE5B/1hx9+CGMLw0+/fv24/fbbAfjggw8AQk46zpMnD8uWLQPgsssuA+Dbb7/lyy+/DGNLgyMcoZuPPvoIgC1btgDQt29fTpw4kenjeonatWsDdih68+bNTJw4MZZNyhA5t7169QLg4YcfDurvJGTtNe6//37ArgACOH78OAAbNmwI6Vh33nknYIfzwE7YlTBarJk8eTLt2rUD4L333gPg0UcfDSqNIE+ePICVkC2Tpn79+gHEfBIF0LZtWyD0wprSpUtzzTXXAPa467X7UEJcMoHNlSuXSXX57rvvgjqGpMjItR5oEpWcnMzcuXMz3d5w4ZxACSIwVKpUCbBDfGfOnKFWrVoAEX9WaGhPURRFURTFJZ5XpFKyceNGs0IMFVlluU3ajjSych0xYoRZCd13331A6G3u2LGjUaKEWFgeTJw40SQuCqGG5y6//HKzArvyyisB6zp49tlnw9NID5AvXz6TfCyh2Pnz53Ps2LFYNitDbr31VsBSO8FONs6IW265JWJtckvTpk1NSEvC4kePHjWJ4l9//XVQx6lXrx5g+YCBFSoSRNWKdSl548aNAStUIsq1jDUZXXMSQvr8888BK1ogxQZ79+6NRHNDQkJV/fv3T/VeMBYbkjgP8PPPP/v99ApvvfUWYKuCYKv2wSb3N2nSBIBZs2al+ZkRI0Z4xvYhV65cDBkyxO+1s2fPmtQdCb1XqFDBvC/WDW+//XZE26aKlKIoiqIoikviTpHavHmza0VKVkteWDU5EfM3SUBNSkri9ddfB4I3kBMaNmwIWCsWySUT+wFZsUSTKlWqGIXFbX7IM888kyr5M9hEylhRvnx5Fi5cCFhJnGAlhv7zzz8BP9+lSxeToyBqpNfK41MyZMgQunfvDth5FhnlxsgKWnIep0yZEsEWhsagQYP81COAxYsXp7tiD0TevHkBf7VACmNEwYsVco29+OKLAPzzzz9GHUzr2kyJuEqLSnzixAm2bt0a7qa6RgofnIr8Z599BqSfPyRjTN++fc24O3To0Eg10zXNmzc3+ZTCH3/8YfKlgqFJkyYBbYAkj/jmm28G7JxUL/DYY48Zx3nhm2++MeOsKMdOgr2mM0vcTaSqVKliBii3Eyov0aJFC6ZOnQrYVW0LFy7k+eefD+k4UvE1Y8YMc6zffvsNsN15o4kMSs5NPsVZPthBt3Tp0oAdKgGMW3Iw1TexpEePHsYl+I477gBg5syZJhwiSFWq03tJKsYOHjwYhZa659JLLzU+VxIeygjxuJHKzfHjx0emcSEgBR3O60z8dAINzulx3nnnmRCZE0kEjmXVaeHChXnuuecA2ydowIABIS3WcuXKxUsvvQRA/vz5AevB7pVq2kKFCgUM6Un4J72JvhQ25c+fn++//x7AUz5uElJ9++23yZcvH2A5jwM88MAD/PTTT0Efq3379gErbKUqzksFWVKYIrtyOHn00UfNM0bCeE6CDcdnFg3tKYqiKIqiuCTuFKlJkya5VqJC9WGKBr169UpV0jl9+vSgPbKEsmXLAv57D8nKLBaJrZIUKIoL2EmQlStXNqtZoXz58ib5UUKcItE7Pyv/LqdOnTLeUl27dgUszyXxJArWkTlSpPQNSomoVRJyEEURYN26dZFrWBiQ8HGXLl0Cqi/pIWEkL4QtZUUuoXVnKE4KM0K9D9u3b+9ndyCItUAsueiii8yqXa676dOnc/bs2aCP0aJFC6PcyTHEt8gLtGnTJlWRzYIFC1i1alWGf3vVVVeZ35cvXx72tmWW+vXrA/5jvDi0i2N7Rogi7HRul/P/1VdfeSpEm5IsWVLrPs5rV1JIYoEqUoqiKIqiKC7xvCKVsky6cOHC3HTTTYDtZLpo0SKT+CirSomr1qpVy8SWveRoLkmMgcwqO3fubGL06bk+S5/Hjh1rDOSEL7/80qwYo4koY5KE7ETKiu+77z4T4w9E375903yvUaNGAMydO5c6deoAdi4V2GXdscptkP2oxF0X4McffwT8z6UkHTuVKClpjoX7fDCIAihJqi+//HIqu4OcOXNy4YUX+r22fft2wFIpJRE7mDL0SNOnTx/A7hfYZe6bNm0K6hhyHUvyvNOVXtizZ48n+jtx4kRjbTB27Fgg+P1KJadt7Nix7NixA7DVjUBkyZLFKMmHDx922+Sgke966KGHjDIheZQ9e/ZMV3WTz4uze1JSkqdyoyTfVM6Zk2BNYiX/S/KMnOqO5Ld5cc9LsI02ncizbcuWLVSuXDnaTUqFpydSHTt25Mknn/R7zemFJImDu3fvNhMnyeqXCydbtmzmopHE1ieffNJIoqEmdWeWGjVqADB48GDAdpd10qRJE1PpJRumFipUiGrVqgH2hSVJds6QhNC9e/eQJPtwIZOlQA60KSvv3CAPLgn7OUlOTo6J30ulSpXMg1LCXj6fz8jnUkzgDHPKFikyiM+bNy/kxOZoI1VBMnHNmzevCcPK/ZcnTx5q1qzp93cSqixfvry5f2O9kS3Y96CT9Cp7ZUGWI0cOs3AZMGAAYBe+OD1shPfee89MJmOBLG7q1q1rJvUrVqwI6RjOSWfnzp2B9ItGunfvbh5wKb1/IoGMC9WqVTNFOzt37gQy3t5FKtTEnd/n84W8e0YkkWtKXLqdyHlp3ry5mVwEurceffRRIPAYLEnqn332mZmYeWGDYsHp6yXIOO+V3S00tKcoiqIoiuISTytS+fPnp1ChQmm+70wuE2ldFJ5ATuB79uwBLG+JMzRvuwAAIABJREFUlStXhrOpQdOpUyfA9prZtGmTSXKUsNVTTz1F9erVgcAhEOl3oFWTfH7fvn1hbnlwuPXo2r59uwkBZM2aFbDVu0Ds3r3b9FFUjtmzZ0fVtb5Dhw6A9W8uqqCck82bN5tQ8qlTpwDr+pPVuaiJsqKKtPNuOJA92eT6a9SoEQsWLADSD0GLqly4cGGzMhZFJFZJvY0bNw6oBsvqP9CmvRL6qVWrVrr3YErE9T1WyFiTlJRkbB2C3QtPVJB7770XgG3btpljOJFxWq7vevXqmVL6WFGxYkXAUvjlepN70UndunX9/v/06dMxd553Ekh9F5o3b25+F3UqVOT6uOqqq8y/mRcUqfPPPx/AuOY7CTRehnJPhhtVpBRFURRFUVziaUVq9+7dAQ21ZDUle60tXrzYzFpFGRA1Y+TIkUbpkL14li1bZtyjo80TTzwB2CvyRYsWmdXPJ598AlgmYi1btgTsPI6CBQuaEleZeffo0QOwVvqitn3xxRfR6EaavPvuuwCULFkSsFTFd955J8O/++2338x5EtdepwuxGMSNHDkSgJUrV8ZMdRNatGgBBM5Rq1y5ssk3kByEdevWmQRdOYdiBrlhwwaToyGl2pLP4hWk7F0Uw71796abSCz5inINT58+nfXr1wP29RFL5Bw4lW1JlM8or0fyLsVVOWWxB8Dq1auB2Owo4ETyRcE2LZR7cvPmzQFzKcUaQowsJSn99ttvN7l+okK1bt3aqJWSKzZw4MCYW5CIovH1118bFU2crp05Ym3btvX7u7179xpl3WkoLEghQiB1KxKIdcb9998PBB5vMoNERP766y+GDRsW1mNnBrm+AuUdirnzF198YfK+AilRzms/kqgipSiKoiiK4pKkaMYTk5KSovZlolB98cUXplrv4Ycfdn08n88XlNtXNPoocfHk5GTAsoGQ1crcuXNdHzeYPkajfy+88AKAn9mj5NZkxsgx3OdQrAsC5SbceuutZkUsCs7/H1va4vf5w4cPm+o+ySlys0WMl65TUYBFkbrmmmuMSpMZwtXHpk2bAraZoeSKpEQsAyQv48svvzTl8WLzMWHChFR/J/vZSWVfKETiXhwzZoyxZ5Br8ocffuDMmTN+n/v7779N/tiVV14JWLYWYKkxYiYr+Yh///23yc2Ue1esadIi3NeptO/ll1/mrrvuCuZPnN8hbQrq87Llk9yvaRHuPorRa+PGjU1FpOyF16VLF7NPpJzbQBYzoj4uXrzY5L0JR48eDbmaNpLjjdjghLo3qxOJaoh1hBtrmaDuxUSdSIlEOWLECDOpWrx4sevjeeEBdcEFFwC2tCyDx6JFi0wo8OTJk66PH+uJlJTVS+Ky+Pvs3bvXJL1mZsPpaJ9DccwWr6XKlSubQVsmFxLWXbhwIZs3b870d3rhOhVJXvYBk/LladOmheX44e6jTHirV69uSvvF02v79u3m4RMolPzGG28A9sMV7H7Lgmfjxo3BNMOPSN2LV1xxBWBvXvzggw8aPzon4p4tD2UJ2f/+++/Mnz8fsPvpZt/LSF2nuXPnNgsR2fGgaNGiGX2HtCnNz3z77bfmYSz9z6j0Plb3oizwAnmhiYXJiBEjwvJdkeyjhM+7desG2JN6J2XKlDHPvvSQxVJGO04EIpg+amhPURRFURTFJZ5ONneDmOaJbA/+ScvxSqFChcyu69JHCZO0b98+U0qUVxBDSqfTNFghlcwoUbFCTDed7uXDhw8H4LnnnotJm6KBqG2imK5ZsyaWzckQUZr27dsXsLQ/PSTJ2okoUG6UqEiTMrT64YcfpvpMuXLljCosPyXM4naf02hx4sQJE76UkFDNmjWNVYkT2TVDEpJlDJ06dWoqx/5t27Z5yhIhVCQMm5nUj2gjbZaQeiDLgzx58phohdjNlClTxrwvKSFTpkyJYEtVkVIURVEURXFNwilSsqVKgwYNAPj444+jVqYaScaPH29iwRLTl1Lyv/76K2btChclSpQIuE8ZwKFDh6LcmszTvHlzE4+X3IvJkycntBIFlvom1gFiyCgJsYmE7KXoLCQQfv311yi3JrwMGDDA9EtUHa8rUYGQbXm2b99ucmSciD2DJGWLNYJbY0uvEGjPPNlyTPZwTRSOHz/OsmXLAMxPMb0G+/qNtBVHwk2kJHFU2L17d0z2nMsskmgnGzR36dLFPJDFDbp///6xaVwYkQHbuaFtSqKx6Wm4EJl51qxZxu9FklMHDRqUEJP69OjcubNJwhVftETktttuAzCVUsLff/9tQvDxhhR7dO3a1ez8IA9gxfuId6IUTDgJtK9kohFLX0EN7SmKoiiKorgk4RQpcd2V0k9J7o03RG4WadIZQpAkZnEbjmfEQVl8sJz89ttvgO3+HQ98/vnngJUEKbvO9+vXD7D9iBIRKU0eNGiQnxVAohJo1Q/Qq1cvTyaZB4NTtZDy+ES+ZhMN8f5yurFLoVUipH9kxPPPPw/EJlKjipSiKIqiKIpLEk6RErVG9uiL1Z56meXmm28GrH30BEk237lzZ0zaFC1kFdy4cWOAuCo7FvO/AwcOGJUm1nsCRgNxXf7nn39YunRpjFsTeS655BIgtYljPKpR1apVA2z7kaVLlwY0c0w08ufPH+smhJU6deoA9h6XgNm/c8eOHbFoUlSRXFrJIZaCs2iQcBMpId6rE2SjW/FP+uSTT/w22kxkBg0aBGBCY/GEFAn8ryGeUatWrQp5m4lE4NtvvwVg//79MW5J6PTu3RuwFyydO3fWkF4ccumllwL2hGLmzJlm4/T/BaQyr1GjRlH/7v/NUV9RFEVRFCUMJOxee+HGC3uYRZpY7LUnMvTgwYNNkmDt2rWB8Icw9RzaaB8zh1iqyPgp+7q9//77YTl+rPe9jDReOIeRRvtok+h9VEVKURRFURTFJapIBYnOvC0SvX+gffQ62keLRO8faB+9jvbRQhUpRVEURVEUl+hESlEURVEUxSVRDe0piqIoiqIkEqpIKYqiKIqiuEQnUoqiKIqiKC7RiZSiKIqiKIpLdCKlKIqiKIriEp1IKYqiKIqiuEQnUoqiKIqiKC7RiZSiKIqiKIpLdCKlKIqiKIriEp1IKYqiKIqiuCRbNL8s0TcuhMTvY6L3D7SPXkf7aJHo/QPto9fRPlqoIqUoiqIoiuISnUgpiqIoiqK4RCdSiqIoiqIoLtGJlKIoiqIoikt0IqUoiqIoiuKSqFbtKYqipEXJkiUB6NmzJ4888ggALVq0AGDx4sUxa5eiAFx22WUAtGrVCoAbbriB3r17A7Bx48aYtUuJPQk3kapSpQoAefPmBWDTpk0cP348lk2KOAULFqRevXoANGnSBIACBQrQt29fAHw+u/L0tddeA+DRRx8F4K+//opmUxXFj6JFi3LnnXcCmJ9Vq1bljz/+AOD333+PWdv+17jgggsAayJbrFixgJ+pWrUqjRo1AmDOnDkAHDx4kLlz5wLwxRdfRKGlkSVLlixcfPHFAHTq1AmAW2+9lYoVKwKQK1cuAN5//322b98em0YqnkJDe4qiKIqiKC5JcqoVEf+yCJpyPfbYYwAMGTIEgDx58gCW5PrBBx8AcO7cOQAOHTpkpNhly5YFdXwvGI+df/75ANxxxx0AVKhQAbBWS4UKFQrpWKJIPfPMM+Y1NQG00D6Gh8KFCwOQPXt2AK6//nrq1q0LQLVq1czP4sWL+/3dsmXLjJr6008/pTqul/oYKaJ5L7Zt2xaAyZMnA1CkSBGSkpKkHfJd5v8DvSe/t2/fHsAoVGkR63OYPXt2c11KSPn6668HoGXLlrRs2dLv8z6fj82bNwPQq1cvANatW8eJEyfS/I5Y9zEaaB8tVJFSFEVRFEVxSUIoUsWKFWPfvn1AcCuoLFmyGHUqSxZrLnnu3DmyZs2a5ndEc+ZdrVo1o6hJO48fP86nn34KQLly5Vwd9++//+bhhx8GYNq0aQD8999/5n2vKlIdOnQwv7dr1w6wV75pMW7cOAAeeugh81o8rp4kwbVWrVoAtGnTxiiR//zzD4DJWYHY9zF79uwmUVx+5siRA7DvtZSIeiHX95QpU8x1H4hY9zEaROtezJs3L6tXrwas/Kf//+5UY+YLL7wAwObNm03+1M0335zqeKNHjwa8oUg1bNgQgPvuu895PMDKpRVVNGVfAfM8kZyvOXPm8PHHH4f0/Xqd2iR6H+M62Vxu6Pnz55ubQEJ2Y8aMAaxk87vvvhuwb+6DBw8aOVvI6MaPBldccQUA06dP58ILLwTsm3v37t2UKVMmw2Ns27YNgF9//ZV58+YBGEn6wIEDAUMlsUSS5MuVK2cmSdLP+vXrp/u3M2fO9Pv/Xbt2UbZsWQAGDBgA+E+kYk3x4sW57rrrAPvaLVWqFACjRo0y/W3atCkA1113nZlIyaRp//79fPjhhwAsWbIkeo0PkhIlSjBy5Mg039+/fz9gJyqPHz+egwcPAold+FCiRAkAjh07BljXu4SW7r//fgDuuuuuqLdryJAhVK5cGbDHGp/PZ85J165dAfjyyy9T/e1TTz0VpVa6Qwpv0lp0yVgpk6Xvv/8esJ4Fp0+fBuDo0aORbqankCKtW2+9FbDGp5UrVwLer5zNls2azkiIVn4CJlSbI0cOc50fPnwYgIULF5r0H1lUhIqG9hRFURRFUVwS16G9J554ArASp8XioFu3bkD4FaZISpgiMa9YsQKwVwUZsXv3bgBeeeUV5s+fD8DevXsBW5oOhWiG9kQ52rlzZ6r3du3aBcCqVasAWLlypemrrI7kMxkd3/m5WMvQY8eONUqZIKuiU6dOmXDujh07AGsF+O677wLw888/A7aikRax6mORIkUAmDBhArfffrvfe6JmzJkzhylTpgD+IeVQifV5DITcs1WrVqVx48bmd4C6detSvnx5wDrPAIUKFTKJytOnTwegT58+5njRuhcnT55sFHtniOvSSy8FbDU73ETjHF555ZWANWaI6rd06VLz/q+//grAv//+6/Yr0sWL12kgpIjplltuoXPnzgBcddVV5n2xILnoootS/a2X+vjNN98AcM0116TXDgLNecTGQiwunGiyuaIoiqIoSgSJyxwpsTqQEn6fzxd0kqMXEZXCqURt2LABsFdQDRo0MHkLMvN+4403AMzr8YTkQznp2LEjAMnJyZk+fkaKVTQRdeziiy82itJvv/0G2Od3/fr1JlfDS23PCFGiJk2aBFiFAZJrIitDWfGnl0AeTzRs2NA4rl9++eUA1KlTB7Dy4E6ePAnYiuqaNWt4//33AVtR3b9/v1FZRZWMBVWrVk21Qp8zZ07ElKho8uqrr5rfJRexUqVKAHz++ecRU6K8Qo4cOUxunhQoVa9eHYBmzZqZIhWxKUlLrdmyZUs0mpshVapUMTmvMqbmyJGD5s2bA3ZOnLMPovqKEpxWHzOjkEMchvYefPBBxo4dK8cDrBv/6aef9vuchIwOHDiQ2a8EIidhdu3alTfffBPAVA2ePn3auDzLAJwjRw7zIDpz5kwoXxE00QonlC1bNlVIb+bMmX7VeZEg2jL0eeedB1gPUoALL7zQhG/EYT7cRLOP5513nvFtGzRoEAAnT540CfUSqg430T6PMsF/9tlnAXsQBytRFeDHH38ErEXO119/DZCpHRUidS/KYm3q1KmA5SEVTKWz+O0NGDCAtWvXhvq1qYhGqoQkDosTuZOTJ0+ahcvQoUMBO3weLqJ9nT7wwAOAXSCQL18+49AeqDIxJXv37uW7774DYPbs2YC1wNu6dSsQeLIRjT5KysPKlSvNuRXmz59P69atAWtuAJa/lyDhW9kpwQ0a2lMURVEURYkgcaNIOd13JZyQ3gpq/fr1gLXpaThCX+GeeefPnx+wQiKS4CfUqlWLH374IdQmZppoKVI7d+5MlQw+cOBAo1JJSCTcRHOFmDNnTqNgSFn7hAkTGD58uLQls18RkGj28dNPP+XGG2/0e+306dNmpS+eZeEOE0Wjj6IOP/rooyaVQJSmV1991dyfcv2GO2wZiXuxSpUqxrKgTZs2coyg3cvBUvgnTpwIZM7+IFLnMHfu3EZBKV26NGBZGIh9iCgUNWvWpGDBgoAd/pF78/nnnw/lK9Mk2oqUXJMpVZv//w7A7uu3335rQsuixL3wwgvpOrUHIhp9FNV79OjR5nqUHTnGjx8ftqhTWqgipSiKoiiKEkE8n2xeu3ZtwN4HqlixYqlWSQcPHjRqRpUqVVL9XaDE5lgj+SOXXHJJqvdGjRrF33//DcDZs2cBa5UkCXOyqoo3JAfKmWMiv8+YMcO8Jgm5zhLceKF79+6AtborUKAAYCdb79q1yyR2Hjp0KDYNDCOBSoWzZ89Oq1atAPselP31ZAUcD4iB38iRI03yvKg4kcpRjBRyHubPn2+MYGUM3bVrlzF2lTyoOXPmGBVfIgGSuF2sWDH69+8PYKw5AlmYxIqkpCSjREm+z3333ZfKaLF8+fLGsFHy+x5//HHAGnMnTJgQpRaHh/Lly5tcMCmEyJs3byq1RqxVMpMzFG2kQMCJmMjmzJkz2s0JiOdDe5JAJwnmhw4dMq7Ir7/+OuA/kZJtKZ588knAGjBE6sxMiCHcEqaEBKSaJBjkgSzbu7z00kuA7eaeWSId2hMX8wEDBjBr1izAlt/LlCljHIhlchXuCVU0ZGhJOq5atWoqp+7ChQubkJE8qD/77DO3XxWQaIYTOnbsaNzY5aF69uxZLrjgAsB2npfJpRROZJZI9lFc9WW7murVq5uxR8adHTt2GL+2SBHOe1EWoT179vQr0AHo27dvUKkPznFYjiGV0sOGDQumGX54yX9INi3+6quvAMsvSa5hceJ3QyT7KNWisnVPnTp1zETq6quvBmD58uWhHjZkonEe5bkwY8aMVCkRx48f55VXXgHsCXG40dCeoiiKoihKBPG8IiVIyA7SV5bkcyJXFylSxDgHS1moG8I98xY3YVktgh3uq1atmgkLpYeE/zp16mRWU5nBK5sWy7+DqB0dO3YMi7dUNFZPsl/ivn37jCOwcPHFFxslrkaNGoC9J+SwYcNMGDczeGmlL0ms0i/pc2aJhiIl+xlWrVqV3Llz+33m2LFjRrESD7g9e/aE+lXpEs57UcJzjz76qBkDQ1XnJSS4evVqo9bIsyO9zd7TwkvXqSAqz7p168z5lY2Z3dybkeyj7BYg+3I6iwYkDNu3b99QDxsy4eqjeECJnUgg2rdvb2yOArmsy9+K/Uq4UEVKURRFURQlgsSNIhUqzrwAsUKQpFc3hHt1Iau4unXrGkMxyTkoVqyY2a9LrADy5cvHPffcA9gzblk9Hz58mI8//hiwS+3dlGOrImURyT7KSuqjjz4C7FLl1q1b88knn2T6+F7ooyB77r388ssAptw8s0Szj9mzZzcqo7hDV65c2dyDYro5ePBgAJOYnlm8ci8KTkVK/h0kN1PcskPBS9epIOaVmzZtMuNz0aJFAVLlOwZDJPso+aaye0BSUpLZT1BypaSwJZKEq49i9in5TiNGjDCWFU6kb/KsvPfee8mWzaqZkx0FJBE9XAn1qkgpiqIoiqJEEM/bH7hFVlBSYeI1JOa+atWqVAaUx44d4+233071N6I2FSpUCLD2iwJL1ZLcB8m9isd9zaRaT5QoQar3vIYogv/880/Q+3bJ6smZgwPe3V9P9uoaOXKkqcyTaqCM1Gyx69i2bRtg9T3erANOnz5t+uusgpKqYMm/lC01KlSoEOUWRgdRF8uVK2fGVLmGvUC+fPkAS6kYN24cYJf6B4tcp999951RfLyA2Bn89NNPgDVWyDNDfrZo0cKoNfFkMyJIBX6/fv0Aay9WyfuVarx9+/YZ1UlyE1etWsWUKVMATC5jz549AdvOIhok3ERKEislSdDn85lS30RBJM/Tp0/HuCVp4wzPycAmG06mhdNLCuwJlFcnGWI/UaVKFVOGLLLyBx98YFyC5QZv0qSJ2WtPBv4XX3wRICZO9sEgCeKdO3emV69egL2n1euvv242YQ7EL7/8AlhO/WCFL724qbiMGaG0TSaEvXv3BuyQXr9+/cx1kUiIu7TP5zOJ6in3N40ld9xxB2AtbkKdQAnFixcHMMn0XkH25ZRNhp2I1cG7775rQucpd8qIB8QGRsbHEiVKmMn7DTfcAFhhdNnk3WnZkVIsicVzUUN7iqIoiqIoLkkYRSqlA7rT9dxLK6dwIG6uXnF1DYQzPCcyrCT/jR8/3rwnbudjx45Ntf+e153NpYihWbNmJqFa6NWrV8Bd56VEV/Zvi9S+guFCjENFQQP7/O3bty9dk83bbrsNsEMS4TKODTeSKH7gwAGza8Cff/6Z7t/IXpli1ikh0Hnz5kWqmTFBwrlOR3QJZ0qifSwRM00J47gxB82ePTtg799WsmRJtm/fDtgJzLFE2ichru+//55bbrkFgGuvvRawwuZipSLWCPGE2E106tQJsM6nWBlJwn9SUpIpdAiEqONvvvlmJJsaEFWkFEVRFEVRXOIZReqxxx4LeTdxydlo27atycMoUqQIgNljSBL1vIYkHV988cVme5ssWax5bVqJy6JwiCIgKhzY5ofRtLMIFcmVat++vUnUdu67F2977I0cORKw1AtZGTr3Tly0aBFgKzIrVqwwqky8FANIErWUVju5/PLLUylSSUlJJuFaFClJBt2yZUskm+qa9957D7ASx8UIV87PwYMHzYpY2l+5cmVz/0q+oqhaKU1Y4xVRTJ25pmBtLRPqOB1JRDkUdezFF19MlX8oRQ9p8dZbbwHQpUsXwMp/k5wwL6husu2ZU2k5cuQIYOfmjR8/PmxbMMUCub5kW6kNGzaYrWEkAf28884z950TyYl69tlnASK+fVMgPOMjtXHjRjPpkcGoSpUq5qYQma9Ro0Z+v4N1EiQcJI7m4jORmf31nITbE0SS6p577jk2bNgA2N5Sy5cvTxUiyJcvn5F2U0409u7da/yIxO3cDeH0rnn++ecBO6yXEStXrvSrxIgEXvSuCTeR6uPTTz9tHi7CyZMnUxVy5MyZk1tvvRWwFwTiGB2uzbYjeR7F50uKBipXrmz8eGrWrAlYCx6p8BoxYgRAQM+bzBCOe1ESkWVz3g8//NAs2lJuZgvWfm1g3bvOsRUst2+wFqbB7M2XEeE+h82bNwcgOTnZhF1l4fbBBx+Yfl944YWAdU3KdSru4FIccscdd5gHemYIVx9z5MgB+G/eK+cgFpMGJ9EcU6tVq2Y2DhfBZMGCBebZLwvWcKM+UoqiKIqiKBHEM4rU2bNnzSpJZphVqlQhT548gL0ycu4ptGzZMsAqW542bRpAWFZLgQj3zPvSSy8F4Mcff8xEqyzeeecd4zGVGSLlpizhO1GpALPnnBAO5/KMUEXKJtQ+1q5dmxtvvBGAgQMHAqS5H6SEQ+Qad1uOnhZ6Hi0y6l/Dhg0BWLx4sRzTjK1SUn/o0CGj8Etoq0iRIqZYR0LRsm+bjLmZJVLnsHnz5iZBXkKyzmeLWBxkyZLFJCeL+i/eYOEqitDr1CbR+6iKlKIoiqIoiks8o0j16tXLGOPJSsLJpk2bAGtFJHkZkVKfAhGpvfb69OljTA9lxV+qVKl0/3bfvn0AZn+9CRMmhCUXzGv7e4UbXT3ZZKaPkqvRrFkzk2QthRCzZ882Sb6RsjvQ82gRbP/EumD+/PkmD0qS6Z0Kv6hQPp/PGMxKrmm4x9pInkN5fsg+dK1atTJ7I4oi9/333zNx4kQgfHm0KdHr1CbR++iZiZTXicYFIxOqL7/80gx+Tg4dOgRYDzAI3wapgk6kLLSP3kb7aBFq/4oWLcoTTzwB2G7uxYoVS5UqMXr06Ih7Eek5tNE+ehsN7SmKoiiKokQQVaSCRGfeFoneP9A+eh3to0Wi9w+0j15H+2ihipSiKIqiKIpLdCKlKIqiKIriEp1IKYqiKIqiuEQnUoqiKIqiKC6JarK5oiiKoihKIqGKlKIoiqIoikt0IqUoiqIoiuISnUgpiqIoiqK4RCdSiqIoiqIoLtGJlKIoiqIoikt0IqUoiqIoiuISnUgpiqIoiqK4RCdSiqIoiqIoLskWzS9L9B2gIfH7mOj9A+2j19E+WiR6/0D76HW0jxaqSCmKoiiKorgkLidS9erVo169euzcuZOdO3fi8/nMa4oSD+TNm5e8efMyZswYxowZQ9euXcmdOze5c+eOddPCzoMPPsiDDz7Izp07ze/xRLVq1ahWrRrz5s1j3rx5nDt3jpYtW9KyZctYN00JgJwvn8+Hz+fj6aefjnWTPMPw4cMZPnw4Pp+Pdu3a0a5du1g3KWiyZcvG448/zuOPP87Zs2c5e/YsY8eOjXWzgDidSCmKoiiKoniBqOZIhYMOHTowY8YMv9c6duzIqlWrYtSi2PPuu+9So0YNAC677LIYtyaylClTBoCxY8fSoUMHAI4dOwZA/vz5Y9auYMmWzbrlpk+fDkDr1q3Ne6tXrwZgy5Yt0W9YBHj++ecBGDBgAADjxo1j1qxZsWxSyOTMmZOHHnoIgBtvvBEAn89n7rdPP/00Zm0LN0lJVipI9+7dad++PQB//fWX32dGjhzJ9u3bATh37lx0Gxgkbdq0Aez23XTTTTzyyCOxbFLMadu2LQAPP/wwAAcOHIi7Z2bPnj0ZOnQoACtWrACsZ58XUEVKURRFURTFJUk+X/SS6TOTuV+2bFkAli9fbl4TRSIaM2svVifUqlULgKVLl5p/l2bNmrk+npcrhUqXLg3Al19+CUCVKlXMe8899xwAQ4YMSfcYsT6HefPmZdq0aYC1SnayaNEibr75ZgCOHDni+jti3Uewlaj69esD8McffwD2/ZpZotnHiRMncs899/i9dvjwYaOMHj9+PLNfEZBY3ItyT23cuDHNzxw9epQKFSoAcPDgQdffFalzmDVrVpYsWQJgcmaPHj1KoUKFQm1ipvHCvSipo5bJAAAgAElEQVRK1JQpUwDIly8fAFdccQXr1q3L9PGj2cePP/6YK6+8EsAopnKuI0lQfZSEvGj8B/jc/pecnOxLTk72+Xw+X4cOHXwdOnRwfSw3/0Wjj8H+V6NGDV+NGjV8+/bt8+3bt8+3fft2X4ECBXwFChSIeB+j+W8u//Xo0cO3a9cu365du3xnz571nT171rd3717fwIEDfQMHDvRly5bNly1bNs+ew5w5c/py5szpmzt3ru/cuXN+/x05csR35MgRX/78+RPiOi1btqxv586dvp07d0bsPo1GH6tUqeKrUqWKb//+/eaak/8aN24c8Ws+mufwzjvv9N15552+06dP+06fPh3w+pw2bZpv2rRpvjp16nj6HObIkSPV+Ro9enTEz1esrtP0/sufP79v3bp1vnXr1vnOnDnjO3PmjG/16tW+1atXx1Ufa9eu7atdu7bv7NmzvqVLl/qWLl3qufOooT1FURRFURSXeD7ZXEJ6IuWtXLmS5OTkWDYpZpQrVw6Ar776CoCiRYsC8Nprr2UqHORVevbsCcCkSZPImjUrAB999BFgJb3+8MMPMWtbMIiVgSSWSxIsYBJ2b731VsAKPyQC9evXN6H2lStXxrg1oSOhj2HDhgFQpEgR896+ffsA+PHHH6PfsAhRqlQpnnnmGQBzjzmRe6xLly5RbZeSeRYuXGiKjyQE7RyD4oW5c+cCVvHAmDFjYtyawKgipSiKoiiK4hLPK1IPPPCA3/9PmDAhRi2JLZUrV2b8+PGArURt2LABIGEM58477zzA7k+vXr0A8Pl8RgXo3LkzACdPnoxBC0NDSnOdq0CxarjrrrsAPK+qBYskks+YMYOOHTsCsGvXrlg2yRU33HADAJ06dTKv7d27F7CLGlJaAsQzb7/9thlPnIjC/dRTT0W7SZkie/bsqV4LR1J1PFGqVCkAatasKflJxqbjzz//jFm7QkXUYDmne/bs8azdiOcnUhLa+1+ndevWZpCXh7FU6EWqciiaFCpUyIRsr732Wr/3pk2bRp8+fQDvT6By5swJWNVerVq18nvvxx9/NJ5E0ag2iQZyf4rD8Lhx4+I69C4pBE5ef/11IDEXccWLFw/4uvj1fP7559FsTqaRxZeTzISYpVq4WLFixt/txIkTro8XDWQhA3Do0CHASo+IN+Q5UKxYMcCuBvYiGtpTFEVRFEVxiecVKVlNBFop/i8gqsaYMWOMEtWtWzfATn6NZ/LmzQtYyeMplagPPvgAgLvvvptTp05FvW2hIP147733AP9wnqhPAwcOZM2aNdFvXASRvbpEmZo5c2Ysm5MpbrrpJlq0aOH32pIlSxImdO4kSxZrDS1u5k6OHTvGsmXLot2ksFCpUiXz++7du4HQFaRixYpx7733AnYIvnTp0nz//fcAxsvozJkzmW5vOJFnhaiJAIMHDwYSQwGXEHuw3H333aZoREKcgPGFC2eYUBUpRVEURVEUl3hekZJVxf8aJUuWBOwE1yxZsvDTTz8BdjloIjBq1CgA+vXrZ16TvRQlsTweuOKKK4DAieXSNzl/iYS4l4tyHG/7d4GtJj7yyCPmd2HMmDGu1VC5Fh577DHjhC5FE7fccktMcxvF8bp69eqp3vvwww+N+hJvNG/e3PwueUHBFgfI344aNYo6deqkev/yyy8HMGrVCy+8kKm2hhtpv7i4nzt3jgMHDsSySZniwQcfBAKrpoEoX748AC+++CIALVu2NEUTkt+WJ08ePv74Y8DOwVq8eHGm2+r5idTOnTtj3YSYMHz4cMCq1gNrQnnHHXfEsEXhRTyiZFACzPYpvXv3jkmb3NKkSRNeeeUVv9eOHDnCoEGDgOAnULJFx+bNm81rMijmypULCF3ejjQyQYjnRGwp4pDJMNgVh6FuIN2qVSuaNm0K2CEEpz/T9ddfD1gPvVguiJz3XUoWLFiQ7t/KmLR161bAP2wSK6TiV4o9AL799tug/la83GT8CVT55+SJJ54ArAewVyaclSpVMknmcj5GjRrl2Sq3YJB+BHt9lShRArA3F1+7di233347YF+rZcqUMVWcch7btm1rkvLdoqE9RVEURVEUl3hekUoZKrjyyivjurw6GJ555hmjypw9exaw/FxCXR17lZIlS/LYY48B9upv69atps9eLy9OSceOHU2Sq9gztG3bloULF/p9LmvWrEZ1uvrqq83nwPJMkWPI6glsRSpHjhwA7N+/35R4r127NiL9CZbnn3/eKFLxeE+KeiGbXSclJRmfHUlS3bFjR1DHEs+bp556iho1agD2vXvs2DHy5MkD2Enes2fPJn/+/OZ9L3D69GkApk6dal4rUKAAYIUi5afYrohytWDBAt5//32AmIWSGjRoANiqRDCIf5Y4uwdSotavX28+KwUVEv51JqDHmnvuuYfChQv7vSaKy/8KTu83sK5V2TBd+OOPP4x3n4T2mjRpwqxZszL13apIKYqiKIqiuMTzipQgZdUDBgww+RiBnJPr1asH2PvSSakq2DFzr66eZaXTunVr89obb7wBwOTJk9P8u1y5cpkydFllHT582Dhre6VMV1bgCxcu5IILLgDg4MGDgGVvEW9KlKyCxdUb4NdffwXwU6OqVasGWE7nsvdVetSsWTPN90qXLm2UgLp16/p9Z7QpW7asWaXLfbd7926TgC5mhlIwsnLlSk+5nYvKV6tWLcDKxZAcNLl30kL25BMl6sMPPwSscy15nQMGDAAs9UnyMoI5/15AcrnEBFGuYSctW7Y0P2X8kZ/79++PRjMDIjmGS5cuTfMzWbJkMTtFXHjhhYCdi/Pjjz+ae1oUjQYNGvDZZ58BgfckjDVSnJQWonbLOXvwwQdNf8VmZu/evebfJNbUrl073XHQiTi59+jRI6jPi7r6zz//ANZ4m1lFKm4mUs4BWAZq2T6mffv2aTqgr1y50twM4kU1duxYBg4cCHhrUnXzzTcDVtJxMNu/iIy9YMECLr300lTvywXmlW0eJLxRsWJFcxO/+uqrQHxWtEnyf6FChUwYp2vXroAVJpCqEwkTpawIAzuccubMGROqC5QkKxOVBg0amHBfkyZNgNhNpJzIwiU5Odnci+PGjQPs+27GjBnmtfQWQ16nUKFCzJ8/H7DPi1zPmzdv5pFHHgEw1UHxgoS2Nm7caMaOggULAnb/kpKSAib/NmzYEICXXnoJ8F9cRBsJ2cpkVx6YTjp27JiqKlgqKmVi7eTrr782i/mUIaRYki2b9QjPkyePqW6TkF62bNlMQYMkYAvnzp0zi1h5FgKemUiBXa2XUdWeTGzl+SLnKWVYL63jN27cODPNBDS0pyiKoiiK4pq4UaRkj7L69esbnyEnUvoZjMKUnJxsjiEyfCz9b8T/QvYrA1umDGT/cNNNNwG20hRIjQJbJfAKo0ePNr9v374dgDfffBOAwoULG8lVuOaaa8x7srKSZG5xEI8l1113nfldnIM3btwIWAnMI0eO9Pv8sWPHWL58OQBvvfUWYKtJhw4dSjexuUKFCgD88ssv5jVRPiT8G0uc92RK1UnuXYAVK1YAthP6Qw89FDNV6qqrrkr1WjCWBHPmzDEpA+fOnQMw57V79+6pzmOLFi3MPS7Mnz8/pm793333HRB4NS4FEZC69DyjUnQJdcYSCdXddtttAKmsScA/fULuQYkIBKJUqVKmQMRLiI1MixYtzL0lyfNz5841Tv2iPkkhy5gxY8wzT1zsK1asGL2GZ8DatWuNdUN65wUw95FYGMg1mDNnzoD3mNzjci1/8sknmW6vKlKKoiiKoiguiRtFSpgwYUIqN+VAK8v06NChg5m9i4LVoEGDmK2Mxbjv/PPPB6wd1yU51YkYx0l5cu7cuc1727ZtA+Diiy8GrNybTZs2Ra7RISDtdOZNSB/EHbhp06bGdFKQGLbP5zOmiUKtWrVMMm+sSJmkCnZOiVONEmfl7t27h2yQJyXNjRo1SvVeSnsFLzBu3Dg/BSololKJ+rp8+fKYKacyBjhxmqGmRMr+ZfwBmDJlCmDt65USuRaefvpp87uwePHimBaByLXz8MMPh/W4//33X1iPFyyiVJ89e9bkzMjYsmDBAjM+Ck41+auvvgICq/9y3vr27WvyxrxEIBVJcjJvvPFGMzaJtYwURYA31MP0EJVbFKm0FDMpbJB8W1HqixUrlipPqmzZssaeRHYWCMc4qoqUoiiKoiiKS+JOkUpOTjbVerIy7NChQ8jVd5LZL/kc9evXj5kiJdVdwosvvmhyL4Qrr7zSmN5JpYYTUaLEIK5nz54BVa1Y0KVLF8CuqgDo06cPYFceHj9+3BgjpnceRAFo2rSpqcr5999/w9/oEClWrBjg30dBcjUy2npDEGXy3nvvNStJOb6T1157zVVbw8WsWbNMRZ6Q0bYcKe/TGTNmGKXSSxW0TqS0XFTRHDlymLwa2SvSieSlSMVtoP3svJDjFwxOVTi9z0huYKy2dxJVadWqVcaWRJSpTz75hEWLFgEwceJE8zdHjx4F7L3ZhGrVqnHfffcBdp6mUw2RcTXWhrjgX2Eoz0OnYvr6668DmCpTJ88++yzgrdwoJ5LPJeN7p06djKLtNC0WRO0XRapWrVpGkRI18dNPPzUqv0QNwpEfHXcTKbBDeTLwOhNdgx2MpcxTJlJeQAatBg0amGRzkaRbtmwZcAIFlg+IJJ7LAC9Stxc5dOiQ8cV6/PHHASv5XDZoDoQkWwslSpQwk7CUsn20kIE4X758xmdHXHOdyE1coUIF4+cSKHFVJiUyaSpQoIBf2bl85+DBgwHbdTlWJCcnG98gp7WIVydEwXDXXXcB+PnKSNiuf//+5jUZc8R3SiaDN954oykGcYbzZBIt3kaZ3dsrs8i1e/LkyVQhdSfpTaCcifbdunUDYr836pNPPmn8noSKFSuayYIUJeXNm9ckIsuiWihTpoxxdA9Eq1atANi3b1/Y2u0WmdQFGk+2bNlC3759AXuBJ4u0YcOGGfsWCXF5ydYB7LF0z549AFStWtXshyg+Z//++68JkcuE6JJLLgEsMUHO49ChQwHrWpDjBZpcukVDe4qiKIqiKC6JS0VKcIYEZIUoK2RnwqvXTf9kZSerP9mHLi3EiVYk6bVr1xpjx3igYMGCfP3114CdIBgo8VaUnNmzZ5tVhigzDz30UMyUKEGusWeffdaoD4FWshKODBWnGiD2Cn379jVhFC8giq4YUzrdztO778T1PJbIHneDBg0CrPMoJqfO0EEgs19Z6YupY0p7A7AVi4kTJ5p/p1haHjgRu4atW7ea5NtgkfMqthte2tPt66+/NjYrUoziVNyc96eoNDK2pMfJkydN+DBW+wkGQvaGDKQcnjp1yjwXZU9Mse1wGqtK+C/UQphoIeHzadOmGbdzKTT75Zdf/Gx1ACpXrgxY96icY+nrwYMHqV27NhBe931VpBRFURRFUVySlJHBWli/LCkpYl8mCdspc5527dplEs4kFr579+5UuR0dO3ZMN7fD5/Ol71P//7jpo6zmpcyzdu3axhRPVu4HDhwwOURO485wEkwf3fRPbBtefvllwMoBmjdvHmCbdB4/ftzkQUkOgpjNnTt3jr///huwE37XrFkTajMidg67d+9uVr+yGpJ93DJCrBFkPzqAb775BoCff/7Z5ASIYpeR8hjJ6zQ95BoeMGCAuafkXpQEdGf+jLz3xx9/GGUvWOU43H2UpOQHHnjAJOAGS8pk7AEDBph+Ll68GLDPcShE6l5MyYwZM1IVDGSEqHGiJrshGtepGFO2adPGdUL1Rx99BFhqZahJydHoo1gYrF271qhOjuOmmeN24sQJY7sjlgiSNxcK0RxvBg8ebK49Z1/TK4iQbX+k2GDSpEkhRzKCuhcTZSIliAzv3DhVBgpnNYMgA3p63jcQ/QeUTDrkwpk6dapJDowUkR68RVJdvXp1UJ+XG+Tbb781+0FJSMIN0TiHsiltxYoVje9QIMTZXJKV03M1D4VYTaScOCdV4D/opfSAGzBggGceUFmyZDHeYMOHD5fvonnz5gAUL1481d9IhZ5cl//9919YwuzRmkjlz5/fhH9SerUF4vHHHzeLn8z4YEXzOi1cuLBZmEqyOdiJ1/J8kAn/d999Z/5N5NpMWUUdDNHsY7du3UzBkVSZOidSEqqW8N24ceNMWDAzRHu8kaKe7t27A9YzRSorU/b1iSeeMK7lbiaJQjB91NCeoiiKoiiKSxJOkYoUsVKkrrjiCsBaLUa6ZDrSq2AJodStW5c5c+YA9qrQydtvvw3YeyB98cUXpkQ3M3hBrYk02kebRO9juPondjJjxowBoGHDhuY9CTmL8rZp0ybOnj2b6e/Uc2ijffQ2qkgpiqIoiqJEEFWkgkRn3haJ3j/QPnod7aNFovcPtI9eR/tooYqUoiiKoiiKS3QipSiKoiiK4hKdSCmKoiiKorhEJ1KKoiiKoigu0YmUoiiKoiiKS3QipSiKoiiK4hKdSCmKoiiKorgkqj5SiqIoiqIoiYQqUoqiKIqiKC7RiZSiKIqiKIpLdCKlKIqiKIriEp1IKYqiKIqiuEQnUoqiKIqiKC7RiZSiKIqiKIpLdCKlKIqiKIriEp1IKYqiKIqiuCRbNL8sKSkpbt0/fT5fUjCfS/Q+Jnr/QPvodbSPFoneP9A+eh3to4UqUoqiKIqiKC7RiZSiKIqiKIpLdCKlKIqiKIriEp1IKYqiKIqiuEQnUoqiKIqiKC7RiZSiKIqiKIpLdCKlKIqiKIriEp1IKYqiKIqiuCSqhpyKkqgULVoUgPLlywOwYcMGOnToAEClSpVSfb5Ro0YALF26FIA+ffrw559/AvDRRx8BsHbtWj755BMAzpw5E7nGK4rieQoVKgRA165dzWtVq1YFoHfv3ua1LFksfeSnn37iueeeA2Dq1KnRaqYr2rZtC8CVV14JQMmSJenWrRsAgwYNAuDNN980nz9+/DgAJ0+eTPe4SUmWl+Z5552X5mdOnTrFsWPHXLbcQhUpRVEURVEUlyT5fNFzbk90m3jIuI9FihQBoH79+qneO3DgAAC33nqr83jy/el9J3Xq1AHslcnWrVuDaa4f4diWIm/evABcffXV5rX58+cDcO7cOQD27NlDr169/P7us88+C7G1oROp7QwKFChglKXq1asH9Tdnz54F4MSJE+a1HDlyAJA9e3ZpBy+//DIA999/P5D+dfD/7+uWDf9PZvpYu3ZtAJYsWULu3LkBmDNnDgB//PFHqs/ffvvtAGzatInhw4cDsHDhQrdfH7EtYuT+PP/88wFLCRUV1TnujBs3Ttrh9/evvfYa27ZtC/VrU6HXqU2wfXznnXcA6NKlS0bHk+/n9OnTAOzcudPvvZ9//tmMwfLccUO4+vjXX38BULBgwaC+d9WqVYDVj/TIls0KunXv3j3Nz2zZsoXrr78egN27d6d6P6h7USdSwZGZC0YG4uuvv5633noLsGXapKSkDCdJ///96X5G3h82bBgATz/9dDDN9SOzg3e7du1o1aoV4H+zi9QsE6lATJo0yfThySefBODgwYPBNDtoIjV4582blw8//BDATGizZs3K77//DvhL0sJvv/0G2GE8sGXtHj16ANCzZ08WL14MQLNmzQDMwJgWXnhAyUSwbNmyfq/Xr1/fvJacnAxA3bp1qVevnt/nXnjhBXbs2JHm8aPRx9tuuw2A9957L61jS1tSvfftt98CcNVVV7n9+ohMpBo2bMj48eMBqFWrlnxPWscO+P5vv/3Giy++CMCMGTMA2L9/fyjNkONG7TrNkSMHnTp1AuCiiy4CrEljtWrVAFizZg1gT5B9Ph9jx44FYMWKFa6/N9x9/PTTTwFo3rx5RseT70/3vbVr1wL2ONuyZctgmuFHuPoo4+ZLL70U1PeWK1cOsBcEaSEToz179qR675JLLgGs8XvAgAGANfakRPfaUxRFURRFiSCabB4FHnnkEQAeffTRiH+XrDTz58/P0aNHI/59Tho2bJih7JwW/fr1M4qVhFVuueUWIHPSczQ4duyYkYZFaSxatGjIIZBASsb69euBjJWocDJy5Eguu+wywJbOT548ac5LRkj4qEmTJml+RlTHQLRq1YqKFSsG29yYIGqMFBQ4+emnn6LdnKC48MILqVmzZqaPIapW586dgcBpCl7gggsuAGDu3Llcfvnlqd4XxUaua1FLixcvzo033ghA06ZNAVi2bFnE25sRs2bNAgIrUq+88gpbtmwB7LQKGT/TIuX93K5dO/Md0UZUwZTqdFpcfPHFQGrVOyWi/IvCXaBAAaNOPvvss4Cl9E2aNCnkNjtRRUpRFEVRFMUlnlGksmXLRpUqVQB79liiRAnKlCkT1N9LXoasJJzs3bsXsHNVwp17kxESp/3666+57rrrQvrbIUOGAFYJqMTIU+Ishy1RogQQm3L5pUuXUqBAAQBq1KgBWMm6EpeXlZK8lxayKhElT+LX8cA///zj9zMUWrRoAcCoUaPC2qZQGTp0qFmtS85bKAST15ceF154oau/Cyd33HGH+X3jxo2AlZgNlkooZdeiOjrV5m+++SZKrXSPtH/mzJnMnj0bsFfvdevWpXHjxoB9DmXcKlmypDmGjNcXXXQR27dvj0q7M6JSpUqMHj0asHNgpJ1O/v33X06dOgXYzwdJuvf5fEaJXbBgAWBdD6JCxoopU6b4/UwLZ55RsWLFAPv6lHuza9euqRK77733XtPfw4cPh6fREULuu1BV/5YtW/LKK6/4vfbNN99kWvH3TLL5pk2bTCKg3OQ5cuQgZ86c8rdA2oPzrl27AEwCnRNJopMJVOnSpUNueziS6nLmzJnq4k1KSjIPUPEEqVOnDpMnTwbsSqGCBQuayoaUnD171vy7yI0gxwyFcFbtieS6efNm854MaBdccIF5QMvDKUuWLKmS0SVZu2PHjgHPa6h4IRE7LfLnz8+GDRsAO5Hy8OHDtG7dGgg+tBCOPjqvJzfIeQy0YJHrX+5rJ//99x8Ad955Z7oPrWgnm0uoTsKdTqRYQK7nI0eOmDCSXL9uiESyee7cubn22msBWL16NRB82Fx8eJYsWWLGKRmTu3fvnmZSflpE6hzedtttTJs2LdXrMll66KGHAFi+fLmpmJVFuPysWrWquXbnzp1r3pOQe7CTRi+PN+XKlePtt98G7AVuUlISN998MwDz5s0L6jhe7qMTGUenT59Onjx5APj1118BK5UiPXFFk80VRVEURVEiiGdCe3fffbdZhYpfxurVq02oTlZQCxcuNLPHYOVHkUJlth0rTp06FbBUWFYG6ZGWGgX+ao6sEmOFOMQ6lShBXtu8eTNffPEFAPfddx9gqXBSEi9JohLiWb16tbEGkKTEREFCoe+++65RooR27drFJMn1nnvuoV+/foCtkn733XepFMMSJUqYlb7w/fff8/nnnwOpV7Vly5bl9ddfB+wkXjk2wMMPPwx4I7E3I0TVbtiwod/r77zzTqaUqEhy4sSJNNMDMkKcpMEeY+ReX7duXeYbFyb69++f6rUxY8YYX6xg0jqc509CnOvXrzc+W+IWHs8cOnQo4LNCvKWCVaS8joSkH3vsMQDy5Mlj1HZxTg9Hqo8qUoqiKIqiKC7xjCK1bNkykw8k5bUHDhwwRoeZIWVMNFGYOHEiYOWkyCw7mjlv4WTNmjV8/PHHgJX0mJIRI0YAdqw73pFcobvvvhvw79ehQ4cA+OGHH6LfMODVV1/lgw8+AOy8tjVr1hg3diGQIpUe33zzTapE8hMnTtCmTRvAnbFjpHAmm0vCriQj79ixw+R4id2FsG/fvqi0L1qImbCs3qtWrWrGmH///Rewk/FjiRgzSlk8wFdffQVYdh6SWB4q8szYtm0bWbNmzWQrvcPVV19t9vtMVFq2bGnmD2IKDbaxqbijhwPPTKTAlthEOu3Vq5dJQHVLvnz5TJJoem7J8USpUqUAOyE2URBvIfE3cXqK5MuXD7AfXG6q4rxESi8TsK9/8X+JdnWpEwmbi7dVIDKaROXKlQuw+ygTEbC3xqlTp46nJlCChNubNm1qHtIyCUyUcSQYZMNt2arISaw8hwIxePBgAAoXLmxSCOQ+cjuJArt4plChQp4N2YaCbDUlYoUT52QjnpFnxJQpU1L1af369fTt2zfs35kY/3KKoiiKoigxwFOKlMioEsbZsmWLqz3jwFYwevbsaVZVEpoYP368cV2WEMsnn3xiEp+9jiTQpQwrxDuiwNx5552AZYkhSImuOKcHWiHHA23btgXgmWeeSfWeWF7EQ7J1esjmy0899RRgJa8LokSJG7a4MXsN8X6bP3++SSyXkGt6eMVPKVyIF5MTCeU98cQT0W5OmjjVa7F0kNBjZmjXrh1gqakpQ9vxhJxHSboOlAKS3l6o8YCMrbLfbJEiRcx733//PQBt2rQJuO9eZlFFSlEURVEUxSWeUqQkMUzyMu677z6zG7Oz/FYoWrQoYJUgS8KqULhwYcA/iVdM5fr372/i6LKXWawSe8ONlJLHM3/++SdgKzRi2gnQu3dvwDIqlc/FC5dffrmx4hDbA+GOO+5g5syZsWhW2JEk1kCl6D/++CPg3f3ohCVLlgCWOih5Ms69K53J6E4SpWxc+hxoVwFRory0B+aXX34JWMqUnLvMIM8PUTcWL17MwoULM33caCP5wVLIJc/MQIrUtm3b/FzR4wVR9+XZkD9/fvPeG2+8AcDjjz8OELFnhipSiqIoiqIoLvGUIiUx7WB3E5fS7ClTprBz504gtTlcUlKSWekH2qk9nhFDtSxZshjlbvjw4bFsUlgQo79ffvkFsPon1ReXXnopYFtaxAMSu3/yySdTKVGyYpo1a5bJH4p3hg4dGvD1FStWxN31ee7cOT8lSpDzKNflkSNHgPi1H0mJrOBr1arl9/rJkydZtGhR9BuUAaL0NmvWzOyp6pYsWbKY48m2Ze3atduORRAAACAASURBVPOUAhcMw4YNM8qpmBynR+XKlSPcovAhthxvv/02N9xwA2DnRQvJyclGUYx0ZbCnJlKhIkm5ErJzIi60rVu3TpiQiZBSnnX6SCUSzv4J8ZQQKQmwkuApLuFgT6AeeOABIHDoOh4ZPny4cYOWcyUTkUGDBoXVuyWWyH560kfZeSCers+0KF68uHk4pRxX7rrrLk9OKMSaQPbDywzXXnutSQmR3TYC7dTgVSSc169fP/OsEGTif+7cOTPhFDfzeEDGVJkgyXXqZNCgQQBMmDAhagUCGtpTFEVRFEVxSVwrUoGQ0muZle7YsSNhEkAFcRl2snbt2hi0JHaULFnS0071V111FRMmTAD8wyOJqkQJ3bp1M6qMqBldu3YFwuskHGuknFoS62UPyNOnT8esTeFi6NChplxezqFYP6Rn0BrviHrjtFaR30+ePBmTNoWCqDSyT2aRIkVSKYpyb06aNMmYmMYLDRs2NEUFsrMAwLRp0wA7rUXUyWhGaVSRUhRFURRFcUnCKVIy45Zk5B07dsTFakIJjdatW3vSuFJsON59991UyY9vvvmmsQRIlMRyQVbBYl4JtnqxYMGCmLQpmsj5jOdcRUmsvv3221O999577wGJvT2O5AxVrFiRqVOnAvGjopYuXZoKFSoAgS0OJK9NojODBw/2vBouCeXXXHMNYOWriRIlz/nXXnvNbCkXy2sz4SZSZ86cAWDXrl2Avd9XIiB7laVMIDx+/LgnNg6NJq+99lqsm+CHeJmIi7f48AC89dZbgOWLlkiT+kKFCpmdB3r06AFYyazbtm3zey2R+pwW4rjfp08fzz+g0kL2YXNWlsoD+JVXXolJm6KBJCzLz5UrVzJ9+nTAfp54FQnBzps3L2Blnpy/5s2bA7aPm9cpX768qRyV1ACwJ4djxowB7HBmrNHQnqIoiqIoiksSTpESJk6cCFh7Zcluz+KcHm+O2ILsGViqVCm/1//9999U/lmJgNMny4s7k4va2b9/fxPaEjkabG+be++9F8jcLvRepG/fvtx9992pXu/YsSMQXyXj4aJSpUomET1eaNasGYDfXqNyv0n5v6iMiUSZMmUAGDduHADZs2cHrLJ5SWr2OqLWSLQC/C0OZN/SeFGiJCXnnXfeoWHDhqneFwXcK0qU4L2nk6IoiqIoSpyQsIrU0qVLAYw5IMSvEiWkZZIXadfWWBHIkFPMVQ8ePBiTNoFtsSEKp6zonXz66aeMHz8eSDwlSnIxAtlwJCcnx50iE05S5i/GA8WKFQP8xxXZVUAsPBIRiVTIDhlip7J169aYtSlUbrnlFsD/3Ml42bRp07ixq5AxVfZXDaRG3XTTTXz++edRbVewJOxESrabWbx4cYxbEj4ksTAlq1atomTJkkD8TxYzQibIhw8fjlkbRFZ2TqBkixCZ7K5Zs8bziaqhkjVrVsD2wKpYsaJ5Tx68AwcOjH7DYoD4KqVkxYoVUW5J5nEm8wpSzJGoVXq5cuWiRYsWfq+JZ1Q8bWAvi+gKFSqYxHIZi3799de4KXyQCmcp2nAiE9yVK1dGzak8VDS0pyiKoiiK4pKEVaT+l6hVq5bZkFk2L1Yih8jQkgx/9OhRmjZtCtgO14mIJJY7k5KFSZMmAbBnz56otilWiJuylGjHK5UqVTL7lzn5+eefY9Ca6NG7d2+zX+J3330HWN5v8cbo0aMBy2Il3iwOnIhFirS9aNGixrvs1VdfBez9LL2IKlKKoiiKoiguUUUqjhgyZAiASbiTxNbGjRtz7NixmLUrGojS44V9E2WPqnjbqyqzLFy4EMBYbdSqVcvk0Mjq8X8V6X+85cX99ttv5nxeffXVgGW++dlnn8WyWRHHOV7KuUsr783LyHmSHNl4RXK5atasGeOWuCMpmlsaJCUlxe3+CT6fLymYzyV6HxO9f6B9zAhJDC1atKjZpHf37t1uDxcyeh4tEr1/ELk+FixY0FQAywb34a42jXUfo4H20UJDe4qiKIqiKC6JqiKlKIqiKIqSSKgipSiKoiiK4hKdSCmKoiiKorhEJ1KKoiiKoigu0YmUoiiKoiiKS3QipSiKoiiK4hKdSCmKoiiKorhEJ1KKoiiKoigu0YmUoiiKoiiKS6K6116i28RD4vcx0fsH2kevo320SPT+gfbR62gfLVSRUhRFURRFcYlOpBRFURRFUVyiEylFURRFURSX6ERKURRFURTFJTqRUhRFURRFcYlOpBRFURRFUVwSVfsDJTxcdtllALz//vsAFC1alK5duwLwxRdfxKxdSvBkz54dgAULFgBQpEgRmjZtCsC+ffti1i4lbbJksdadzZo1488//wTgggsuAGDq1KkAVK5cOe7PX48ePUy/HnvsMcDu+y+//MKNN94IwLZt22LTQCUg9erVA6BFixYA3HbbbVSsWBGAjRs3AtCtWzcAdu/ezd69e2PQysQkISZS559/vrmIrr76ar/3OnXqRIkSJQCYN28eADfffHN0GxgG8uTJQ506dQDo0qULYA3agjyE420ilSNHDgAaNWrE0KFDAWjcuDEA586dS/X5V199FYB77rknOg0MMwULFgTg3XffBax+C1999RUA1atXj37Dwki5cuUAGDBgAAD9+/cHYPPmzVx77bUAZiLidZKSkrjlllsAGDVqFACXXHJJqs/99ddfQOBr1uvkypULgCeeeAKA+++/n6xZswLg81n2P9Kviy66yIxD8T6RqlGjBmD1F+z7rm7duuYzW7duBaBKlSpRbl3oPPfcc4A9oVq5cqU5p8OHDwegVq1aAFx//fXMmjULiP/zWLp0aQD69OkDwMCBAwHImTNnqs8uX77cTDT//fffsLVBQ3uKoiiKoiguiUtFqnz58oAlQQP07t2bwoULA9YKEuyVlPN3UabiCQnjzZgxg0qVKgH+fRN++umnqLYrszRp0gTAqFDXXHONeU9Wv4H62aFDBwBmz57NqVP/1965B1g17v//NSKRSYroghPdKNJxKSq6UVROqSjpJHRBKLmUvqqjkMsRolJudURTJkQiKjoJIZfcKqLQ6aSoyFGq+f2xfu9n7Zm9Z9p7zey91x6f1z8ye8/ezzPPWs/6PO/PbQcAa9asAWD9+vXJG3AJIRds+/bto16LpXRkCkOHDgU8NVEn+ooVKwL+etapU4dXXnkF8N0PYV+zI4880p3cv//+e8Bzmdx8880AnHTSSQB89913AGzevDkNowzOfvvt59x3gwcPTvNokodck1qvxx57jFq1agFQvnx5wN9vduzYwfbt2wFv/TOByy+/3ClRl112GeCr3gBvvPEGAH379gU8r4aUnGuuuSaFIy1ZatWqxfLlywE46KCDAH8dYz0/mjZtysUXXwzA5MmTS2wcpkgZhmEYhmEEJGMUqdatWwNw00030aBBA8CLjSrIrFmzAP+k++OPP3L22WcDfmxRJiDr+qGHHgKgdu3a3HXXXQDuhCwF48knn0z9AIvBueeey/PPPw/gYjEAPvvsMwDmz58PwNSpUwFPfTr22GMBX+WYOXOm+xspkLJ9+/ahVzgUq1DauPLKKwGoUaNGke/TvTt37lwA2rZty8aNG5M7uGIQGU+puLwXX3yRtm3bAr7CoXlXqlSJTZs2pXiUwTnnnHMYNmxYuoeRVCpXruz2USnaAGvXrgX82MRPPvkE8K5N7SlSrcJOly5dnDLz1FNPRb0uxXTUqFGA5+lQfKaeI5pzJqB9ZMGCBU5RFDNnzgS8mMx99/VMHMVpHnTQQc4O0HNz586dxR5PqA2p7OxsnnvuOQDOOOMMwAtOluw6btw4wAuW08Ys+b1atWoALFmyxAW/dujQAYApU6akaAbB0Q2sec+dOzdqw+vYsWPKx1USDBs2zBlQ3377LeBlB2mtdWFLVj/wwAOjPkNB24Bz8YU9W+rKK6/k0EMPBWLLzkLBvO+//35KxlUSzJgxA/ADPQFWrlwJwK5duwCoWbOmW0sF+p5wwgksWLAglUNNiGOOOYYff/wRgDfffBPwjKauXbsC8NtvvwFw6623AmSMEaX95d577435ug5rN954Y9Rrmea+vP/++/MZUABPP/00I0eOBPzQgEj095EL+qOPPkryKINx2GGHAV7SigyhovYWudlzc3OdIaEg9VjhBmFD4TkSFQ499FB++eUXwDsUAM6g1L4DfpD9ueeeS7NmzQCcwCLboTiYa88wDMMwDCMgoVSkFHj8z3/+k0aNGgE4+f+KK66Iy4KU9Kd6KJmGJEkFzy9dujTqPVJsdu/ezbvvvpu6wQVEp7x69eo5FUmniDVr1rjTn5g3bx7gBaSr7IH+LoCbswK4d+/encTRF5/DDz/cqan9+vUDYPr06VHvU5p9ly5d+N///pe6AQZAAasqWRGJ7mPdg9OnT49ylRxwwAHJHWAxOe6441zNL431hBNOcCUDNMdY92eYkaKmOkORbN++3SlVcgllIvJA9OzZ091HSlDKzc3ljz/+iPl7Rx99NEOGDAG8Z1CY0X5Y0L21NypXrpyM4SSdSy+9FIB27doBnjtWzxApx5FI+T755JOjXivJv4EpUoZhGIZhGAEJpSKleINGjRo5/6cKTioguTBUhEu+/by8PKdgZUJslNBpSv7uyAKG8hPLEh87dixffvllikeYOIoPqlSpklNmIuMTNFfFI2ieV111VT4lCrxAUSlY27ZtS+7AS4j+/fuzbNkyAFcGQGUrqlWrRnZ2NuCftrp06RIzcDRMKMha8SaRarHSqhV7EStwVzGNYWXWrFnuPlNcRs2aNVm9ejWQeUpUPFx++eUZFZ9XGEpGysrKcgq4ElliqVH9+/cHYMyYMW5vCXuxSs1j69atTvmVNyZWSRyVnbntttv44YcfAL9wbthp2LCh22cUD3v22WcXGpfYoUMHHn/8ccB/9mRlZbF48WIAF5NbEoTSkIqUm2VA6CGzN6pUqQL4Fwz41Wkzmcjg6u7duwN+QH2mBX8WhowLBVDKtXfSSSc5I0sPro4dO2aMAaVDQMWKFV32qXjppZcALxgyspZWpiD3qtxfb731Fk2bNgX89iKxkCEddtflnDlzXMZX79693c+/+OKLdA2pRNCBdPv27VFuIRkbsahSpQoVKlQA8lcA14Ho9NNPB7wA73QTWY9O+6eyS2+//Xb3Prkxu3TpAnj7jYyMsCMjYujQoUycOBHw9822bdu6AHQ9F7Uu5cuX56abbgL8pJCwU61aNdcJQ4JJ+fLlowwpufGmTZvm1l3Pj6+++sq5tSXSlATm2jMMwzAMwwhIKBWpRx99FPBqXkidUnPX119/nTFjxgDwwQcfxPV5jz32WBJGmVxWrFiR7//POussHnzwQcCvqaVeQe+9915qB5ckpNzcd999QP5K31IFMrH6slSXJ554Iuq1W265BfAC5pWO+/vvvwPwzTffpGiExUcuhtWrVztFKhZysWiNw34a3rhxo2v0+vDDDwOem0CuArmApAJkCuo7+tNPP0UpUnPnzo3qGaikl+zsbFfCQrXdVqxYQbdu3QC/bEIYUFmNtWvXuhI4qgX2+OOPu3pCUhpVTiZT1KhIPv30U6cKHnPMMYD3zJTb7s477wT8riCLFy92Te8zhdWrV7ukMylsgwcPZtCgQQCuPJDCeiK9OG+99RYAAwcOTEqJElOkDMMwDMMwApJVVPGuEv+yrKyEvmzEiBEuXVUWaNmyZV3fJAWqdu7c2QUFKkBZMQC//fab61FXnI7zeXl5WfG8L9E5FoZODlKmNm3a5NQLFVKbNm0a4PdPKi7xzLE489MJYcmSJW5NdGLaunWr67snFVLX5sMPP+zeV5wSB6lew4KUL1/exQYJrfNrr73mTpL/+Mc/AL8MQiKkeo6KXdRJf+TIkTG7roOnWik2RV3pg5CudVRMiYLOwQ967dOnD1ByylSy70Vx5513xiy6GeO7NK6Yr0vNiXf+qVzDJk2auE4Kir/cunWr248eeOABoOSDrlN9nWq/0D4aC3lnJk6c6ApXFodUz/Hqq68GcN6ZXbt2uXk0btxYY3Lv1z6j2LAtW7Yk/J1x3YthNqQiUfB4mzZtnNRetWpV97qyaHTDK0OoT58++Zo3BiVdm3ePHj0ArwaP5iZjQm4FuT2LS6o27/79+zs3iVxCu3btcnV6FASojeHhhx8ukTL+6TakXnzxRddy5LjjjnM/A692jTLeilOxPpVzPPDAA121b1UOjoWMx969e5dIpky61lFGYM+ePZ2xqwDfn3/+GfCCrVWtvzgk+17Ues2fP59DDjlkr+/X4XXPnj1ccsklgOcWBM+VpL9DvKR6DRUmUDBkArzahBDb9V4cUj1HuVyV9a7G2pHoeZKTk1MSX5nyOapFmNZRCVf//zs0JsB7Zl577bVAMANKxDNHc+0ZhmEYhmEEJJTB5rFYtGiR++/kyZMBv0rtgAEDnDuooFUqpSrTkOz89ddfA17/OaV+qhp2SSlRqWbbtm1unTSnsmXLunIGkthL+oSYbk455RTXk+6CCy7I99pzzz2XMU21pU48+uijRSpRQrVfSrJuSzpQMsDOnTvdtRn5twBP4VHdqZJQppKFxlixYsUie7MJBZ9/8MEHroK0mv1mAgpS1r4TiQKxpYSHKWA+EdT3UWpTLEVKSR4rVqzIqCbFhRG5nvq3XNWprEpvipRhGIZhGEZAMkaRimTt2rWAF4wO8Oqrr7r4koKFOxs1asQ777yT2gGWAIoXUiyGCh6C399M7wl7UUMhle3iiy+OeQpWTFRpU6IiUdE/zX/27NkAXHbZZa7sQdiRMnPRRRfF9f5YPbD2hqo016tXD/Du8XShOIyGDRsC/v4Dfoxbbm4u4K2vFHOpPpnG1q1bXUmEgh0Frrrqqoysei5VVPdd7969XdzMzJkzARg3bhwAX375Zcyq4JmC1FHwEnvAq8YP/rPj1ltvdfFSmcKZZ57J3XffDfj3ZKzniLwcqcQUKcMwDMMwjIBkpCJVkMg+ZZHZJeAVclTsjWKLwk6lSpVcMTmdyCNRBqPUjbD3Y5Ny9vTTTwNe64JYJNrBPOzo1DR16lTAW1epTspalApXku0KkoVaL6QihkTxDgUVkXSgMeik+/HHH7vXFHujtOwTTzzR9TpTaYswxUqVK1cO8E73hVGzZk2X7TRq1Kh8r9177720aNEiWcNLCvXq1XOZ3uvXrwfghRdecPecylnIw3HbbbdFxTBmEio/8vXXXztVVPFSUubOP/98l8kY9lgpFfh98cUXOeigg/K9tnHjRqe66XmorGfFvqWC9O9SJUD79u2dxCc3gtKyO3fuzEknnQRkjiHVo0ePKANq/fr13HHHHYBf5VvB9rNnz3aBhmFENVxUkX3Hjh2u3oxcshMmTHD1TxQQGeY57Y0qVaq4PnpyCYGfMh5P7Z6woXIjSrOORO7YypUrc/7556d0XKkmVgKL6kn17duXN954A/DT0HWfhgE9ZPfmdpQLRWVkevXqBXguVrmHMqUCeOXKld0DWPOKPLjIFaY+fAcddBBlypQBile3LtVordRIfMGCBe7gpmeHSuacdtppMe/jMKFrTnW+srOzXbmcCRMmAF59KJVXkSGVDsy1ZxiGYRiGEZBSoUip9AH4gddKfezcuTPdu3cHwq8CqNjYyJEjnWtD87nppptcb6QmTZoAvsVepUqVULkPImnZsmWUK2DixInccMMN+X7WqlUrF/woZUqV3DMJBWCPGDHCFd2MZPHixakeUlJ59913Af/euvfee91rOg1/+OGHCX/uunXrAPjuu++KO8SU8uWXX7ogZikEmcaQIUOYNGkSQFQl/tGjR7sekCrlEXbat2/v/h2rIKeUNanFrVu3dnvx5s2bUzDCkkG9SpXUE9mLVkqOkiNOO+00t1eFMXmgevXqTonSfH744Qf3bFC4RFgwRcowDMMwDCMgpUKRikT+7g0bNgBef73IVjJhRu1DKleu7OIwFFekvoLgB28rLiyVbX4SZdiwYS7eQP0PCwawAsyaNcspUhdeeCGQWYrUpZdeCnhd5cFbk1jr8vLLL6dyWEln+PDhgK+OtmrVyr0mRUrrnghKFsk0ypcvH4oA+cLQmqicRKzEj+HDh9OvXz8g/76TqVSpUsUp/Icffnih79N7PvrooyglLhMoqN7Guu8iyzoo8SCMjBgxwqmCb731FuDtMbE8L4rZ0/opvrhatWouuSDZhPeOjwO5uPRf8DcIPdiqVavmAs/DTqNGjdy/JcFGbmQKptPmt2zZMqB4zZiTxRlnnAFA8+bN3c/UOPLXX3+Nev8pp5ySmoElCWWIFNXcde3ataGTpIuLmqBqM4s0IpSlWdro1KlTlJEv98PAgQNdYHMY78sdO3YAONdds2bNYmbLqrmv/pvp6H6U63nWrFnOBdu/f3/Az7KcN29extR0i0QJD2LIkCHu4KZnpJr9gl9jKoy0aNHCuSPlzotlRB111FHMnz8f8NdYc0yVEQXm2jMMwzAMwwhMRitSIi8vj+XLlwNw3nnnAb4Vm5eXly/oLsxMmzYN8E61Up0i65noxCTLW/Wjdu7cmcJRxsfSpUsBWLVqFfXr1wfg0EMPBbz0eZU2kJKhatmAq6GVicRSotasWQP46fClCVUgj8WcOXNSOJLkIZeJ9pTRo0c7xViqhvadSpUqOQX8iiuuSPVQ40bjb926Nddddx3gBysXpUJ98sknGXd/Rrq8pBYOGjTIKeMqiSC1LlOVVCkw6s/avHlzlxigNVUttClTpoS+96XUfYV8qH4i+OvYq1cv91yRO1YV6lOJKVKGYRiGYRgByUploHJWVlaJflmdOnUAz9dbqVIlfQfgKwMLFy5k8ODBQLCgV5GXlxfdNjwGxZmjTgu33347119/fdTrCrpTfMbbb78NlFzRuHjmmOj8+vTp4+IxFHT+wQcfuGrzCgysWrWqOz0pYLmkU9+TtYb169d3Cpwq7Ofl5bkK4FovnRSTSTKv08aNGwPePQWxg1V/+eUXp6yqJIJO+iVFKu7FWOgUPG7cOJcQUaFCBcAP4n377bfdflOcgOVk3It7Q+v7+uuvu1IAKuSoAogDBw5k4sSJxf6uVK5h06ZNiyw7omeGugzESoYJQrqu03bt2gFwww03uHtPhTgVo9mvXz8Xg1QckjXHFStWuMrrEZ8Rpfjv2bPHlVdRwsvKlSsT+aq9Ete9mMmGlBg/fryrSqubQlWlr7322nxNRoOSrpsilSRr89YmpgD0SOQmGD16NF988QWQvNottoY+xZlj165dAcjJyXFuyzFjxgCwaNEiVwMqWdg6epT2+UHJzLFChQr06dMH8LsmgBd2AH6LmJycHKDkmsDbdeqT6Bw7d+5M586dgfwuvUWLFgF+ZfrZs2cn3dUczxzNtWcYhmEYhhGQUqFIpQI7XXiU9vmBzTHs2Bw9Svv8wOYYdmyOHqZIGYZhGIZhBMQMKcMwDMMwjICYIWUYhmEYhhEQM6QMwzAMwzACktJgc8MwDMMwjNKEKVKGYRiGYRgBMUPKMAzDMAwjIGZIGYZhGIZhBMQMKcMwDMMwjICYIWUYhmEYhhEQM6QMwzAMwzACYoaUYRiGYRhGQMyQMgzDMAzDCMi+qfyy0t4BGkr/HEv7/MDmGHZsjh6lfX5gcww7NkcPU6QMwzAMwzACYoaUYRiGYRhGQMyQMgzDMAzDCIgZUoZhGIZhGAExQ8owDMMwDCMgZkgZhmEYhmEExAwpwzAMwzCMgJghZRiGYRiGEZCUFuSMl4MPPhiAq666iiFDhgBQqVKlqPdlZXl1sqZOncrYsWMB+PLLL1M0yvRx4oknAnDNNdcA0LVrV/c3E4sWLaJ169YpH5vx5+Lyyy8HYOjQoQAcc8wx7LOPdz7bs2dPvve++uqrjB8/HoB58+alcJTxs+++3pb4t7/9DYAWLVpw9dVXA/5+k5eXx7p16wCYNm0aAI8//jgA3377bSqHa8RBuXLlOOeccwA4++yzAahevbpb44LX6++//86YMWMAuO+++wDYsWNHSsdsxEfVqlV58sknAdwa79q1C4Dp06czcOBAAH799dekjsMUKcMwDMMwjIBk5eWlrnL73srE16lTB/AsSYCTTz6Zt956C4CnnnoKgHXr1rF7924A1q5dC8CkSZOcSnPHHXcA8MQTTwDw008/lcjY010K/9xzz6V79+4A9OrVS2Nyr3///fcAVKxYEYADDjiA0047DYCPPvooru9IVVuKZs2aOSWjZ8+e7uedO3cGYO7cucX9ipikcg0rVqxIrVq1ALjkkkuiXu/YsSMAv/32GwB169Zl06ZNAIwePRrwVJtEFY5kzlGqsFSl+vXrc9xxxwFQpkyZyM/WWKI+45dffgFg8ODBAO40mQjJmmNWVhaPPPII4Ctt8bJmzRoAcnNzefjhhwH47rvvEvqMSJJ9L1apUgWA5s2b0759ewDatGkDeIqMrt2I7wK80/6//vUvAJYvXw54qty2bdsS+v5U3Itt27YF4MILL6R3795FfYfGFPWars8rrrgi4e9P1zOjbNmyAOy///5xvV8Kzv/+97+EvyvVczzggAMAXwG/+uqrY3qrxIgRIwC45557gGDKYlz3YpgMqbPOOgvw3FIA33zzjduod+7cWejv7bfffk5+10Po5ZdfBmDAgAH8/PPPxRx56i+Yww47DPDn07t3b3eDrF69GsDJzytWrOCLL74A4JlnngE8w+v4448H4nc3JGPzLlOmDE2bNgX8i79169bu5t28ebM+190Q2tjfeOONRL5qr6RiDU855RQATj/9dO6///5Yn62x7PWzdu/ezaRJkwC49tpr4/r+ZM2xXLly7oAjl0gRn62xFPqe7du3A56R+eKLLyYylBKfY7ly5QAYN24c/fr1y/far7/+6g4pOpQ1aNCAChUqFPp5OTk5AFx88cXxfH1MkmVIybUlY0gGVXFYuHChJrBf5gAAEqJJREFUuya0rnsjWddp+fLlnTvu0ksvBTx37e+//w7AggULAJgzZw4vvPBCzM+oXLkyn376ab6f9e3b1x3O4yVZcyxbtiyVK1cGoF69egB06tTJvR65B8Wzz3z11VcAjBo1yj0/4iWVz8W6desyY8YMABo2bFjo+7TW++67r3PVX3TRRQDMmjUr4e+1XnuGYRiGYRhJJFTB5tnZ2fn+/+OPPy5SiRJ//PGHO/1/9tlngO/+mjBhgvu3VJCwc+SRR/Laa68BULt2bQC2bNnCrbfeCnhzAt8tBDBs2DAAzj//fMA7cYUh8LVXr14uEFfk5uY6pe2TTz4BvNODlMi77roL8IPply1blqrhBuKAAw5wJ3y5Ew488MBif26ZMmWikgjSxaBBg/aqRAklfih4V78nhRQ85QA8xXj+/PlA6gN6pZz16NEDIJ8aJfWpadOmrFq1Kt/vHXHEEc4NLZfPSSedlPTxFpf69eu7vaMklCjRqlUrp4hItUwXLVq0iHLLLlu2zLmS33nnnb1+xrZt23j77bcBT9UB7xpOVJEqKQ4//HAA555s3rw55513XqHvlwfmq6++corUSy+9BHjeC/D2LHkI5MZt3bp1wopUKpDqtmDBAqpWrRr1+scffwzgbADNtVWrVm5fbty4MRBMkYoHU6QMwzAMwzACEipFSsGOIoj1KCVH8TVbt251J06dlgqmZYcFWdvTp093StS7774LeCd3qTcFGTp0KLfddhvgp5UrMD1dKIZk1KhRLg5KsQtjx46N8t3v2rWLjRs3An7QuU6FDz74oDtRhpGbb77ZjTkWUlW3bNnCgw8+GPfnDhs2LO1KxwMPPADEDpiP5NlnnwXgxhtvjAqy/stf/gLkV6RE27ZtufvuuwG47rrrijvchLjgggsAePTRR93PlMgyZcoUgCg1CmDDhg1MnDgR8ONLXnnlFfe61Lb99tsP8BTzMHDeeedx7LHHRv38hx9+AGDx4sWAt3d+/vnngL//iI4dO3LooYcC3loDUYHp6UBK8OzZs93PpOA/8sgjCSUd7dixg/Xr10d9vmKP3n///eIOd68oeeOGG26gf//+gB83u2rVqqgEokmTJjklSiWACsZ5RfLXv/41Lm9POmnQoAHgxztHqlEffPAB4MUQv/rqq0C0oj1//nz3NygqIL0kCJUh9dxzzwF+YG2imSCRaPN64IEHmDp1KoD7g+uBHTb0QGnevDlz5swB4O9//zvgGYRCmXnKDurRo4cLnLzwwguB9G/eypaoUaOGk5DvvffeQt8/YMCAqDog4sQTT3Qbix50YUDukVgur+XLlzujVq6rJUuWxPW52kCuv/56J02nmvr16wO+UVuYi1EPriuvvBLw3Apy0evBoySSWGzZsiVt7oRYAasyoG655ZYif1euWwWxRtKhQwfAe1hBtDGSapTpJMMR/AytUaNGucy0H3/8ca+f9fzzz3PyySfn+9y8vLx8YQbpQAc3BRcDbk8MkrmtEJGuXbu6n2l/VkC9Mm+TgYzAW2+91V0/OswsXbq02J+/fPly95yVKyxsyICsUaOG+5muVe03kcaTnhFy440ZM8bdgyeccAIAkydPjsu9myjm2jMMwzAMwwhIqBSpggwZMqTYNYXGjx9Ply5d8v2sQYMG1KxZEyDh1OtkIPeN1KSffvrJ1cOKVKLEm2++CfjKRV5eHoMGDQLSr0QJnRA3btxYpBLVt29fwAsw37JlC+CXdTjqqKMAz8UlWXvDhg1JG3OiqLaZapgBPPTQQ4AnySeyFg0bNuT6668HfBVo3333dSptqjnkkEMAqFatWqHvmTdvnkvk0MmwU6dO7iSpVPuiuPvuu5NyQowHXVPi119/Zdy4cXH9roJ9+/TpU+h7dGpOtyKl/a9x48ZOiZLSkmiF+eOPP94lf8hj0Lt377RdpwWJDBnQ/TR27Fjngo2HcuXKOVehPm/hwoVO7ZLSmkyk6C9btsztB//973+T+p01atRwtafSXcm9W7du7tkglixZ4n4WyzOhTh6RbnahazVZe40pUoZhGIZhGAEJlSKlIDn5p2vXru1iM2IpM/Gwfv16mjVrBvhlAzp16uRSnxW/EW/8SjJQzJNOPC+88EKRp1j5e1NZTDUoBx98sDtRvf7664Cnvjz22GOAX95h9+7d7uSs96nIKuAUN8VbpRP54iODH1VlX+rb3tQofYZOt6+88krMIo+KCWjevHnxBp0gipdRCQ0FjEeybds27rzzTsAvfliuXDl3qo2nIKeCmtNBwdi2nTt3xq1cFCzcGYsWLVoAnvIVT/xRKtD8gvY6jCwt8J///Afwldl0outo8+bNrlilrsk2bdrwf//3fwA8/fTTgJdwpABkxVBpHz7iiCNcnI1o166dK7QqT0AqmD59etKVKHH22Wc7JTrdyn/Xrl1dAepvvvkG8NZTSpT2mEaNGrlnwrnnnlvo5yX7Gg2VIaULRq6dZ555xtWIUEZfIhKtaNWqFZA/2FIZcOmW3ROhsDomOTk5LvMmLMho7dChA7m5uUD+h7IyRvT337ZtmzOgRGTWierUhMGQ0gMyMgtNwcmx2oLIQDrqqKNcZppcZu3atSvyu4K0UCkJVq5cCfiZk7EMqViB1vGiTM50BroqW2/48OEJ/V6bNm3iyqY88sgjAT8oOwxEBmPHgx5mOuQUZ82TiQ7fLVu2dIcuHT7q1Knj7iPNY+fOnS65RZneZ555JoDLSoxk9OjRLrlH124q0KH5z8y///1vwEuqkotOa1yzZk3nhlRYhfYUVbEHvzVVsjDXnmEYhmEYRkBCpUgJBVM/++yzLijyvffeA7yTv/qPxevaikxhBe8EokDQsARnR9KtWzf3b7k2W7ZsWWh/r/vuuy90VdtV5+TMM890gbk68b399tsuqLeomixa87Dx4YcfAvkVNqkaWqMNGzY4F4NUqBNOOCGhXnthQCf5Aw88MO7K5vEQxgrK2dnZDBw4EPBPt+DXhVJZg/Hjx7u1Ve0eBTb36tXLKeCiZ8+ezgWaDqTi79mzxyXZqGvAhAkTogJ3GzZs6Hqcqn7bqaeemqrhFovPP//cuV3lujviiCPc+sjdF0ndunUBP3ykfPnyTkVUGYJ0rV+3bt1cxe6SdoPH278z3agEEPili+Tl+Oijj9xzQs+cyB6zcqlH1opLBqZIGYZhGIZhBCQrlSfjRDtA77///s4nr2JoVapUcbEzskqfffZZp3YoYFBxLBUqVHCnqX328ezGU0891VVGjZdUdLmWD75v375RsQx//PGHq5QsVUPFGiMt9uKQrI7zQSlXrhzgJQxI6TnjjDOAYP33SmoNVZZB8UNHHHFE3GOQr17xYAq67t69e9Sa9+vXj5kzZ+b7vb2RzOtU/RFjVQlW1frTTjuNAQMGALgkj0gUlK+ekEVVXy6MkpqjShdEnlYLxu6Br0ipuF8kkydPBvxSB82aNXOVmPV7GzZscLEu8RaHLMl7UfvGwoULadq0ab7Xfvnllyh1tFy5ci42qigU36eCs4mQiv00UVS2Q/Gd4McoBVGDijNHJeg8++yzLvBb6f1KygqC9qoRI0a4/VVej06dOlG9enUg/mDzZK3jyJEjnSdDz+9PPvmEiy++GCh6PVS9f/Xq1W79FEMXhHjmaIqUYRiGYRhGQEKtSIGfNaJTcOQJVuqTlCbw++jJXzp16lTuuecewLNQwbNwf//994TGkcoTVIMGDVyKrVKM+/Tp44ofas10uiypzMOwKVIiJyfHxY1JkQpSWK2k11Br9Pzzz3PMMcdEvb5p0ybAV5OefPJJp+qIyJgqnRBFu3btEj7th+Gkr/stVn9EFdgtTrxVSc1R+4ZS40eOHBn3GBT3Nm3aNCB/O6sVK1YA+bM6Ey2zkox7cfz48flKigRFsYHKXAzSyisM16lQRqq8FAcffLDzdlx22WWArxwnQknM8ZxzznFxWirJkJuby5o1awBfnXr++eeL/I727dsDfvZ78+bN3b9PP/10AGbMmMHRRx8NxM4+jkUy11HPfrXi2rx5sysoWxRS+1u1auWKPEsdT5ayGMpg80gktUtqjExNbdmyJeAFkyuoTD2f5ArMyclxC6K0yESNqFTz6aefOoNRgZCRfZ3uuusuILNKNxSHdevWuX9L3k5XNexItEb169d3qdRar3Xr1rkxRo5f6GCgsgn777+/M5BVBiSI+zLd9OnTxwX2hj2gXoeu22+/HfA24GHDhgF+5XLwDQXVonnsscf4+uuvgdguV7kK5e4Ev2xAOuvVjRo1isMPPxyITsApiJJwdBhQzbQdO3a4nm/F6YUaJoYMGQL4Lq7Nmze7cIl0V/ieP3++O0ypTlKtWrXcGih4fuzYsc7g0AFhn332cde4AullBJ9zzjnOWJIhlZeX5/4WKi+QTvTs//777xP6PfXBBF9skXs7WZhrzzAMwzAMIyChV6SKYtGiRfn+G4ms8sheYemu1hoEpWPLsobgVYkzlchTf2ElINLJjh07XM/GeHs3yk0b6f6SqqpAbMnSmcTe+pAFCUxONkr/X7p0qSvaW6dOHfe6VIl4iwGnqhJ1omzevJnu3bsDfi/FJk2aUK9evXzvmz59ulMDpkyZAvjBz5s2bWLp0qWpGnJSUWD1VVddBfgK6rx589KuRMUict8fP3581OtaW4UIZGVluTlJCY11DUv53rhxo1P8w9JzL1MwRcowDMMwDCMgGa1IFYXSJCPTfTMppkiBzDplgJ9+ns44i3SgXksQv+ITVrKzs4HovlB79uxxbX4SjQnIJObMmZPuIRSJYoOKky6tOBQpqdnZ2fTs2RPwA9uD9g4tLoqZUZuTuXPnugSASAqWkRGKPc10qlevziuvvJLvZ1JLb7zxxnQMqdjMmDEj0O9FXq9KkFApFlOk4qPUGlLqqQSwatUqwO+vF3aysrIYMWIEgKug/OGHH7qg2D8bCngF3yWWicZk1apVnRtazZrFU0895eoalVa2bt0augr8yUBJBnpQd+vWzQUyy5U7atSotIwtXnr37g34LkAFJk+fPj1tYyoJZCAMGjTIGQ1yY86aNQsgNA2mU01OTo5LflGfyOLUrPozYa49wzAMwzCMgJRaRUqnDPDTllUaIey0b9/eKWqSVocPHx7KvoCpoHbt2q6aeyZz4YUXuiDmgqUBVJOnNDNhwgRXF+3PgCqcR/bOVHr5PffcE6g2UbqQKzLeyuxhZcyYMYCnDOoeXLhwIeCXx/mzotpUgHNFq4ZVpqKEnWQn7pgiZRiGYRiGEZBSq0gpgLJbt26uqrBOI2Hn8ssvd/+ePXs2QFRg5J+J2rVru9OjTvmZhE51sYJYlV4e9piZvaHg5Fq1ahX6ngYNGrj+c5mkxgQlJycH8JJEatSoAfjFAps0acKCBQvSNra9UXAdFWeaqaggqhTBSE4++WTA7434Z42RmjNnTsYrjgVRTKZKnCSLUmtIZSI9evQAvPYZK1euBPzK139GypQpA/gZjBDeGj1FoZu5fPnyzkVZsL5LJtY4i0Ru56KaK3fs2NFVdP8zGFJyyy9ZsiRf9i14B722bdsC8Oabb6Z8bEXRpEkTl/Us1DIlU1FD4lghAmpppOy1PyubN292bXJKCzKgkp3kYq49wzAMwzCMgJRaRUqBc4sXL3aNRcOO+jsBPPHEEwCsXbs2XcNJO1KizjrrLLeeH374YTqHFIjIRptSonTC31uz0UxBCtP999/vqmAX5NVXX+Xnn39O5bBCQa9evVxfUPW622+//WjVqhUQPkUqNzfXlT147bXXAJg5c2Y6h1RsVGPp1FNPBbwG4uqV+cgjj6RtXGFDlf1jNRzPFDT2Z555xilsyVb8TZEyDMMwDMMISFYqO7RnZWWFux18EeTl5cWVf1/a55jK+anf05w5c/j888+B4p2U0rWGr7/+OgAtW7Z0KdaKhyvpkhx2nfqU9jmW9PwGDBgAwJVXXkndunUBv8deSff3tDX0CeMcy5YtC+QvIxSLTJ5jvMR1L5ohFR92wXiU9vmBzTHs2Bw9Svv8wOYYdmyOHubaMwzDMAzDCEhKFSnDMAzDMIzShClShmEYhmEYATFDyjAMwzAMIyBmSBmGYRiGYQTEDCnDMAzDMIyAmCFlGIZhGIYREDOkDMMwDMMwAmKGlGEYhmEYRkDMkDIMwzAMwwiIGVKGYRiGYRgBMUPKMAzDMAwjIGZIGYZhGIZhBMQMKcMwDMMwjICYIWUYhmEYhhEQM6QMwzAMwzACYoaUYRiGYRhGQMyQMgzDMAzDCIgZUoZhGIZhGAExQ8owDMMwDCMgZkgZhmEYhmEExAwpwzAMwzCMgJghZRiGYRiGERAzpAzDMAzDMAJihpRhGIZhGEZA/h9BwI4hCKf2uAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x720 with 100 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def P1(num_examples=10):\n",
    "    #Defining the colormap for image properties\n",
    "    plt.rc('image', cmap='gray')\n",
    "    plt.figure(figsize=(num_examples,num_examples))\n",
    "\n",
    "    # for each digit in 0-9 -> the unique labels in Y covers this part\n",
    "    for i in np.unique(Y):\n",
    "        features = X[Y == i][:num_examples]\n",
    "        for j in range(num_examples):\n",
    "               # Create subplot by specifying position from 1 to each digit * columns. Add j to get all columns\n",
    "                plt.subplot(num_examples, num_examples, 1 + int (i) * num_examples + j  )\n",
    "                # Hide axes\n",
    "                plt.axis('off')\n",
    "                # Plot the corresponding digit ( reshaped to 2D matrix)\n",
    "                vector_size = int(np.sqrt(features.shape[1]))\n",
    "                digit = features[j].reshape((vector_size,vector_size))            \n",
    "                plt.imshow(digit)\n",
    "\n",
    "P1(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(2) Evaluate a K-Nearest-Neighbors model with k = [1,3,5,7,9] using the mini training set. Report accuracy on the dev set. For k=1, show precision, recall, and F1 for each label. Which is the most difficult digit?\n",
    "\n",
    "- KNeighborsClassifier() for fitting and predicting\n",
    "- classification_report() for producing precision, recall, F1 results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy for k-> 1 neighbours: 0.884\n",
      "\n",
      "Precission, recall, and F1 for each digit when k = 1:\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.95      0.95      0.95       106\n",
      "           1       0.89      0.98      0.93       118\n",
      "           2       0.90      0.79      0.84       106\n",
      "           3       0.93      0.87      0.90        97\n",
      "           4       0.91      0.85      0.88        92\n",
      "           5       0.86      0.88      0.87        88\n",
      "           6       0.92      0.92      0.92       102\n",
      "           7       0.85      0.94      0.89       102\n",
      "           8       0.83      0.77      0.80        94\n",
      "           9       0.80      0.86      0.83        95\n",
      "\n",
      "   micro avg       0.88      0.88      0.88      1000\n",
      "   macro avg       0.88      0.88      0.88      1000\n",
      "weighted avg       0.89      0.88      0.88      1000\n",
      "\n",
      "Accuracy for k-> 3 neighbours: 0.876\n",
      "Accuracy for k-> 5 neighbours: 0.882\n",
      "Accuracy for k-> 7 neighbours: 0.877\n",
      "Accuracy for k-> 9 neighbours: 0.875\n"
     ]
    }
   ],
   "source": [
    "def P2(k_values):\n",
    "    for i in k_values:\n",
    "        # create KNN neigh model from sklearn\n",
    "        neigh = KNeighborsClassifier(n_neighbors=i)\n",
    "        # fit the KNN model using mini training data and mini training labels set\n",
    "        neigh.fit(mini_train_data,mini_train_labels)\n",
    "        # prediction on the dev data set\n",
    "        dev_prediction_labels = neigh.predict(dev_data)\n",
    "        # Report the accuracy on the dev data set\n",
    "        print 'Accuracy for k-> {0:d} neighbours: {1:.3f}'.format(i,neigh.score(dev_data,dev_labels)) \n",
    "        #For k=1, show precision, recall, and F1 for each label\n",
    "        if i==1:\n",
    "            target_names = np.unique(Y)\n",
    "            print '\\nPrecission, recall, and F1 for each digit when k = 1:'\n",
    "            print classification_report(dev_labels, dev_prediction_labels, target_names = target_names)\n",
    "            \n",
    "k_values = [1, 3, 5, 7, 9]\n",
    "P2(k_values)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "ANSWER: Which is the most difficult digit for K=1?\n",
    "\n",
    "    From the classification report, looking at the f1-score which considers both precision and recall, the difficult digit is 8. \n",
    "    The recall value is @ 0.77 which means there are lot of false negatives.     \n",
    "    The precision is very low for digit 9 which is 0.8 implying that there are lot of false positives.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(3) Using k=1, report dev set accuracy for the training set sizes below. Also, measure the amount of time needed for prediction with each training size.\n",
    "\n",
    "- time.time() gives a wall clock value you can use for timing operations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " For Sample Size   100, \t  The Accuracy is -> 0.702 , Processing time is 0.141 seconds\n",
      " \n",
      " For Sample Size   200, \t  The Accuracy is -> 0.791 , Processing time is 0.264 seconds\n",
      " \n",
      " For Sample Size   400, \t  The Accuracy is -> 0.811 , Processing time is 0.513 seconds\n",
      " \n",
      " For Sample Size   800, \t  The Accuracy is -> 0.866 , Processing time is 1.033 seconds\n",
      " \n",
      " For Sample Size  1600, \t  The Accuracy is -> 0.905 , Processing time is 2.109 seconds\n",
      " \n",
      " For Sample Size  3200, \t  The Accuracy is -> 0.927 , Processing time is 4.336 seconds\n",
      " \n",
      " For Sample Size  6400, \t  The Accuracy is -> 0.939 , Processing time is 8.679 seconds\n",
      " \n",
      " For Sample Size 12800, \t  The Accuracy is -> 0.952 , Processing time is 18.121 seconds\n",
      " \n",
      " For Sample Size 25000, \t  The Accuracy is -> 0.962 , Processing time is 36.578 seconds\n",
      " \n"
     ]
    }
   ],
   "source": [
    "def P3(train_sizes, accuracies):\n",
    "    # create KNN neigh model from sklearn with k value 1\n",
    "    neigh = KNeighborsClassifier(n_neighbors=1)\n",
    "    # holder for times taken\n",
    "    time_taken = []\n",
    "    for size in train_sizes:\n",
    "        #start the timer\n",
    "        start = time.time()\n",
    "        # Get training sample from original data set\n",
    "        train_data_size, train_labels_size = X[:size], Y[:size]\n",
    "        # fit the KNN model using training sizes set\n",
    "        neigh.fit(train_data_size,train_labels_size)\n",
    "        #append the accuracy to accuracies\n",
    "        accuracies.append(neigh.score(dev_data,dev_labels))\n",
    "        end = time.time()\n",
    "        #calculate time and append it to time_taken\n",
    "        time_taken.append(end-start)\n",
    "    #printing the report with times taken for each training size\n",
    "    for size, accuracy, time_value in zip(train_sizes, accuracies, time_taken):\n",
    "        print(\" For Sample Size %5d, \\t  The Accuracy is -> %0.03f , Processing time is %.3f seconds\\n \"\n",
    "              % (size, accuracy,time_value))\n",
    "    \n",
    "train_sizes = [100, 200, 400, 800, 1600, 3200, 6400, 12800, 25000]\n",
    "accuracies = []\n",
    "\n",
    "P3(train_sizes, accuracies)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(4) Fit a regression model that predicts accuracy from training size. What does it predict for n=60000? What's wrong with using regression here? Can you apply a transformation that makes the predictions more reasonable?\n",
    "\n",
    "- Remember that the sklearn fit() functions take an input matrix X and output vector Y. So each input example in X is a vector, even if it contains only a single value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy for n = 60000 for esitmated function y=a+bX is [1.2446923405015078]\n",
      "Accuracy for n = 60000 for esitmated function y=a+blogX is [1.0337099653785087]\n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjAAAACZCAYAAAAxfQBgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xl81fWd7/HXJ4SwhLCFQEggBETWAMGyaNW6KyoaajtTkGlrb1tmOjq9tZ1x2tJpvb3afW7HmXY6QzveTi2gdbkJVqxLW5dWZZMDCC4gSkhOWELYQkhIcj73j3OiIZ5AQnJylryfj0cecH7nd36/D7a/bz7n893M3RERERFJJmnxDkBERESks5TAiIiISNJRAiMiIiJJRwmMiIiIJB0lMCIiIpJ0lMCIiIhI0lECI1GZ2aVm9ma84+gIM3vXzK6OdxwiktjM7Atmtt/Mas0sO97xSNcogenl2vvl7+4vuvvkeMQUK2Z2uZlVxDsOkd4o3l80zKwv8H+Aa919kLsfavP+bDM7amYTWx37kJkdMbPCno1WOkIJjCQUM0uPdwwikpJGAf2B7dHedPfNwE+Bn1tYX+B+4Jvu/m6PRSkdpgRGompbrYh8e/p7M9sa+ZbykJn1b/X+QjMLRL6tvGRmM1u991Uze9vMjpvZDjP7aKv3bjOzP5vZj82sBrg7Six3m9kjkXseN7NXzWxWO3H3M7N/MbNg5OdfIscygSeBvEj5uNbM8rrnv5aIdIWZfd7MdplZjZmtaf1smtm1ZvZmpN35dzN73sw+18512nv+JwEtXeJHzOwP7YTyv4DRwDLg60At8JPu+5dKd1ICI53xl8ACYDwwE7gNwMwuIPxN5a+BbOA/gTVm1i/yubeBS4EhhBuIX5vZ6FbXnQ/sBkYC97Zz7xLgYWA4sAoojXxDams5cCFQDMwC5gHfcPcTwPVAMFI+HuTuwc7+BxCR7mVmVwLfJdy+jAb2AA9G3hsBPAJ8jXDb8ibw4TNcrr3n/y1geuScoe5+ZbQPu3sD8Fng+8BXgM+6e6gr/z6JHSUw0hn/6u5Bd68BHifcSAB8HvhPd1/n7s3u/t9AA+GGBHd/OPK5kLs/BOwk3LC0CLr7v7l7k7ufbOfem9z9EXdvJNyP3b/l+m0sBb7t7gfc/SDhhOmTXfx3i0jsLAXud/dXIwnE14CLIuNObgC2u/tj7t4E/Cuw7yzX6urz/xrQBGxz9zc6+VnpQUpgpDNaNxx1wKDI38cBX4l0Hx0xsyPAWCAPwMw+1ap76QhQBIxoda29Hbj3e+dEvhFVtFy/jTzC3+Ba7GnnPBFJDKc9s+5eCxwC8iPvtX72nfCz36FrcW7P/z8DzwNjzGxxJz8rPUgDJqU77AXudfcPdP+Y2Tjg58BVwMvu3mxmAcBandaRLdHHtrpmGjAGiNYFFCScULUM1CtodZ62XhdJPC3PLACR8WrZQCVQRfhZb3nPWr8+w7WiPf9nZWZXEe6ungbMAX5pZk9Hqs6SYFSBEYC+Zta/1U9nE9ufA39jZvMjo/czzexGM8sCMgknDgcBzOwzhCswnfUhM7slEtuXCHdRvRLlvNXAN8wsJ9J//k3g15H39gPZZjbkHO4vIl0Xra1ZBXzGzIoj4+a+A6yLzPx5AphhZosi594O5J7h+md6/s8okjj9HPiSux909yeBZ4Afn+O/VWJMCYwArAVOtvq5uzMfdveNhMfB/AQ4DOwiMsDX3XcQLsm+TDiBmAH8+RxiLAM+Ebn+J4FbIuNh2roH2AhsBbYBr0aOEenPXg3sjnRnqWtJpGd9oK1x998D/wQ8Srjich6wGMDdq4G/AH5AuFtpGuHnu6Gd67f7/HfAd4A33H1lq2NfAq43s2s7eA3pQRbuUhRJXGZ2NzDR3f8q3rGISPxEuo8rgKXu/sd4xyPxpQqMiIgkLDO7zsyGRrqXvk54/Fy07mPpZZTAiIhIIruI8FpS1cBNwKIzLLcgvYi6kERERCTpqAIjIiIiSUcJjIiIiCSdpFvIbsSIEV5YWBjvMESklU2bNlW7e0684+gstSciiaej7UnSJTCFhYVs3Lgx3mGISCtmtufsZyUetSciiaej7Ym6kERERCTpKIERERGRpKMERkROU3PiFA+8/C4f/9lLlB+qi3c4IpLEqo6e5L5nd/LJ/1pHdy/bknRjYESk+9WdauKZHfsp3VzJizuraQo5k0dlcbC2noLsgfEOT0SSSHPIef6tA6xat5c/vLGfkMOl54/g2Mkmhgzs2233UQIj0ks1Nof4085qSgOVPL19Pycbm8kb0p/PXTqBRbPzmJI7ON4hikgSqTp6kt9sqOChDeUEj9YzYlAGf33ZeSyZWxCTL0JKYER6EXdn057DlAWCPLGtipoTpxg6sC8fvSCfkll5zC0cTlqaxTtMEUkS7VVbvrFwGldPHUVGeuxGqiiBEekF3tp/nNLNlazZEqTi8En6903jqqmjWFScz2WTcmLayIhI6tl3tJ7fbNzLQxv2Unnk5HvVlsVzxzIuO7NHYohZAmNm9wMLgQPuXhTl/aXAP0Ze1gJfcPctsYpHpLepOnqSNYEgpYEgr1cdI83gkvNzuPPqSVxXlMugfvr+IiId1xxyXnjrICvXlb9Xbblk4giW3zg15tWWaGLZgv0S+Anwq3befwe4zN0Pm9n1wApgfgzjEUl5R+saWftaFaWbK1n/bg3uMGvsUL510zQWzswjJ6tfvEMUkSSz/1g9D22Ib7UlmpglMO7+gpkVnuH9l1q9fAUYE6tYRFJZfWMzv3/9AKWBSp578wCNzc6EEZl86apJlBTnUTgifg1Md+lARdeA+4AbgDrgNnd/tWejFEkdLdWWVevL+cMbB2gOeVyrLdEkSg35s8CT8Q5CJFk0NYd4efchSjcHeWr7PmobmhiZ1Y9PXVTIouJ8ivIHE/6dnjJ+yZkrutcD50d+5gM/QxVdkU6LNrbl85dOYMm8+FZbool7AmNmVxBOYC45wznLgGUABQUFPRSZSGJxd7ZUHKUsUMlvt1Zx8HgDWf3Sub4ol0Wz87lwQjZ9UnQG0dkqukAJ8CsPr5T1ipkNNbPR7l7VIwGKJLH2qi1fv2Eq10xLjGpLNHFNYMxsJvAL4Hp3P9Teee6+gvAYGebMmdO9S/mJJLjdB2spCwQpC1Ty7qE6MvqkccWUHBYV53PFlJH079sn3iEmgnxgb6vXFZFjSmBE2rH/WD2/2bCXB5Og2hJN3BIYMysAHgM+6e5vxSsOkUR04Hg9j2+poixQydaKo5jBRROy+cLl57GgaDRDBnTfapYpIlrpKeqXHVV0pTdrDjkv7DzIqnXvV1sunpid8NWWaGI5jXo1cDkwwswqgG8BfQHc/T+AbwLZwL9H+uqb3H1OrOIRSXTH6xv53Wv7WLMlyJ93VRNymJ43mOU3TOWmWXnkDukf7xATWQUwttXrMUAw2omq6Epv1Lbakp2ZwecuHc+SuQVJO9A/lrOQlpzl/c8Bn4vV/UWSQUNTM8+9eZA1gSDPvr6fhqYQBcMHcvsVEykpzmPiyKx4h5gs1gB3mNmDhAfvHtX4F+ntmkPOi5Fqy+9bVVu+dsMUrp2Wm1TVlmjiPohXpLcJhZx179SwZkslT2yt4lh9E9mZGSyeO5aS2fnMHjs01WYQdVkHKrprCU+h3kV4GvVn4hOpSPztP1bPwxv3snr9+9WWz186gcVzxyZttSUaJTAiPcDd2VF1jLJAkDWBIPuO1TMwow/XTc+lpDiPSyaOIL1Pcn8biqUOVHQduL2HwhFJOKlebYlGCYxIDO2tqaMsUElpIMiuA7WkpxmXTcrh6zdO5eqpIxmYoUdQRM5dS7XlwQ17qTicGmNbOkqtp0g3O1TbwNptVZQGgmzacxiAuYXDuGdRETfMGM3wzIw4RygiySzUaiZRS7Xlw+dl89XrU7faEo0SGJFuUHeqiae376c0UMmLO6tpDjmTR2Vx14LJ3DwrjzHDBsY7RBFJcgeOhVfJbam2DM/M4HOXjGfxvALGp3i1JRolMCLnqLE5xIs7D1IWCPL09v2cbGwmb0h/Pn/pBBbNzmNK7uB4hygiSS4Ucl7cVc2qdXt49vX3qy3/uGAK104fRb/0Ti5kuXIlLF8O5eVQUAD33gtLl8Ym+BhTAiPSCe7Opj2HKQ2EZxAdrmtkyIC+LJqdz6LiPOYWDictRZfzF5Ge07ba0jK2ZfHcLlRbVq6EZcugri78es+e8GtIyiRGCYxIB+zcf5zSQCVlgSAVh0/SLz2Na6aNoqQ4n8sm5fSaPmcRiZ2WasvqdeU8+/p+mkLORRO6UG1pa/ny95OXFnV14eNKYERSR/DISdZsCVIWCPJ61THSDC45P4c7r57EdUW5DOqnx0dEuu7A8Xoe3ljB6vXl741t+R+XjGfxvgATvn1793X3lJd37niCUwss0sqRulOs3baPskAl69+twR1mjR3Kt26axsKZeeRk9Yt3iCKSAtqrtty1YArXTR9Fv4cehL/r5u6egoLwdaIdT0JKYKTXq29s5tnX91O6Ocjzbx2gsdmZkJPJl66aRElxXsqvpSAiPael2vLghnL21rSqtswdy4ScQe+fGIvunnvvPX0MDMDAgeHjSUgJjPRKTc0hXt59iNLNQZ7avo/ahiZGZvXj0xcVUlKcT1H+YC3nLyLdIhRy/rSrmlWtqi0XThjOP1wXqbZEG9sSi+6elsRHs5BEkou7s6XiKGWBSh7fUkV1bQNZ/dK5viiXRbPzuXBCNn00g0hEOuMM05LbVluGDewbvdoSTay6e5YuTdqEpS0lMJLydh+sDe9BtCXIO9UnyOiTxpVTRlJSnMcVU0bSv28XR/aLSO8UZVpyaNlf86fj6azKnNjxaks0KdbdEwtKYCQlHThez+NbqigLVLK14ihmcOH4bP7msgksKBrNkAF94x2iiCS7VuNUDmQO5eEZ1/DgrOvY++4ghg08xGcuLmTJvIKzV1uiSbHunlhQAiMp41h9I0+9to+yQJCX3q4m5DA9bzDLb5jKTbPyyB3SP94hiki8xGAF2lD5Xv5cWMyq4ut5ZuJ8mvqkc+Gerfz9iw+wYMvvu75uSwp198SCEhhJag1NzTz35kHKApU8+/oBTjWFKBg+kNuvmEhJcR4TR2bFO0QRibduXoH2wPF6HtlUwYN/+1+UD8phWN1RPrNpDYu3PMV5NZUwbhx0NXmRs1ICI0knFHLWvVNDWaCStduqOFbfRHZmBkvmjqVkdj6zxw7VDCIReV83TEkOhZw/v13N6vXlPL09PLZl/qhhfOXR+1jw2h/p19wUPlHjVHqMEhhJCu7Ojqpj4cG4gSD7jtUzMKMP103PpaQ4j0smjiC9j5bzF5EoujAl+eDxBh7etJcH1++lvKaOYQP7ctuHC1kyv4DzcgbBhFpY/pbGqcSBEhhJaHtr6iiL7EG080At6WnG5ZNz+PqNU7lm6igGZKhMKyJn0ckpyaGQ89Lbh1i1fs/71Zbxw/nKtZO4bnru6TMXNU4lbpTASMI5VNvAE9uqKN1cyavlRwCYWziMexYVceOM0QzLzIhzhCKSVDo4Jfng8Ybw2JYN5ew5FKXaIglFCYwkhBMNTTyzYz9lgUpe2FlNc8iZkpvFXQsmc/OsPMYMGxjvEEWkJ8RgttCZpiRHHdsyfjhfviZKtUUSihIYiZvG5hAv7jxI6eYgz+zYz8nGZvKHDmDZRyZQUpzHlNzB8Q5REoiZLQDuA/oAv3D377V5fxxwP5AD1AB/5e4VPR6onLtuni10mjZdPQePN/DIc2+/V20ZqmpL0lECIz3K3dm05zBlgSBPbKui5sQphg7sy0cvyGdRcT5zxg0jTcv5Sxtm1gf4KXANUAFsMLM17r6j1Wk/An7l7v9tZlcC3wU+2fPRyjmLxQaGrUQb2zJv/HDuvHoSC4pUbUk2SmCkR7y1/zilm8ODcSuPnKR/3zSumjqKRcX5XDYph4x0zSCSM5oH7HL33QBm9iBQArROYKYBd0b+/kegtEcjlK6LxQaGQHVtw3t7ErVUWz794fAquRNHqtqSrJTASMwEj5xkzZYgpZsreWPfcfqkGRdPHBHuWy7KZVA//d9POiwf2NvqdQUwv805W4CPEe5m+iiQZWbZ7n6oZ0KULuvGDQxDIefl3YdYtb6cp7fvo7FZ1ZZUo98g0q2O1J1i7bZ9lAYqWf9ODQDFY4dy903TuHFmHjlZ/eIcoSSpaP2K3ub13wM/MbPbgBeASqDpAxcyWwYsAyjo6s6+0r26YQPD6trITKL15bwbqbZ86iJVW1JRzBIYM7sfWAgccPeiKO8b4W9KNwB1wG3u/mqs4pHYqW9s5tnX91MWCPLcmwdobHYm5GTy5WsmcfOsPApHZMY7REl+FcDYVq/HAMHWJ7h7ELgFwMwGAR9z96NtL+TuK4AVAHPmzGmbBEk8neMGhu1WWzSTKKXFsgLzS+AnwK/aef964PzIz3zgZ3ywJCwJqqk5xMu7D1G6OchT2/dR29DEyKx+fPqiQkqK8ynKH6zl/KU7bQDON7PxhCsri4FbW59gZiOAGncPAV8jPCNJkk0nFoZTtaV3i1kC4+4vmFnhGU4pITxjwIFXzGyomY1296pYxSRd4+5sqThK6eZKfru1iuraBrL6pXN9US6LZudz4YRs+mgGkcSAuzeZ2R3AU4SnUd/v7tvN7NvARndfA1wOfNfMnHAX0u1xC1hiJmq1pXA4//Pq87m+aLSqLb1IPMfARBuUlw8ogUkwuw/WUhYIUhao5N1DdWT0SePKKSNZNDuPyyePVIMhPcLd1wJr2xz7Zqu/PwI80tNxSc+orm3g0U0VrI5UW4YM6MsnLyzk1vljtet8LxXPBKYjg/LCJ2rQXY87cKyex7dWURaoZGvFUczgwvHZfOHy81hQNJohA/rGO0QRSXGhkPNKpNryVKTaMrdwmKotAsQ3gTnroLwWGnTXM47XN/K71/ZRFgjy0tvVhByK8gez/Iap3DQrj9wh/eMdooj0AociY1tUbZEziWcCswa4I7Ig1XzgqMa/9LyGpmaee/MgZYFKnn39AKeaQhQMH8jtV0ykpDhfA+FE5INisF9RS7Vlpca2SAedNYGJDJxb6e6HO3NhM1tNeFDdCDOrAL4F9AVw9/8g3Jd9A7CL8DTqz3QqcjlnoZCz7p0aygKVPLGtiuP1TWRnZrBk7lhKZucze+zQM88gisVma9IrnGt7Igmkm/crija25a8uHMet8wo4f5SqLdK+jlRgcgnvO/Iq4WmJT0VmDp2Ruy85y/uOZgn0GHdne/AYa7YEWRMIsu9YPZkZfbhuei43F+dxycQRpPfpwHL+sdxsTXqDc2pPJIF0w35F7pGZROtOH9vyxavO54YZqrZIx1hH2o7IonPXEq6SzAF+A/yXu78d2/A+aM6cOb5x48aevm3PiEFlY29NHWWBSkoDQXYdqCU9zbh8cg43F+dzzdRRDMjoZENRWBh9qe9x4+Ddd7sUqyQvM9vk7nM6eK7ak2SWlgbRfm+YQSh0xo8eqm3g0VcrWL1+L+9Un2DIgL7cckG+qi1ymo62Jx0aA+Pubmb7gH2El+YeBjxiZs+4+11dC1WAbq1sHKpt4IltVZRuruTV8iMAzC0cxj2LirhxxmiGZWace5wx2mxNeg+1J0muk/sVtVRbVq/fy1Ov7eNUc4g544bxd1dOVLVFuqQjY2C+CHwaqAZ+AfyDuzeaWRqwE+h9DU4sxoB0sSx7oqGJZ3bspzRQyYs7q2kOOVNys7hrwWRunpXHmGEDuxZfi27cbE16H7UnKaCD+xVFq7YsvbBA1RbpNh2pwIwAbnH3035ruXvIzBbGJqwEFqsxIOdQ2WhsDvHizoOUbg7yzI79nGxsJm9Ifz5/6QRKivOYOnrwucfTnm7YbE16NbUnye4M+xWp2iI9qUNjYBJJ3PusYzUGpIPXDYWcV8sPUxqo5ImtVRyua2TowL7cMGM0i4rzmTNuGGmxXs5fs5Ckjc6MgUkkcW9PUkTNiVM8smnve9WWwf3TueWCMdw6v4BJqrZIJ3XrGJheoaO/lGM1BuQslY239h+ndHMlZYEglUdO0r9vGldPHcWi4nw+MimHjPQOzCDqLp3YbE1EUpO788ruGlavL+d3kWrLh8YN444rJnLjTFVbJPaUwEDnuoViNQYkSlk2+K3v8PiY+ZTe9yKvVx0jzeCS83P4yrWTuHZ6LoP66X8+EelZNSdOvbduy+5IteXW+QWqtkiPUxcSdK5bqG2yA+FKyYoV3VKVOFJ3irXb9lEaqGT9OzUAFI8dyqLiPG6cmUdOVr8u30Oku6kLKbVFq7bMGTeMJfMKVG2RbqcupM7oTLfQGQawnav6xmaefX0/pZuDPP/WARqbnQk5mXz5mkmUFOcxLjvznK8tInKuVG2RRNY7E5i2412GD4dDhz54XnvdQt0wBqSpOcRLbx+iLBDkqe37qG1oYmRWPz59USElxfkU5Q8+83L+IiIx0N7Ylh9dMZEbZ4zu/OKXIjHS+xKYaONd+vaFjAw4der982IwNdjd2VJxlNLNlfx2axXVtQ1k9U/nhhm5LCrOZ/6EbPrEegaRiEgUNSdO8dirFaxaX87ugyfIilRblswrYHKuqi2SeHpfAhNtwbjGRsjOhkGDYjI1ePfBWsoCQdZsCfJO9Qky+qRx5ZSRLJqdx+WTR6r/WETiwj28sevq9eU8uS1cbbmgYCg/+otZqrZIwut9CUx7411qaqC6uttuc+B4PY9vqaIsUMnWiqOYwYXjs/mbyyawoGg0Qwb07bZ7iYh0xuETp3hU1RZJcr0vgYnhUvjH6hv53Wv7WBMI8tLb1YQcivIHs/yGqdw0K4/cIf27fA8RkQ5rNd7PCwpY97XvsXrIZFVbJCX0vgSmm5fCb2hq5rk3D1IWqOTZ1w9wqinEuOyB3H7FREqK85k4clA3BS4i0gmR8X41ns5jc0pYNes6dr+TRVZaJUvmF7J4XkFsthsR6SG9J4FpPfNo+HAYMCDcbXQO411CIeeVdw5RtjnIk69Vcay+iezMDG6dV8DNxXnMHjtUM4hEupmZLQDuA/oAv3D377V5vwD4b2Bo5JyvuvvaHg80Abg76/7l/7L6yr/lyckXcyq9L7Mr3+CHT/yYhXV7GPCdnfEOUaTLekcC03bm0aFD4arLAw90OHFxd3ZUHQsPxg0E2XesnsyMPlw3PZeS2flcfF426X16cDl/kV7EzPoAPwWuASqADWa2xt13tDrtG8Bv3P1nZjYNWAsU9niwcdQytmX1+nLevupOsuprWbzlKW4NPMmU6kjXub5cSYroHQlMtJlHdXXh42dJYPbW1FEWqKQ0EGTXgVrS04zLJ+fw9Runcs3UUeo3FukZ84Bd7r4bwMweBEqA1gmMAy19IkOAYI9GGCfuzvrITKK1r+3jVFN4bMsP1z3Awj+VMqCp4fQPdMN4P5FE0DsSmE5uwHiotoEntlVRurmSV8uPADC3cBj3LCrixhmjGZaZEatIRSS6fGBvq9cVwPw259wNPG1mfwdkAlf3TGjxcVq1JTKTaMncsSyZX8CU3MEw5F1Y/zg0tfpQDNa3EomX3pHAdGDm0YmGJp7ZsZ/SQCUv7qymOeRMyc3irgWTuXlWHmOGDezBgEWkjWj9Hm03clsC/NLd/9nMLgIeMLMidw+ddiGzZcAygIJEqUa0XR28nXF50aotswuG8sOPz2ThzLzTK8Ix2PZEJJH0jgSmnZlHjffcy4tvhPcgembHfk42NpM/dADLPjKBkuK88LcYEUkEFcDYVq/H8MEuos8CCwDc/WUz6w+MAA60PsndVwArILyZY6wC7rBoq4MvWxb+eyTZOFJ3ikc2taq29Etn8dyxLDnbTKJu2PZEJFH1jgSm1TcRLy9n0wWXU/aXd/DEO1nUvLaRoQP78tEL8llUnM+cccNI03L+IolmA3C+mY0HKoHFwK1tzikHrgJ+aWZTgf7AwR6N8ly0M0bPly9nw4evZ/X6cp7YVsWpphDFY4fyg4/PZOHM0QzM6B3Nt0h7es0T8NbVN1Oa/SHWbAlScfgk/Y+ncfXUbBYV5/ORSTlkpGsGkUiicvcmM7sDeIrwFOn73X27mX0b2Ojua4CvAD83szsJdy/d5u7xr7CcTZuxeEf6D+LRoitZPWsBu/7zZbL6pfOJOWO5db7WbRFpLeUTmF0HjnPHqs28se84fdKMiyeO4MvXTOLa6bkM6pfy/3yRlBFZ02Vtm2PfbPX3HcDFPR1XlxUU4Hv2sGHMdFbPuo4nplzCqfQMiqvf4Qcfm8nCWaq2iEST8k/F6CEDGDKgL3ffNI0bZ+aRk9Uv3iGJiADhsS2P3vkjVm8/xK7hY8hqOMEntj7NkjefZ9p3lsPcsWe/iEgvlfIJTGa/dB4atBuWLtFIfBGJO3dn457DrFrXMrZlALPyc/nBc79m4UulDBw9Cr6jNkrkbGKawCTE0t8dGOEvIhJrR+sa31u3ZeeBWrL6pfOXc8Zw67xxTMsbDN8qiXeIIkklZglMwiz93YVVeEVEuqKl2rI6Um1paJlJpLEtIl0Wy6cnMZb+7uQqvCIiXXWk7hSPvVrZptoSXrdlWp5mEol0h1gmMImx9Pfw4eHNG9tKlBU4RSQluDubWo1tUbVFJLZi+UTFf+nvlSvh2LEPHs/I0H4gItIt2lZbBvVL5y/mjGHJvAKm5w2Jd3giKSuWCUz8l/5evhwaGz94PCtL419E5JxFq7bMGjOE739sBgtn5pGpNaZEYi6WT1n8l/5ub5xLTU233UJEeo+jdY08tjk8k+it/eFqy8c/NIZb56vaItLTYpbAJMTS3x3YhVpEpKN+vW4PP3zqTWaNGcL3bpnBTbNUbRGJl5g+eXFf+rudXag1/kVEzsWSeQVcNimHonxVW0TiLbV3MFy6FFasgHHjwCz854oVGv8iIudkeGaGkheRBJH6tc+lS5WwiIiIpJjUrsCIiIhISlICIyIiIklHCYyIiIgkHSUwIiIiknSUwIiIiEjSUQIjIiIiSUcJjIiIiCQdJTAikhTMbIGZvWlmu8zsq1He/7GZBSI/b5nZkW4NYOVKKCyEtLTwnytXduvlRaRzUj/fkpP8AAADl0lEQVSBUaMjkvTMrA/wU+B6YBqwxMymtT7H3e9092J3Lwb+DXis2wJYuTK8LcmePeAe/nPZMrUnInGU2gmMGh2RVDEP2OXuu939FPAgUHKG85cAq7vt7suXn76nGoRfL1/ebbcQkc5J7QRGjY5IqsgH9rZ6XRE59gFmNg4YD/yh2+5eXt654yISc6mdwKjREUkVFuWYt3PuYuARd2+OeiGzZWa20cw2Hjx4sGN3Lyjo3HERibnUTmDU6IikigpgbKvXY4BgO+cu5gzdR+6+wt3nuPucnJycjt393nth4MDTjw0cGD4uInGR2gmMGh2RVLEBON/MxptZBuEkZU3bk8xsMjAMeLlb7750KaxYAePGgVn4zxUrtNO9SBylxzuAmGppXJYvD3cbFRSEkxc1OiJJxd2bzOwO4CmgD3C/u283s28DG929JZlZAjzo7u11L527pUvVdogkkNROYECNjkiKcPe1wNo2x77Z5vXdPRmTiMRPanchiYiISEpSAiMiIiJJx2LRVRxLZnYQ2BPvOETkNOPcvYNTehKH2hORhNSh9iTpEhgRERERdSGJiIhI0lECIyIiIklHCYyIiCQkM6uNdwySuJTAiIiISNJRAiM9yszmmtlWM+tvZplmtt3MiuIdl4gkLgv7oZm9ZmbbzOwTkeNpZvbvkXbkt2a21sw+3uaz6Wa2wcwuj7z+rplpP5kUkPor8UpCcfcNZrYGuAcYAPza3V+Lc1gikthuAYqBWcAIYIOZvQBcDBQCM4CRwOvA/a0/GNmG4jbgETP7IrAAmN9jkUvMKIGRePg24c356oEvxjkWEUl8lwCr3b0Z2G9mzwNzI8cfdvcQsM/M/hjtw5F9sx4AHgcucvdTPRW4xI66kCQehgODgCygf5xjEZHEZ508Hs0M4AgwquvhSCJQAiPxsAL4J2Al8P04xyIiie8F4BNm1sfMcoCPAOuBPwEfi4yFGQVcHu3DZnYLkB353L+a2dCeCVtiSV1I0qPM7FNAk7uvMrM+wEtmdqW7/yHesYlIwvp/wEXAFsCBu9x9n5k9ClwFvAa8BawDjrb+oJmNAL4HXOXue83sJ8B9wKd7MH6JAW0lICIiScvMBrl7rZllE67KXOzu++Idl8SeKjAiIpLMfhvpEsoA/reSl95DFRgRERFJOhrEKyIiIklHCYyIiIgkHSUwIiIiknSUwIiIiEjSUQIjIiIiSUcJjIiIiCSd/w8Dy6/PI7SuFAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1008x144 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def P4():\n",
    "    lr = LinearRegression(fit_intercept=True)\n",
    "    size = np.asarray([60000])\n",
    "    X = np.asarray(train_sizes)[:, np.newaxis]\n",
    "    y = np.asarray(accuracies)[:, np.newaxis]\n",
    "    lr.fit(X, y)\n",
    "    print 'Accuracy for n = 60000 for esitmated function y=a+bX is {0:s}'.format(lr.predict(size.reshape(1, -1)).tolist()[0])\n",
    "    \n",
    "    # The predicted value for n=60000 is greater than 1 which is not possible. \n",
    "    \n",
    "    plt.figure(figsize=(14, 2))\n",
    "  \n",
    "    #plotting the linear function\n",
    "    \n",
    "    ax = plt.subplot(1, 3, 1)\n",
    "    # Turn off tick marks to keep things clean.\n",
    "    plt.setp(ax, xticks=())\n",
    "    \n",
    "    x = np.linspace(X.min(), 60000, 100)[:,np.newaxis]\n",
    "    plt.plot(x, lr.predict(x))\n",
    "    plt.scatter(X, y, color='red')\n",
    "    plt.xlabel(\"x\")\n",
    "    plt.ylabel(\"y\")\n",
    "    plt.title('Linear plot')\n",
    "    \n",
    "    # Applying log-transformation for the training values.\n",
    "  \n",
    "    logX = np.log(X)\n",
    "    size = np.log(size)\n",
    "    lr.fit(logX, y)\n",
    "    print 'Accuracy for n = 60000 for esitmated function y=a+blogX is {0:s}\\n'.format(lr.predict(size.reshape(1, -1)).tolist()[0])\n",
    "    ax = plt.subplot(1, 3, 2)\n",
    "    # Turn off tick marks to keep things clean.\n",
    "    plt.setp(ax, xticks=())\n",
    "    \n",
    "    x = np.linspace(np.log(X.min()), np.log(60000), 100)[:,np.newaxis]\n",
    "    plt.plot(x, lr.predict(x))\n",
    "    plt.scatter(np.log(X), y, color='red')\n",
    "    plt.xlabel(\"log x\")\n",
    "    plt.ylabel(\"y\")\n",
    "    plt.title('Log of X')\n",
    "    \n",
    "\n",
    "P4()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "ANSWER: \n",
    "\n",
    "    The predicted value for n=60000 is 1.24 and that is greater than 1 which is not possible. \n",
    "    \n",
    "    Even applying log transformation to our training values, the predicted value for n=60000 is 1.033 and that is still\n",
    "    greater than 1.\n",
    "    \n",
    "    We are using the size of the data set to predict the accuracy instead of values in the data set. That is something not\n",
    "    right when applying regression here.\n",
    "  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Fit a 1-NN and output a confusion matrix for the dev data. Use the confusion matrix to identify the most confused pair of digits, and display a few example mistakes.\n",
    "\n",
    "- confusion_matrix() produces a confusion matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "confusion matrix when training the model with mini training data\n",
      "\n",
      "[[101   0   1   0   0   0   1   1   2   0]\n",
      " [  0 116   1   0   0   0   0   0   1   0]\n",
      " [  1   4  84   2   2   0   2   4   6   1]\n",
      " [  0   2   0  84   0   6   0   2   3   0]\n",
      " [  0   0   1   0  78   0   0   2   0  11]\n",
      " [  2   0   0   1   1  77   5   0   2   0]\n",
      " [  1   2   1   0   1   2  94   0   1   0]\n",
      " [  0   1   1   0   0   0   0  96   0   4]\n",
      " [  1   5   4   3   1   3   0   1  72   4]\n",
      " [  0   1   0   0   3   2   0   7   0  82]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAATwAAACOCAYAAACscLTMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAFX9JREFUeJzt3XdwVFUbx/HvIgIRg4oiqPgiylhRkVhQUKPYEQUVexQR0LGCRuxAxIoauyCjY1dUxF4GG7ZYAHsbcZQig4oV0FCS3PePnefuJrtZdpPdu3dzfp8ZJrB773JudvPkOec859yI53mIiLigVb4bICISFAU8EXGGAp6IOEMBT0ScoYAnIs5QwBMRZyjgiYgzFPBExBkKeCLijNZB/meRSKTgl3V4nhdZ0zEuXKcL1wi6zkKSznUqwxMRZyjgiYgzFPBExBkKeCLiDAU8EXFGoLO0IslMmDABgKVLlwJQWVlJbW1tPpskLZQyPBFxh+d5gf0BvHT+FBUVeUVFRV6XLl28Ll26pHVOsj8lJSVeSUmJt2zZMq+8vNwrLy9v8mvZn2xeZ3P/lJaWeqWlpZ4ZP3581l47iGu092fhwoXewoULvZqaGq+mpsbbeOONA/n+5fu97NOnj//e1dbWerW1tV5VVZVXVVXlDRkypMVcZ/yfyy+/3Lv88su9uro6r66uzisrKwv0OkPZpT3zzDMBuPHGGwFo3bppzdx+++0BKCoqyk7DQmbcuHGN/nv8+PEBtyZz5513HgCbbLJJnluSH6NGjaKurg7A/7rHHnsA8Nhjj9G1a1cAbrnllvw0MAfWWWcdAAuydOnSJdD/X11aEXFG6DK8DTfckLPPPjvfzQi1VDdesiwv7BleUVER3bt3r/fYjz/+CMCKFSvy0aTADBkyxP8aiURXQ7VqFc094v/dp0+f/DSwCTp06MBFF10EwK233grAH3/8kXCc9bryRRmeiDgjdBleSUkJW2yxBQDffPNN1l73wQcfzNpr5UNpaSmQOG5XqPr3789ee+1V77GqqiogVp7SUo0aNQqIjttZZmdjePH/tgzPvn744YdBNzVtjz76KIcddhgQ7aUBnHXWWflsUlLK8ETEGaHL8ObNm8fvv/8O4Gd62XDMMccAMGnSpKy9ZpAss7NMb01sDC+sY3ljx471x6u+//57AH8MKF3FxcUA1NTUAFBdXZ3FFmbP5ptvDsDUqVMB2HPPPYHoWKx9Dyx769u3LxDN9Ow8m60NE8tE7777bgAGDBjAqlWrAJgyZUqj5w0aNAhIPQ6dS6ELeFtssQWdOnUC4O23327Wa1mXuLq6mvbt2ze7bfnw1ltvAckD3cyZM9f4XNgceuihQHTo4r///gNg4sSJACxZsiSj17DzLGAOHTqUZcuWZbW92WCBbvfddwdiP+x1dXV+oDvhhBMAOP/884Fot9eCij02bdq04Bq9BnYtI0aMAKLXtO+++wLw2WefJRxvv5zs2u3rDz/8kPO2xlOXVkScEboMb968ef5v+lmzZjXrteILjy0ruOmmm5rXwIBYVzRVF7aiogKIZcJhntDYdNNNAbj66qv9x1577TUA7r///oxea4cddgBi7+92220HRAf37TXzLb4ba5MzNjGxaNEiIDpJc/zxx9c77+OPPwaiXUbr7jac3AmDfv361fv3J598wuzZsxs9fv/990/6eDYnJtOhDE9EnBG6DK+4uJh27doBiRMN8+bNa/Lrfv31181uW67Fl56kk9mZZJmdnR+Wsbz+/fsDsPPOOwPR4uKmZNuHH344I0eOrPfYX3/9BcDixYub2crmGz16NBD77O6+++4Jy8es/KZhdhcvWcnKE088AUTHvz766CMgf8vO1l133Xr/bteuHWuttRZAqHe6UYYnIs4IXYY3Z84cXnrpJQCOO+44AJ5//nkAdtpppya/7oIFC5rfuByzGdlUZs6cmVbJSVgyOystuvLKK+s9XlNTw99//w3Ellp169Yt4fxdd90VgN9++w2AkSNHJmwmMX36dAC++uqr7DU8A3369PGvwYqKbfwtvvTEllzZON2QIUP8ouJU51mmZ/+H53kce+yxQGxM2rKroNx8880AnHHGGUB0XNV6YKeccgoQ/VmGWAYOseszrVu39nt0QSwpDF3AAxg+fDgQXZ8HsM8++wDRQeqgBzlzybqd6QS6/fbbD0g/kIUl4J1++ukAbLXVVvUe79ChA19++WWj5zXsziVjQdB+6IJmwWrq1Kn+JEWyFRP2dwtqyZ5LdV46zwXNyn8OPPBAAIYNG+Z342fMmAHAv//+C0TLVDp27AgklqVMnz6du+66C4Dbb7895+1Wl1ZEnBHKDM8q5l9++WUgOlAN8Oabb3LIIYcAyYsbG7rwwguBaBo9YMAAIDagmu89xkpLS9PK7OKPj/8KqctQ7DdopplhtlmXtmFlfV1dXcpq+59//hmIFSM//fTTAPTq1YvBgwcnfc2gffDBB0D0WlLtepKL56y0xbq2+WLDCBdccAFjx44F8N8f2+uuc+fO7L333vXOs/f1ggsu4NVXXw2qucrwRMQdoczwjJWj2DKWsrIyXnjhBSC6jAjgnXfeAWD16tX+zsZbb701AG3btgWimYCNA9p6vzBkeJnItKg4LGN4Nm5lRamWoVdWVqY8b+7cuUBiKdKnn36a5RY2XXy5SbbG4lI9Z8vPPM/zM7ww7aCyfPlyAB5++OGE52zCxUpr/vzzTwB/gjIoyvBExBmRIMdBIpFIk/4zy9TuuOMOhg0bVu+5+AzPiiHtvgDxrARixx13BJpepOp5XmRNx6RznaWlpRnvgNIUDcsA0rWm62zqe9lcn376qf8e2iytLVvLVHPfyyeffBKIft4s07Kfp2TlJfZcqoJjy4SmTp2acF5TS0+y9ZltjiOPPBKAZ599FoBvv/0WyO4OyOlcZ6i7tGblypVAdNcI69LaVkLWVY1EIv4Hw4LZTz/9BETXItokRxiq8SHa5Uw2EdHwuWTnmWTH5HuSwiU2YdCnT5+sdy3DUHqSTT169ADyt0uKUZdWRJxREBmeqa6u9jO8119/HYjtsxWf4dnExMCBA4Fohhfk1He6bKVEw33t1rRpZ6qdVFp6ZheJRPyuXlO769mWi4mDZGUphcx2tDH5WkBQ+N9JEZE0FVSGF8+Kk1Nt623T4xMnTvR3Yw3jfniWlTU3O2u4i0pL9NJLL9GzZ08g/4XHuRQ/hmfrVgtZr1698t0EQBmeiDikYDO8THie55eqnHTSSUD0tnKFyjLBMO9wnCthvVFPtsWP4dkSyTFjxuSzSc1iO+Hke9zViYAHsXtl2oenJQQ868Ja4Bs3blxo71KWLfGD3/aeWvW+bSfWErS0spSGu6TY6qmOHTv6qy6CoC6tiDjDmQyvJUrWtQ37/Wibq7Ky0t+Fw8pybBVNS9KSylJ69+7N+uuvX+8xWxVl62+DUtjfSRGRDDiR4U2ePNnPBmzPrpagYTlLLtfkhsWcOXP8GwK1RLbHXlVVlX97xkIvS1m0aJE/2WTr4q+44gogtkggKMrwRMQZBbFbSpiEYeeJIIR1t5Rs0nsZ48p1KsMTEWco4ImIMxTwRMQZCngi4oxAJy1ERPJJGZ6IOEMBT0ScoYAnIs5QwBMRZyjgiYgzFPBExBkKeCLiDAU8EXGGAp6IOEMBT0ScoYAnIs4IdIt3VzYZdOE6XbhG0HUWEm0AKiISRwFPRJyhgCcizlDAExFnKOCJiDMU8ETEGQp4IuKMgg94JSUllJSU8M8//+B5Hp7nMWHCBCZMmJDvpolkbPLkyUyePJna2loGDx7M4MGD892kFiXQwuNsaNOmDQAnnXQSALfccgsAxcXF2A2JLr74YgDWWmstAC677LKgm5kz7dq1Y++99wZg2bJlAHz44YcJx22++eYAvPrqqwCUl5fzyiuvBNTKptl0000BePfdd5k/fz4ARx99NAB//fVXwvFt27YFoKKiwj/mhhtuCKKpjdp2222B2OeyrKyM33//PePX8TyPyspKIPr9AJr0OvlUXFwMwPHHH+8/Vl1dDcAjjzyScPxFF10E4L+Hp5xyStLjmqPgMzwRkXQVXIbXrl07AO67775Gj2ndOnpZI0aMAKBTp06sWLECgNGjRwNQU1OTy2bmzMSJExk4cCAA/fr1S3jesp67774bgE8++QQg9NkdwDHHHANA9+7d6dq1KwB9+/YF4MUXX0w4fpdddgFimcH8+fP967bsN2j77LMPAIcccggAb7/9NqWlpQAsWbIk7deJRCJ069YNgP/9739A+DM8y9A7deoEwJgxY4D6Gd7q1asBaN++PQD33HOPnwmeffbZAOTy1rHK8ETEGQWX4dlvu3RsuOGGAJx++un+Yzael68MoKnGjx8PwGmnneZnDIsWLUo4btiwYQAMGDAAgKOOOiqQ9jXHQQcdBMBVV13lP2bjOG+88UbC8ZYRNBybtSw+DOrq6gDYZptt/ImHKVOmrPG87bbbDshtlpMLnTt35oUXXgCgV69ejR639tprA7HP6VNPPeV/r2zcOZeU4YmIMwoqw1t77bVTzrj+/PPP9f5t40CFrEePHkBs5vmpp55izpw5SY9t27YtZ5xxBgDPPfccAM8//3wArWyaDTbYAIhlPpa5vf/++9x8881AbFYv3pVXXgnEslhz3XXX5S1zP/nkk4FY9UCrVtFcwrKXdNkMvOd5RCJr3O0o77bccksg+nnbfvvt6z33+uuvA7B48WLKy8uBxJ7VqlWrePTRR+s99t577wHw+OOPZ729BRHwrLykvLy83gBovNWrV3PYYYcBsUkLG7AvRHYN999/PwC//vorEJ2qb8zJJ5/MTjvtBMADDzwAZP4DFyTrmjccpjj11FP5559/kp7Ts2dPDj30UAA/ILz//vsAPPzwwzlq6Zq98847AAwaNAiIfd8z7Zra8WHv0nbu3BmI/WKND3YW1KxcqKqqKuVr2c+3se9dbW1tdhobR11aEXFGQWR41oUZO3Zso8cMHz6cr776Ckg9aFoorr76aiBWlpFOxf2pp57K119/DUSn+8OsuLjYn5hYtWoVECsZ+vHHHxOOt+7vAw884GcTv/32GxD7fOTTggULAJgxYwaQ+WSRlbPEd2P/+++/el/DwEpPbIIiPrOz0qF7770XWHNmB9FJHeuVBEEZnog4I9QZng1Kp/oNfueddwLJl6oUqvbt2/uD37fddhuQvPDW9O/fH4gOeF9yySVA8sH+MNlkk038Nlo5yQcffNDo8VZc3Lt3b398y7LYmTNn5rClTZPpWJwtSYs/77vvvgPwv4bBY489BiT2oubOncuxxx4LwMqVK9N+vW7durH11lunffyuu+4KwOzZs9M+J54yPBFxRigzPFswbjuepJqet7GEsM9qZaKiooL11lsPgJtuuglIPWN1zjnnANHfsjY7G3bLly/3l/dZOYqV2wwdOpRp06YBsTGi4cOH++dauYN9b8Io/jNrS61SsXKUMJeilJWVsfPOO9d7zN6LioqKjDK7VP744w8g+r2wGfxx48YBcMABBwCZLUCIFwkyUKR7Kzhbc2grJVKx9YXxqw6KiooAkqbKNrFxzTXXAPDnn3/6z9k6v1RdpFze8s7Wwc6bN48vvvgCgIMPPrjR460b9PnnnwPRbn38qpLmCOI2jTYR8/TTT2d03l577QUk3yUmE7l4L0eOHAnApEmT7P9g4cKFAOy2225A/TWx9j146KGHAFhnnXX886zLb11aOy/THViydZ2e5yWUOVkp0NChQ9NuT7yDDjooYZ335MmTAVi6dKm/Htc8++yzQCwpatA+3aZRRMSErks7evRoOnTokPbxG220Ub2va9KzZ08geRW37blm3Sgr9g2K7bDRuXNnvw22F9pnn30GREs2rNjTdtOw9Yk//PBDoO1tLpuIsWu1AfFu3bqx/vrrJz0nEolQUlICND/DywVbNWKTTv369fPfJ/s8Wbc1fjWF9bTiu7S2o0jv3r2B/BfSJ+sN2mqmI444IuW51gtp2CW2FTXxzjzzzITHrAD/3HPPTa+xjVCGJyLOCN0Ynm3Tng82PmGTADaWEC+XY3jrrrsuAGeddZZfcGzZgRVnRiIRf3B4+fLlAHTs2BGIjgdZ9tdcQYzhNWbWrFl+VmMs+54/f76/fMvGxpoql++lDaqPGDHCX//dMItLleHFP2fjzbfffjuQ+b542brOG264gfPPPx+I9SrSZWPSqYqM//33XyD2uV69erW/R54t3Vu6dGmj52sMT0QkTugyvJqaGn+niaD98ssvQGz5TDK5zAqSsd+kNq7Zpk0bf6zPfvN36dIFKPwMz0oOZsyY4Wc8tlTuxBNPBGKz7NkQ1HtpZSk2I2slKPHjew0zvCVLlrDvvvsCzS88zuZ12vJOKxPJJiszsp2BMpXOdYZu0qK0tNSvqretzM20adPSevNtwLh79+4Jz02cOBGIrd+MN3369Izbm2tWKmO1SRBbkWCBziYrrGtbaGz3E1uDCbEffBuszmagC5qVWdmEhn0dOXJkvfKVeGVlZaFaYWGuv/56gIR6z4033jjlnQJteygrvZo1a5b/nNVjzp07N5tNTUpdWhFxRugyvPfee4+PPvoIiN2wx1RXV6d1850999wTqJ/h2VbhlpIny/AKhW0Kamy7bNs4sVDssMMOQKxI1zJWgFGjRgGxmxG1RJ06dUpYWWETMfkuQWmM/dzY7jBmwYIFfqaeSvx7bGySIj7DzxVleCLijNBleBAbt7Kv2bDZZpsBiburFiIr1LUp+i+//DKfzWkyKzBtePOW++67z8/sCvV2mukYNGhQwq4qtidg2G/JmE1Brh9WhicizghlhtdUe+yxBxBbpB3Pdt8I+z5x6bC9yCyza+z+D2FWUlJSbwcUiO10PHbs2Bad2VkW17t374Sb/TzzzDN5a1e+fP/994H9Xy0q4FmtWiZrcQtNhw4d/B1Uvvnmmzy3JnNW4zh79my/G2flF7aR6eLFi/PTuByxXW2sDs/qKON3H2lJ25tlKshtvtSlFRFntKgMr+FkR/x6vzDdCKU52rRp49/QptB2R4FYAarnef5ehNbFa2mZXUO2ptbWTNfV1fldWrtpk+SWMjwRcUaLyvBsp2Jbi1dZWQlEl4zZkpiWpJDG8Ow2hFYkvXLlSj/bs50wWiobo7z22muBWDbneZ6/2/N1112Xn8YFzJZIXnHFFUDqW6/mgjI8EXFG6HZLCbugd0tpqFWrVv5vRSu1ycXC+mzvlmIzlNbmSy+91N/IIV/y/V4GJczXWVFR4ZdX2WejqdK6TgW8zIT5w5NN+dwANCh6L2NcuU51aUXEGQp4IuIMBTwRcUagY3giIvmkDE9EnKGAJyLOUMATEWco4ImIMxTwRMQZCngi4gwFPBFxhgKeiDhDAU9EnKGAJyLOUMATEWco4ImIMxTwRMQZCngi4gwFPBFxhgKeiDhDAU9EnKGAJyLOUMATEWco4ImIMxTwRMQZCngi4gwFPBFxxv8B3Y1e/PM/t/oAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 360x144 with 10 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def P5():\n",
    "    # create KNN neigh model from sklearn with k value 1\n",
    "    neigh = KNeighborsClassifier(n_neighbors=1)\n",
    "   \n",
    "    # fit the KNN model using mini training set \n",
    "    neigh.fit(mini_train_data,mini_train_labels)\n",
    "    #prediction for dev data\n",
    "    pred_dev_labels = neigh.predict(dev_data)\n",
    "    #creating confusion matrix for the dev data\n",
    "    confusion_mat = confusion_matrix(dev_labels,pred_dev_labels)\n",
    "    #outputting confusion matrix\n",
    "    print \"confusion matrix when training the model with mini training data\\n\"\n",
    "    print confusion_mat\n",
    "\n",
    "    #Defining the colormap for image properties\n",
    "    plt.rc('image', cmap='gray')\n",
    "    plt.figure(figsize=(5,2))\n",
    "\n",
    "    # for each digit in 0-9 -> the unique labels in Y covers this part\n",
    "    \n",
    "    count = 1\n",
    "    for j in range(len(pred_dev_labels)):\n",
    "        if pred_dev_labels[j] == '9' and dev_labels[j] == '4' and count<=10: \n",
    "            features = np.array(dev_data[j]).reshape(28,28)\n",
    "            # Create subplot by specifying position from 1 to each digit * columns. Add j to get all columns\n",
    "            plt.subplot(2, 5, count)\n",
    "            # Hide axes\n",
    "            plt.axis('off')\n",
    "            plt.imshow(features)\n",
    "            count +=1\n",
    "                            \n",
    "                            \n",
    "P5()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Examples: \n",
    "\n",
    "    Using Mini Training data:\n",
    "        When using the mini training data, the success rate seems to be little low compared to the training data.\n",
    "        For Digit 2, 6 times it is confused with digit 8 and 4 times each with 1 and 7.\n",
    "        For Digit 4, 11 times it is confused with digit 9.\n",
    "        For Digit 8, 22 times it is confused with other digits specially with 1,2 and 9.\n",
    "        For Digit 9, 7 times it is confused with digit 7.\n",
    "        \n",
    "        Digits 2, 4, 8 and 9 seems to be confused more compared to other digits.\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(6) A common image processing technique is to smooth an image by blurring. The idea is that the value of a particular pixel is estimated as the weighted combination of the original value and the values around it. Typically, the blurring is Gaussian -- that is, the weight of a pixel's influence is determined by a Gaussian function over the distance to the relevant pixel.\n",
    "\n",
    "Implement a simplified Gaussian blur by just using the 8 neighboring pixels: the smoothed value of a pixel is a weighted combination of the original value and the 8 neighboring values. Try applying your blur filter in 3 ways:\n",
    "- preprocess the training data but not the dev data\n",
    "- preprocess the dev data but not the training data\n",
    "- preprocess both training and dev data\n",
    "\n",
    "Note that there are Guassian blur filters available, for example in scipy.ndimage.filters. You're welcome to experiment with those, but you are likely to get the best results with the simplified version I described above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy for k-> 8 neighbours: 0.877\n",
      "Accuracy for k-> 8 neighbours: 0.866\n",
      "Accuracy for k-> 8 neighbours: 0.873\n"
     ]
    }
   ],
   "source": [
    "def Gaussian_blur(image,sigma):\n",
    "    #get the nearest 9 neighbors to a pixel from the image\n",
    "    #create a KNN with 9 neighbors\n",
    "    neigh = KNeighborsClassifier(n_neighbors=9)\n",
    "    vector_size = int(np.sqrt(image.shape[0]))\n",
    "    #Take a copy of the image\n",
    "    blurred_image = np.copy(image)\n",
    "    #create a matrix with the vector_size\n",
    "    \n",
    "    \n",
    "def P6():\n",
    "    from scipy.ndimage.filters import gaussian_filter\n",
    "    \n",
    "    # create KNN neigh model from sklearn using 8 neighbors\n",
    "    neigh = KNeighborsClassifier(n_neighbors=8)\n",
    "    \n",
    "    #preprocessing the training data but not dev data\n",
    "    blur_mini_train_data = gaussian_filter(mini_train_data, sigma=0.5)\n",
    "\n",
    "    # fit the KNN model using blurred mini training data and mini training labels set\n",
    "    neigh.fit(blur_mini_train_data,mini_train_labels)\n",
    "    # Report the accuracy on the dev data set\n",
    "    print 'Accuracy for k-> 8 neighbours: {0:.3f}'.format(neigh.score(dev_data,dev_labels)) \n",
    "        \n",
    "    #preprocessing the dev data but not training data\n",
    "    blur_dev_data = gaussian_filter(dev_data, sigma=0.5)\n",
    "    # fit the KNN model using mini training data and mini training labels set\n",
    "    neigh.fit(mini_train_data,mini_train_labels)\n",
    "    # Report the accuracy on the dev data set\n",
    "    print 'Accuracy for k-> 8 neighbours: {0:.3f}'.format(neigh.score(blur_dev_data,dev_labels)) \n",
    "    \n",
    "    #preprocessing the both training data and dev data\n",
    "    # Report the accuracy on the dev data set\n",
    "     # fit the KNN model using blurred mini training data and mini training labels set\n",
    "    neigh.fit(blur_mini_train_data,mini_train_labels)\n",
    "    print 'Accuracy for k-> 8 neighbours: {0:.3f}'.format(neigh.score(blur_dev_data,dev_labels)) \n",
    "    \n",
    "Gaussian_blur(X[0],0.5)\n",
    "    \n",
    "P6()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "ANSWER: \n",
    "\n",
    "    Accuracy seems to be improved when we pre-process the training data but not the dev data\n",
    "    However accuracy is reduced when we only pre-process the dev data but not the training data\n",
    "    When both data sets are pre-processed, accuracy seems to be improved as well. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(7) Fit a Naive Bayes classifier and report accuracy on the dev data. Remember that Naive Bayes estimates P(feature|label). While sklearn can handle real-valued features, let's start by mapping the pixel values to either 0 or 1. You can do this as a preprocessing step, or with the binarize argument. With binary-valued features, you can use BernoulliNB. Next try mapping the pixel values to 0, 1, or 2, representing white, grey, or black. This mapping requires MultinomialNB. Does the multi-class version improve the results? Why or why not?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy using Bernoulli NB for dev set-> 0.831\n",
      "Accuracy using Multinomial NB for dev set-> 0.815\n"
     ]
    }
   ],
   "source": [
    "def P7():\n",
    "    #For Binary Valued features using BernoulliNB\n",
    "    Binary_Bernoulli_model = BernoulliNB(binarize=0.333)\n",
    "    #Using Mini Training data to fit the model\n",
    "    Binary_Bernoulli_model.fit(mini_train_data, mini_train_labels)\n",
    "    print 'Accuracy using Bernoulli NB for dev set-> {0:.3f}'.format(Binary_Bernoulli_model.score(dev_data, dev_labels))\n",
    "\n",
    "    #For multinominal values, I am dividing the values  into 3 parts \n",
    "    #0 for values less than 0.333\n",
    "    #1 for values between 0.333 and 0.667\n",
    "    #2 for values greater than 0.667\n",
    "    Multinomial_model = MultinomialNB()\n",
    "    changed_mini_train_data_for_multinomial = np.where(mini_train_data < 0.333,0,\n",
    "                                      np.where((mini_train_data > 0.333) & (mini_train_data < 0.667), 1, 2))\n",
    "    changed_dev_data_for_multinomial = np.where(dev_data < 0.333,0,\n",
    "                                      np.where((dev_data > 0.333) & (dev_data < 0.667), 1, 2))\n",
    "    \n",
    "    #Using Transformed Mini Training data to fit the model\n",
    "    Multinomial_model.fit(changed_mini_train_data_for_multinomial, mini_train_labels)\n",
    "    \n",
    "    print 'Accuracy using Multinomial NB for dev set-> {0:.3f}'.format(\n",
    "        Multinomial_model.score(changed_dev_data_for_multinomial, dev_labels))\n",
    "\n",
    "P7()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "ANSWER: Does the multi-class version improve the results? Why or why not?\n",
    "\n",
    "    The multi-class version doesn't improve the results but they are not farther apart. It may fall more if we map the pixel values to more than 3 classes. Accuracy usually improves when the categories are less.\n",
    "    \n",
    "    I did try with binarize value 0.25 for Binomial NB, which gave accuracy 0.826 and 0.25,0.75,1 threshold limits for Multinomial NB which gave accuracy to 0.809. \n",
    "    \n",
    "    This looks less when compared to binarize threshold values 0.333 and 0.667 and 1."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(8) Use GridSearchCV to perform a search over values of alpha (the Laplace smoothing parameter) in a Bernoulli NB model. What is the best value for alpha? What is the accuracy when alpha=0? Is this what you'd expect?\n",
    "\n",
    "- Note that GridSearchCV partitions the training data so the results will be a bit different than if you used the dev data for evaluation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " For {'alpha': 0.0} \t Accuracy -> 0.814 \n",
      " For {'alpha': 0.0001} \t Accuracy -> 0.820 \n",
      " For {'alpha': 0.001} \t Accuracy -> 0.825 \n",
      " For {'alpha': 0.01} \t Accuracy -> 0.824 \n",
      " For {'alpha': 0.1} \t Accuracy -> 0.820 \n",
      " For {'alpha': 0.5} \t Accuracy -> 0.812 \n",
      " For {'alpha': 1.0} \t Accuracy -> 0.816 \n",
      " For {'alpha': 2.0} \t Accuracy -> 0.806 \n",
      " For {'alpha': 10.0} \t Accuracy -> 0.765 \n"
     ]
    }
   ],
   "source": [
    "def P8(alphas):\n",
    "    #creating a GridSearchCV using BernoulliNB estimator and passing the \n",
    "    nb = GridSearchCV(BernoulliNB(binarize=0.333), alphas)\n",
    "    nb.fit(mini_train_data, mini_train_labels)\n",
    "    means = nb.cv_results_['mean_test_score']\n",
    "    stds = nb.cv_results_['std_test_score']\n",
    "    for params, accuracy in zip(nb.cv_results_['params'], means ):\n",
    "        print(\" For %r \\t Accuracy -> %0.03f \"\n",
    "              % (params, accuracy))\n",
    "    return nb\n",
    "\n",
    "alphas = {'alpha': [0.0, 0.0001, 0.001, 0.01, 0.1, 0.5, 1.0, 2.0, 10.0]}\n",
    "nb = P8(alphas)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'alpha': 0.001}\n"
     ]
    }
   ],
   "source": [
    "print nb.best_params_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "ANSWER: What is the best value for alpha? What is the accuracy when alpha=0? Is this what you'd expect?\n",
    "\n",
    "    The best value for alpha is 0.001\n",
    "    The accuracy at alpha=0 is 0.814 little less compared to the best value. When alpha=0, we are suggesting the Model, that \n",
    "    no smoothing is required, and so, it makes sense that the accuracy fall to an extent. And we are only looking \n",
    "    at P(label) instead of P(label|feature). Yes, I would expect it to be that way."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(9) Try training a model using GuassianNB, which is intended for real-valued features, and evaluate on the dev data. You'll notice that it doesn't work so well. Try to diagnose the problem. You should be able to find a simple fix that returns the accuracy to around the same rate as BernoulliNB. Explain your solution.\n",
    "\n",
    "Hint: examine the parameters estimated by the fit() method, theta\\_ and sigma\\_."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GaussianNB(priors=None, var_smoothing=1e-09)\n",
      "Accuracy using Gaussian NB for dev set-> 0.593\n",
      "[0.17307555 0.07855835 0.15131058 0.14127578 0.12075603 0.13209409\n",
      " 0.13926231 0.11201644 0.14564696 0.12939563]\n",
      "[0.06069296 0.02779091 0.06625564 0.05829797 0.05256061 0.05983694\n",
      " 0.05189687 0.04626576 0.05681175 0.04956135]\n",
      "Accuracy using Gaussian NB for dev set using with sigma 0.000 -> 0.106 \n",
      "Accuracy using Gaussian NB for dev set using with sigma 0.050 -> 0.803 \n",
      "Accuracy using Gaussian NB for dev set using with sigma 0.100 -> 0.802 \n",
      "Accuracy using Gaussian NB for dev set using with sigma 0.150 -> 0.802 \n",
      "Accuracy using Gaussian NB for dev set using with sigma 0.200 -> 0.802 \n",
      "Accuracy using Gaussian NB for dev set using with sigma 0.250 -> 0.802 \n",
      "Accuracy using Gaussian NB for dev set using with sigma 0.300 -> 0.802 \n",
      "Accuracy using Gaussian NB for dev set using with sigma 0.350 -> 0.801 \n",
      "Accuracy using Gaussian NB for dev set using with sigma 0.400 -> 0.801 \n",
      "Accuracy using Gaussian NB for dev set using with sigma 0.450 -> 0.801 \n",
      "Accuracy using Gaussian NB for dev set using with sigma 0.500 -> 0.801 \n",
      "Accuracy using Gaussian NB for dev set using with sigma 0.550 -> 0.801 \n",
      "Accuracy using Gaussian NB for dev set using with sigma 0.600 -> 0.801 \n",
      "Accuracy using Gaussian NB for dev set using with sigma 0.650 -> 0.801 \n",
      "Accuracy using Gaussian NB for dev set using with sigma 0.700 -> 0.801 \n",
      "Accuracy using Gaussian NB for dev set using with sigma 0.750 -> 0.801 \n",
      "Accuracy using Gaussian NB for dev set using with sigma 0.800 -> 0.799 \n",
      "Accuracy using Gaussian NB for dev set using with sigma 0.850 -> 0.799 \n",
      "Accuracy using Gaussian NB for dev set using with sigma 0.900 -> 0.799 \n",
      "Accuracy using Gaussian NB for dev set using with sigma 0.950 -> 0.799 \n",
      "Accuracy using for dev set when adding random noise with sigma 0.050 -> 0.801 \n",
      "Accuracy using for dev set when adding random noise with sigma 0.100 -> 0.817 \n",
      "Accuracy using for dev set when adding random noise with sigma 0.150 -> 0.818 \n",
      "Accuracy using for dev set when adding random noise with sigma 0.200 -> 0.796 \n",
      "Accuracy using for dev set when adding random noise with sigma 0.250 -> 0.786 \n",
      "Accuracy using for dev set when adding random noise with sigma 0.300 -> 0.770 \n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAskAAAEICAYAAACtaWlhAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAGRhJREFUeJzt3Xu0nXWd3/H3RwJq5RIkgTIhY1Si46VVaMpg7cyoWMtlJMxa4sJqiax0sjpe6q11cMbqMJcuaEcZWDOiKNbgFaRVMoqjLIRldQk1DIpysUREkkkkUSCIDDrgt3/s3xk3v5zk7CRnn5PkvF9rnbWf5/f8nr2/z/7l8tnP/p3nSVUhSZIk6ZceN9sFSJIkSXsaQ7IkSZLUMSRLkiRJHUOyJEmS1DEkS5IkSR1DsiRJktQxJEuaFUluSfKi2a5jNiX5nSTrkzyY5JjZrmdHkrwoyYYZeq0HkzxtJl5LkrbHkCxp2iW5K8lLu7bXJvnqxHpVPaeqrpvieZYkqSTzxlTqbPtz4A1VdWBV3TTbxewp2vtx52zXIWluMyRLmrP2gPD9FOCWcTxxkv3G8bySNFcYkiXNiuGzzUmOS7I2yQNJ7kny3tbtK+3x/vYV/AuSPC7JO5P8IMnmJJcmOWToec9s236c5L92r/NHSa5I8rEkDwCvba/99ST3J9mU5C+THDD0fJXkdUnuSPKTJH+S5OltnweSXD7cvzvGSWtN8vgkDwL7Ad9K8r3t7H9Bm47xQJIbk/zGDt7PjyS5KMlVSX4KvDjJKUluavuvT/JHQ/0nztKvSHJ3kh8l+cOh7U9sz3lfkluBf9m93rOSXNfet1uSnNrV8r4kX2jj9rUk/zTJX7Tnu31H00taXUfvynMlOTvJ99pY3Zrkd4a27ZfkPe1Yv5/kDcPfVLSxuaT9Ofi7JH/qhw1p7jIkS9oTXABcUFUHA08HLm/tv9ke57ev4L8OvLb9vBh4GnAg8JcASZ4NvA94NXAkcAiwqHut5cAVwHzg48CjwFuABcALgBOA13X7nAj8C+B44O3Axe01FgPPBV61neOatNaq+llVHdj6PK+qnr6d/b8BPB94MvAJ4NNJnrCdvgD/Dvgz4CDgq8BPgTPbsZ4C/F6S07p9/jXwzHbc70ryrNb+bgZj8XTg3wIrJnZIsj/w18CXgMOBNwIfT/LMoed9JfBOBu/rz4CvA3/b1q8A3svodua5vgf8BoOxPwf4WJIj27bfBU5i8J4eC/TvxWrgEeBo4BjgZcB/2Ik6Je1DDMmSxuWz7Szj/UnuZxBet+cfgKOTLKiqB6vq+h30fTXw3qq6s6oeBN4BnNHOBr4C+Ouq+mpV/Rx4F1Dd/l+vqs9W1S+q6u+r6saqur6qHqmqu4APAL/V7XNeVT1QVbcA3wG+1F5/K/AFBoFqZ2udUlV9rKp+3Gp7D/B4BoF2e66sqq+1Y3u4qq6rqm+39ZuBT05ybOe09+FbwLeA57X2VwJ/VlX3VtV64MKhfY5nEPjPraqfV9WXgc/x2A8Ln2nv7cPAZ4CHq+rSqnoUuIztv2eTGfm5qurTVbWxHfNlwB3AcUPHdEFVbaiq+4BzJ/ZLcgSDAP3mqvppVW0GzgfO2Ik6Je1DDMmSxuW0qpo/8cO2Z2eHrQSeAdye5BtJfnsHfX8F+MHQ+g+AecARbdv6iQ1V9RDw427/9cMrSZ6R5HNJftimYPw3Bmcoh90ztPz3k6wfyOR2VOuUkrwtyW1JtrYPGodMUtuw/th+Pcm1SbYk2Qr8x0n2/+HQ8kP88lge8152x/ErwPqq+kW3ffis/a6+Z5MZ+bkymG7zzaEPZ8/ll8fcH9Pw8lOA/YFNQ/t+gMGZcklzkCFZ0qyrqjuq6lUMAsl5wBVJnsS2Z4EBNjIINBN+lcFX5PcAm4CjJjYkeSJwWP9y3fpFwO3A0jbd4w+A7PrRjFzrDrX5x7/P4Oznoe2DxtYpauuP7RPAGmBxVR0CvH+K/YdtYjCdZLj2CRuBxUke123/uxGfeyySPAX4IPAG4LD2nn2HXx7zY/588NjjW89gKseCoQ93B1fVc2agdEl7IEOypFmX5DVJFrYzk/e35keBLcAvGMznnfBJ4C1JnprkQAZnfi+rqkcYzE99eZJ/1X6Z7hymDoUHAQ8ADyb5NeD3pu3AdlzrVA5iEKi3APOSvAs4eCdf/yDg3qp6OMlxDOYsj+py4B1JDk1yFIN5xxNuYDDf+e1J9s/getcvBz61k/VNt4kPVlsAkpzF4EzyhMuBNyVZlGQ+gw8hAFTVJgZzrN+T5OAMfuny6Un66SmS5ghDsqQ9wYnALRlc8eEC4Iw2p/YhBr+I9rX2FfjxwIeBjzK48sX3gYdpAa7NGX4jg7C2CfgJsJnBGcLt+c8MwuNPGJyFvGwaj2u7tY7giwzmO/8/BlMZHqabTjGC1wF/nOQnDOZnXz5F/2HntNf9PoPw+NGJDW2+96kM5vD+iMF88zOr6vadrG9aVdWtwHsY/GLfPcA/A7421OWDDI7lZuAm4CoGH0QebdvPBA4AbgXuY/Ch60gkzUmpmuzbTEna+7Wzt/czmErx/dmuR3uWJCcB76+qp0zZWdKc45lkSfuUJC9P8k/anOY/B74N3DW7VWlP0K79fHKSeUkWMbjM3Wdmuy5JeyZDsqR9zXIGv1i2EVjKYOqGX5kJBvPTz2EwleIm4DYG01AkaRtOt5AkSZI6nkmWJEmSOiPd9WncFixYUEuWLJntMiRJkrSPu/HGG39UVQun6rdHhOQlS5awdu3a2S5DkiRJ+7gkP5i6l9MtJEmSpG0YkiVJkqSOIVmSJEnqGJIlSZKkjiFZkiRJ6hiSJUmSpI4hWZIkSeoYkiVJkqSOIVmSJEnq7BF33JtNS87+/GyXMKPuOveU2S5BkiRpj+eZZEmSJKljSJYkSZI6I4XkJPOTXJHk9iS3JXlBkicnuTrJHe3x0NY3SS5Msi7JzUmOHe8hSJIkSdNr1DPJFwB/U1W/BjwPuA04G7imqpYC17R1gJOApe1nFXDRtFYsSZIkjdmUITnJwcBvApcAVNXPq+p+YDmwunVbDZzWlpcDl9bA9cD8JEdOe+WSJEnSmIxyJvlpwBbgfya5KcmHkjwJOKKqNgG0x8Nb/0XA+qH9N7S2x0iyKsnaJGu3bNmyWwchSZIkTadRQvI84Fjgoqo6Bvgpv5xaMZlM0lbbNFRdXFXLqmrZwoULRypWkiRJmgmjhOQNwIaquqGtX8EgNN8zMY2iPW4e6r94aP+jgI3TU64kSZI0flOG5Kr6IbA+yTNb0wnArcAaYEVrWwFc2ZbXAGe2q1wcD2ydmJYhSZIk7Q1GvePeG4GPJzkAuBM4i0HAvjzJSuBu4PTW9yrgZGAd8FDrK0mSJO01RgrJVfVNYNkkm06YpG8Br9/NuiRJkqRZ4x33JEmSpI4hWZIkSeoYkiVJkqSOIVmSJEnqGJIlSZKkjiFZkiRJ6hiSJUmSpI4hWZIkSeoYkiVJkqSOIVmSJEnqGJIlSZKkjiFZkiRJ6hiSJUmSpI4hWZIkSeoYkiVJkqSOIVmSJEnqGJIlSZKkjiFZkiRJ6hiSJUmSpI4hWZIkSeoYkiVJkqSOIVmSJEnqGJIlSZKkjiFZkiRJ6hiSJUmSpM5IITnJXUm+neSbSda2ticnuTrJHe3x0NaeJBcmWZfk5iTHjvMAJEmSpOm2M2eSX1xVz6+qZW39bOCaqloKXNPWAU4ClrafVcBF01WsJEmSNBN2Z7rFcmB1W14NnDbUfmkNXA/MT3LkbryOJEmSNKNGDckFfCnJjUlWtbYjqmoTQHs8vLUvAtYP7buhtT1GklVJ1iZZu2XLll2rXpIkSRqDeSP2e2FVbUxyOHB1ktt30DeTtNU2DVUXAxcDLFu2bJvtkiRJ0mwZ6UxyVW1sj5uBzwDHAfdMTKNoj5tb9w3A4qHdjwI2TlfBkiRJ0rhNGZKTPCnJQRPLwMuA7wBrgBWt2wrgyra8BjizXeXieGDrxLQMSZIkaW8wynSLI4DPJJno/4mq+psk3wAuT7ISuBs4vfW/CjgZWAc8BJw17VVLkiRJYzRlSK6qO4HnTdL+Y+CESdoLeP20VCdJkiTNAu+4J0mSJHUMyZIkSVLHkCxJkiR1DMmSJElSx5AsSZIkdQzJkiRJUseQLEmSJHUMyZIkSVLHkCxJkiR1DMmSJElSx5AsSZIkdQzJkiRJUseQLEmSJHUMyZIkSVLHkCxJkiR1DMmSJElSx5AsSZIkdQzJkiRJUseQLEmSJHUMyZIkSVLHkCxJkiR1DMmSJElSx5AsSZIkdQzJkiRJUmfkkJxkvyQ3JflcW39qkhuS3JHksiQHtPbHt/V1bfuS8ZQuSZIkjcfOnEl+E3Db0Pp5wPlVtRS4D1jZ2lcC91XV0cD5rZ8kSZK01xgpJCc5CjgF+FBbD/AS4IrWZTVwWlte3tZp209o/SVJkqS9wqhnkv8CeDvwi7Z+GHB/VT3S1jcAi9ryImA9QNu+tfV/jCSrkqxNsnbLli27WL4kSZI0/aYMyUl+G9hcVTcON0/StUbY9suGqourallVLVu4cOFIxUqSJEkzYd4IfV4InJrkZOAJwMEMzizPTzKvnS0+CtjY+m8AFgMbkswDDgHunfbKJUmSpDGZ8kxyVb2jqo6qqiXAGcCXq+rVwLXAK1q3FcCVbXlNW6dt/3JVbXMmWZIkSdpT7c51kn8feGuSdQzmHF/S2i8BDmvtbwXO3r0SJUmSpJk1ynSLf1RV1wHXteU7geMm6fMwcPo01CZJkiTNCu+4J0mSJHUMyZIkSVLHkCxJkiR1DMmSJElSx5AsSZIkdQzJkiRJUseQLEmSJHUMyZIkSVLHkCxJkiR1DMmSJElSx5AsSZIkdQzJkiRJUseQLEmSJHUMyZIkSVLHkCxJkiR1DMmSJElSx5AsSZIkdQzJkiRJUseQLEmSJHUMyZIkSVLHkCxJkiR1DMmSJElSx5AsSZIkdQzJkiRJUmfKkJzkCUn+b5JvJbklyTmt/alJbkhyR5LLkhzQ2h/f1te17UvGewiSJEnS9BrlTPLPgJdU1fOA5wMnJjkeOA84v6qWAvcBK1v/lcB9VXU0cH7rJ0mSJO01pgzJNfBgW92//RTwEuCK1r4aOK0tL2/rtO0nJMm0VSxJkiSN2UhzkpPsl+SbwGbgauB7wP1V9UjrsgFY1JYXAesB2vatwGHTWbQkSZI0TiOF5Kp6tKqeDxwFHAc8a7Ju7XGys8bVNyRZlWRtkrVbtmwZtV5JkiRp7Hbq6hZVdT9wHXA8MD/JvLbpKGBjW94ALAZo2w8B7p3kuS6uqmVVtWzhwoW7Vr0kSZI0BqNc3WJhkvlt+YnAS4HbgGuBV7RuK4Ar2/Katk7b/uWq2uZMsiRJkrSnmjd1F44EVifZj0GovryqPpfkVuBTSf4UuAm4pPW/BPhoknUMziCfMYa6JUmSpLGZMiRX1c3AMZO038lgfnLf/jBw+rRUJ0mSJM0C77gnSZIkdQzJkiRJUseQLEmSJHUMyZIkSVLHkCxJkiR1DMmSJElSx5AsSZIkdQzJkiRJUseQLEmSJHUMyZIkSVLHkCxJkiR1DMmSJElSx5AsSZIkdQzJkiRJUseQLEmSJHUMyZIkSVLHkCxJkiR1DMmSJElSx5AsSZIkdQzJkiRJUseQLEmSJHUMyZIkSVLHkCxJkiR1DMmSJElSx5AsSZIkdaYMyUkWJ7k2yW1Jbknyptb+5CRXJ7mjPR7a2pPkwiTrktyc5NhxH4QkSZI0nUY5k/wI8LaqehZwPPD6JM8GzgauqaqlwDVtHeAkYGn7WQVcNO1VS5IkSWM0ZUiuqk1V9bdt+SfAbcAiYDmwunVbDZzWlpcDl9bA9cD8JEdOe+WSJEnSmOzUnOQkS4BjgBuAI6pqEwyCNHB467YIWD+024bW1j/XqiRrk6zdsmXLzlcuSZIkjcnIITnJgcD/At5cVQ/sqOskbbVNQ9XFVbWsqpYtXLhw1DIkSZKksRspJCfZn0FA/nhV/e/WfM/ENIr2uLm1bwAWD+1+FLBxesqVJEmSxm+Uq1sEuAS4rareO7RpDbCiLa8ArhxqP7Nd5eJ4YOvEtAxJkiRpbzBvhD4vBP498O0k32xtfwCcC1yeZCVwN3B623YVcDKwDngIOGtaK5YkSZLGbMqQXFVfZfJ5xgAnTNK/gNfvZl2SJEnSrPGOe5IkSVLHkCxJkiR1DMmSJElSx5AsSZIkdQzJkiRJUseQLEmSJHUMyZIkSVLHkCxJkiR1DMmSJElSx5AsSZIkdQzJkiRJUseQLEmSJHUMyZIkSVLHkCxJkiR1DMmSJElSx5AsSZIkdQzJkiRJUseQLEmSJHUMyZIkSVLHkCxJkiR1DMmSJElSx5AsSZIkdQzJkiRJUseQLEmSJHWmDMlJPpxkc5LvDLU9OcnVSe5oj4e29iS5MMm6JDcnOXacxUuSJEnjMMqZ5I8AJ3ZtZwPXVNVS4Jq2DnASsLT9rAIump4yJUmSpJkzZUiuqq8A93bNy4HVbXk1cNpQ+6U1cD0wP8mR01WsJEmSNBN2dU7yEVW1CaA9Ht7aFwHrh/ptaG3bSLIqydoka7ds2bKLZUiSJEnTb7p/cS+TtNVkHavq4qpaVlXLFi5cOM1lSJIkSbtu3i7ud0+SI6tqU5tOsbm1bwAWD/U7Cti4OwVKkiTty5ac/fnZLmHG3XXuKbNdwpR29UzyGmBFW14BXDnUfma7ysXxwNaJaRmSJEnS3mLKM8lJPgm8CFiQZAPwbuBc4PIkK4G7gdNb96uAk4F1wEPAWWOoWZIkSRqrKUNyVb1qO5tOmKRvAa/f3aIkSZKk2eQd9yRJkqSOIVmSJEnqGJIlSZKkjiFZkiRJ6hiSJUmSpI4hWZIkSeoYkiVJkqSOIVmSJEnqGJIlSZKkjiFZkiRJ6hiSJUmSpI4hWZIkSeoYkiVJkqSOIVmSJEnqGJIlSZKkjiFZkiRJ6hiSJUmSpI4hWZIkSeoYkiVJkqSOIVmSJEnqGJIlSZKkjiFZkiRJ6hiSJUmSpM682S5AkiRpwpKzPz/bJUiAIVnSPsL/WOeGu849ZbZLmHH+2ZZmx1hCcpITgQuA/YAPVdW543gdaRT+ByNJknbWtIfkJPsBfwX8G2AD8I0ka6rq1ul+LUnS3OKHXkkzZRxnko8D1lXVnQBJPgUsBwzJewD/g5EkSZraOELyImD90PoG4Nf7TklWAava6oNJvjuGWkaxAPjRLL22ZoZjPDc4znOD47zvc4zngJw3q+P8lFE6jSMkZ5K22qah6mLg4jG8/k5Jsraqls12HRofx3hucJznBsd53+cYzw17wziP4zrJG4DFQ+tHARvH8DqSJEnSWIwjJH8DWJrkqUkOAM4A1ozhdSRJkqSxmPbpFlX1SJI3AF9kcAm4D1fVLdP9OtNo1qd8aOwc47nBcZ4bHOd9n2M8N+zx45yqbaYLS5IkSXPaOKZbSJIkSXs1Q7IkSZLUmRMhOcmJSb6bZF2SsyfZ/vgkl7XtNyRZMvNVaneNMM5vTXJrkpuTXJNkpOskas8y1TgP9XtFkkqyR19iSNsaZYyTvLL9fb4lySdmukbtvhH+zf7VJNcmuan9u33ybNSpXZfkw0k2J/nOdrYnyYXtz8DNSY6d6Rp3ZJ8PyUO3yT4JeDbwqiTP7rqtBO6rqqOB84HzZrZK7a4Rx/kmYFlV/XPgCuC/z2yV2l0jjjNJDgL+E3DDzFao3TXKGCdZCrwDeGFVPQd484wXqt0y4t/ldwKXV9UxDK6U9b6ZrVLT4CPAiTvYfhKwtP2sAi6agZpGts+HZIZuk11VPwcmbpM9bDmwui1fAZyQZLKbomjPNeU4V9W1VfVQW72ewTW8tXcZ5e8zwJ8w+BD08EwWp2kxyhj/LvBXVXUfQFVtnuEatftGGecCDm7Lh+A9F/Y6VfUV4N4ddFkOXFoD1wPzkxw5M9VNbS6E5Mluk71oe32q6hFgK3DYjFSn6TLKOA9bCXxhrBVpHKYc5yTHAIur6nMzWZimzSh/l58BPCPJ15Jcn2RHZ6q0ZxplnP8IeE2SDcBVwBtnpjTNoJ39v3tGjeO21HuaUW6TPdKttLVHG3kMk7wGWAb81lgr0jjscJyTPI7BlKnXzlRBmnaj/F2ex+Dr2Rcx+Ebo/yR5blXdP+baNH1GGedXAR+pqvckeQHw0TbOvxh/eZohe3T+mgtnkke5TfY/9kkyj8HXOjv6ekB7npFuh57kpcAfAqdW1c9mqDZNn6nG+SDgucB1Se4CjgfW+Mt7e5VR/82+sqr+oaq+D3yXQWjW3mOUcV4JXA5QVV8HngAsmJHqNFNG+r97tsyFkDzKbbLXACva8iuAL5d3WdnbTDnO7Wv4DzAIyM5h3DvtcJyramtVLaiqJVW1hMHc81Orau3slKtdMMq/2Z8FXgyQZAGD6Rd3zmiV2l2jjPPdwAkASZ7FICRvmdEqNW5rgDPbVS6OB7ZW1abZLmrCPj/dYnu3yU7yx8DaqloDXMLga5x1DM4gnzF7FWtXjDjO/wM4EPh0+73Mu6vq1FkrWjttxHHWXmzEMf4i8LIktwKPAv+lqn48e1VrZ404zm8DPpjkLQy+gn+tJ7D2Lkk+yWBa1II2t/zdwP4AVfV+BnPNTwbWAQ8BZ81OpZPzttSSJElSZy5Mt5AkSZJ2iiFZkiRJ6hiSJUmSpI4hWZIkSeoYkiVJkqSOIVmSJEnqGJIlSZKkzv8HTZYypIruOsoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 864x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def P9():\n",
    "    gnb = GaussianNB()\n",
    "    print gnb\n",
    "    gnb.fit(mini_train_data, mini_train_labels)\n",
    "    print 'Accuracy using Gaussian NB for dev set-> {0:.3f}'.format(gnb.score(dev_data, dev_labels))\n",
    "    \n",
    "    #The theta_ gives mean of each feature per class\n",
    "    print gnb.theta_.mean(axis=1)\n",
    "    #The sigma_ gives mean of each feature per class\n",
    "    print gnb.sigma_.mean(axis=1)\n",
    "\n",
    "    #For a Gaussian NB to work accurately, we need the data to be normally distributed. \n",
    "    # check the normality for an example image\n",
    "    image_example = X[0]\n",
    "    plt.figure(figsize=(12, 4))\n",
    "    plt.subplot(1,1,1)\n",
    "    fig = plt.hist(image_example)\n",
    "    plt.title('Histogram of a random image')\n",
    "    \n",
    "    #we can see that the data is not normally distributed as most values are either 0 or 1, lets change some variance \n",
    "    #and see if the Gaussian Classifier improves the accuracy.\n",
    "    \n",
    "    new_sigma_values = np.arange(0,1,0.05)\n",
    "    \n",
    "    for i in new_sigma_values:\n",
    "        for digit in range(10):\n",
    "            for feature in range(X.shape[1]):\n",
    "                gnb.sigma_[digit][feature] = i\n",
    "            \n",
    "        print 'Accuracy using Gaussian NB for dev set using with sigma {0:.3f} -> {1:.3f} '.format(i,gnb.score(dev_data, dev_labels))\n",
    "     \n",
    "    \n",
    "    # Observing the accuracies by altering sigma values, we found 0.05,0.1,0.15,0.2,0.25,0.3 \n",
    "    \n",
    "    updated_sigma_values = [0.05,0.1,0.15,0.2,0.25,0.3]\n",
    "    #lets trying adding normal random noise to the mini training data using the updated sigma values\n",
    "    for i in range(len(updated_sigma_values)):\n",
    "        noised_mini_train_data = mini_train_data + np.random.normal(0.0, updated_sigma_values[i], mini_train_data.shape)\n",
    "        gnb.fit(noised_mini_train_data, mini_train_labels)\n",
    "        print 'Accuracy using for dev set when adding random noise with sigma {0:.3f} -> {1:.3f} '.format(updated_sigma_values[i],gnb.score(dev_data, dev_labels))\n",
    "            \n",
    "\n",
    "gnb = P9()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "ANSWER: \n",
    "\n",
    "    Two ways, I could improve the Gaussian NB\n",
    "    - One by adding variance to the data\n",
    "    - Two by adding some random normal noise to the training data\n",
    "    \n",
    "    As Gaussian distribution works best if the data is normalized, our image data consists mostly values either 0 or 1. So, to improve the classifier, I tried the above two ways, which got the accuracy 0.803 when added variance with the sigma value 0.05 and 0.818 when adding random normal noise with the sigma value 0.15"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(10) Because Naive Bayes is a generative model, we can use the trained model to generate digits. Train a BernoulliNB model and then generate a 10x20 grid with 20 examples of each digit. Because you're using a Bernoulli model, each pixel output will be either 0 or 1. How do the generated digits compare to the training digits?\n",
    "\n",
    "- You can use np.random.rand() to generate random numbers from a uniform distribution\n",
    "- The estimated probability of each pixel is stored in feature\\_log\\_prob\\_. You'll need to use np.exp() to convert a log probability back to a probability."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlQAAAIrCAYAAAAp7Ac9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAHmpJREFUeJzt3dFu28i2RVHWRf7/l+s+HDBNy5Rje0u1F6UxgEYS2yfTpfOyUFS7x5xzAwDg9/6v+xsAALg6gwoAoMigAgAoMqgAAIoMKgCAIoMKAKDIoAIAKDKoAACKDCoAgKI/K2NjjKf+WPY559B7nV5HU08vudfR1NNL7nU1z7ihAgAoihxU+39fcM759x89vaSmnl5yr6Opp5fcW9GMHFRnVr3geq/R62jq6SX3Opp6esm9RzfjBtWccxvj/uPKR7/YetfudTT19JJ7HU09veTeqmbUoLq9jjvaX4gxxsNebL1r9zqaenrJvY6mnl5yb2WzdVAdv/l9Pd5bkMev3Q/+08PrXbvX0dTTS+51NPX0kntdzW1b/GMTbu0HPK7Hr77u+DVfXd3pvWavo6mnl9zraOrpJfe6mtvWPKiOjtdtZwe6fUH21an3nr2Opp5ecq+jqaeX3FvdbH8P1f7N/+uK7XjAygusd+1eR1NPL7nX0dTTS+51NdsH1bZ9fxHeXuPpvWevo6mnl9zraOrpJfc6mhGDal+RP1mHj7h21Ltmr6Opp5fc62jq6SX3Oprtg+p4Jfevgx8//9slqXftXkdTTy+519HU00vudTXbB9VPHF+g4+/19FKaenrJvY6mnl5y75HNmJ9D9Z11eLsyf3o1p3ftXkdTTy+519HU00vudTW3rXFQHQ+wYpHqXbvX0dTTS+51NPX0kntdzd1YcZ32NzbGl7HvfC9fLcc554dP6l2719HU00vudTT19JJ7Xc0zEe+hmvO/H/V+e6gx/vuR8b+9htN7rV5HU08vudfR1NNL7nU0IwbVPcdDPvJF1nvNXkdTTy+519HU00vuPbMZ85+e2R2v5m6v6Z7xYutdu9fR1NNL7nU09fSSe6uaEYNqP8zxzWR6eklNPb3kXkdTTy+519GMeuS36kXWe41eR1NPL7nX0dTTS+6tbEYNKgCAK1r6YxMAAF6RGyoAgCKDCgCgyKACACgyqAAAigwqAIAigwoAoMigAgAoMqgAAIoMKgCAIoMKAKDIoAIAKDKoAACKDCoAgCKDCgCgyKACACgyqAAAigwqAIAigwoAoMigAgAoMqgAAIoMKgCAIoMKAKDIoAIAKDKoAACKDCoAgCKDCgCgyKACACgyqAAAigwqAIAigwoAoMigAgAo+rMyNsaYz/z755xD73V6HU09veReR1NPL7nX1TzjhgoAoOhyg2rOuc351DGq90K9jqaeXnKvo6mnl9x7VDN+UN0ecIxv3bzpvWmvo6mnl9zraOrpJfee1YwaVGfrcIzxtBdb79q9jqaeXnKvo6mnl9xb2YwaVPcOc3ZwPb2Opp5ecq+jqaeX3FvZXPpv+f3E2SHnnE+7CtS7dq+jqaeX3Oto6ukl957djLqhOhpj/D3g7a/PWs161+11NPX0knsdTT295N6zm7GD6niw2/X4jPWqd+1eR1NPL7nX0dTTS+49uxk7qLbteQtV7zV7HU09veReR1NPL7n3zGbsoHrmM1S91+t1NPX0knsdTT295N6zm5FvSr9dj89+0fWu3eto6ukl9zqaenrJvRXN2BsqAICriB1U95bjM9/5r3fdXkdTTy+519HU00vuPbsZO6juHW7FO//1rtfraOrpJfc6mnp6yb1nN+MG1X7Y258PoaeX0tTTS+51NPX0knurmjGD6njYMcannw+hp9fd1NNL7nU09fSSe6ubrYPqePW2H/D4sTnn33/09DqaenrJvY6mnl5yr6u5bds2Hv0Xfhkb46mxOeeH2al37V5HU08vudfR1NNL7nU1z8Q98jv++ZljT+/avY6mnl5yr6Opp5fcW92MGVTHZ5r7YffnnfcOX3lR9K7d62jq6SX3Opp6esm91c32R35zPu4NYvMbV4961+l1NPX0knsdTT295F5X80z7DdWz3+Gv91q9jqaeXnKvo6mnl9zrarYPKgCAq1v6yA8A4BW5oQIAKDKoAACKDCoAgCKDCgCgyKACACgyqAAAigwqAIAigwoAoMigAgAoMqgAAIoMKgCAIoMKAKDIoAIAKDKoAACKDCoAgCKDCgCgyKACACgyqAAAigwqAIAigwoAoMigAgAoMqgAAIoMKgCAIoMKAKDIoAIAKDKoAACKDCoAgCKDCgCgyKACACgyqAAAiv6sjI0x5jP//jnn0HudXkdTTy+519HU00vudTXPuKECACiKHFRzPnVs6r1Yr6Opp5fc62jq6SX3VjTjBtXtgeecHz62//5RL4zetXsdTT295F5HU08vubequfQ9VN8xxv8eVe6HGmN8OOD++f1XvffudTT19JJ7HU09veTeqmbcDdW23T/wI9eq3uv0Opp6esm9jqaeXnJvRTNuUM057y7JR65VvdfodTT19JJ7HU09veTeqmbcoNq2j4c9/vn293p6XU09veReR1NPL7m3ohn7Hqoz+5J85KLUu3avo6mnl9zraOrpJfdWNWNuqM7W4dnhHvUi612719HU00vudTT19JJ7q5sxg+rsIGeL8ZkLWe86vY6mnl5yr6Opp5fcW92MGVTbdv4885EvrN5r9TqaenrJvY6mnl5yb2Uz5j1Ut9dy+2HPruv09DqaenrJvY6mnl5yb3UzZlAd1+JxQR4P/6xrQL3r9TqaenrJvY6mnl5yb3Wz9ZHffrjvLMXj4X+7LPWu3eto6ukl9zqaenrJva7mtjUPqtvleDzMGOPTi3Jckr85uN61ex1NPb3kXkdTTy+519XctrA3pZ8ddHc88P61eu/d62jq6SX3Opp6esm9lc22QXVcjWfL8OyAt1+v9z69jqaeXnKvo6mnl9zrav79+27X2jONMU5j+0K8/V72j333kHPOD1+od+1eR1NPL7nX0dTTS+51Nc/EPfLbfz3+ftv+W5CPHIB61+51NPX0knsdTT295N7KZsSPTTguyK8W408Wpd7r9jqaenrJvY6mnl5yr6MZMai27evDfufzeu/V62jq6SX3Opp6esm91c2oR34AAFdkUAEAFC39t/wAAF6RGyoAgCKDCgCgyKACACgyqAAAigwqAIAigwoAoMigAgAoMqgAAIoMKgCAIoMKAKDIoAIAKDKoAACKDCoAgCKDCgCgyKACACgyqAAAigwqAIAigwoAoMigAgAoMqgAAIoMKgCAIoMKAKDIoAIAKDKoAACKDCoAgCKDCgCgyKACACgyqAAAigwqAIAigwoAoMigAgAo+rMyNsaYv/3fzjm3Mca/vubDF+hdu9fR1NNL7nU09fSSe13NM5e5ofrXgfX0upt6esm9jqaeXnLv0c2lN1TfNefHsXk88HfWpN579TqaenrJvY6mnl5yb0Uz4obq3iH3X/fPP+pF1rt2r6Opp5fc62jq6SX3OpoRg2qMsc05n/KC6r1er6Opp5fc62jq6SX3OpoRg+poP/D+Ajz7Bde7dq+jqaeX3Oto6ukl91Y1I95D9Z3VeHt199PP671Or6Opp5fc62jq6SX3OpoRN1THAx/X4/Gq7l8vyk/Wpt61ex1NPb3kXkdTTy+519GMGFRHZ9/88ZrupwtV77V7HU09veReR1NPL7m3qhnxyG93PNDt4X+6TPVev9fR1NNL7nU09fSSeyubUYNq2z5fx23b5+X4yBdA79q9jqaeXnKvo6mnl9xb1Vw+qOY8f5PY/vGvnmv+5kpO79q9jqaeXnKvo6mnl9zrat5a/h6qewvw9uO3K3J/UY4vjN7r9zqaenrJvY6mnl5yr6t5K/JN6bcvwNmfH7Uo9a7d62jq6SX3Opp6esm9Vc2YQXV7iHtr87uf13vtXkdTTy+519HU00vurW5GDKp7zz7Pvm7/tbIi9a7d62jq6SX3Opp6esm9jmbEoNq2z881bx2v686u7vTeq9fR1NNL7nU09fSSe6ubEYPqeIB9Ue5L8bgc9fS6mnp6yb2Opp5ecq+j2f5zqM7W470VqafX0dTTS+51NPX0kntdzfYbquNV2788YknqXbvX0dTTS+51NPX0knttzUdfsX0ZG+OpsTnnh1dO79q9jqaeXnKvo6mnl9zrap5pv6ECALi6uEG18sZM7/q9jqaeXnKvo6mnl9xb1Vz6yA8A4BXF3VABAFyNQQUAUGRQAQAUGVQAAEUGFQBAkUEFAFBkUAEAFBlUAABFBhUAQJFBBQBQZFABABQZVAAARQYVAECRQQUAUGRQAQAUGVQAAEUGFQBAkUEFAFBkUAEAFBlUAABFBhUAQJFBBQBQZFABABQZVAAARQYVAECRQQUAUGRQAQAUGVQAAEUGFQBAkUEFAFBkUAEAFP1ZGRtjzGf+/XPOofc6vY6mnl5yr6Opp5fc62qecUMFAFBkUAEAFEUMqjnPb+vOPv6Tr9V7zV5HU08vudfR1NNL7nU0x0+/wYrvPuecc25jfOuR5e3/7lfPcvUyex1NPb3kXkdTTy+519U8E3FDtW3/XoFzzk9fUxmDetfudTT19JJ7HU09veTe6mbMoDquxnsLcozx96C/XZp6r9HraOrpJfc6mnp6yb3VzaU/NuE7jstwP9TxgPvBqy+y3mv0Opp6esm9jqaeXnJvVbN9UO0H2A97dpjbg1YOrHftXkdTTy+519HU00vudTXbH/ndHmDOj88z7/1e7z17HU09veReR1NPL7nX1WwfVMf1eG8lHhek3nv3Opp6esm9jqaeXnKvq9k6qG6v5Lbt47PN26/Ve+9eR1NPL7nX0dTTS+51NbeteVAdl+F+qNtrueOyvPeC6L1Hr6Opp5fc62jq6SX3uprbFnBDtS/Jo7PDHV+M317R6V2719HU00vudTT19JJ7Xc1t2zJ/Uvq2fTzwGN/71xln4b96rZfX62jq6SX3Opp6esm9ruaZ9jelf8fK0ad3/V5HU08vudfR1NNL7j2jGTmozq7fqldxeq/b62jq6SX3Opp6esm9Fc32H+x55t5zTz29lKaeXnKvo6mnl9xb0Yy8oQIAuJKYQXX7rvvds1ar3rV7HU09veReR1NPL7m3uhkzqFY+R9W7fq+jqaeX3Oto6ukl91Y3YwYVAMBVGVQAAEVLf7AnAMArckMFAFBkUAEAFBlUAABFBhUAQJFBBQBQZFABABQZVAAARQYVAECRQQUAUGRQAQAUGVQAAEUGFQBAkUEFAFBkUAEAFBlUAABFBhUAQJFBBQBQZFABABQZVAAARQYVAECRQQUAUGRQAQAUGVQAAEUGFQBAkUEFAFBkUAEAFBlUAABFBhUAQJFBBQBQZFABABQZVAAARX9WxsYY85l//5xz6L1Or6Opp5fc62jq6SX3uppn3FABABQZVAAARbGDas77N3hzzi8/r/d+vY6mnl5yr6Opp5fce3YzclDNObcxxoeDHQ86xtjG+NYjTb036HU09fSSex1NPb3k3opm1KC6PeT+6/4i3H6N3nv3Opp6esm9jqaeXnJvZTNqUN0ux6Pjgtz/XH0B9K7d62jq6SX3Opp6esm9lc2IQfXVN392/bavyt9ezeldu9fR1NNL7nU09fSSex3NpT+H6szxym23r8n942ef13vPXkdTTy+519HU00vudTXbb6iOBziuyXsf13vvXkdTTy+519HU00vudTXbB9XR7UH3f/ZVqafX3dTTS+51NPX0knsrm62D6niQ29+PMf7+s3+uenC9a/c6mnp6yb2Opp5ecq+ruW3Ng+q4Gu/9/njQ48f13q/X0dTTS+51NPX0kntdzW0LeuS3H24/2NlhH3k1p3ftXkdTTy+519HU00vurW7GDKrb5bj/+Xgt96gVqXf9XkdTTy+519HU00vurW7GDKqj40GPzzyPHrli9a7d62jq6SX3Opp6esm9Fc2oQXV2Fbf/envIRyxKvWv3Opp6esm9jqaeXnJvZXPpoPrX8rt3wOPn9N6n19HU00vudTT19JJ7Xc0zSwfV2Td+POCcn382xL1f9V6/19HU00vudTT19JJ7Xc0zbY/89m/+uBzH+Pw8c38Bbq/p9N6r19HU00vudTT19JJ7Xc1d23/L71+L8vhi6Ol1NPX0knsdTT295F5Xc9f+H0fe3VuHj1iNeq/X62jq6SX3Opp6esm91c2of8uv4hlrU+91ex1NPb3kXkdTTy+599PmywyqZy5cvdfrdTT19JJ7HU09veTeT5ujY/EBALySl7mhAgDoYlABABQZVAAARQYVAECRQQUAUGRQAQAUGVQAAEUGFQBAkUEFAFBkUAEAFBlUAABFBhUAQJFBBQBQZFABABQZVAAARQYVAECRQQUAUGRQAQAUGVQAAEUGFQBAkUEFAFBkUAEAFBlUAABFBhUAQJFBBQBQZFABABQZVAAARQYVAECRQQUAUGRQAQAUGVQAAEV/VsbGGPM7Xzfn3MYYP/7755wf/kd61+51NPX0knsdTT295F5X80zkDdXtgef81mul96a9jqaeXnKvo6mnl9xb0WwfVGcH+uqQ1RdA79q9jqaeXnKvo6mnl9zrai595Hfm7Ppt/9jZAX9zXaf3Or2Opp5ecq+jqaeX3Otqtt9Q3bM/6/zqBbj9er337XU09fSSex1NPb3k3rOb7TdUX9kP8p3lOMZ42LWk3jV7HU09veReR1NPL7n3zGbcDdWc8++C1NNLbOrpJfc6mnp6yb1VzZhBdVyAxwM/6/B61+51NPX0knsdTT295N7qZtQjv/3gP7mO03vfXkdTTy+519HU00vurWxGDaqzQz7zBdC7dq+jqaeX3Oto6ukl91Y2owbV2dXc/oawe9d2eu/b62jq6SX3Opp6esm9lc2I91DN+fFfYzx+fNs+H/L4Aui9X6+jqaeX3Oto6ukl9zqaEYNqX4rHP+9uD77/erss9d6n19HU00vudTT19JJ7Hc2IQbVt/x38eKjbwx9fnLPVqfc+vY6mnl5yr6Opp5fcW90clfX349gX/0Xo2+/jbEnefvzk7/jwSb1r9zqaenrJvY6mnl5yr6t5JupN6UffPaieXldTTy+519HU00vuPbsZ9cjv3uH2F+CRt2l61+51NPX0knsdTT295N7qZsyg2v3r4Hp63U09veReR1NPL7m3qhn7yG/b1l0B6r1Gr6Opp5fc62jq6SX3ntmMHFSv/n+o3vWbenrJvY6mnl5yb0Uz7pEfAMDVxAyq3zzHrDz71Lt2r6Opp5fc62jq6SX3VjdjBtVvruIq13d61+51NPX0knsdTT295N7q5tIf7AkA8IpibqgAAK7KoAIAKDKoAACKDCoAgCKDCgCgyKACACgyqAAAigwqAIAigwoAoMigAgAoMqgAAIoMKgCAIoMKAKDIoAIAKDKoAACKDCoAgCKDCgCgyKACACgyqAAAigwqAIAigwoAoMigAgAoMqgAAIoMKgCAIoMKAKDIoAIAKDKoAACKDCoAgCKDCgCgyKACACgyqAAAiv6sjI0x5jP//jnn0HudXkdTTy+519HU00vudTXPuKECACgyqAAAigwqAICi2EE151Mfieq9WK+jqaeX3Oto6ukl957djBhUc86/h9x/HWN8+vzxhbj9s9779DqaenrJvY6mnl5yr6MZMaiOB9y2+wvy+HVjjE//O7336HU09fSSex1NPb3kXkczYlDdLsj997eH/2o1/mRR6l2719HU00vudTT19JJ7Hc2lP4fqaM75dwXuvx4/9lNjjH++KHrX7XU09fSSex1NPb3kXldz13ZDdTzc/s1+58A/Xah6r9HraOrpJfc6mnp6yb2u5q7thuro3mHPlqaeXkdTTy+519HU00vudTQj3kP1HWfPPfX0kpp6esm9jqaeXnLv0c2IG6oztwd89HLVe61eR1NPL7nX0dTTS+49uxl7Q3V8M9nxVz29lKaeXnKvo6mnl9x7djNyUJ0d8PZF0NPrbOrpJfc6mnp6yb0VzchHfmP8719TvH23/hi1H/Kl95q9jqaeXnKvo6mnl9xb0Yy8ododF+PqZ6t61+t1NPX0knsdTT295N4zm5GD6rggVzxT1bt2r6Opp5fc62jq6SX3VjQjB9W2/XfY/YpOTy+tqaeX3Oto6ukl957djBtUZ88zn7ko9a7d62jq6SX3Opp6esm9Vc2YQXVcjfc88lmn3rV7HU09veReR1NPL7m3uhkzqG4P9OzrP71r9zqaenrJvY6mnl5yb3Uz8scmHN84pqeX2NTTS+51NPX0knsrmmPVG8EAAF5VzCM/AICrMqgAAIoMKgCAIoMKAKDIoAIAKDKoAACKDCoAgCKDCgCgyKACACgyqAAAigwqAIAigwoAoMigAgAoMqgAAIoMKgCAIoMKAKDIoAIAKDKoAACKDCoAgCKDCgCgyKACACgyqAAAigwqAIAigwoAoMigAgAoMqgAAIoMKgCAIoMKAKDIoAIAKDKoAACKDCoAgKI/K2NjjHn885xzG2M87O+fc374y/Su3eto6ukl9zqaenrJva7mmdYbqkceWO/1ex1NPb3kXkdTTy+519XcNo/8AADKogfVnPNHH9d7715HU08vudfR1NNL7j2zGTGozg6xPwOdcz78hdW7dq+jqaeX3Oto6ukl9zqaEYPq+LzzeMB/vRh679nraOrpJfc6mnp6yb2O5tJ/y++7jgfaX5D9Y/ufH/wOfr0L9zqaenrJvY6mnl5yb0Wz/YbqbA1+dbB/rUy91+51NPX0knsdTT295F5Xs/2G6nYl7r+/tx7PFqbe+/Q6mnp6yb2Opp5ecq+r2T6otu3zwfaPjTFOP6b33r2Opp5ecq+jqaeX3Ototj/y27b7a3DO+eGfMf57w9hxTeq9V6+jqaeX3Oto6ukl9zqao/oN/yh28+Phb91bicfv8asVOf/xI/D1rtXraOrpJfc6mnp6yb2u5pmYR377Fdy9gfeoK0C96/c6mnp6yb2Opp5ecq+jGf3I72i/mtPT62jq6SX3Opp6esm9jmbEoDq69wLsK1NPr7upp5fc62jq6SX3VjVjBtXZs8z9oPufH7kk9a7d62jq6SX3Opp6esm91c2YQTXG+HTw/ZBnL4jee/c6mnp6yb2Opp5ecm91s2VQnS3B/WPHQx7X5PFXvffqdTT19JJ7HU09veReV/OoZVCdffPHj90uytslefai6b1ur6Opp5fc62jq6SX3uppHMY/8zg51uyb19DqbenrJvY6mnl5yb3UzZlD96+rtuCwf8SLoXbvX0dTTS+51NPX0knurm+0/KX3Oh/53ez78RXrX7nU09fSSex1NPb3kXlfzTPsN1X7gVcNO79q9jqaeXnKvo6mnl9zrarYPKgCAq1v6yA8A4BW5oQIAKDKoAACKDCoAgCKDCgCgyKACACgyqAAAigwqAIAigwoAoMigAgAoMqgAAIoMKgCAIoMKAKDIoAIAKDKoAACKDCoAgCKDCgCgyKACACgyqAAAigwqAIAigwoAoMigAgAoMqgAAIoMKgCAIoMKAKDIoAIAKDKoAACKDCoAgCKDCgCgyKACACgyqAAAiv6sjI0x5jP//jnn0HudXkdTTy+519HU00vudTXPuKECACgyqAAAigwqAIAigwoAoKh1UM357/eR7V/zna/Ve+1eR1NPL7nX0dTTS+51NbeteVCN8fmN83POTwe9PfBvXwi9a/c6mnp6yb2Opp5ecq+ruW3Bj/xuD3R8gc5eLL337nU09fSSex1NPb3k3rObMYPquB53++HGGKeLsXJ4vWv3Opp6esm9jqaeXnJvdTNiUM05Pxxw/+f247+9htN7rV5HU08vudfR1NNL7nU0l/6k9N965Aus9/q9jqaeXnKvo6mnl9x7RjPihupoP+BxQZ59Xk+vq6mnl9zraOrpJfdWNSNuqG6v3O4d7OxF0Hu/XkdTTy+519HU00vudTRjbqjuLcb9BXj0s1W9a/c6mnp6yb2Opp5ecm91M2ZQHe1vHDu6d02np9fR1NNL7nU09fSSeyuaUYPq9lDHP69YrnrX6nU09fSSex1NPb3k3spm1KDatq+fZT5juepdu9fR1NNL7nU09fSSe6uacYNqX4rHZ5y3n9PT62zq6SX3Opp6esm9Vc24QbVfv624BtS7fq+jqaeX3Oto6ukl91Y1lw+q73zzx4NXr+L0rt3raOrpJfc6mnp6yb2u5qe//5mL8FNsjC9jt+vxp+acH/7HetfudTT19JJ7HU09veReV/NM1CO//cCrRp7etXsdTT295F5HU08vubeyGTWodl8tyWe8IHrX7nU09fSSex1NPb3k3opm5KD6yjOee+q9bq+jqaeX3Oto6ukl9x7VXPoeKgCAV3S5GyoAgDQGFQBAkUEFAFBkUAEAFBlUAABFBhUAQJFBBQBQZFABABQZVAAARQYVAECRQQUAUGRQAQAUGVQAAEUGFQBAkUEFAFBkUAEAFBlUAABFBhUAQJFBBQBQZFABABQZVAAARQYVAECRQQUAUPT/mffB9UsQLLkAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x720 with 200 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def P10(num_examples):\n",
    "\n",
    "    #For Binary Valued features using BernoulliNB\n",
    "    Binary_Bernoulli_model = BernoulliNB(binarize=0.333)\n",
    "    #Using Mini Training data to fit the model\n",
    "    Binary_Bernoulli_model.fit(mini_train_data, mini_train_labels)\n",
    "    #Get the probability values from the model\n",
    "    probs = np.exp(Binary_Bernoulli_model.feature_log_prob_)\n",
    "    #Defining the colormap for image properties\n",
    "    plt.rc('image', cmap='gray')\n",
    "    plt.figure(figsize=(10,10))\n",
    "\n",
    "    # for each digit in 0-9 -> the unique labels in Y covers this part\n",
    "    for i in range(len(np.unique(Y))):\n",
    "        # using np.random.rand(784L) here as Bernoulli NB model needs 0 and 1 values\n",
    "        # we turn on if the value of the random is less than the probability the model has given us\n",
    "        features = np.where(probs[i,]>np.random.rand(X.shape[1]),1,0)\n",
    "        for j in range(num_examples):\n",
    "               # Create subplot by specifying position from 1 to each digit * columns. Add j to get all columns\n",
    "                plt.subplot(len(np.unique(Y)), num_examples, 1 + int (i) * num_examples + j  )\n",
    "                # Hide axes\n",
    "                plt.axis('off')\n",
    "                # Plot the corresponding digit ( reshaped to 2D matrix)\n",
    "                vector_size = int(np.sqrt(X.shape[1]))\n",
    "                digit = features.reshape((vector_size,vector_size))            \n",
    "                plt.imshow(digit)\n",
    "\n",
    "P10(20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "ANSWER: How do the generated digits compare to the training digits?\n",
    "    \n",
    "    The generated digits are much lighter in color and unable to read clearly when compared to the training digits. \n",
    "    But given the probabilities, they show that those digits stay in the right place for the given training."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(11) Remember that a strongly calibrated classifier is rougly 90% accurate when the posterior probability of the predicted class is 0.9. A weakly calibrated classifier is more accurate when the posterior is 90% than when it is 80%. A poorly calibrated classifier has no positive correlation between posterior and accuracy.\n",
    "\n",
    "Train a BernoulliNB model with a reasonable alpha value. For each posterior bucket (think of a bin in a histogram), you want to estimate the classifier's accuracy. So for each prediction, find the bucket the maximum posterior belongs to and update the \"correct\" and \"total\" counters.\n",
    "\n",
    "How would you characterize the calibration for the Naive Bayes model?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "p(pred) <= 0.5000000000000    total =   0  correct =   0  accuracy = 0.000\n",
      "p(pred) <= 0.9000000000000    total =  32  correct =  13  accuracy = 0.406\n",
      "p(pred) <= 0.9990000000000    total =  86  correct =  46  accuracy = 0.535\n",
      "p(pred) <= 0.9999900000000    total =  60  correct =  33  accuracy = 0.550\n",
      "p(pred) <= 0.9999999000000    total =  59  correct =  44  accuracy = 0.746\n",
      "p(pred) <= 0.9999999990000    total =  55  correct =  39  accuracy = 0.709\n",
      "p(pred) <= 0.9999999999900    total =  59  correct =  48  accuracy = 0.814\n",
      "p(pred) <= 0.9999999999999    total =  57  correct =  46  accuracy = 0.807\n",
      "p(pred) <= 1.0000000000000    total = 592  correct = 566  accuracy = 0.956\n"
     ]
    }
   ],
   "source": [
    "def P11(buckets, correct, total):\n",
    "    \n",
    "    #Using BinomialNB with the best alpha and a binarize value of 0.333\n",
    "    Binary_Bernoulli_model = BernoulliNB(alpha=0.001, binarize=0.333)\n",
    "    #Using Mini Training data to fit the model\n",
    "    Binary_Bernoulli_model.fit(mini_train_data, mini_train_labels)\n",
    "    #Get the predicted labels for the dev data set\n",
    "    dev_predicted_labels = Binary_Bernoulli_model.predict(dev_data)\n",
    "    #Get the predicted probabilites for the dev data set.\n",
    "    #This returns the probability estimate for the dev data set\n",
    "    dev_predicted_probs = Binary_Bernoulli_model.predict_proba(dev_data)\n",
    " \n",
    "\n",
    "    #for each bucket - need to look at previous value to get the correct bin\n",
    "    for i in range(len(buckets)):\n",
    "        #for all the probabilites for the entire set\n",
    "        #set counter for total_value and correct_value\n",
    "        total_value = 0\n",
    "        correct_value = 0\n",
    "        for j in range(dev_predicted_probs.shape[0]):\n",
    "            # increment correct value only if total is incremented\n",
    "            increment_value = False\n",
    "            #Get the maximum probability, that the model predicts\n",
    "            prob_of_digit = dev_predicted_probs[j, dev_predicted_probs[j].argmax()]\n",
    "            \n",
    "            #place it in the bucket\n",
    "            #if the probability of the digit is in the current bucket\n",
    "            if i == 0:\n",
    "                increment_correct = np.where((prob_of_digit <= buckets[i]),True, False)\n",
    "            else:     \n",
    "                increment_correct = np.where((prob_of_digit <= buckets[i]) & (prob_of_digit > buckets[i-1]), True, False)\n",
    "            \n",
    "            #if the prediction of the label is correct for the given bucket\n",
    "            if increment_correct: \n",
    "                total_value +=1\n",
    "                if(dev_predicted_labels[j] == dev_labels[j]):\n",
    "                    correct_value +=1\n",
    "                \n",
    "        \n",
    "        #place the total_value and correct_value in the respective buckets\n",
    "        correct[i] = float(correct_value)\n",
    "        total[i] = float(total_value)\n",
    "\n",
    "\n",
    "buckets = [0.5, 0.9, 0.999, 0.99999, 0.9999999, 0.999999999, 0.99999999999, 0.9999999999999, 1.0]\n",
    "correct = [0 for i in buckets]\n",
    "total = [0 for i in buckets]\n",
    "\n",
    "P11(buckets, correct, total)\n",
    "\n",
    "for i in range(len(buckets)):\n",
    "    accuracy = 0.0\n",
    "    if (total[i] > 0): accuracy = correct[i] / total[i]\n",
    "    print 'p(pred) <= %.13f    total = %3d  correct = %3d  accuracy = %.3f' %(buckets[i], total[i],  correct[i], accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "ANSWER: How would you characterize the calibration for the Naive Bayes model?\n",
    "\n",
    "    The accuracy is low at probability bucket 0.5-0.9 which is 0.406 compared to the accuracy at 0.9999999999900 which is \n",
    "    0.814.\n",
    "    This shows that the naive bayes model's precision is different from accuracy.\n",
    "    As Accuracy is precision with calibration, when the posterior probability of the predicted class is 0.9 our model \n",
    "    shows only 0.406 accuracy. This shows that our model is not perfectly calibrated shows signs of weakly calibrated \n",
    "    classifier. So, we cannot simply infer the correct digit even though the posterior probability is at 0.9     \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(12) EXTRA CREDIT\n",
    "\n",
    "Try designing extra features to see if you can improve the performance of Naive Bayes on the dev set. Here are a few ideas to get you started:\n",
    "- Try summing the pixel values in each row and each column.\n",
    "- Try counting the number of enclosed regions; 8 usually has 2 enclosed regions, 9 usually has 1, and 7 usually has 0.\n",
    "\n",
    "Make sure you comment your code well!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "#def P12():\n",
    "\n",
    "### STUDENT START ###\n",
    "\n",
    "\n",
    "### STUDENT END ###\n",
    "\n",
    "#P12()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}