{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Neural Networks for Classification, and Clustering\n",
    "\n",
    "In this notebook we are going to explore the use of Neural Networks for image classification. We are going to use a dataset of small images of clothes and accessories, the Fashion MNIST. You can find more information regarding the dataset here: https://pravarmahajan.github.io/fashion/\n",
    "\n",
    "Each instance in the dataset consist of an image, in a format similar to the digit images you have seen in the previous homework, and a label. The labels correspond to the type of clothing, as follows:\n",
    "\n",
    "| Label | Description |\n",
    "| --- | --- |\n",
    "| 0 | T-shirt/top |\n",
    "| 1 | Trouser |\n",
    "| 2 | Pullover |\n",
    "| 3 | Dress |\n",
    "| 4 | Coat |\n",
    "| 5 | Sandal |\n",
    "| 6 | Shirt |\n",
    "| 7 | Sneaker |\n",
    "| 8 | Bag |\n",
    "| 9 | Ankle boot |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's first load the required packages."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "#load the required packages\n",
    "\n",
    "%matplotlib inline  \n",
    "\n",
    "import numpy as np\n",
    "import scipy as sp\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "import sklearn\n",
    "from sklearn.neural_network import MLPClassifier\n",
    "from sklearn.model_selection import GridSearchCV"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following is a function to load the data, that we are going to use later in the notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# helper function to load Fashion MNIST dataset from disk\n",
    "def load_fashion_mnist(path, kind='train'):\n",
    "    import os\n",
    "    import gzip\n",
    "    import numpy as np\n",
    "    labels_path = os.path.join(path, '%s-labels-idx1-ubyte.gz' % kind)\n",
    "    images_path = os.path.join(path, '%s-images-idx3-ubyte.gz' % kind)\n",
    "    with gzip.open(labels_path, 'rb') as lbpath:\n",
    "        labels = np.frombuffer(lbpath.read(), dtype=np.uint8,offset=8)\n",
    "    with gzip.open(images_path, 'rb') as imgpath:\n",
    "        images = np.frombuffer(imgpath.read(), dtype=np.uint8,offset=16).reshape(len(labels), 784)\n",
    "    return images, labels"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TO DO 0\n",
    "Place your ID (\"numero di matricola\") that will be used as seed for random generator. Change the ID number in case you observe unexpected behaviours and want to test if this is due to randomization (e.g., train/test split). If you change the ID number explain why you have change it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "ID = 2051998\n",
    "np.random.seed(ID)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we load the dataset using the function above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "#load the fashion MNIST dataset and normalize the features so that each value is in [0,1]\n",
    "X, y = load_fashion_mnist(\"data\")\n",
    "# rescale the data\n",
    "X = X / 255.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we split the data into training and test. Make sure that each label is present at least 10 times\n",
    "in the training set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Labels in training dataset:  [0 1 2 3 4 5 6 7 8 9]\n",
      "Frequencies in training dataset:  [54 42 54 57 50 42 41 49 51 60]\n"
     ]
    }
   ],
   "source": [
    "#random permute the data and split into training and test taking the first 500\n",
    "#data samples as training and the rest as test\n",
    "permutation = np.random.permutation(X.shape[0])\n",
    "\n",
    "X = X[permutation]\n",
    "y = y[permutation]\n",
    "\n",
    "m_training = 500\n",
    "\n",
    "X_train, X_test = X[:m_training], X[m_training:]\n",
    "y_train, y_test = y[:m_training], y[m_training:]\n",
    "\n",
    "labels, freqs = np.unique(y_train, return_counts=True)\n",
    "print(\"Labels in training dataset: \", labels)\n",
    "print(\"Frequencies in training dataset: \", freqs)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following function plots an image and the corresponding label, to be used to inspect the data when needed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "#function for plotting a image and printing the corresponding label\n",
    "def plot_input(X_matrix, labels, index):\n",
    "    print(\"INPUT:\")\n",
    "    plt.imshow(\n",
    "        X_matrix[index].reshape(28,28),\n",
    "        cmap          = plt.cm.gray_r,\n",
    "        interpolation = \"nearest\"\n",
    "    )\n",
    "    plt.show()\n",
    "    print(\"LABEL: %i\"%labels[index])\n",
    "    return"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's test the function above and check few images."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INPUT:\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAQPUlEQVR4nO3dbYyV9ZnH8d8lgiAPCs7wICgUJaTGWNqMxIRNda3boDFiX9TUxMZNDNMYSdqk0SXdF/WFL8xmabMvNk1wNWU3KNa0Rl/4AEGj9k3jaCiOS3ZRMlhknAdRHOQZrn0xt80U5r7+x3Ofp+X//SSTM3Ou8z/nP/ec35yZc933/Td3F4AL30XtngCA1iDsQCYIO5AJwg5kgrADmbi4lQ/W1dXly5Yta+VDAlkZGBjQ6OioTVarFHYzWyvp3yRNkfQf7v54dPtly5apr6+vykMCCPT09JTW6v4z3symSPp3SbdLuk7SvWZ2Xb33B6C5qvzPvlrSB+6+z91PStomaV1jpgWg0aqEfbGkv0z4+kBx3d8ws14z6zOzvpGRkQoPB6CKKmGf7E2A8/a9dffN7t7j7j3d3d0VHg5AFVXCfkDSVRO+XiLpYLXpAGiWKmF/W9IKM/uGmU2T9CNJLzZmWgAare7Wm7ufNrMNkl7VeOvtKXd/v2EzA9BQlfrs7v6SpJcaNBcATcTuskAmCDuQCcIOZIKwA5kg7EAmCDuQCcIOZIKwA5kg7EAmCDuQCcIOZIKwA5kg7EAmCDuQCcIOZIKwA5kg7EAmCDuQCcIOZIKwA5kg7EAmCDuQCcIOZIKwA5kg7EAmCDuQCcIOZIKwA5kg7EAmKq3iCrzyyithfevWraW1NWvWhGOXLl0a1q+99tqwvmLFirCem0phN7MBSWOSzkg67e49jZgUgMZrxCv737v7aAPuB0AT8T87kImqYXdJ283sHTPrnewGZtZrZn1m1jcyMlLx4QDUq2rY17j7dyTdLukhM/vuuTdw983u3uPuPd3d3RUfDkC9KoXd3Q8Wl8OSnpe0uhGTAtB4dYfdzGaa2eyvPpf0fUn9jZoYgMaq8m78AknPm9lX9/O0u8dNV1xwHn744bDe31/++/+5554Lx544caKuOTXCrFmzwnrqX9IrrrgirK9cubK0tnHjxnDs9ddfH9bL1B12d98n6Vv1jgfQWrTegEwQdiAThB3IBGEHMkHYgUxwiOsF7uzZs2H9oouq/b4fGhoK6wsXLiytFW3bUhdfXO3pOTY2VlpLfd+nT58O60eOHAnrl156aViPDv3dv39/OPatt94K62V4ZQcyQdiBTBB2IBOEHcgEYQcyQdiBTBB2IBP02S9w7t7U+0+daqyrq6u0NmPGjHBsah+BqVOnhvUqY1N9+EceeSSsv/HGG2H98OHDpbUHH3wwHFsvXtmBTBB2IBOEHcgEYQcyQdiBTBB2IBOEHcgEffYL3JQpUyqN//jjjyuNj45Jj3rNtUgdMx71ylPHo0enepak7du3h/UdO3aE9eXLl5fWNmzYEI696667SmvRvgm8sgOZIOxAJgg7kAnCDmSCsAOZIOxAJgg7kAn67B0gdcx5ql713O+R9evXh/WZM2eG9TNnzpTWTp48GY5NHe+e2i7RY8+ePTscu2/fvrB+zz33hPXjx4+H9eHh4dLa2rVrw7HRctLRcyH5LDGzp8xs2Mz6J1w3z8x2mNne4nJu6n4AtFctLwm/lXTur5qNkna6+wpJO4uvAXSwZNjd/U1Jh865ep2kLcXnWyTd3dhpAWi0ev/ZW+Dug5JUXM4vu6GZ9ZpZn5n1pc5XBqB5mv5uvLtvdvced+/p7u5u9sMBKFFv2IfMbJEkFZflby0C6Aj1hv1FSfcXn98v6YXGTAdAsyT77Gb2jKRbJHWZ2QFJv5T0uKTfmdkDkj6S9MNmTvL/u1Q/OLVOecqpU6dKa6nzo7/22mth/eWXXw7rS5cuDeujo6OltVSve/r06WE91YePpNZ+P3HiRFjftGlTWE9tl6iP39/fX1qrIhl2d7+3pPS9Bs8FQBOxuyyQCcIOZIKwA5kg7EAmCDuQCQ5xbYFUa63KoZpS3F5LLXt82223hfVUC+nLL78M69GprKOWoSRdcsklYT21XaPWXaolmWr7jY2NhfW9e/eG9eh737ZtWzj2vvvuC+tleGUHMkHYgUwQdiAThB3IBGEHMkHYgUwQdiAT9NkbINUnT0n1i1OHY0an+1q4cGE49rLLLgvrqbmdPn06rE+bNq20ljqVdGq56aNHj4b16P5TP7PUNk8tFx1931K8f8LWrVvDsfTZAYQIO5AJwg5kgrADmSDsQCYIO5AJwg5kgj57IdXzjfquzVwyWZKeeOKJsN7b21taW7BgQTg21Wf/5JNPwnrqmPNUHz5y7NixsJ7qZUc/l9T+A6kll6v+zKM+/auvvlrpvsvwyg5kgrADmSDsQCYIO5AJwg5kgrADmSDsQCY6qs9e5fjm1LHPKamebRW7d+8O6+vWrQvrQ0NDYX3RokWltVQfPLU0caqfnNru0fnRU2Orzi16PqX6/6nz7aeOd6/Sp08dax+dkz7aZslXdjN7ysyGzax/wnWPmtnHZrar+LgjdT8A2quWP+N/K2ntJNf/2t1XFR8vNXZaABotGXZ3f1PSoRbMBUATVXmDboOZ7S7+zJ9bdiMz6zWzPjPri86VBqC56g37byRdI2mVpEFJm8pu6O6b3b3H3Xu6u7vrfDgAVdUVdncfcvcz7n5W0hOSVjd2WgAara6wm9nEXs8PJPWX3RZAZ0j22c3sGUm3SOoyswOSfinpFjNbJcklDUj6SSMm08xed8qhQ/F7kK+//npp7bHHHgvH7tq1K6x3dXWF9fnz54f1qJ+c6vem1ilPraGeGh/1o1P3XfWY8Sr7ZaT67FXXCkj16SMffvhhaS3qsycf0d3vneTqJ2uaFYCOwe6yQCYIO5AJwg5kgrADmSDsQCY66hDX6NA9SVq/fn1pLXXI4pEjR8L6p59+GtY/++yz0lqqZbh06dKwnmrzpA79jb73VHur6nLRKdH4M2fOhGNTh+em2n6R1M9s+vTpYT213aoeGhwZHBwsrUXPFV7ZgUwQdiAThB3IBGEHMkHYgUwQdiAThB3IREv77KdOnQp7hHfeeWc4fv/+/aW1efPmhWNThySmerbRWXZS/eJUnzzVk03NPer5pvrkqXq0tLAkHT16NKxHc0ttt1QvOrVdon0MUo8d7VfRCDNmzKh7bPRcjJ7HvLIDmSDsQCYIO5AJwg5kgrADmSDsQCYIO5CJlvbZDx06pGeffba0vmTJknD85ZdfXlqL+vdSutc9NjYW1qNjzlPHNqdUWfa4qtHR0bCe2v8g9b1H2y31fafqqV55tA9AanWiFStWhPXU/gkLFy4M63PmzCmtRfuTSNI111xTWovOAcArO5AJwg5kgrADmSDsQCYIO5AJwg5kgrADmWhpn33OnDm69dZbS+upc3V//vnnpbWDBw+GY1PnjU8t2Xz48OG65iVJQ0NDYf3YsWNhPdXrjuqpfvDKlSvD+syZM8N6tO+DJM2dO7e0ljqmO9WrXrx4cVj/6KOPSmupcwgsWLAgrKek9j+IjsUfGBgIx0b7F0T3m3xlN7OrzOx1M9tjZu+b2U+L6+eZ2Q4z21tclv9UAbRdLX/Gn5b0c3f/pqSbJD1kZtdJ2ihpp7uvkLSz+BpAh0qG3d0H3f3d4vMxSXskLZa0TtKW4mZbJN3dpDkCaICv9QadmS2T9G1Jf5K0wN0HpfFfCJLml4zpNbM+M+tr9nm9AJSrOexmNkvS7yX9zN2/qHWcu2929x5374nerAHQXDWF3cymajzoW939D8XVQ2a2qKgvkjTcnCkCaIRk683GewhPStrj7r+aUHpR0v2SHi8uX0jd14wZM3TDDTeU1pcvXx6O37NnT2kt1d5Ktda++KLmP1bOk1ouOnUoZuoQ1uHh+Pdo9PipU0Gn5pZq+1155ZVhvaurq7R29dVXh2NTLainn346rEdtwdQS3VGrVUov+Zw6zfXs2bNLaydOnAjHRqdcj36etfTZ10j6saT3zGxXcd0vNB7y35nZA5I+kvTDGu4LQJskw+7uf5RUtofA9xo7HQDNwu6yQCYIO5AJwg5kgrADmSDsQCZaeohryqxZs8L6jTfe2KKZnC86RHZkZCQcmzqENTrdsiQdP348rEe99OiUxZI0f/6kezn/Vaqf3Ew33XRTWE8d4lrv0sa1SB0imzpcO9quqR599DOLMsQrO5AJwg5kgrADmSDsQCYIO5AJwg5kgrADmeioPnsni/qXqf0DUJ9UL/vmm29u0UwuDLyyA5kg7EAmCDuQCcIOZIKwA5kg7EAmCDuQCcIOZIKwA5kg7EAmCDuQCcIOZIKwA5kg7EAmCDuQiWTYzewqM3vdzPaY2ftm9tPi+kfN7GMz21V83NH86QKoVy0nrzgt6efu/q6ZzZb0jpntKGq/dvd/bd70ADRKLeuzD0oaLD4fM7M9kuKlOAB0nK/1P7uZLZP0bUl/Kq7aYGa7zewpM5tbMqbXzPrMrC+1TBKA5qk57GY2S9LvJf3M3b+Q9BtJ10hapfFX/k2TjXP3ze7e4+490dpbAJqrprCb2VSNB32ru/9Bktx9yN3PuPtZSU9IWt28aQKoqpZ3403Sk5L2uPuvJly/aMLNfiCpv/HTA9Aotbwbv0bSjyW9Z2a7iut+IeleM1slySUNSPpJE+YHoEFqeTf+j5JsktJLjZ8OgGZhDzogE4QdyARhBzJB2IFMEHYgE4QdyARhBzJB2IFMEHYgE4QdyARhBzJB2IFMEHYgE4QdyIS5e+sezGxE0v4JV3VJGm3ZBL6eTp1bp85LYm71auTclrr7pOd/a2nYz3twsz5372nbBAKdOrdOnZfE3OrVqrnxZzyQCcIOZKLdYd/c5sePdOrcOnVeEnOrV0vm1tb/2QG0Trtf2QG0CGEHMtGWsJvZWjP7HzP7wMw2tmMOZcxswMzeK5ah7mvzXJ4ys2Ez659w3Twz22Fme4vLSdfYa9PcOmIZ72CZ8bZuu3Yvf97y/9nNbIqk/5X0D5IOSHpb0r3u/t8tnUgJMxuQ1OPubd8Bw8y+K+mIpP909+uL6/5F0iF3f7z4RTnX3f+pQ+b2qKQj7V7Gu1itaNHEZcYl3S3pH9XGbRfM6x61YLu145V9taQP3H2fu5+UtE3SujbMo+O5+5uSDp1z9TpJW4rPt2j8ydJyJXPrCO4+6O7vFp+PSfpqmfG2brtgXi3RjrAvlvSXCV8fUGet9+6StpvZO2bW2+7JTGKBuw9K408eSfPbPJ9zJZfxbqVzlhnvmG1Xz/LnVbUj7JMtJdVJ/b817v4dSbdLeqj4cxW1qWkZ71aZZJnxjlDv8udVtSPsByRdNeHrJZIOtmEek3L3g8XlsKTn1XlLUQ99tYJucTnc5vn8VSct4z3ZMuPqgG3XzuXP2xH2tyWtMLNvmNk0ST+S9GIb5nEeM5tZvHEiM5sp6fvqvKWoX5R0f/H5/ZJeaONc/kanLONdtsy42rzt2r78ubu3/EPSHRp/R/5DSf/cjjmUzGu5pD8XH++3e26SntH4n3WnNP4X0QOSrpC0U9Le4nJeB83tvyS9J2m3xoO1qE1z+zuN/2u4W9Ku4uOOdm+7YF4t2W7sLgtkgj3ogEwQdiAThB3IBGEHMkHYgUwQdiAThB3IxP8BbEk4U0oIvFYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LABEL: 9\n",
      "INPUT:\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAQe0lEQVR4nO3dX4yV9ZkH8O9XBET+CTKyA0Wn23gh2WRpc0I2oo2bZhv1BnrRplw0bKILF5i0sRdrbCJ6p03/pBebRqr86aYradKixBi3BpuY3gBHZBWWuCKwdHDCnBETQBBm4OnFvOyOOOf3DOf3vuc95fl+EjIz55lzzjOH+fIeznN+v5dmBhG58d1UdwMi0h0Ku0gQCrtIEAq7SBAKu0gQN3fzzhYtWmQDAwPdvMvwRkdHk/WRkZFk3ZvWkEzW+/v7k3Up1/HjxzEyMjLpX0pW2Ek+COAXAKYBeMHMnk19/8DAAJrNZs5dhpQTuKGhoeR1t27dmqxfvHgxWZ82bVqy/tRTTyXrKVeuXEnWb7pJT0yv1Wg02tY6frRITgPwbwAeArAcwFqSyzu9PRGpVs4/jSsBHDGzo2Z2CcAOAKvLaUtEypYT9qUA/jzh68Hiss8huZ5kk2Sz1Wpl3J2I5MgJ+2T/UfzCfy7NbLOZNcys0dfXl3F3IpIjJ+yDAJZN+PpLAD7Ka0dEqpIT9n0A7ib5ZZIzAHwXwK5y2hKRsnU8ejOzMZKPAfhPjI/etpjZodI6CyR3xPT666+3rW3bti153eeeey5Zv+uuu5L1F154IVnfsGFD29rzzz+fvK5Ga+XKmrOb2WsAXiupFxGpkP7pFAlCYRcJQmEXCUJhFwlCYRcJQmEXCaKr69mjqnqp5quvvtq2tmPHjqzb9jz66KPJempJ8zPPPJO87qZNm5J1b/ntzJkzk/VodGQXCUJhFwlCYRcJQmEXCUJhFwlCYRcJQqO3LhgbG0vWZ8yYkazv3bs3WT9//vx193SVNxb0drb1dpe955572tZ27tyZvK43epProyO7SBAKu0gQCrtIEAq7SBAKu0gQCrtIEAq7SBCas3eBN8v2DA4OJuv79u3r+La95bUXLlxI1mfNmpWsp94DcOTIkeR1PVrCen10ZBcJQmEXCUJhFwlCYRcJQmEXCUJhFwlCYRcJQnP2Lrj55ryHeXh4OFnv6+vLuv0Ub62957777mtb27p1a9Zte1Lvb4h4Ouis30KSxwGcBXAZwJiZNcpoSkTKV8aR/R/NbKSE2xGRCsV7LiMSVG7YDcAfSL5Ncv1k30ByPckmyWar1cq8OxHpVG7YV5nZ1wA8BGAjya9f+w1mttnMGmbWqPKFJBFJywq7mX1UfBwGsBPAyjKaEpHydRx2krNJzr36OYBvAjhYVmMiUq6cV+MXA9hJ8urt/IeZvV5KVzeY4jHqmLemfOXKzp9QeXva575H4P77729bW7JkSdZte7w976Pp+G/SzI4C+PsSexGRCmn0JhKEwi4ShMIuEoTCLhKEwi4ShJa4dkHu6O3YsWPJ+vz587NuP8XbBjtnqeiZM2eS9YsXLybr3lbSGr19no7sIkEo7CJBKOwiQSjsIkEo7CJBKOwiQSjsIkFozt4FudsWV7mVdO57AHJcunQpWf/www+T9eXLl5fZzg1PR3aRIBR2kSAUdpEgFHaRIBR2kSAUdpEgFHaRIDRnL3hrn3PWRufO2T/77LNk/dy5cx3fdpXr1T3e6aBPnDiRrGvOfn10ZBcJQmEXCUJhFwlCYRcJQmEXCUJhFwlCYRcJQnP2greuu85132+++Wayfu+993Z829OnT+/4urneeeedZP3kyZNZt597uukbjXtkJ7mF5DDJgxMuW0jyDZIfFB8XVNumiOSaytP4bQAevOayJwDsNrO7AewuvhaRHuaG3czeAnD6motXA9hefL4dwJpy2xKRsnX6At1iMxsCgOLjHe2+keR6kk2SzVar1eHdiUiuyl+NN7PNZtYws0bOxogikqfTsJ8i2Q8Axcf09qciUrtOw74LwLri83UAXimnHRGpijuIJPkSgAcALCI5CGATgGcB/JbkIwBOAPh2lU1Gt2BBerI5b968LnVSrltuuSVZr3ItfURu2M1sbZvSN0ruRUQqpH86RYJQ2EWCUNhFglDYRYJQ2EWCYM4Wyder0WhYs9ns2v1N9PjjjyfrL7/8crKe2q7ZO/WwN0KaOXNmsu5tmXz69LVLF/7f0aNHk9fNXdrrbQedetfkrbfemnXfZ86cSdZTv9sLFy5MXtfbYttbfrtnz55kfcmSJcl6pxqNBprN5qQPnI7sIkEo7CJBKOwiQSjsIkEo7CJBKOwiQSjsIkGE2Wt3//79yfrAwECyPjY21rZ29uzZ5HVzt6H2Tsmcmid7M3rvdNCjo6PJurcV9axZs9rWLly4kHXb3lbRqZ/NW157++23J+up9zYAwNDQULJe1Zw9RUd2kSAUdpEgFHaRIBR2kSAUdpEgFHaRIBR2kSDCzNk906ZNS9bnzp3btubN2b09A7z17DlbKnvrzUdGRpJ1bw5/2223Jetz5sxpW/PeP+CtOffm7DmnbPYec+9xvXz5csf3XRUd2UWCUNhFglDYRYJQ2EWCUNhFglDYRYJQ2EWCCDNn9+amqfXqQHpm683RvT3IvfXuOb17e9qvWrUqWU+tRweAwcHBZP3jjz9uW/Pe2+CtpffWw8+fP79tzXtvw+zZs5P1Q4cOJeuffPJJsl4H98hOcgvJYZIHJ1z2NMmTJA8Ufx6utk0RyTWVp/HbADw4yeU/N7MVxZ/Xym1LRMrmht3M3gKQ3oNHRHpezgt0j5F8t3iav6DdN5FcT7JJstlqtTLuTkRydBr2XwL4CoAVAIYA/LTdN5rZZjNrmFkjdZI/EalWR2E3s1NmdtnMrgD4FYCV5bYlImXrKOwk+yd8+S0AB9t9r4j0BnfOTvIlAA8AWERyEMAmAA+QXAHAABwHsKG6FsvhrT/2zvWdmgl7+5t7M/zz588n6968OTWH//TTT5PX9fbT93r31m2n5tXe+wu89wh4s/LU3vDeueFz5Z4roApu2M1s7SQXv1hBLyJSIb1dViQIhV0kCIVdJAiFXSQIhV0kCC1xLXhbJuds5+yNp7zb9rZETi2h9UZA3s+du/w2NTb0lrh6S4O90Vvq+t6yZG/c6fF6r4OO7CJBKOwiQSjsIkEo7CJBKOwiQSjsIkEo7CJBhJmzp5Y7AnnzZm8ZqLcE1lvK6c3pU/Nmrzdv1u3Vvd5Sj5s3w/eWHc+bNy9ZT22D7c3Bvb8TT+6cvgo6sosEobCLBKGwiwShsIsEobCLBKGwiwShsIsEEWbO7q1f9taM5/Bmurnr3VMzXW+W7a0Jz5nxA+k5f86M3rttIO80254qf1+qoiO7SBAKu0gQCrtIEAq7SBAKu0gQCrtIEAq7SBB/fcPCDl28eDFZ9+au3kw3peo9xFPzau/nqnJfeI+3Vt6bZXv7BKR+dm/G7/Xm/T7kzvGr4B7ZSS4j+UeSh0keIvn94vKFJN8g+UHxcUH17YpIp6byNH4MwA/N7B4A/wBgI8nlAJ4AsNvM7gawu/haRHqUG3YzGzKz/cXnZwEcBrAUwGoA24tv2w5gTUU9ikgJrusFOpIDAL4KYA+AxWY2BIz/gwDgjjbXWU+ySbLZarUy2xWRTk057CTnAPgdgB+YWXonwAnMbLOZNcys0dfX10mPIlKCKYWd5HSMB/03Zvb74uJTJPuLej+A4WpaFJEyuKM3js9mXgRw2Mx+NqG0C8A6AM8WH1+ppMOSeKMSbxSTqnsjonPnziXr3virSt5YMHd5bupnyx1/eWO/CxcutK3NmTMn67Y93liwDlOZs68C8D0A75E8UFz2JMZD/luSjwA4AeDblXQoIqVww25mfwLQ7p/nb5TbjohURW+XFQlCYRcJQmEXCUJhFwlCYRcJIswSV28W7s2Tc5a4erzlkN48OsVbopq7VDPnlM+5c/acvzPvvQ3eKbw9Xu910JFdJAiFXSQIhV0kCIVdJAiFXSQIhV0kCIVdJIgwc/ZZs2Yl6zlbSefO4L379ubJ3iw957a93rz7ztnm2pvDz5gxI1lPrWf3rpu7Hv2vcitpEbkxKOwiQSjsIkEo7CJBKOwiQSjsIkEo7CJBhJmzV7mPd+4pmXPn6Knr587JvXXfOT+7t+Y7dz/9VG9V/lyAv39CHXRkFwlCYRcJQmEXCUJhFwlCYRcJQmEXCUJhFwliKudnXwbg1wD+BsAVAJvN7BcknwbwLwBaxbc+aWavVdVorqVLlybre/fuTdZz5vQzZ85M1qvckz5nz3nAn0fnrKX35uw57y8A0rPu3MfFs3jx4kpvvxNTmfyPAfihme0nORfA2yTfKGo/N7OfVNeeiJRlKudnHwIwVHx+luRhAOnDpIj0nOt6DkZyAMBXAewpLnqM5Lskt5Bc0OY660k2STZbrdZk3yIiXTDlsJOcA+B3AH5gZmcA/BLAVwCswPiR/6eTXc/MNptZw8wafX19+R2LSEemFHaS0zEe9N+Y2e8BwMxOmdllM7sC4FcAVlbXpojkcsPO8ZdjXwRw2Mx+NuHy/gnf9i0AB8tvT0TKMpVX41cB+B6A90geKC57EsBakisAGIDjADZU0F9pFi1alKwfO3YsWU8tFfVei8hdqpmzLbE3nsodf+X+bFXedur63jj0zjvvzLpvb9Rbh6m8Gv8nAJM9aj07UxeRL9I76ESCUNhFglDYRYJQ2EWCUNhFglDYRYLovf1uK7Jx48Zk/f3330/W582b17bmzcG9eXHurDt1/6Ojo8nreks9vTm997NVOYf3pB5X75TN3uO2Zs2aZH3BgkmXitRKR3aRIBR2kSAUdpEgFHaRIBR2kSAUdpEgFHaRIJizVvq674xsAfjfCRctAjDStQauT6/21qt9AeqtU2X2dpeZTbr/W1fD/oU7J5tm1qitgYRe7a1X+wLUW6e61ZuexosEobCLBFF32DfXfP8pvdpbr/YFqLdOdaW3Wv/PLiLdU/eRXUS6RGEXCaKWsJN8kOT7JI+QfKKOHtoheZzkeyQPkGzW3MsWksMkD064bCHJN0h+UHysZeF0m96eJnmyeOwOkHy4pt6WkfwjycMkD5H8fnF5rY9doq+uPG5d/z87yWkA/gfAPwEYBLAPwFoz+++uNtIGyeMAGmZW+xswSH4dwDkAvzazvysu+zGA02b2bPEP5QIz+9ce6e1pAOfqPo13cbai/omnGQewBsA/o8bHLtHXd9CFx62OI/tKAEfM7KiZXQKwA8DqGvroeWb2FoDT11y8GsD24vPtGP9l6bo2vfUEMxsys/3F52cBXD3NeK2PXaKvrqgj7EsB/HnC14PorfO9G4A/kHyb5Pq6m5nEYjMbAsZ/eQDcUXM/13JP491N15xmvGceu05Of56rjrBPtilZL83/VpnZ1wA8BGBj8XRVpmZKp/HulklOM94TOj39ea46wj4IYNmEr78E4KMa+piUmX1UfBwGsBO9dyrqU1fPoFt8HK65n//TS6fxnuw04+iBx67O05/XEfZ9AO4m+WWSMwB8F8CuGvr4ApKzixdOQHI2gG+i905FvQvAuuLzdQBeqbGXz+mV03i3O804an7saj/9uZl1/Q+AhzH+ivyHAH5URw9t+vpbAP9V/DlUd28AXsL407pRjD8jegTA7QB2A/ig+Liwh3r7dwDvAXgX48Hqr6m3+zD+X8N3ARwo/jxc92OX6Ksrj5veLisShN5BJxKEwi4ShMIuEoTCLhKEwi4ShMIuEoTCLhLEXwBNE6OxcGf3fQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LABEL: 8\n",
      "INPUT:\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAQU0lEQVR4nO3dX4he9Z3H8c/XJEaTTE00owwxmrYEXFldK4MsuNRIUdQb04uG5qK4GEgFhRYqrHaReuGFLGvLCosQV2lcu5aCihFktyKV0JviKGqi0Y3G2EaHmYkmJjGJJpnvXsxxmcY5v+94zvMPv+8XDDPzfOc85zfneT7zzMz3/M7P3F0Avv7O6PcAAPQGYQeSIOxAEoQdSIKwA0ks7OXOVq5c6WvWrOnlLoFU9u7dq/3799tctVZhN7MbJP2bpAWS/sPd7y99/Zo1azQ2NtZmlwAKRkdHa2uNf403swWS/l3SjZIulbTRzC5ten8AuqvN3+xXSXrH3fe4++eSfivp5s4MC0CntQn7Kkl/mfX5vuq2v2Jmm81szMzGpqamWuwOQBttwj7XPwG+dO6tu29x91F3Hx0eHm6xOwBttAn7PkmrZ31+oaQP2w0HQLe0CftLktaa2TfN7ExJP5S0rTPDAtBpjVtv7n7SzO6Q9D+aab096u5vdGxkADqqVZ/d3Z+T9FyHxgKgizhdFkiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5BEqyWbzWyvpMOSTkk66e6jnRgUgM5rFfbKte6+vwP3A6CL+DUeSKJt2F3S783sZTPbPNcXmNlmMxszs7GpqamWuwPQVNuwX+3uV0q6UdLtZvbd07/A3be4+6i7jw4PD7fcHYCmWoXd3T+s3k9KelrSVZ0YFIDOaxx2M1tqZkNffCzpekk7OzUwAJ3V5r/xF0h62sy+uJ//cvf/7sioGjh69Gix/tRTTxXrk5OTxfqqVatqa+5e3PbkyZPF+hlnlH/mRvXS/Uf7jlSPb+N6NPaSxYsXF+snTpwo1o8fP15bO3jwYHHbI0eOFOtDQ0PF+oYNG4r10vMpMj093Wi7xmF39z2S/q7p9gB6i9YbkARhB5Ig7EAShB1IgrADSXRiIsxAuPXWW4v1t99+u1iP2mdnn312ba3U4pHatZ+kuNVy7Nix2lo0tiVLlhTr0dgXLFhQrE9MTNTWorZd6ZhL8XEpPaYjIyPFbaO2X+n7kqQ333yzWH/44YeL9ZKmzyde2YEkCDuQBGEHkiDsQBKEHUiCsANJEHYgiZ722Y8dO6bXXnuttr5u3bri9vfcc09t7fPPPy9ue/755xfrUb/4rLPOalST4um30dgjp06darzvqB5ZuLD8FLroootqa1EvO+qzR49ZqY8fPWaRM888s1h/9913i/XHH3+8tnbfffcVt922bVttrXReBa/sQBKEHUiCsANJEHYgCcIOJEHYgSQIO5CERfO4O2nZsmV++eWX19Y/++yz4valXvmKFSuK25bmfEvlXrVU7ulG27adzx71k0uisUU9/ui4RX320v6jsUV9+KhXvmjRosb7js4/iM7beP/994v1/fvr10IdHx8vbrtx48ba2pNPPqnJyck5TzDglR1IgrADSRB2IAnCDiRB2IEkCDuQBGEHkujpfPbp6eniUrgXX3xxcftDhw7V1qJrkEd91eh8g1I9uu+oH9xmXrZU7nVHc8Ijn3zySbEejb10bKJr2kfLTUf7Ln3vbZfR/vTTT4v1AwcOFOulcwii58uzzz5bWystRR2+spvZo2Y2aWY7Z912rpk9b2a7q/flM1oA9N18fo3/taQbTrvtLkkvuPtaSS9UnwMYYGHY3X27pI9Pu/lmSVurj7dKWt/ZYQHotKZ/s1/g7uOS5O7jZlZ7orCZbZa0WSqfqwygu7r+33h33+Luo+4+Gk2aANA9TcM+YWYjklS9n+zckAB0Q9Owb5N0S/XxLZKe6cxwAHRL+Hu1mT0haZ2klWa2T9IvJN0v6XdmtknSnyX9YD47W716tR588MHa+t13313cvtQ3jXqu0dzoqFce9YRLomuMRz3dNtcciP5PEs1XbzOXXirPl4/GFtXbzEmPjmn0mJT62ZJ0+PDhxve/fPny4rYPPfRQbe22226rrYVhd/e6mfLfi7YFMDg4XRZIgrADSRB2IAnCDiRB2IEkenpK29DQUHFZ5ksuuaS4fbQMbknUSonaPKWz/6I2TtQimp6eLtZPnDhRrJemwEZnLUaX747GFh3XUusuaklGUz2jdmiprdj2MtbR9tH3tm/fvtranXfeWdz22muvra0NDQ3V1nhlB5Ig7EAShB1IgrADSRB2IAnCDiRB2IEkBurSMVFPt9TPji4N3PYqOUuWLKmtRcseRz3ZSHQp6dJxiS55HJ0jEPWyo/MTStOSo+m1bY9r6XuLpu5Gz6fouVpaklmSRkZGamsbNmwobtsUr+xAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kMRA9dkfeOCBYv26666rrUWX3436ptH2pTnlUb+42yvhlL63aL551KtuM189Es1Xjy7H3Ob8hej8gdLS4pK0cuXKYj16zNeuXVtbu+yyy4rbNsUrO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kMVB99uuvv75YL/U2o7nPpXnVknTOOecU63v27KmtRb3maD56JLr/0rztaN512z57pDS26NrqpWugS/GyyaVeeXQt/ui4vffee8X66tWri/Xt27fX1jZt2lTc9pFHHinW64SPpJk9amaTZrZz1m33mtkHZvZq9XZTo70D6Jn5/Nj+taQb5rj9V+5+RfX2XGeHBaDTwrC7+3ZJH/dgLAC6qM0fZHeY2evVr/kr6r7IzDab2ZiZjU1NTbXYHYA2mob9IUnflnSFpHFJtTNY3H2Lu4+6++jw8HDD3QFoq1HY3X3C3U+5+7SkhyVd1dlhAei0RmE3s9nXwf2+pJ11XwtgMIR9djN7QtI6SSvNbJ+kX0haZ2ZXSHJJeyX9eD47O3bsmHbs2FFbX7p0aXH78847r7Z26NCh4rYXXnhhsd5mrfCoj952jfNIm3ndba9pH63vXrr/aM53dE37SOlxiebSR8+H6NyH6DEvndexe/fu4rZNhWF3941z3Nysqw+gbzhdFkiCsANJEHYgCcIOJEHYgSR6OsXV3YtTC6Nphx988EFtrbSksiRNTEyUBxcoXa45ahFF7a2ozdNmKeto39Fxi7RpK0Ytx+i4Ru2vUj26/HdUP3DgQLEeHffSlOuojVy6xHbp8eCVHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeS6GmffcmSJbryyiuL9ZJSHz7qe+7fv79YX7ZsWbFemn4bTZeMlouOLoMdKfV0o+m34+PjxXp07kOb5aij+47OP2jTp287LTm6NHl0/6WpwYsXLy5uW7rEdvG8huK9AvjaIOxAEoQdSIKwA0kQdiAJwg4kQdiBJHraZz958qQ++uij2nrU6y7NT44uaRzNT47mHx8/frxr992mV93WN77xjb7tO9L2EtulXnnp8ZSko0ePFutRL7w051wqP2ei8wua4pUdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5LoaYN34cKFxWWXI2+99VZtLeqLHjlypFiP5i+X5qRH9x31TaOebZt+czSvetGiRcV6dG32SOkcg+i+o+sARPPh2+w7Om7RNe2j+e4HDx6srUXfV1Phs8jMVpvZH8xsl5m9YWY/qW4/18yeN7Pd1fsVXRkhgI6Yz0vGSUk/c/e/kfT3km43s0sl3SXpBXdfK+mF6nMAAyoMu7uPu/sr1ceHJe2StErSzZK2Vl+2VdL6Lo0RQAd8pT8GzWyNpO9I+pOkC9x9XJr5gSDp/JptNpvZmJmNTU1NtRwugKbmHXYzWybpSUk/dffyynOzuPsWdx9199Hh4eEmYwTQAfMKu5kt0kzQf+PuT1U3T5jZSFUfkTTZnSEC6ISw9WYzPYhHJO1y91/OKm2TdIuk+6v3z7QdzGOPPVasv/jii43vO2pvlS4VHYmm17Zt80TaXDI5aiG1XVa5TfsrmvobtahKSx9H20bfdzS2aOpw6RLe11xzTXHbpubTZ79a0o8k7TCzV6vbfq6ZkP/OzDZJ+rOkH3RlhAA6Igy7u/9RUt3Lw/c6OxwA3cLpskAShB1IgrADSRB2IAnCDiTRv2sYz2H58uXF+vr163syDuDriFd2IAnCDiRB2IEkCDuQBGEHkiDsQBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kARhB5Ig7EAShB1IIgy7ma02sz+Y2S4ze8PMflLdfq+ZfWBmr1ZvN3V/uACams8iEScl/czdXzGzIUkvm9nzVe1X7v6v3RsegE6Zz/rs45LGq48Pm9kuSau6PTAAnfWV/mY3szWSviPpT9VNd5jZ62b2qJmtqNlms5mNmdnY1NRUu9ECaGzeYTezZZKelPRTdz8k6SFJ35Z0hWZe+R+Yazt33+Luo+4+Ojw83H7EABqZV9jNbJFmgv4bd39Kktx9wt1Pufu0pIclXdW9YQJoaz7/jTdJj0ja5e6/nHX7yKwv+76knZ0fHoBOmc9/46+W9CNJO8zs1eq2n0vaaGZXSHJJeyX9uAvjA9Ah8/lv/B8l2Ryl5zo/HADdwhl0QBKEHUiCsANJEHYgCcIOJEHYgSQIO5AEYQeSIOxAEoQdSIKwA0kQdiAJwg4kQdiBJMzde7czsylJ78+6aaWk/T0bwFczqGMb1HFJjK2pTo7tYnef8/pvPQ37l3ZuNubuo30bQMGgjm1QxyUxtqZ6NTZ+jQeSIOxAEv0O+5Y+779kUMc2qOOSGFtTPRlbX/9mB9A7/X5lB9AjhB1Ioi9hN7MbzOxtM3vHzO7qxxjqmNleM9tRLUM91uexPGpmk2a2c9Zt55rZ82a2u3o/5xp7fRrbQCzjXVhmvK/Hrt/Ln/f8b3YzWyDpfyVdJ2mfpJckbXT3N3s6kBpmtlfSqLv3/QQMM/uupCOSHnP3v61u+xdJH7v7/dUPyhXu/k8DMrZ7JR3p9zLe1WpFI7OXGZe0XtI/qo/HrjCuDerBcevHK/tVkt5x9z3u/rmk30q6uQ/jGHjuvl3Sx6fdfLOkrdXHWzXzZOm5mrENBHcfd/dXqo8PS/pimfG+HrvCuHqiH2FfJekvsz7fp8Fa790l/d7MXjazzf0ezBwucPdxaebJI+n8Po/ndOEy3r102jLjA3Psmix/3lY/wj7XUlKD1P+72t2vlHSjpNurX1cxP/NaxrtX5lhmfCA0Xf68rX6EfZ+k1bM+v1DSh30Yx5zc/cPq/aSkpzV4S1FPfLGCbvV+ss/j+X+DtIz3XMuMawCOXT+XP+9H2F+StNbMvmlmZ0r6oaRtfRjHl5jZ0uofJzKzpZKu1+AtRb1N0i3Vx7dIeqaPY/krg7KMd90y4+rzsev78ufu3vM3STdp5j/y70r6536MoWZc35L0WvX2Rr/HJukJzfxad0IzvxFtknSepBck7a7enztAY/tPSTskva6ZYI30aWz/oJk/DV+X9Gr1dlO/j11hXD05bpwuCyTBGXRAEoQdSIKwA0kQdiAJwg4kQdiBJAg7kMT/ATUivi8jflfhAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LABEL: 8\n"
     ]
    }
   ],
   "source": [
    "#let's try the plotting function\n",
    "plot_input(X_train,y_train,10)\n",
    "plot_input(X_test,y_test,50)\n",
    "plot_input(X_test,y_test,300)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TO DO 1\n",
    "\n",
    "Now use a (feed-forward) Neural Network for prediction. Use the multi-layer perceptron (MLP) classifier MLPClassifier(...) in scikit-learn, with the following parameters: max_iter=300, alpha=1e-4, solver='sgd', tol=1e-4, learning_rate_init=.1, random_state=ID (this last parameter ensures the run is the same even if you run it more than once). The alpha parameter is the regularization parameter for L2 regularization that is used by the MLP in sklearn.\n",
    "\n",
    "Then, using the default activation function, pick four or five architectures to consider, with different numbers of hidden layers and different sizes. It is not necessary to create huge neural networks, you can limit to 3 layers and, for each layer, its maximum size can be of 100. You can evaluate the architectures you chose using the GridSearchCV with a 5-fold cross-validation, and use the results to pick the best architecture. The code below provides some architectures you can use, but you can choose other ones if you prefer.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RESULTS FOR NN\n",
      "\n",
      "Best parameters set found:\n",
      "{'hidden_layer_sizes': (50,)}\n",
      "Score with best parameters:\n",
      "0.792\n",
      "\n",
      "All scores on the grid:\n",
      "{'mean_fit_time': array([0.67497864, 0.96655545, 0.26948581, 0.59302826]), 'std_fit_time': array([0.16834269, 0.03805584, 0.03209772, 0.22659953]), 'mean_score_time': array([0.00099649, 0.00119662, 0.00079808, 0.00099711]), 'std_score_time': array([7.62939453e-07, 3.97920966e-04, 3.99042696e-04, 6.64157308e-07]), 'param_hidden_layer_sizes': masked_array(data=[(10,), (50,), (10, 10), (50, 50)],\n",
      "             mask=[False, False, False, False],\n",
      "       fill_value='?',\n",
      "            dtype=object), 'params': [{'hidden_layer_sizes': (10,)}, {'hidden_layer_sizes': (50,)}, {'hidden_layer_sizes': (10, 10)}, {'hidden_layer_sizes': (50, 50)}], 'split0_test_score': array([0.38, 0.8 , 0.63, 0.75]), 'split1_test_score': array([0.71, 0.81, 0.55, 0.79]), 'split2_test_score': array([0.74, 0.79, 0.78, 0.77]), 'split3_test_score': array([0.72, 0.77, 0.6 , 0.79]), 'split4_test_score': array([0.7 , 0.79, 0.1 , 0.78]), 'mean_test_score': array([0.65 , 0.792, 0.532, 0.776]), 'std_test_score': array([0.1356466 , 0.0132665 , 0.22920733, 0.01496663]), 'rank_test_score': array([3, 1, 4, 2])}\n"
     ]
    }
   ],
   "source": [
    "#MLPclassifier requires in input the parameter hidden_layer_sizes, that is a tuple specifying the number of \n",
    "#neurons in the hidden layers; for example: (10,) means that there is only 1 hidden layer with 10 neurons; \n",
    "#(10,50) means that there are 2 hidden layers, the first with 10 neurons, the second with 50 neurons\n",
    "\n",
    "#these are examples of possible architectures you can test, but feel free to use different architectures! \n",
    "hl_parameters = {'hidden_layer_sizes': [(10,), (50,), (10,10,), (50,50,)]}\n",
    "\n",
    "mlp_cv = MLPClassifier(max_iter=300, alpha=1e-4, solver='sgd', tol=1e-4, learning_rate_init=.1, random_state=ID)\n",
    "gridSearch = GridSearchCV(mlp_cv, hl_parameters, cv=5)\n",
    "gridSearch.fit(X_train,y_train)\n",
    "    \n",
    "print ('RESULTS FOR NN\\n')\n",
    "\n",
    "print(\"Best parameters set found:\")\n",
    "print(gridSearch.best_params_)\n",
    "\n",
    "print(\"Score with best parameters:\")\n",
    "print(gridSearch.best_score_)\n",
    "\n",
    "print(\"\\nAll scores on the grid:\")\n",
    "print(gridSearch.cv_results_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TO DO 2\n",
    "\n",
    "What do you observe for different architectures and their scores? How do the number of layers and their sizes affect the performances?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The result for 10 neurons and 1 hidden layer the score is 0.65\n",
    "The result for 50 neurons and 1 hidden layer the score is 0.792\n",
    "The result for 10 neurons and 2 hidden layer the score is 0.532\n",
    "The result for 50 neurons and 2 hidden layer the score is 0.776 \n",
    "\n",
    "Best result is obtained when the number of neurons are increased whereas there is no much change in result when hidden layer is increased"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TO DO 3\n",
    "\n",
    "Now get training and test error (according to the initial split) for a NN with best parameters chosen from the cross-validation above (and learning the NN weights from the entire training set). Use verbose=True\n",
    "in input so to see how loss changes in iterations. (Note that the loss used by the MLPclassifier may be different from the 0-1 loss, also called *accuracy*.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iteration 1, loss = 2.23491903\n",
      "Iteration 2, loss = 1.65312581\n",
      "Iteration 3, loss = 1.39238340\n",
      "Iteration 4, loss = 0.94868744\n",
      "Iteration 5, loss = 0.95214853\n",
      "Iteration 6, loss = 0.82783418\n",
      "Iteration 7, loss = 0.68006396\n",
      "Iteration 8, loss = 0.60126940\n",
      "Iteration 9, loss = 0.55428932\n",
      "Iteration 10, loss = 0.50118175\n",
      "Iteration 11, loss = 0.48019322\n",
      "Iteration 12, loss = 0.42746870\n",
      "Iteration 13, loss = 0.40621904\n",
      "Iteration 14, loss = 0.32164951\n",
      "Iteration 15, loss = 0.32727930\n",
      "Iteration 16, loss = 0.38149245\n",
      "Iteration 17, loss = 0.26987695\n",
      "Iteration 18, loss = 0.27374162\n",
      "Iteration 19, loss = 0.23157931\n",
      "Iteration 20, loss = 0.31955497\n",
      "Iteration 21, loss = 0.20953823\n",
      "Iteration 22, loss = 0.19146667\n",
      "Iteration 23, loss = 0.17265256\n",
      "Iteration 24, loss = 0.18594496\n",
      "Iteration 25, loss = 0.27227754\n",
      "Iteration 26, loss = 0.21490140\n",
      "Iteration 27, loss = 0.16040825\n",
      "Iteration 28, loss = 0.14964326\n",
      "Iteration 29, loss = 0.17283541\n",
      "Iteration 30, loss = 0.20290663\n",
      "Iteration 31, loss = 0.17095906\n",
      "Iteration 32, loss = 0.09971637\n",
      "Iteration 33, loss = 0.08826478\n",
      "Iteration 34, loss = 0.08538536\n",
      "Iteration 35, loss = 0.07678185\n",
      "Iteration 36, loss = 0.08431189\n",
      "Iteration 37, loss = 0.07979650\n",
      "Iteration 38, loss = 0.16448143\n",
      "Iteration 39, loss = 0.14163033\n",
      "Iteration 40, loss = 0.06614859\n",
      "Iteration 41, loss = 0.05559328\n",
      "Iteration 42, loss = 0.05971223\n",
      "Iteration 43, loss = 0.05334380\n",
      "Iteration 44, loss = 0.04477627\n",
      "Iteration 45, loss = 0.04780368\n",
      "Iteration 46, loss = 0.03780384\n",
      "Iteration 47, loss = 0.03618613\n",
      "Iteration 48, loss = 0.03317051\n",
      "Iteration 49, loss = 0.03438012\n",
      "Iteration 50, loss = 0.03317725\n",
      "Iteration 51, loss = 0.03470568\n",
      "Iteration 52, loss = 0.02813227\n",
      "Iteration 53, loss = 0.02649521\n",
      "Iteration 54, loss = 0.02694411\n",
      "Iteration 55, loss = 0.02455538\n",
      "Iteration 56, loss = 0.02638552\n",
      "Iteration 57, loss = 0.03315480\n",
      "Iteration 58, loss = 0.02160265\n",
      "Iteration 59, loss = 0.02012935\n",
      "Iteration 60, loss = 0.02028987\n",
      "Iteration 61, loss = 0.01864363\n",
      "Iteration 62, loss = 0.01998649\n",
      "Iteration 63, loss = 0.01780610\n",
      "Iteration 64, loss = 0.01812111\n",
      "Iteration 65, loss = 0.01588105\n",
      "Iteration 66, loss = 0.01517374\n",
      "Iteration 67, loss = 0.01502516\n",
      "Iteration 68, loss = 0.01454002\n",
      "Iteration 69, loss = 0.01388756\n",
      "Iteration 70, loss = 0.01382663\n",
      "Iteration 71, loss = 0.01312566\n",
      "Iteration 72, loss = 0.01306304\n",
      "Iteration 73, loss = 0.01221908\n",
      "Iteration 74, loss = 0.01241014\n",
      "Iteration 75, loss = 0.01169710\n",
      "Iteration 76, loss = 0.01174548\n",
      "Iteration 77, loss = 0.01102452\n",
      "Iteration 78, loss = 0.01096250\n",
      "Iteration 79, loss = 0.01042870\n",
      "Iteration 80, loss = 0.01021668\n",
      "Iteration 81, loss = 0.00997063\n",
      "Iteration 82, loss = 0.00981628\n",
      "Iteration 83, loss = 0.00968432\n",
      "Iteration 84, loss = 0.00935086\n",
      "Iteration 85, loss = 0.00931229\n",
      "Iteration 86, loss = 0.00915253\n",
      "Iteration 87, loss = 0.00879651\n",
      "Iteration 88, loss = 0.00870157\n",
      "Iteration 89, loss = 0.00847841\n",
      "Iteration 90, loss = 0.00833403\n",
      "Iteration 91, loss = 0.00817054\n",
      "Iteration 92, loss = 0.00798444\n",
      "Iteration 93, loss = 0.00795310\n",
      "Iteration 94, loss = 0.00784930\n",
      "Iteration 95, loss = 0.00756874\n",
      "Iteration 96, loss = 0.00754739\n",
      "Iteration 97, loss = 0.00746602\n",
      "Iteration 98, loss = 0.00714852\n",
      "Iteration 99, loss = 0.00711265\n",
      "Iteration 100, loss = 0.00695263\n",
      "Iteration 101, loss = 0.00687172\n",
      "Iteration 102, loss = 0.00669678\n",
      "Iteration 103, loss = 0.00674397\n",
      "Iteration 104, loss = 0.00653825\n",
      "Iteration 105, loss = 0.00648908\n",
      "Iteration 106, loss = 0.00640641\n",
      "Iteration 107, loss = 0.00639659\n",
      "Iteration 108, loss = 0.00617655\n",
      "Iteration 109, loss = 0.00616551\n",
      "Iteration 110, loss = 0.00610756\n",
      "Iteration 111, loss = 0.00612010\n",
      "Iteration 112, loss = 0.00598451\n",
      "Iteration 113, loss = 0.00575291\n",
      "Iteration 114, loss = 0.00569262\n",
      "Iteration 115, loss = 0.00561780\n",
      "Iteration 116, loss = 0.00551546\n",
      "Iteration 117, loss = 0.00552306\n",
      "Iteration 118, loss = 0.00536921\n",
      "Iteration 119, loss = 0.00532779\n",
      "Iteration 120, loss = 0.00523108\n",
      "Iteration 121, loss = 0.00513925\n",
      "Iteration 122, loss = 0.00514707\n",
      "Iteration 123, loss = 0.00508249\n",
      "Iteration 124, loss = 0.00496849\n",
      "Iteration 125, loss = 0.00493705\n",
      "Iteration 126, loss = 0.00482176\n",
      "Iteration 127, loss = 0.00485244\n",
      "Iteration 128, loss = 0.00474512\n",
      "Iteration 129, loss = 0.00471414\n",
      "Iteration 130, loss = 0.00466621\n",
      "Iteration 131, loss = 0.00455180\n",
      "Iteration 132, loss = 0.00457542\n",
      "Iteration 133, loss = 0.00446377\n",
      "Iteration 134, loss = 0.00443313\n",
      "Iteration 135, loss = 0.00434429\n",
      "Iteration 136, loss = 0.00431274\n",
      "Iteration 137, loss = 0.00426423\n",
      "Iteration 138, loss = 0.00424446\n",
      "Iteration 139, loss = 0.00417999\n",
      "Iteration 140, loss = 0.00421593\n",
      "Iteration 141, loss = 0.00417801\n",
      "Iteration 142, loss = 0.00411158\n",
      "Training loss did not improve more than tol=0.000100 for 10 consecutive epochs. Stopping.\n",
      "\n",
      "RESULTS FOR BEST NN\n",
      "\n",
      "Best NN training error: 0.000000\n",
      "Best NN test error: 0.212403\n"
     ]
    }
   ],
   "source": [
    "#get training and test error for the best NN model from CV\n",
    "\n",
    "mlp = MLPClassifier(max_iter=300, alpha=1e-4, solver='sgd', tol=1e-4, learning_rate_init=.1, random_state=ID, hidden_layer_sizes=(25,), verbose=True)\n",
    "mlp.fit(X_train,y_train)\n",
    "\n",
    "training_error = 1. - mlp.score(X_train, y_train)\n",
    "\n",
    "test_error = 1. - mlp.score(X_test, y_test)\n",
    "\n",
    "print ('\\nRESULTS FOR BEST NN\\n')\n",
    "\n",
    "print (\"Best NN training error: %f\" % training_error)\n",
    "print (\"Best NN test error: %f\" % test_error)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## More data \n",
    "Now let's do the same but using 10000 (or less if it takes too long on your machine) data points for training. Use the same NN architectures as before, but you can try more if you want!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Labels and frequencies in training dataset: \n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=uint8),\n",
       " array([1034,  994,  997, 1009,  993,  995,  993, 1021,  993,  971],\n",
       "       dtype=int64))"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = X[permutation]\n",
    "y = y[permutation]\n",
    "\n",
    "m_training = 10000\n",
    "\n",
    "X_train, X_test = X[:m_training], X[m_training:]\n",
    "y_train, y_test = y[:m_training], y[m_training:]\n",
    "\n",
    "print(\"Labels and frequencies in training dataset: \")\n",
    "np.unique(y_train, return_counts=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TO DO 4\n",
    "\n",
    "Now train the NNs with the added data points. Feel free to try more different architectures than before if you want, or less if it takes too much time. You can use 'verbose=True' so have an idea of how long it takes to run 1 iteration (eventually reduce also the number of iterations to 50)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 4 candidates, totalling 20 fits\n",
      "Iteration 1, loss = 1.23587398\n",
      "Iteration 2, loss = 0.65936105\n",
      "Iteration 3, loss = 0.54151440\n",
      "Iteration 4, loss = 0.49385153\n",
      "Iteration 5, loss = 0.48494783\n",
      "Iteration 6, loss = 0.45150179\n",
      "Iteration 7, loss = 0.43163933\n",
      "Iteration 8, loss = 0.41310924\n",
      "Iteration 9, loss = 0.41431347\n",
      "Iteration 10, loss = 0.39808539\n",
      "Iteration 11, loss = 0.39731951\n",
      "Iteration 12, loss = 0.38004774\n",
      "Iteration 13, loss = 0.36683556\n",
      "Iteration 14, loss = 0.37763441\n",
      "Iteration 15, loss = 0.35815620\n",
      "Iteration 16, loss = 0.35895180\n",
      "Iteration 17, loss = 0.35689411\n",
      "Iteration 18, loss = 0.34525345\n",
      "Iteration 19, loss = 0.35070440\n",
      "Iteration 20, loss = 0.34649555\n",
      "Iteration 21, loss = 0.33038446\n",
      "Iteration 22, loss = 0.32675327\n",
      "Iteration 23, loss = 0.32080759\n",
      "Iteration 24, loss = 0.31963739\n",
      "Iteration 25, loss = 0.33073655\n",
      "Iteration 26, loss = 0.31867702\n",
      "Iteration 27, loss = 0.31356816\n",
      "Iteration 28, loss = 0.31014858\n",
      "Iteration 29, loss = 0.30637664\n",
      "Iteration 30, loss = 0.31438370\n",
      "Iteration 31, loss = 0.29798886\n",
      "Iteration 32, loss = 0.31016254\n",
      "Iteration 33, loss = 0.29558043\n",
      "Iteration 34, loss = 0.28373417\n",
      "Iteration 35, loss = 0.28224505\n",
      "Iteration 36, loss = 0.28231645\n",
      "Iteration 37, loss = 0.28412738\n",
      "Iteration 38, loss = 0.28435457\n",
      "Iteration 39, loss = 0.28335815\n",
      "Iteration 40, loss = 0.28409554\n",
      "Iteration 41, loss = 0.28290619\n",
      "Iteration 42, loss = 0.27143385\n",
      "Iteration 43, loss = 0.28191156\n",
      "Iteration 44, loss = 0.27640979\n",
      "Iteration 45, loss = 0.27513747\n",
      "Iteration 46, loss = 0.27475180\n",
      "Iteration 47, loss = 0.27790929\n",
      "Iteration 48, loss = 0.26216122\n",
      "Iteration 49, loss = 0.26681233\n",
      "Iteration 50, loss = 0.25372820\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "A:\\InstallationDriver\\Anaconda\\anaconda3\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:614: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (50) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iteration 1, loss = 1.25726189\n",
      "Iteration 2, loss = 0.66855680\n",
      "Iteration 3, loss = 0.53680780\n",
      "Iteration 4, loss = 0.49045834\n",
      "Iteration 5, loss = 0.48981272\n",
      "Iteration 6, loss = 0.45749934\n",
      "Iteration 7, loss = 0.44039103\n",
      "Iteration 8, loss = 0.42130033\n",
      "Iteration 9, loss = 0.42144670\n",
      "Iteration 10, loss = 0.39993472\n",
      "Iteration 11, loss = 0.40494530\n",
      "Iteration 12, loss = 0.37465901\n",
      "Iteration 13, loss = 0.38146512\n",
      "Iteration 14, loss = 0.37827126\n",
      "Iteration 15, loss = 0.36655644\n",
      "Iteration 16, loss = 0.36725868\n",
      "Iteration 17, loss = 0.36140347\n",
      "Iteration 18, loss = 0.35081333\n",
      "Iteration 19, loss = 0.35022994\n",
      "Iteration 20, loss = 0.34213559\n",
      "Iteration 21, loss = 0.34614536\n",
      "Iteration 22, loss = 0.34979740\n",
      "Iteration 23, loss = 0.32829712\n",
      "Iteration 24, loss = 0.32939645\n",
      "Iteration 25, loss = 0.33373366\n",
      "Iteration 26, loss = 0.32015835\n",
      "Iteration 27, loss = 0.32405950\n",
      "Iteration 28, loss = 0.31224441\n",
      "Iteration 29, loss = 0.30837069\n",
      "Iteration 30, loss = 0.31267110\n",
      "Iteration 31, loss = 0.31259982\n",
      "Iteration 32, loss = 0.30787576\n",
      "Iteration 33, loss = 0.30532307\n",
      "Iteration 34, loss = 0.28999011\n",
      "Iteration 35, loss = 0.30014887\n",
      "Iteration 36, loss = 0.29316484\n",
      "Iteration 37, loss = 0.28590764\n",
      "Iteration 38, loss = 0.29521087\n",
      "Iteration 39, loss = 0.29619921\n",
      "Iteration 40, loss = 0.29684911\n",
      "Iteration 41, loss = 0.28760052\n",
      "Iteration 42, loss = 0.28404509\n",
      "Iteration 43, loss = 0.27958895\n",
      "Iteration 44, loss = 0.27581128\n",
      "Iteration 45, loss = 0.27074622\n",
      "Iteration 46, loss = 0.28412877\n",
      "Iteration 47, loss = 0.29037676\n",
      "Iteration 48, loss = 0.28556805\n",
      "Iteration 49, loss = 0.26591713\n",
      "Iteration 50, loss = 0.26836327\n",
      "Iteration 1, loss = 1.25882218\n",
      "Iteration 2, loss = 0.66430507\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "A:\\InstallationDriver\\Anaconda\\anaconda3\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:614: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (50) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iteration 3, loss = 0.56480652\n",
      "Iteration 4, loss = 0.50085580\n",
      "Iteration 5, loss = 0.49854607\n",
      "Iteration 6, loss = 0.46649683\n",
      "Iteration 7, loss = 0.44780806\n",
      "Iteration 8, loss = 0.41994749\n",
      "Iteration 9, loss = 0.41006418\n",
      "Iteration 10, loss = 0.40460474\n",
      "Iteration 11, loss = 0.39546968\n",
      "Iteration 12, loss = 0.37240638\n",
      "Iteration 13, loss = 0.37805694\n",
      "Iteration 14, loss = 0.37384895\n",
      "Iteration 15, loss = 0.36150245\n",
      "Iteration 16, loss = 0.34677600\n",
      "Iteration 17, loss = 0.35083096\n",
      "Iteration 18, loss = 0.34951451\n",
      "Iteration 19, loss = 0.34287402\n",
      "Iteration 20, loss = 0.33651471\n",
      "Iteration 21, loss = 0.33988896\n",
      "Iteration 22, loss = 0.33439686\n",
      "Iteration 23, loss = 0.32448438\n",
      "Iteration 24, loss = 0.31781136\n",
      "Iteration 25, loss = 0.31203681\n",
      "Iteration 26, loss = 0.31249348\n",
      "Iteration 27, loss = 0.31793986\n",
      "Iteration 28, loss = 0.30961641\n",
      "Iteration 29, loss = 0.30364679\n",
      "Iteration 30, loss = 0.32068186\n",
      "Iteration 31, loss = 0.29978208\n",
      "Iteration 32, loss = 0.30462007\n",
      "Iteration 33, loss = 0.29399890\n",
      "Iteration 34, loss = 0.29085137\n",
      "Iteration 35, loss = 0.28603925\n",
      "Iteration 36, loss = 0.28255823\n",
      "Iteration 37, loss = 0.28215683\n",
      "Iteration 38, loss = 0.28261378\n",
      "Iteration 39, loss = 0.28308682\n",
      "Iteration 40, loss = 0.28936598\n",
      "Iteration 41, loss = 0.27760208\n",
      "Iteration 42, loss = 0.26315881\n",
      "Iteration 43, loss = 0.27324739\n",
      "Iteration 44, loss = 0.26930160\n",
      "Iteration 45, loss = 0.26716015\n",
      "Iteration 46, loss = 0.27418343\n",
      "Iteration 47, loss = 0.25384500\n",
      "Iteration 48, loss = 0.27040876\n",
      "Iteration 49, loss = 0.27482216\n",
      "Iteration 50, loss = 0.26416137\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "A:\\InstallationDriver\\Anaconda\\anaconda3\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:614: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (50) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iteration 1, loss = 1.24567495\n",
      "Iteration 2, loss = 0.63442133\n",
      "Iteration 3, loss = 0.53288675\n",
      "Iteration 4, loss = 0.47039829\n",
      "Iteration 5, loss = 0.45439512\n",
      "Iteration 6, loss = 0.44070402\n",
      "Iteration 7, loss = 0.43161351\n",
      "Iteration 8, loss = 0.40379499\n",
      "Iteration 9, loss = 0.40560964\n",
      "Iteration 10, loss = 0.39292444\n",
      "Iteration 11, loss = 0.38335170\n",
      "Iteration 12, loss = 0.37458042\n",
      "Iteration 13, loss = 0.36426545\n",
      "Iteration 14, loss = 0.36204614\n",
      "Iteration 15, loss = 0.36573608\n",
      "Iteration 16, loss = 0.35025439\n",
      "Iteration 17, loss = 0.33526211\n",
      "Iteration 18, loss = 0.34980969\n",
      "Iteration 19, loss = 0.33844228\n",
      "Iteration 20, loss = 0.32909306\n",
      "Iteration 21, loss = 0.33285837\n",
      "Iteration 22, loss = 0.32269523\n",
      "Iteration 23, loss = 0.33350310\n",
      "Iteration 24, loss = 0.31526877\n",
      "Iteration 25, loss = 0.31531497\n",
      "Iteration 26, loss = 0.31068174\n",
      "Iteration 27, loss = 0.32296418\n",
      "Iteration 28, loss = 0.30516691\n",
      "Iteration 29, loss = 0.29803251\n",
      "Iteration 30, loss = 0.30803810\n",
      "Iteration 31, loss = 0.30439571\n",
      "Iteration 32, loss = 0.30146680\n",
      "Iteration 33, loss = 0.30078061\n",
      "Iteration 34, loss = 0.29407722\n",
      "Iteration 35, loss = 0.28237979\n",
      "Iteration 36, loss = 0.27150145\n",
      "Iteration 37, loss = 0.28371185\n",
      "Iteration 38, loss = 0.27395914\n",
      "Iteration 39, loss = 0.28011491\n",
      "Iteration 40, loss = 0.28045460\n",
      "Iteration 41, loss = 0.27568192\n",
      "Iteration 42, loss = 0.26725151\n",
      "Iteration 43, loss = 0.27804991\n",
      "Iteration 44, loss = 0.26584110\n",
      "Iteration 45, loss = 0.27424995\n",
      "Iteration 46, loss = 0.25890470\n",
      "Iteration 47, loss = 0.26594668\n",
      "Iteration 48, loss = 0.27099261\n",
      "Iteration 49, loss = 0.26108780\n",
      "Iteration 50, loss = 0.27804844\n",
      "Iteration 1, loss = 1.30948067\n",
      "Iteration 2, loss = 0.67971655\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "A:\\InstallationDriver\\Anaconda\\anaconda3\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:614: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (50) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iteration 3, loss = 0.61032328\n",
      "Iteration 4, loss = 0.54860803\n",
      "Iteration 5, loss = 0.52952644\n",
      "Iteration 6, loss = 0.51848945\n",
      "Iteration 7, loss = 0.48460041\n",
      "Iteration 8, loss = 0.45719661\n",
      "Iteration 9, loss = 0.44266024\n",
      "Iteration 10, loss = 0.45234613\n",
      "Iteration 11, loss = 0.42037097\n",
      "Iteration 12, loss = 0.41549225\n",
      "Iteration 13, loss = 0.41661733\n",
      "Iteration 14, loss = 0.40809612\n",
      "Iteration 15, loss = 0.40565847\n",
      "Iteration 16, loss = 0.39056021\n",
      "Iteration 17, loss = 0.38709168\n",
      "Iteration 18, loss = 0.37590114\n",
      "Iteration 19, loss = 0.37736672\n",
      "Iteration 20, loss = 0.37750103\n",
      "Iteration 21, loss = 0.38363303\n",
      "Iteration 22, loss = 0.36499897\n",
      "Iteration 23, loss = 0.37173106\n",
      "Iteration 24, loss = 0.35957142\n",
      "Iteration 25, loss = 0.35759035\n",
      "Iteration 26, loss = 0.35429087\n",
      "Iteration 27, loss = 0.34860665\n",
      "Iteration 28, loss = 0.35514605\n",
      "Iteration 29, loss = 0.34553763\n",
      "Iteration 30, loss = 0.34903192\n",
      "Iteration 31, loss = 0.33948746\n",
      "Iteration 32, loss = 0.33143027\n",
      "Iteration 33, loss = 0.33483746\n",
      "Iteration 34, loss = 0.33940775\n",
      "Iteration 35, loss = 0.31548699\n",
      "Iteration 36, loss = 0.32329209\n",
      "Iteration 37, loss = 0.31819712\n",
      "Iteration 38, loss = 0.32314009\n",
      "Iteration 39, loss = 0.32364978\n",
      "Iteration 40, loss = 0.30842670\n",
      "Iteration 41, loss = 0.31406647\n",
      "Iteration 42, loss = 0.30753535\n",
      "Iteration 43, loss = 0.31933669\n",
      "Iteration 44, loss = 0.30834456\n",
      "Iteration 45, loss = 0.31216225\n",
      "Iteration 46, loss = 0.30992621\n",
      "Iteration 47, loss = 0.30738717\n",
      "Iteration 48, loss = 0.31731703\n",
      "Iteration 49, loss = 0.30683002\n",
      "Iteration 50, loss = 0.30742934\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "A:\\InstallationDriver\\Anaconda\\anaconda3\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:614: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (50) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iteration 1, loss = 1.02204925\n",
      "Iteration 2, loss = 0.52846944\n",
      "Iteration 3, loss = 0.47154031\n",
      "Iteration 4, loss = 0.42223781\n",
      "Iteration 5, loss = 0.40212757\n",
      "Iteration 6, loss = 0.38273035\n",
      "Iteration 7, loss = 0.35142492\n",
      "Iteration 8, loss = 0.34452651\n",
      "Iteration 9, loss = 0.33769786\n",
      "Iteration 10, loss = 0.31860835\n",
      "Iteration 11, loss = 0.30582747\n",
      "Iteration 12, loss = 0.29510442\n",
      "Iteration 13, loss = 0.29111403\n",
      "Iteration 14, loss = 0.28135521\n",
      "Iteration 15, loss = 0.27076683\n",
      "Iteration 16, loss = 0.25606552\n",
      "Iteration 17, loss = 0.25187788\n",
      "Iteration 18, loss = 0.24518380\n",
      "Iteration 19, loss = 0.23577627\n",
      "Iteration 20, loss = 0.24206249\n",
      "Iteration 21, loss = 0.23233894\n",
      "Iteration 22, loss = 0.21781768\n",
      "Iteration 23, loss = 0.21698257\n",
      "Iteration 24, loss = 0.21587081\n",
      "Iteration 25, loss = 0.23187125\n",
      "Iteration 26, loss = 0.19548814\n",
      "Iteration 27, loss = 0.19236970\n",
      "Iteration 28, loss = 0.18282426\n",
      "Iteration 29, loss = 0.18801362\n",
      "Iteration 30, loss = 0.18943795\n",
      "Iteration 31, loss = 0.17717575\n",
      "Iteration 32, loss = 0.18247080\n",
      "Iteration 33, loss = 0.17143786\n",
      "Iteration 34, loss = 0.17263383\n",
      "Iteration 35, loss = 0.15719093\n",
      "Iteration 36, loss = 0.15491916\n",
      "Iteration 37, loss = 0.14938968\n",
      "Iteration 38, loss = 0.14709868\n",
      "Iteration 39, loss = 0.14558968\n",
      "Iteration 40, loss = 0.14272278\n",
      "Iteration 41, loss = 0.14254849\n",
      "Iteration 42, loss = 0.13923524\n",
      "Iteration 43, loss = 0.12667952\n",
      "Iteration 44, loss = 0.12399331\n",
      "Iteration 45, loss = 0.12308288\n",
      "Iteration 46, loss = 0.12297097\n",
      "Iteration 47, loss = 0.12364132\n",
      "Iteration 48, loss = 0.12298727\n",
      "Iteration 49, loss = 0.11518959\n",
      "Iteration 50, loss = 0.11036485\n",
      "Iteration 1, loss = 0.97585508\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "A:\\InstallationDriver\\Anaconda\\anaconda3\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:614: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (50) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iteration 2, loss = 0.53182140\n",
      "Iteration 3, loss = 0.46414198\n",
      "Iteration 4, loss = 0.42823289\n",
      "Iteration 5, loss = 0.40332828\n",
      "Iteration 6, loss = 0.38605251\n",
      "Iteration 7, loss = 0.35325965\n",
      "Iteration 8, loss = 0.34517668\n",
      "Iteration 9, loss = 0.32801200\n",
      "Iteration 10, loss = 0.31721743\n",
      "Iteration 11, loss = 0.30514699\n",
      "Iteration 12, loss = 0.29973250\n",
      "Iteration 13, loss = 0.28771119\n",
      "Iteration 14, loss = 0.29683461\n",
      "Iteration 15, loss = 0.27255984\n",
      "Iteration 16, loss = 0.26717333\n",
      "Iteration 17, loss = 0.25453813\n",
      "Iteration 18, loss = 0.25186008\n",
      "Iteration 19, loss = 0.24410071\n",
      "Iteration 20, loss = 0.23633097\n",
      "Iteration 21, loss = 0.22871941\n",
      "Iteration 22, loss = 0.22368264\n",
      "Iteration 23, loss = 0.23170120\n",
      "Iteration 24, loss = 0.21802035\n",
      "Iteration 25, loss = 0.21295836\n",
      "Iteration 26, loss = 0.19552538\n",
      "Iteration 27, loss = 0.19079388\n",
      "Iteration 28, loss = 0.19060359\n",
      "Iteration 29, loss = 0.20147476\n",
      "Iteration 30, loss = 0.18654306\n",
      "Iteration 31, loss = 0.17193075\n",
      "Iteration 32, loss = 0.17967590\n",
      "Iteration 33, loss = 0.17168587\n",
      "Iteration 34, loss = 0.17031909\n",
      "Iteration 35, loss = 0.16577684\n",
      "Iteration 36, loss = 0.15473001\n",
      "Iteration 37, loss = 0.15556070\n",
      "Iteration 38, loss = 0.15337017\n",
      "Iteration 39, loss = 0.16800401\n",
      "Iteration 40, loss = 0.14416282\n",
      "Iteration 41, loss = 0.13979444\n",
      "Iteration 42, loss = 0.13899102\n",
      "Iteration 43, loss = 0.13323407\n",
      "Iteration 44, loss = 0.12509203\n",
      "Iteration 45, loss = 0.11819994\n",
      "Iteration 46, loss = 0.13441720\n",
      "Iteration 47, loss = 0.13105934\n",
      "Iteration 48, loss = 0.12057777\n",
      "Iteration 49, loss = 0.12699677\n",
      "Iteration 50, loss = 0.12175675\n",
      "Iteration 1, loss = 0.96094659\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "A:\\InstallationDriver\\Anaconda\\anaconda3\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:614: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (50) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iteration 2, loss = 0.51974390\n",
      "Iteration 3, loss = 0.45325911\n",
      "Iteration 4, loss = 0.41473674\n",
      "Iteration 5, loss = 0.39674561\n",
      "Iteration 6, loss = 0.36618403\n",
      "Iteration 7, loss = 0.34419002\n",
      "Iteration 8, loss = 0.33715489\n",
      "Iteration 9, loss = 0.32245070\n",
      "Iteration 10, loss = 0.30553541\n",
      "Iteration 11, loss = 0.29209471\n",
      "Iteration 12, loss = 0.29317372\n",
      "Iteration 13, loss = 0.26462440\n",
      "Iteration 14, loss = 0.27508225\n",
      "Iteration 15, loss = 0.26029545\n",
      "Iteration 16, loss = 0.25612316\n",
      "Iteration 17, loss = 0.24115598\n",
      "Iteration 18, loss = 0.24173323\n",
      "Iteration 19, loss = 0.23183028\n",
      "Iteration 20, loss = 0.22474933\n",
      "Iteration 21, loss = 0.21061501\n",
      "Iteration 22, loss = 0.22825001\n",
      "Iteration 23, loss = 0.21832399\n",
      "Iteration 24, loss = 0.20124838\n",
      "Iteration 25, loss = 0.19930125\n",
      "Iteration 26, loss = 0.19193117\n",
      "Iteration 27, loss = 0.17539607\n",
      "Iteration 28, loss = 0.18501286\n",
      "Iteration 29, loss = 0.17898176\n",
      "Iteration 30, loss = 0.17332242\n",
      "Iteration 31, loss = 0.16104996\n",
      "Iteration 32, loss = 0.16261552\n",
      "Iteration 33, loss = 0.17476651\n",
      "Iteration 34, loss = 0.14933432\n",
      "Iteration 35, loss = 0.15353900\n",
      "Iteration 36, loss = 0.15331781\n",
      "Iteration 37, loss = 0.15939206\n",
      "Iteration 38, loss = 0.15093825\n",
      "Iteration 39, loss = 0.14245524\n",
      "Iteration 40, loss = 0.13844049\n",
      "Iteration 41, loss = 0.13472301\n",
      "Iteration 42, loss = 0.12298521\n",
      "Iteration 43, loss = 0.12250876\n",
      "Iteration 44, loss = 0.12139805\n",
      "Iteration 45, loss = 0.10833346\n",
      "Iteration 46, loss = 0.11405142\n",
      "Iteration 47, loss = 0.12239429\n",
      "Iteration 48, loss = 0.11171105\n",
      "Iteration 49, loss = 0.10840508\n",
      "Iteration 50, loss = 0.10168689\n",
      "Iteration 1, loss = 0.95892386\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "A:\\InstallationDriver\\Anaconda\\anaconda3\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:614: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (50) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iteration 2, loss = 0.52175089\n",
      "Iteration 3, loss = 0.44924300\n",
      "Iteration 4, loss = 0.41527248\n",
      "Iteration 5, loss = 0.38806532\n",
      "Iteration 6, loss = 0.36411046\n",
      "Iteration 7, loss = 0.34045827\n",
      "Iteration 8, loss = 0.33171795\n",
      "Iteration 9, loss = 0.31323272\n",
      "Iteration 10, loss = 0.29896372\n",
      "Iteration 11, loss = 0.29178119\n",
      "Iteration 12, loss = 0.27879445\n",
      "Iteration 13, loss = 0.26551544\n",
      "Iteration 14, loss = 0.25804216\n",
      "Iteration 15, loss = 0.25668616\n",
      "Iteration 16, loss = 0.24412757\n",
      "Iteration 17, loss = 0.23954891\n",
      "Iteration 18, loss = 0.23136672\n",
      "Iteration 19, loss = 0.22183790\n",
      "Iteration 20, loss = 0.21302625\n",
      "Iteration 21, loss = 0.20949064\n",
      "Iteration 22, loss = 0.19858183\n",
      "Iteration 23, loss = 0.20047975\n",
      "Iteration 24, loss = 0.19244317\n",
      "Iteration 25, loss = 0.17979579\n",
      "Iteration 26, loss = 0.19814194\n",
      "Iteration 27, loss = 0.17806842\n",
      "Iteration 28, loss = 0.17185939\n",
      "Iteration 29, loss = 0.16195061\n",
      "Iteration 30, loss = 0.16017796\n",
      "Iteration 31, loss = 0.14474421\n",
      "Iteration 32, loss = 0.15335383\n",
      "Iteration 33, loss = 0.15015871\n",
      "Iteration 34, loss = 0.13779954\n",
      "Iteration 35, loss = 0.14258985\n",
      "Iteration 36, loss = 0.13652991\n",
      "Iteration 37, loss = 0.14201725\n",
      "Iteration 38, loss = 0.12368117\n",
      "Iteration 39, loss = 0.12939435\n",
      "Iteration 40, loss = 0.12166353\n",
      "Iteration 41, loss = 0.11794014\n",
      "Iteration 42, loss = 0.11609789\n",
      "Iteration 43, loss = 0.10669622\n",
      "Iteration 44, loss = 0.10682459\n",
      "Iteration 45, loss = 0.09665414\n",
      "Iteration 46, loss = 0.09766512\n",
      "Iteration 47, loss = 0.10298660\n",
      "Iteration 48, loss = 0.10355953\n",
      "Iteration 49, loss = 0.09616742\n",
      "Iteration 50, loss = 0.09007099\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "A:\\InstallationDriver\\Anaconda\\anaconda3\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:614: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (50) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iteration 1, loss = 0.96600744\n",
      "Iteration 2, loss = 0.53171388\n",
      "Iteration 3, loss = 0.46750628\n",
      "Iteration 4, loss = 0.42301923\n",
      "Iteration 5, loss = 0.40428242\n",
      "Iteration 6, loss = 0.37839088\n",
      "Iteration 7, loss = 0.35415706\n",
      "Iteration 8, loss = 0.33286763\n",
      "Iteration 9, loss = 0.32382092\n",
      "Iteration 10, loss = 0.31302465\n",
      "Iteration 11, loss = 0.30524384\n",
      "Iteration 12, loss = 0.29467402\n",
      "Iteration 13, loss = 0.27544181\n",
      "Iteration 14, loss = 0.27205985\n",
      "Iteration 15, loss = 0.25919888\n",
      "Iteration 16, loss = 0.25725857\n",
      "Iteration 17, loss = 0.24281670\n",
      "Iteration 18, loss = 0.23384900\n",
      "Iteration 19, loss = 0.22801817\n",
      "Iteration 20, loss = 0.21696062\n",
      "Iteration 21, loss = 0.21218997\n",
      "Iteration 22, loss = 0.20751432\n",
      "Iteration 23, loss = 0.20113966\n",
      "Iteration 24, loss = 0.20591105\n",
      "Iteration 25, loss = 0.18255101\n",
      "Iteration 26, loss = 0.19970217\n",
      "Iteration 27, loss = 0.17548770\n",
      "Iteration 28, loss = 0.17053012\n",
      "Iteration 29, loss = 0.17441495\n",
      "Iteration 30, loss = 0.16341984\n",
      "Iteration 31, loss = 0.16657656\n",
      "Iteration 32, loss = 0.15616780\n",
      "Iteration 33, loss = 0.16031924\n",
      "Iteration 34, loss = 0.15965583\n",
      "Iteration 35, loss = 0.16212723\n",
      "Iteration 36, loss = 0.14236236\n",
      "Iteration 37, loss = 0.13783089\n",
      "Iteration 38, loss = 0.14379320\n",
      "Iteration 39, loss = 0.13061418\n",
      "Iteration 40, loss = 0.12052991\n",
      "Iteration 41, loss = 0.12540855\n",
      "Iteration 42, loss = 0.12770946\n",
      "Iteration 43, loss = 0.11244005\n",
      "Iteration 44, loss = 0.11126138\n",
      "Iteration 45, loss = 0.10375458\n",
      "Iteration 46, loss = 0.11694992\n",
      "Iteration 47, loss = 0.09817465\n",
      "Iteration 48, loss = 0.11264426\n",
      "Iteration 49, loss = 0.10122205\n",
      "Iteration 50, loss = 0.09426802\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "A:\\InstallationDriver\\Anaconda\\anaconda3\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:614: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (50) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iteration 1, loss = 1.50596079\n",
      "Iteration 2, loss = 0.78306291\n",
      "Iteration 3, loss = 0.66862017\n",
      "Iteration 4, loss = 0.61425531\n",
      "Iteration 5, loss = 0.58161571\n",
      "Iteration 6, loss = 1.10381399\n",
      "Iteration 7, loss = 0.74807862\n",
      "Iteration 8, loss = 0.68002147\n",
      "Iteration 9, loss = 0.63949818\n",
      "Iteration 10, loss = 0.61660400\n",
      "Iteration 11, loss = 0.63157803\n",
      "Iteration 12, loss = 0.59300351\n",
      "Iteration 13, loss = 0.66610799\n",
      "Iteration 14, loss = 0.54659553\n",
      "Iteration 15, loss = 0.55575851\n",
      "Iteration 16, loss = 0.66114856\n",
      "Iteration 17, loss = 0.52863999\n",
      "Iteration 18, loss = 0.51462429\n",
      "Iteration 19, loss = 0.52268634\n",
      "Iteration 20, loss = 0.49797843\n",
      "Iteration 21, loss = 0.48704602\n",
      "Iteration 22, loss = 0.50661741\n",
      "Iteration 23, loss = 0.50777080\n",
      "Iteration 24, loss = 0.48971024\n",
      "Iteration 25, loss = 0.48611527\n",
      "Iteration 26, loss = 0.45183718\n",
      "Iteration 27, loss = 0.51057214\n",
      "Iteration 28, loss = 0.46187869\n",
      "Iteration 29, loss = 0.47391236\n",
      "Iteration 30, loss = 0.46809656\n",
      "Iteration 31, loss = 0.46336839\n",
      "Iteration 32, loss = 0.57206749\n",
      "Iteration 33, loss = 0.72110750\n",
      "Iteration 34, loss = 0.52240417\n",
      "Iteration 35, loss = 0.49203012\n",
      "Iteration 36, loss = 0.48474009\n",
      "Iteration 37, loss = 0.46548140\n",
      "Training loss did not improve more than tol=0.000100 for 10 consecutive epochs. Stopping.\n",
      "Iteration 1, loss = 1.48750784\n",
      "Iteration 2, loss = 0.73618081\n",
      "Iteration 3, loss = 0.61483931\n",
      "Iteration 4, loss = 0.55407019\n",
      "Iteration 5, loss = 0.52193140\n",
      "Iteration 6, loss = 0.49962742\n",
      "Iteration 7, loss = 0.47062381\n",
      "Iteration 8, loss = 0.47258315\n",
      "Iteration 9, loss = 0.44271921\n",
      "Iteration 10, loss = 0.44069825\n",
      "Iteration 11, loss = 0.42940523\n",
      "Iteration 12, loss = 0.41482167\n",
      "Iteration 13, loss = 0.42048456\n",
      "Iteration 14, loss = 0.45806478\n",
      "Iteration 15, loss = 0.38697696\n",
      "Iteration 16, loss = 0.37556057\n",
      "Iteration 17, loss = 0.38009025\n",
      "Iteration 18, loss = 0.37034703\n",
      "Iteration 19, loss = 0.36497068\n",
      "Iteration 20, loss = 0.36520950\n",
      "Iteration 21, loss = 0.35862645\n",
      "Iteration 22, loss = 0.36361351\n",
      "Iteration 23, loss = 0.36165391\n",
      "Iteration 24, loss = 0.35141268\n",
      "Iteration 25, loss = 0.34926668\n",
      "Iteration 26, loss = 0.34683364\n",
      "Iteration 27, loss = 0.34265338\n",
      "Iteration 28, loss = 0.34418021\n",
      "Iteration 29, loss = 0.34769456\n",
      "Iteration 30, loss = 0.34170370\n",
      "Iteration 31, loss = 0.33500391\n",
      "Iteration 32, loss = 0.32728501\n",
      "Iteration 33, loss = 0.32516021\n",
      "Iteration 34, loss = 0.33129035\n",
      "Iteration 35, loss = 0.32651349\n",
      "Iteration 36, loss = 0.31285517\n",
      "Iteration 37, loss = 0.32382849\n",
      "Iteration 38, loss = 0.30559876\n",
      "Iteration 39, loss = 0.30754627\n",
      "Iteration 40, loss = 0.32464322\n",
      "Iteration 41, loss = 0.31127617\n",
      "Iteration 42, loss = 0.31249142\n",
      "Iteration 43, loss = 0.30733111\n",
      "Iteration 44, loss = 0.30794308\n",
      "Iteration 45, loss = 0.31640767\n",
      "Iteration 46, loss = 0.30365742\n",
      "Iteration 47, loss = 0.29571778\n",
      "Iteration 48, loss = 0.31040693\n",
      "Iteration 49, loss = 0.29935034\n",
      "Iteration 50, loss = 0.29717260\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "A:\\InstallationDriver\\Anaconda\\anaconda3\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:614: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (50) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iteration 1, loss = 1.61212130\n",
      "Iteration 2, loss = 0.84165622\n",
      "Iteration 3, loss = 0.73136499\n",
      "Iteration 4, loss = 0.61192339\n",
      "Iteration 5, loss = 0.58014223\n",
      "Iteration 6, loss = 0.52982466\n",
      "Iteration 7, loss = 0.51492270\n",
      "Iteration 8, loss = 0.48876669\n",
      "Iteration 9, loss = 0.47158576\n",
      "Iteration 10, loss = 0.45716926\n",
      "Iteration 11, loss = 0.45061351\n",
      "Iteration 12, loss = 0.45783216\n",
      "Iteration 13, loss = 0.45511450\n",
      "Iteration 14, loss = 0.43157310\n",
      "Iteration 15, loss = 0.41386226\n",
      "Iteration 16, loss = 0.41814264\n",
      "Iteration 17, loss = 0.56434483\n",
      "Iteration 18, loss = 0.56990229\n",
      "Iteration 19, loss = 0.52493504\n",
      "Iteration 20, loss = 0.48556350\n",
      "Iteration 21, loss = 0.42142954\n",
      "Iteration 22, loss = 0.41291576\n",
      "Iteration 23, loss = 0.39988230\n",
      "Iteration 24, loss = 0.40518525\n",
      "Iteration 25, loss = 0.37978262\n",
      "Iteration 26, loss = 0.39033700\n",
      "Iteration 27, loss = 0.37290709\n",
      "Iteration 28, loss = 0.36744534\n",
      "Iteration 29, loss = 0.37513233\n",
      "Iteration 30, loss = 0.38773153\n",
      "Iteration 31, loss = 0.35670741\n",
      "Iteration 32, loss = 0.37197005\n",
      "Iteration 33, loss = 0.36705113\n",
      "Iteration 34, loss = 0.37091030\n",
      "Iteration 35, loss = 0.36117313\n",
      "Iteration 36, loss = 0.34972240\n",
      "Iteration 37, loss = 0.35569489\n",
      "Iteration 38, loss = 0.35411527\n",
      "Iteration 39, loss = 0.36224098\n",
      "Iteration 40, loss = 0.36580690\n",
      "Iteration 41, loss = 0.34734537\n",
      "Iteration 42, loss = 0.34379709\n",
      "Iteration 43, loss = 0.39697675\n",
      "Iteration 44, loss = 0.34850803\n",
      "Iteration 45, loss = 0.34508361\n",
      "Iteration 46, loss = 0.34315703\n",
      "Iteration 47, loss = 0.33341565\n",
      "Iteration 48, loss = 0.33099732\n",
      "Iteration 49, loss = 0.33624408\n",
      "Iteration 50, loss = 0.34430641\n",
      "Iteration 1, loss = 1.51881196\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "A:\\InstallationDriver\\Anaconda\\anaconda3\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:614: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (50) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iteration 2, loss = 0.72590740\n",
      "Iteration 3, loss = 0.59085393\n",
      "Iteration 4, loss = 0.62251764\n",
      "Iteration 5, loss = 0.50552388\n",
      "Iteration 6, loss = 0.47575259\n",
      "Iteration 7, loss = 0.44577893\n",
      "Iteration 8, loss = 0.43104103\n",
      "Iteration 9, loss = 0.45651430\n",
      "Iteration 10, loss = 0.41914389\n",
      "Iteration 11, loss = 0.42053119\n",
      "Iteration 12, loss = 0.39916406\n",
      "Iteration 13, loss = 0.39131062\n",
      "Iteration 14, loss = 0.37703261\n",
      "Iteration 15, loss = 0.36904458\n",
      "Iteration 16, loss = 0.36867115\n",
      "Iteration 17, loss = 0.37710297\n",
      "Iteration 18, loss = 0.36749427\n",
      "Iteration 19, loss = 0.34980918\n",
      "Iteration 20, loss = 0.34558213\n",
      "Iteration 21, loss = 0.35132048\n",
      "Iteration 22, loss = 0.34189141\n",
      "Iteration 23, loss = 0.34546951\n",
      "Iteration 24, loss = 0.34815492\n",
      "Iteration 25, loss = 0.33855280\n",
      "Iteration 26, loss = 0.32424476\n",
      "Iteration 27, loss = 0.33000597\n",
      "Iteration 28, loss = 0.31560710\n",
      "Iteration 29, loss = 0.33255892\n",
      "Iteration 30, loss = 0.30405490\n",
      "Iteration 31, loss = 0.31691217\n",
      "Iteration 32, loss = 0.45449737\n",
      "Iteration 33, loss = 0.34884748\n",
      "Iteration 34, loss = 0.29872583\n",
      "Iteration 35, loss = 0.29970063\n",
      "Iteration 36, loss = 0.29566792\n",
      "Iteration 37, loss = 0.29395850\n",
      "Iteration 38, loss = 0.29058405\n",
      "Iteration 39, loss = 0.29161186\n",
      "Iteration 40, loss = 0.28183430\n",
      "Iteration 41, loss = 0.28064025\n",
      "Iteration 42, loss = 0.28073392\n",
      "Iteration 43, loss = 0.30484289\n",
      "Iteration 44, loss = 0.29408262\n",
      "Iteration 45, loss = 0.28961766\n",
      "Iteration 46, loss = 0.27292929\n",
      "Iteration 47, loss = 0.27816627\n",
      "Iteration 48, loss = 0.26553391\n",
      "Iteration 49, loss = 0.28006050\n",
      "Iteration 50, loss = 0.26295555\n",
      "Iteration 1, loss = 1.36732745\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "A:\\InstallationDriver\\Anaconda\\anaconda3\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:614: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (50) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iteration 2, loss = 0.68920707\n",
      "Iteration 3, loss = 0.59043888\n",
      "Iteration 4, loss = 0.53608848\n",
      "Iteration 5, loss = 0.48489437\n",
      "Iteration 6, loss = 0.47016579\n",
      "Iteration 7, loss = 0.46567266\n",
      "Iteration 8, loss = 0.43416912\n",
      "Iteration 9, loss = 0.43761624\n",
      "Iteration 10, loss = 0.41512321\n",
      "Iteration 11, loss = 0.39641129\n",
      "Iteration 12, loss = 0.40919292\n",
      "Iteration 13, loss = 0.38361126\n",
      "Iteration 14, loss = 0.39041898\n",
      "Iteration 15, loss = 0.37258137\n",
      "Iteration 16, loss = 0.39026287\n",
      "Iteration 17, loss = 0.36274936\n",
      "Iteration 18, loss = 0.35579164\n",
      "Iteration 19, loss = 0.36106502\n",
      "Iteration 20, loss = 0.34763550\n",
      "Iteration 21, loss = 0.33679279\n",
      "Iteration 22, loss = 0.33498936\n",
      "Iteration 23, loss = 0.32817132\n",
      "Iteration 24, loss = 0.32928483\n",
      "Iteration 25, loss = 0.32155907\n",
      "Iteration 26, loss = 0.32369708\n",
      "Iteration 27, loss = 0.33086357\n",
      "Iteration 28, loss = 0.31961061\n",
      "Iteration 29, loss = 0.30641233\n",
      "Iteration 30, loss = 0.29554403\n",
      "Iteration 31, loss = 0.31277990\n",
      "Iteration 32, loss = 0.32660223\n",
      "Iteration 33, loss = 0.30875269\n",
      "Iteration 34, loss = 0.29677099\n",
      "Iteration 35, loss = 0.30462555\n",
      "Iteration 36, loss = 0.30065432\n",
      "Iteration 37, loss = 0.28055431\n",
      "Iteration 38, loss = 0.28216700\n",
      "Iteration 39, loss = 0.28394085\n",
      "Iteration 40, loss = 0.30976863\n",
      "Iteration 41, loss = 0.31419204\n",
      "Iteration 42, loss = 0.29560222\n",
      "Iteration 43, loss = 0.27613403\n",
      "Iteration 44, loss = 0.26569823\n",
      "Iteration 45, loss = 0.26684121\n",
      "Iteration 46, loss = 0.26594767\n",
      "Iteration 47, loss = 0.26708168\n",
      "Iteration 48, loss = 0.29411483\n",
      "Iteration 49, loss = 0.27883418\n",
      "Iteration 50, loss = 0.26005961\n",
      "Iteration 1, loss = 1.15953982\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "A:\\InstallationDriver\\Anaconda\\anaconda3\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:614: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (50) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iteration 2, loss = 0.56632102\n",
      "Iteration 3, loss = 0.48196393\n",
      "Iteration 4, loss = 0.41749618\n",
      "Iteration 5, loss = 0.38656963\n",
      "Iteration 6, loss = 0.37464719\n",
      "Iteration 7, loss = 0.34993015\n",
      "Iteration 8, loss = 0.32449722\n",
      "Iteration 9, loss = 0.32141753\n",
      "Iteration 10, loss = 0.30603026\n",
      "Iteration 11, loss = 0.29595672\n",
      "Iteration 12, loss = 0.26898977\n",
      "Iteration 13, loss = 0.26634978\n",
      "Iteration 14, loss = 0.25530582\n",
      "Iteration 15, loss = 0.25209399\n",
      "Iteration 16, loss = 0.24151609\n",
      "Iteration 17, loss = 0.25650905\n",
      "Iteration 18, loss = 0.23438690\n",
      "Iteration 19, loss = 0.21976503\n",
      "Iteration 20, loss = 0.21112415\n",
      "Iteration 21, loss = 0.20575605\n",
      "Iteration 22, loss = 0.21161933\n",
      "Iteration 23, loss = 0.18771231\n",
      "Iteration 24, loss = 0.18814050\n",
      "Iteration 25, loss = 0.17546694\n",
      "Iteration 26, loss = 0.19530557\n",
      "Iteration 27, loss = 0.17760052\n",
      "Iteration 28, loss = 0.16650848\n",
      "Iteration 29, loss = 0.17099761\n",
      "Iteration 30, loss = 0.17989832\n",
      "Iteration 31, loss = 0.17657126\n",
      "Iteration 32, loss = 0.17333515\n",
      "Iteration 33, loss = 0.14826805\n",
      "Iteration 34, loss = 0.14217004\n",
      "Iteration 35, loss = 0.14885145\n",
      "Iteration 36, loss = 0.13295324\n",
      "Iteration 37, loss = 0.12930306\n",
      "Iteration 38, loss = 0.12355839\n",
      "Iteration 39, loss = 0.11600059\n",
      "Iteration 40, loss = 0.11961616\n",
      "Iteration 41, loss = 0.12397739\n",
      "Iteration 42, loss = 0.13486168\n",
      "Iteration 43, loss = 0.14797559\n",
      "Iteration 44, loss = 0.10134303\n",
      "Iteration 45, loss = 0.11306352\n",
      "Iteration 46, loss = 0.10687999\n",
      "Iteration 47, loss = 0.09611245\n",
      "Iteration 48, loss = 0.13456278\n",
      "Iteration 49, loss = 0.21421670\n",
      "Iteration 50, loss = 0.14621816\n",
      "Iteration 1, loss = 1.22681759\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "A:\\InstallationDriver\\Anaconda\\anaconda3\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:614: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (50) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iteration 2, loss = 0.57400179\n",
      "Iteration 3, loss = 0.48418348\n",
      "Iteration 4, loss = 0.42596435\n",
      "Iteration 5, loss = 0.40048281\n",
      "Iteration 6, loss = 0.39244516\n",
      "Iteration 7, loss = 0.35358400\n",
      "Iteration 8, loss = 0.33211892\n",
      "Iteration 9, loss = 0.32478152\n",
      "Iteration 10, loss = 0.31416860\n",
      "Iteration 11, loss = 0.29627333\n",
      "Iteration 12, loss = 0.28722963\n",
      "Iteration 13, loss = 0.27924726\n",
      "Iteration 14, loss = 0.26671204\n",
      "Iteration 15, loss = 0.25894926\n",
      "Iteration 16, loss = 0.25058976\n",
      "Iteration 17, loss = 0.24308499\n",
      "Iteration 18, loss = 0.26029938\n",
      "Iteration 19, loss = 0.24197642\n",
      "Iteration 20, loss = 0.21538307\n",
      "Iteration 21, loss = 0.21821822\n",
      "Iteration 22, loss = 0.21002745\n",
      "Iteration 23, loss = 0.20351883\n",
      "Iteration 24, loss = 0.19432393\n",
      "Iteration 25, loss = 0.20700930\n",
      "Iteration 26, loss = 0.18643055\n",
      "Iteration 27, loss = 0.19737954\n",
      "Iteration 28, loss = 0.18157952\n",
      "Iteration 29, loss = 0.18573056\n",
      "Iteration 30, loss = 0.17085285\n",
      "Iteration 31, loss = 0.16228089\n",
      "Iteration 32, loss = 0.16192393\n",
      "Iteration 33, loss = 0.15994653\n",
      "Iteration 34, loss = 0.16228769\n",
      "Iteration 35, loss = 0.13679650\n",
      "Iteration 36, loss = 0.14010523\n",
      "Iteration 37, loss = 0.13464130\n",
      "Iteration 38, loss = 0.13712784\n",
      "Iteration 39, loss = 0.15749104\n",
      "Iteration 40, loss = 0.14673589\n",
      "Iteration 41, loss = 0.13541778\n",
      "Iteration 42, loss = 0.11960193\n",
      "Iteration 43, loss = 0.11715896\n",
      "Iteration 44, loss = 0.11622320\n",
      "Iteration 45, loss = 0.11892814\n",
      "Iteration 46, loss = 0.12764482\n",
      "Iteration 47, loss = 0.10196238\n",
      "Iteration 48, loss = 0.10466290\n",
      "Iteration 49, loss = 0.16800817\n",
      "Iteration 50, loss = 0.10778782\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "A:\\InstallationDriver\\Anaconda\\anaconda3\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:614: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (50) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iteration 1, loss = 1.27442977\n",
      "Iteration 2, loss = 0.59334351\n",
      "Iteration 3, loss = 0.48517185\n",
      "Iteration 4, loss = 0.42785186\n",
      "Iteration 5, loss = 0.40944527\n",
      "Iteration 6, loss = 0.37956785\n",
      "Iteration 7, loss = 0.35450925\n",
      "Iteration 8, loss = 0.35173307\n",
      "Iteration 9, loss = 0.32688195\n",
      "Iteration 10, loss = 0.31983840\n",
      "Iteration 11, loss = 0.31884149\n",
      "Iteration 12, loss = 0.29888497\n",
      "Iteration 13, loss = 0.29311039\n",
      "Iteration 14, loss = 0.27587639\n",
      "Iteration 15, loss = 0.26616819\n",
      "Iteration 16, loss = 0.25673775\n",
      "Iteration 17, loss = 0.25699396\n",
      "Iteration 18, loss = 0.25647055\n",
      "Iteration 19, loss = 0.24336693\n",
      "Iteration 20, loss = 0.23636497\n",
      "Iteration 21, loss = 0.22575869\n",
      "Iteration 22, loss = 0.22341463\n",
      "Iteration 23, loss = 0.21042687\n",
      "Iteration 24, loss = 0.23134642\n",
      "Iteration 25, loss = 0.20460474\n",
      "Iteration 26, loss = 0.20431327\n",
      "Iteration 27, loss = 0.19825042\n",
      "Iteration 28, loss = 0.18874913\n",
      "Iteration 29, loss = 0.17352506\n",
      "Iteration 30, loss = 0.19318278\n",
      "Iteration 31, loss = 0.19682077\n",
      "Iteration 32, loss = 0.17450868\n",
      "Iteration 33, loss = 0.16746956\n",
      "Iteration 34, loss = 0.15930031\n",
      "Iteration 35, loss = 0.16410274\n",
      "Iteration 36, loss = 0.15284037\n",
      "Iteration 37, loss = 0.14985137\n",
      "Iteration 38, loss = 0.14194933\n",
      "Iteration 39, loss = 0.19264507\n",
      "Iteration 40, loss = 0.16190958\n",
      "Iteration 41, loss = 0.14229525\n",
      "Iteration 42, loss = 0.14224007\n",
      "Iteration 43, loss = 0.14130542\n",
      "Iteration 44, loss = 0.14719724\n",
      "Iteration 45, loss = 0.14294073\n",
      "Iteration 46, loss = 0.12056841\n",
      "Iteration 47, loss = 0.13546350\n",
      "Iteration 48, loss = 0.12867914\n",
      "Iteration 49, loss = 0.11479498\n",
      "Iteration 50, loss = 0.16617138\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "A:\\InstallationDriver\\Anaconda\\anaconda3\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:614: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (50) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iteration 1, loss = 1.11538056\n",
      "Iteration 2, loss = 0.55035770\n",
      "Iteration 3, loss = 0.48005690\n",
      "Iteration 4, loss = 0.41490155\n",
      "Iteration 5, loss = 0.38592948\n",
      "Iteration 6, loss = 0.36138885\n",
      "Iteration 7, loss = 0.34769045\n",
      "Iteration 8, loss = 0.32641757\n",
      "Iteration 9, loss = 0.31005210\n",
      "Iteration 10, loss = 0.29148371\n",
      "Iteration 11, loss = 0.29213832\n",
      "Iteration 12, loss = 0.27479646\n",
      "Iteration 13, loss = 0.26050868\n",
      "Iteration 14, loss = 0.25638823\n",
      "Iteration 15, loss = 0.25821308\n",
      "Iteration 16, loss = 0.22843871\n",
      "Iteration 17, loss = 0.26086209\n",
      "Iteration 18, loss = 0.22830327\n",
      "Iteration 19, loss = 0.23725947\n",
      "Iteration 20, loss = 0.21445582\n",
      "Iteration 21, loss = 0.20485804\n",
      "Iteration 22, loss = 0.19438702\n",
      "Iteration 23, loss = 0.19551805\n",
      "Iteration 24, loss = 0.19816549\n",
      "Iteration 25, loss = 0.17714320\n",
      "Iteration 26, loss = 0.17580968\n",
      "Iteration 27, loss = 0.16259834\n",
      "Iteration 28, loss = 0.16105974\n",
      "Iteration 29, loss = 0.16104625\n",
      "Iteration 30, loss = 0.15613805\n",
      "Iteration 31, loss = 0.15466963\n",
      "Iteration 32, loss = 0.15942157\n",
      "Iteration 33, loss = 0.14615678\n",
      "Iteration 34, loss = 0.16803248\n",
      "Iteration 35, loss = 0.13871575\n",
      "Iteration 36, loss = 0.13766299\n",
      "Iteration 37, loss = 0.12135482\n",
      "Iteration 38, loss = 0.13310634\n",
      "Iteration 39, loss = 0.14294225\n",
      "Iteration 40, loss = 0.15625352\n",
      "Iteration 41, loss = 0.14622816\n",
      "Iteration 42, loss = 0.11501413\n",
      "Iteration 43, loss = 0.10127007\n",
      "Iteration 44, loss = 0.10505904\n",
      "Iteration 45, loss = 0.10453928\n",
      "Iteration 46, loss = 0.11700670\n",
      "Iteration 47, loss = 0.09566447\n",
      "Iteration 48, loss = 0.12035684\n",
      "Iteration 49, loss = 0.08717201\n",
      "Iteration 50, loss = 0.08452620\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "A:\\InstallationDriver\\Anaconda\\anaconda3\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:614: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (50) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iteration 1, loss = 1.03082416\n",
      "Iteration 2, loss = 0.55992167\n",
      "Iteration 3, loss = 0.48646284\n",
      "Iteration 4, loss = 0.42393545\n",
      "Iteration 5, loss = 0.39132982\n",
      "Iteration 6, loss = 0.36758175\n",
      "Iteration 7, loss = 0.34755549\n",
      "Iteration 8, loss = 0.33545037\n",
      "Iteration 9, loss = 0.32215803\n",
      "Iteration 10, loss = 0.29800005\n",
      "Iteration 11, loss = 0.29564909\n",
      "Iteration 12, loss = 0.28186965\n",
      "Iteration 13, loss = 0.25744827\n",
      "Iteration 14, loss = 0.26045685\n",
      "Iteration 15, loss = 0.25319811\n",
      "Iteration 16, loss = 0.23052456\n",
      "Iteration 17, loss = 0.23379113\n",
      "Iteration 18, loss = 0.21781193\n",
      "Iteration 19, loss = 0.20544571\n",
      "Iteration 20, loss = 0.21762216\n",
      "Iteration 21, loss = 0.20694070\n",
      "Iteration 22, loss = 0.19541282\n",
      "Iteration 23, loss = 0.20108694\n",
      "Iteration 24, loss = 0.18310905\n",
      "Iteration 25, loss = 0.18218900\n",
      "Iteration 26, loss = 0.17781469\n",
      "Iteration 27, loss = 0.16438105\n",
      "Iteration 28, loss = 0.19216311\n",
      "Iteration 29, loss = 0.16776784\n",
      "Iteration 30, loss = 0.18089437\n",
      "Iteration 31, loss = 0.15785309\n",
      "Iteration 32, loss = 0.14732536\n",
      "Iteration 33, loss = 0.14220030\n",
      "Iteration 34, loss = 0.13937829\n",
      "Iteration 35, loss = 0.12862834\n",
      "Iteration 36, loss = 0.14192243\n",
      "Iteration 37, loss = 0.12642765\n",
      "Iteration 38, loss = 0.11410477\n",
      "Iteration 39, loss = 0.11255568\n",
      "Iteration 40, loss = 0.11933455\n",
      "Iteration 41, loss = 0.11107101\n",
      "Iteration 42, loss = 0.10971539\n",
      "Iteration 43, loss = 0.09388885\n",
      "Iteration 44, loss = 0.12094420\n",
      "Iteration 45, loss = 0.11440281\n",
      "Iteration 46, loss = 0.10210654\n",
      "Iteration 47, loss = 0.09528841\n",
      "Iteration 48, loss = 0.09160486\n",
      "Iteration 49, loss = 0.07993320\n",
      "Iteration 50, loss = 0.07200011\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "A:\\InstallationDriver\\Anaconda\\anaconda3\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:614: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (50) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iteration 1, loss = 0.88319614\n",
      "Iteration 2, loss = 0.50793429\n",
      "Iteration 3, loss = 0.46196928\n",
      "Iteration 4, loss = 0.42098273\n",
      "Iteration 5, loss = 0.38674721\n",
      "Iteration 6, loss = 0.36751516\n",
      "Iteration 7, loss = 0.34335619\n",
      "Iteration 8, loss = 0.32552195\n",
      "Iteration 9, loss = 0.32341516\n",
      "Iteration 10, loss = 0.30618748\n",
      "Iteration 11, loss = 0.29888448\n",
      "Iteration 12, loss = 0.28812727\n",
      "Iteration 13, loss = 0.28183777\n",
      "Iteration 14, loss = 0.27011055\n",
      "Iteration 15, loss = 0.26127752\n",
      "Iteration 16, loss = 0.25157114\n",
      "Iteration 17, loss = 0.25552899\n",
      "Iteration 18, loss = 0.23619496\n",
      "Iteration 19, loss = 0.23859845\n",
      "Iteration 20, loss = 0.23047925\n",
      "Iteration 21, loss = 0.21871712\n",
      "Iteration 22, loss = 0.21462198\n",
      "Iteration 23, loss = 0.21084839\n",
      "Iteration 24, loss = 0.21433333\n",
      "Iteration 25, loss = 0.19576466\n",
      "Iteration 26, loss = 0.20028233\n",
      "Iteration 27, loss = 0.18297649\n",
      "Iteration 28, loss = 0.19082379\n",
      "Iteration 29, loss = 0.18536914\n",
      "Iteration 30, loss = 0.17883075\n",
      "Iteration 31, loss = 0.17432974\n",
      "Iteration 32, loss = 0.16349988\n",
      "Iteration 33, loss = 0.16660378\n",
      "Iteration 34, loss = 0.16455106\n",
      "Iteration 35, loss = 0.15799562\n",
      "Iteration 36, loss = 0.16193027\n",
      "Iteration 37, loss = 0.15305652\n",
      "Iteration 38, loss = 0.14825680\n",
      "Iteration 39, loss = 0.15209548\n",
      "Iteration 40, loss = 0.14582625\n",
      "Iteration 41, loss = 0.14437471\n",
      "Iteration 42, loss = 0.12871896\n",
      "Iteration 43, loss = 0.13979650\n",
      "Iteration 44, loss = 0.13506464\n",
      "Iteration 45, loss = 0.11998005\n",
      "Iteration 46, loss = 0.14388038\n",
      "Iteration 47, loss = 0.13242641\n",
      "Iteration 48, loss = 0.11844929\n",
      "Iteration 49, loss = 0.10345062\n",
      "Iteration 50, loss = 0.10845943\n",
      "RESULTS FOR NN\n",
      "\n",
      "Best parameters set found:\n",
      "{'hidden_layer_sizes': (50,)}\n",
      "Score with best parameters:\n",
      "0.8472000000000002\n",
      "\n",
      "All scores on the grid:\n",
      "{'mean_fit_time': array([3.48991451, 5.10629439, 3.45933599, 5.89379749]), 'std_fit_time': array([0.14558508, 0.07054004, 0.42420765, 0.07141195]), 'mean_score_time': array([0.00848894, 0.0111835 , 0.00686765, 0.01117058]), 'std_score_time': array([0.00544379, 0.00640661, 0.0050469 , 0.0061501 ]), 'param_hidden_layer_sizes': masked_array(data=[(10,), (50,), (10, 10), (50, 50)],\n",
      "             mask=[False, False, False, False],\n",
      "       fill_value='?',\n",
      "            dtype=object), 'params': [{'hidden_layer_sizes': (10,)}, {'hidden_layer_sizes': (50,)}, {'hidden_layer_sizes': (10, 10)}, {'hidden_layer_sizes': (50, 50)}], 'split0_test_score': array([0.8195, 0.8435, 0.798 , 0.8445]), 'split1_test_score': array([0.8315, 0.8445, 0.8145, 0.8375]), 'split2_test_score': array([0.8185, 0.8445, 0.8195, 0.846 ]), 'split3_test_score': array([0.788 , 0.845 , 0.7625, 0.849 ]), 'split4_test_score': array([0.822 , 0.8585, 0.8415, 0.856 ]), 'mean_test_score': array([0.8159, 0.8472, 0.8072, 0.8466]), 'std_test_score': array([0.01468809, 0.00567098, 0.02632033, 0.00602827]), 'rank_test_score': array([3, 1, 4, 2])}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "A:\\InstallationDriver\\Anaconda\\anaconda3\\lib\\site-packages\\sklearn\\neural_network\\_multilayer_perceptron.py:614: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (50) reached and the optimization hasn't converged yet.\n",
      "  warnings.warn(\n"
     ]
    }
   ],
   "source": [
    "#for NN we try the same architectures as before\n",
    "hl_parameters = {'hidden_layer_sizes': [(10,), (50,), (10,10,), (50,50,)]}\n",
    "\n",
    "mlp_large_cv = MLPClassifier(max_iter=50, alpha=1e-4, solver='sgd', tol=1e-4, learning_rate_init=.1, random_state=ID, verbose=True)\n",
    "GS_CV = GridSearchCV(mlp_large_cv, hl_parameters, cv=5, verbose=True)\n",
    "GS_CV.fit(X_train, y_train)\n",
    "\n",
    "\n",
    "print ('RESULTS FOR NN\\n')\n",
    "\n",
    "print(\"Best parameters set found:\")\n",
    "print(GS_CV.best_params_)\n",
    "\n",
    "print(\"Score with best parameters:\")\n",
    "print(GS_CV.best_score_)\n",
    "\n",
    "print(\"\\nAll scores on the grid:\")\n",
    "print(GS_CV.cv_results_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TO DO 5\n",
    "Describe your architecture choices and the results you observe with respect to the architectures you used."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The score for 10 neurons and 1 hidden layer is 0.8159\n",
    "The score for 50 neurons and 1 hidden layer is 0.8472\n",
    "The score for 10 neurons and 2 hidden layer is 0.8072\n",
    "The score for 50 neurons and 2 hidden layer is 0.8466"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TO DO 6\n",
    "\n",
    "Get the train and test error for the best NN you obtained with 10000 points. This time you can run for 100 iterations if you cannot run for 300 iterations. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RESULTS FOR BEST NN\n",
      "\n",
      "Best NN training error: 0.000000\n",
      "Best NN test error: 0.212403\n"
     ]
    }
   ],
   "source": [
    "#get training and test error for the best NN model from CV\n",
    "\n",
    "best_mlp_large = MLPClassifier(max_iter=300, alpha=1e-4, solver='sgd', tol=1e-4, learning_rate_init=.1, random_state=ID,hidden_layer_sizes=(50,50,))\n",
    "\n",
    "best_mlp_large.fit(X_train, y_train)\n",
    "\n",
    "training_error_large = 1. - best_mlp_large.score(X_train, y_train)\n",
    "\n",
    "test_error_large = 1. - best_mlp_large.score(X_test, y_test)\n",
    "\n",
    "print ('RESULTS FOR BEST NN\\n')\n",
    "\n",
    "print (\"Best NN training error: %f\" % training_error)\n",
    "print (\"Best NN test error: %f\" % test_error)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TO DO 7\n",
    "\n",
    "Compare the train and test error you got with a large number of samples with the best one you obtained with only 500 data points. Are the architectures the same or do they differ? What about the errors you get?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The two architectures are different with respect to the number of samples in the training set:\n",
    "\n",
    "1 hidden layer and 50 neurons for 500 data points.\n",
    "1 hidden layers and 50 neurons for each layer for 10000 data points.\n",
    "\n",
    "The Result of the 10000 data points is better than the 500 data points."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TO DO 8\n",
    "\n",
    "Plot an image that was missclassified by NN with m=500 training data points and it is now instead correctly classified by NN with m=10000 training data points."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INPUT:\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAASBElEQVR4nO3dbWxU55UH8P/hLQFswMbGtoyzhgblXYFqhFZiRbJqAiRfSBWxKh8qqqBSKYlUon7YKPuh+Rittq36YVOJblDpqpuqUokgUrIlIkUJUgIxgRBY8gqmMRhs82rAQMBnP/imMuB7zmTuzNyB8/9Jlu05fmYej/lz7Tn3uY+oKojo1jcu7wkQUXUw7ERBMOxEQTDsREEw7ERBTKjmgzU1NWlnZ2c1H/KWcPHiRbN++fLlku97eHg4031PmTLFrA8ODqbWmpqazLETJ04063Sj7u5uDAwMyFi1TGEXkWUAfg1gPID/UtWXrK/v7OxEV1dXloesSV5gxo3L9gvUZ599Zta7u7tTayJj/tz/7sKFC2b9q6++MusPPvigWd+2bVtq7amnnjLHtre3m3W6UaFQSK2V/K9QRMYD+E8AjwG4F8BKEbm31PsjosrKcshZCOALVT2oqpcB/BHA8vJMi4jKLUvY2wGM/h2vJ7ntGiKyRkS6RKSrv78/w8MRURZZwj7WH4M3nHurqutUtaCqhebm5gwPR0RZZAl7D4COUZ/PBnA023SIqFKyhP0DAPNEZI6ITALwAwCbyzMtIiq3kltvqnpFRJ4F8BeMtN7Wq+r+ss3sJlLplYNr164164cOHSr5vidMsP8JeL1uq48OAJcuXUqtvf322+bYjRs3mvWGhgazfvXq1dTa+PHjzbG3okx9dlV9A8AbZZoLEVUQT5clCoJhJwqCYScKgmEnCoJhJwqCYScKoqrr2Wls3jLT8+fPm/VZs2al1vbu3WuOHRoaMuveElmrjw7Y5yDMnDnTHHvq1Cmz7vXZeeXka/HIThQEw04UBMNOFATDThQEw04UBMNOFARbb2Vw5coVs+4tp/Raa94y0gceeCC15l26+7bbbjPrb775plm/7777zLp1qelp06aZYwcGBsz63LlzzTpbb9fikZ0oCIadKAiGnSgIhp0oCIadKAiGnSgIhp0oCPbZy8DrVXu8bY/r6+vN+rFjx1Jr3i6sK1asMOtz5swx60uXLjXrW7duTa1Zu88CwHvvvWfWFy5caNa55fO1eGQnCoJhJwqCYScKgmEnCoJhJwqCYScKgmEnCoJ99oS39tm6pPLly5fNsa+//rpZ37Rpk1kfHh426+3t7am15uZmc+zu3bvN+rhx9vFg586dZn3y5MmpNW9uu3btMusvv/yyWV+8eHFq7f777zfH3ooyhV1EugEMArgK4IqqFsoxKSIqv3Ic2f9ZVe1LihBR7vg3O1EQWcOuALaIyC4RWTPWF4jIGhHpEpGu/v7+jA9HRKXKGvZFqvpdAI8BeEZEbnhFRFXXqWpBVQveCzJEVDmZwq6qR5P3fQBeA2AvQyKi3JQcdhGZKiL133wMYAmAfeWaGBGVV5ZX41sAvJb0nycA+B9V/d+yzCoH3tbE1rbKTz75pDn2jjvuMOveevhJkyaZdavPP2GC/SP2vu/GxkazfubMGbPe2tqaWvPOT/C+708++cSsb9myJbX20EMPmWOfe+45s57lvIy8lBx2VT0I4MEyzoWIKoitN6IgGHaiIBh2oiAYdqIgGHaiILjEtUjWMlWrvQT4WzZ7Ojo6zLq1DPXixYvmWK/uXY65rq7OrFvLcxsaGsyx3lbYXntrwYIFqbXe3l5zbNa2YC3ikZ0oCIadKAiGnSgIhp0oCIadKAiGnSgIhp0oCPbZi2Rdcvn22283x546dcqst7W1mXXv/q2esLcU0zsHYGhoyKw3NTWVfP/eZaq9Hr53DoD1vHuP/f7775t16zLVtYpHdqIgGHaiIBh2oiAYdqIgGHaiIBh2oiAYdqIg2Gcv0uHDh1NrLS0t5tisa8pnzJhh1r/++uvUmrfm21uL76379vrVly5dSq15z5u13TNgf98AcPXq1dSad37BgQMHzDr77ERUsxh2oiAYdqIgGHaiIBh2oiAYdqIgGHaiINhnT3z00Udm/fTp06m1efPmmWO9Pnp/f79Zb25uNuvWum9r3oDfw7/zzjvNunf/1nbU9fX15ljv2u1Hjhwx69Z21d41Ao4fP27Wb0bukV1E1otIn4jsG3Vbo4i8JSKfJ+/tq/0TUe6K+TX+dwCWXXfb8wC2quo8AFuTz4mohrlhV9V3AJy87ublADYkH28A8ER5p0VE5VbqC3QtqtoLAMn7WWlfKCJrRKRLRLq8v02JqHIq/mq8qq5T1YKqFrwXmoiockoN+3ERaQOA5H1f+aZERJVQatg3A1iVfLwKwKbyTIeIKsXts4vIqwAeBtAkIj0Afg7gJQB/EpHVAP4GYEUlJ1kNXl/V6kd7+4h7vWiv55uFt97cOwdg5syZZt3r0x89ejS15l333eqTA/ZaecD+uXh7wx87dsys34zcsKvqypTS98o8FyKqIJ4uSxQEw04UBMNOFATDThQEw04UBJe4JqzLDgN+m8hy5swZs97R0WHWz507Z9atuc2ZM8cc633fAwMDZt27FHVPT09qbXBw0Bzb2Nho1r22orVls9dS9C5T7bVbvbZhHnhkJwqCYScKgmEnCoJhJwqCYScKgmEnCoJhJwqi9pqBOfEumWX1Tb2e7D333GPWh4aGMtVV1axbvF61t+WzZ/78+ak1r1ft9eG9Kx9Z5ydk2e4ZsHv4gD+3PPDIThQEw04UBMNOFATDThQEw04UBMNOFATDThQE++yJEydOmHVr6+Hh4WFzbKFQMOvbtm0z697lnq317N62x966a68X7s3Nun+vl+2t48/Sy/b65N75BSdPXr/94bXYZyei3DDsREEw7ERBMOxEQTDsREEw7ERBMOxEQbDPnjh//rxZt3rZXp/d29Y4S68asHvpdXV1me7bWyvv1a3799bSe+vZva2u29raUmuffvqpOda73r53/YO77rrLrOfBPbKLyHoR6RORfaNue1FEjojInuTt8cpOk4iyKubX+N8BWDbG7b9S1fnJ2xvlnRYRlZsbdlV9B4B9biAR1bwsL9A9KyJ7k1/zG9K+SETWiEiXiHR5f+cQUeWUGvbfAPgOgPkAegH8Iu0LVXWdqhZUtVCLiwOIoigp7Kp6XFWvquowgN8CWFjeaRFRuZUUdhEZ3dP4PoB9aV9LRLXB7bOLyKsAHgbQJCI9AH4O4GERmQ9AAXQD+Enlplgd3rptqyfsrcuur6836+PHj89Unzx5cmrNuz66t++8t67bG289r16ffOrUqWbde96nT59e0ryKkXV8Htywq+rKMW5+pQJzIaIK4umyREEw7ERBMOxEQTDsREEw7ERBcIlrwmvjWK0373LNniztK8BeCjpt2jRzrPd9e0tYvWWq1nOTdbtob1mytcTV+7685yXrzzwPPLITBcGwEwXBsBMFwbATBcGwEwXBsBMFwbATBcE+e8JbCmotM7W2c/bGAsCFCxfMunf/Vj/aG+v1uj1eL9zqZ3tbMk+aNMmsnz171qxb35s3b+9nNjQ0ZNZrEY/sREEw7ERBMOxEQTDsREEw7ERBMOxEQTDsREGwz57w1i9bdetSzoC/9bC3nj3LtsonTpwwx3qXa/Z42017/ewsY71zI/btS9/OwLuMtYd9diKqWQw7URAMO1EQDDtREAw7URAMO1EQDDtREOyzJ7ye7vDwcGrNW3c9ZcoUs+6tObceG7B7vt59e7zzD7xzBKw+vDfWW2vv9cqt5917bM/NuGWze2QXkQ4R+auIHBCR/SLy0+T2RhF5S0Q+T943VH66RFSqYn6NvwLgZ6p6D4B/BPCMiNwL4HkAW1V1HoCtyedEVKPcsKtqr6p+mHw8COAAgHYAywFsSL5sA4AnKjRHIiqDb/UCnYh0AlgAYAeAFlXtBUb+QwAwK2XMGhHpEpGu/v7+jNMlolIVHXYRqQPwZwBrVdW+0t8oqrpOVQuqWmhubi5ljkRUBkWFXUQmYiTof1DVjcnNx0WkLam3AeirzBSJqBzc1puM9KReAXBAVX85qrQZwCoALyXvN1VkhlXitVKs9pfXIpo+fbpZ97b/9ZZTWuOztpi8ZaTe3Kzlv97S3yw/EwBobW1Nre3evdscm+US2bWqmD77IgA/BPCxiOxJbnsBIyH/k4isBvA3ACsqMkMiKgs37Kq6HUDaf3PfK+90iKhSeLosURAMO1EQDDtREAw7URAMO1EQXOKa8JZyWrxe9s6dO8363LlzzXpHR4dZt/r8AwMD5lhv+a3XT85yqWjv+/bu+/Tp02bd6uN7WzJ7Pf5bcokrEd0aGHaiIBh2oiAYdqIgGHaiIBh2oiAYdqIg2GdPeGvKrb5qY2OjOXb//v1m3dpaGAAOHjxo1q1+cltbmzn27rvvNuvelsx1dXVm/fDhw6m17du3m2NbWlrMuve8rV69OrXmbVV96dIls+6t869FPLITBcGwEwXBsBMFwbATBcGwEwXBsBMFwbATBcE+e8Lb/tdaW+1dF95z4cIFs+6tKbfGz5492xzrXXvd6zd76+GtdePeenSvl11fX2/WrfMfvLHe9fC977sW8chOFATDThQEw04UBMNOFATDThQEw04UBMNOFEQx+7N3APg9gFYAwwDWqeqvReRFAD8G0J986Quq+kalJlppX375pVm31mV3dnaaY999912z7l2z3us3W71wbw/08+fPm/VDhw6Zda+Pf/LkydTa2bNnzbHeNQbOnDlj1q259/T0mGP37Nlj1hcvXmzWa1ExJ9VcAfAzVf1QROoB7BKRt5Lar1T1Pyo3PSIql2L2Z+8F0Jt8PCgiBwC0V3piRFRe3+pvdhHpBLAAwI7kpmdFZK+IrBeRhpQxa0SkS0S6+vv7x/oSIqqCosMuInUA/gxgraqeBfAbAN8BMB8jR/5fjDVOVdepakFVC83NzdlnTEQlKSrsIjIRI0H/g6puBABVPa6qV1V1GMBvASys3DSJKCs37DKy3OsVAAdU9Zejbh992dLvA7Av9UlEuSrm1fhFAH4I4GMR2ZPc9gKAlSIyH4AC6AbwkwrMr2q85ZZWq8ZqLwHAjh07zLrX5vEu12y1z6ZNm2aOnTFjhll/+umnzbrXFjx16lRqzXvOGxrGfBno77y24ZIlS1Jrjz76qDm2r6/PrB85csSs16JiXo3fDmCsxdw3bU+dKCKeQUcUBMNOFATDThQEw04UBMNOFATDThQELyWdeOSRR8y6dcnlBQsWmGNbW1vN+rJly8w6ld+iRYvMureV9dKlS8s5nargkZ0oCIadKAiGnSgIhp0oCIadKAiGnSgIhp0oCPG2Ay7rg4n0Axh9TeYmAANVm8C3U6tzq9V5AZxbqco5t39Q1TGv/1bVsN/w4CJdqlrIbQKGWp1brc4L4NxKVa258dd4oiAYdqIg8g77upwf31Krc6vVeQGcW6mqMrdc/2YnourJ+8hORFXCsBMFkUvYRWSZiHwqIl+IyPN5zCGNiHSLyMciskdEunKey3oR6RORfaNuaxSRt0Tk8+S9fXH16s7tRRE5kjx3e0Tk8Zzm1iEifxWRAyKyX0R+mtye63NnzKsqz1vV/2YXkfEAPgPwKIAeAB8AWKmq/1fViaQQkW4ABVXN/QQMEVkM4ByA36vq/clt/w7gpKq+lPxH2aCq/1ojc3sRwLm8t/FOditqG73NOIAnAPwIOT53xrz+BVV43vI4si8E8IWqHlTVywD+CGB5DvOoear6DoDrt5tZDmBD8vEGjPxjqbqUudUEVe1V1Q+TjwcBfLPNeK7PnTGvqsgj7O0Avhr1eQ9qa793BbBFRHaJyJq8JzOGFlXtBUb+8QCYlfN8rudu411N120zXjPPXSnbn2eVR9jH2kqqlvp/i1T1uwAeA/BM8usqFaeobbyrZYxtxmtCqdufZ5VH2HsAdIz6fDaAoznMY0yqejR53wfgNdTeVtTHv9lBN3lv70BYRbW0jfdY24yjBp67PLc/zyPsHwCYJyJzRGQSgB8A2JzDPG4gIlOTF04gIlMBLEHtbUW9GcCq5ONVADblOJdr1Mo23mnbjCPn5y737c9VtepvAB7HyCvyXwL4tzzmkDKvuQA+St725z03AK9i5Ne6rzHyG9FqADMBbAXwefK+sYbm9t8APgawFyPBastpbv+EkT8N9wLYk7w9nvdzZ8yrKs8bT5clCoJn0BEFwbATBcGwEwXBsBMFwbATBcGwEwXBsBMF8f95dSGGNi/b5QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LABEL: 4\n",
      "For 500 samples: 2\n",
      "For 10000 samples: 4 \n"
     ]
    }
   ],
   "source": [
    "for i in range(y_test.size):\n",
    "    yp = mlp.predict(X_test)\n",
    "    ypl = best_mlp_large.predict(X_test)\n",
    "    if yp[i] != y_test[i] and ypl[i] == y_test[i]:\n",
    "        plot_input(X_test,y_test,i)\n",
    "        print('For 500 samples: %s' % yp[i])\n",
    "        print('For 10000 samples: %s ' % ypl[i])\n",
    "        break \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's plot some of the weigths of the multi-layer perceptron classifier, for the best NN we get with 500 data points and with 10000 data points. The code below plots the weights in a matrix form, where a figure represents all the weights of the edges entering in a hidden node. Notice that the code assumes that the NNs are called \"mlp\" and \"best_mlp_large\": you may need to replace such variables with your variable names. \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Weights with 500 data points:\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUcAAADuCAYAAACqLcX5AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAACMY0lEQVR4nO39eXjcZ3U2jt8zo9FIGu2rZUnWasn7Kq/xbseOTRISIIQABQqBkjYEStn7vn2B0vKW0lIoS5uWtUBImgYSEm9xTJx4i2LZlq1dtmTZkixLsvaZ0Wikmd8fc933jCxhfuIi7Sff9znXlUuOZvSZ+ZzPeZ5zn/ssjy0UCsGIESNGjEwV+//0FzBixIgRK4rZHI0YMWJkBjGboxEjRozMIGZzNGLEiJEZxGyORowYMTKDmM3RiBEjRmYQszkaMWLEyAxiNkcjRowYmUHM5mjEiBEjM0jMbN6ckJAQSk5ORigUgt1uv/U1AIDP59PvJicnAQButxvj4+NTfhcTE/7oYDCIiYkJAEBcXBwAYGxsDA6HY8r7AoEAAMDlcuk1foexsTGMjY0BAGJjYwFA73E6nbDb7bhx4waGh4dts7nf/05JSEgIpaamwu12w+v1AgjfK4ApuouPjwcQ0XMwGNT7qCve++DgoHTE9zudTumIf8f3A9CzcDqdAAC/36/3DQ8PA4jo2Ol0wuFwoKenx9K6TUlJCWVnZ8Pv98uOEhMTAQAej0f/PzIyAgDScbRtjo6OAgDYUZaQkKBrUC/R64I6pb3PZLeTk5P6Ha/Ba46Njelve3p6+kKhUNYfTCF/YElMTAxlZGQgGAzCZgubAW2EdhcfHy/d8/79fr9slu+n7ff398vuKampqbLF5ORkABE7HRwc1GfzPSMjI9qXqGc+P7vdjpGREQwPD8Pr9c5ou7PaHNPT0/Gxj30M5eXlMqT09HQAkQVcX1+PoaEhANBPv9+v13Nzc6d82bq6OqxevVo3DwCvvvoqsrLCtpCfnw8AuvGsrCwptLu7GwBw8+ZNlJWVTVFWRkaGrh8MBvFv//Zvs7nV/3bJzc3F3/3d38Fms8mgLly4AADYuXMnAKC5uRnXrl0DEDGA2tpavOc97wEQWYjUwZEjR5CSkgIAuHjxIgCgrKwM5eXlAKC/6+3t1ft/+tOfAgDe//73AwhvCnRafCZHjx4FAOzevRuHDx/G008//YdUxR9c4uLicM899yArK0t2x02IOhsYGJBdlZSUAAB6enqwaNEiAEBjYyMA4NKlS3r/9u3bAQDHjh0DEHYsn/rUpwAA3/72twFE7HDJkiWorq4GADz44IMAgL6+PvT19QEATp8+DQDIzs7WawUFBQCAL3zhC+1/MGW8AZKRkYHPfvazqK2txb333gsgYlPcEN1uN0pLSwFANrxgwQIsXLgQAPDzn/8cANDe3q73cKPke6JbnXfs2AEAWLt2LYCw7qnDuro6AEBNTY2eUWdnJwBgzpw5AMLP/8qVK/ja1772W+9rVpvjxMQE+vr6kJaWhnnz5gGIeAEajdfrxYIFCwBEFOPz+eB2uwEALS0tAIC5c+cCACoqKmQgzc3NuoEVK1boBgFoM25oaNBGSG/LzQCIbJhJSUkAgPLyckxOTmqBW1WCwSC8Xi8uX74sZ0HncuXKFQDhDY6LjYt6dHRUz4D3SLSzb98+tLa2AoCM9uWXX9bm29TUBCCsUyDsjYuLiwEAVVVVAMIen0gqMzNzyud0dXUhPT19CvK0osTExCAzMxNpaWm6l/PnzwOANqBly5Zp8XGR5ebmalOk/XKxNTU14fLlywCAnJwcAGGAQCd81113AYDeMzg4qE33xRdf1GdzsVZUVACIbCoFBQW4efPmH1ALb5wEAgH09PRg/vz5eO211wBATpnrduXKlXI+XLculwt/+qd/CiCsn2iZmJiQo/jJT34CIOysuFf81V/9FYAwkAKAjRs3yu6XLl0KAFi8eDGuX78OIPL80tLSAISdW2pqqr7TTGI4RyNGjBiZQWaFHEOhECYnJ1FXV6fw+KWXXgIQCUUqKyvFz7S1tQEIIw2G1UQZZ8+eBRD2KFevXgUQCUG6u7uFIrnT5+Xl6TucO3duyu/WrFmD/fv363pAOJTntfLz82H16UN+vx+tra1ITEzEjRs3AEBhCKmLzMxMochNmzYBCHtY6plonn83MDCAO++8E0CEM9u5cycGBgYAQF6YCMXpdGL+/PkAIpFAKBQSv0OEylBmbGwM6enpt/W+VpCYmBhkZGTA6XQKCfI+qeuOjg4h4zvuuAMA8I1vfAMbNmwAEEFCjGj6+vrwyiuvAIBC77GxMSHREydOTHnt+vXrQui00aqqKmzduhVABI0vW7ZM1+L3sbrEx8dj4cKFePnll2UrpGAWL16s9zBS4ZoeGxvTumS0SfnP//xP3HfffQCAwsJCAMC3vvUt3H///QCgiIi22NnZqdCZkaTP51ME+fzzzwMANm/erO9TVlZmkKMRI0aMzFZm5fKdTidyc3PR0dEhEpocDAn/lpYW8X7ktpYsWSLkyN2diObmzZviG4LBIIBw0uDUqVNTPpucQUFBgXgMev3u7m5xNvTc/f39AMLc3LVr16Zlvqwm9L6hUEh8H++JBPbg4KB4lePHjwMIo22+j7olau7p6ZmWHCssLBRHRu/LjK3b7RYXTGQ/OTk5peoAiPC5DocDJSUlSg5ZVWw2G5xOJ2pqalBUVAQASkoxS+rxeGQzjGT27t0rDpBCsn9yclI2xSTC7t27hcKJCLkubty4Ifvm52RlZYmDJxfK7GpHR4eehdXF6/WipqYGeXl5sg1GO7SZnJwcPPXUUwCAd7/73QDCeiCKZqRI2y0rKxP3S7S/evVqvZ/7D+16YmJCuQiul7KyMiVy+dyZ9L1+/TrGxsb0TGYSgxyNGDFiZAaZFXJ0OBxwu92YP3+++Cdmp1hyc/bsWaXgKbW1teIUyJWxrKGnp0cIkxxiTEwM1q1bByCSvWMGanBwUEiJHNv4+Lg8BL1yV1cXgDAnUVVVZXnOMRgMwu/3o7q6WuiXeiRSXrZsGZ555hkAwKpVqwCEkQwR45YtWwBAnCIALF++HECYwwHC6J9lELdmvp944gm85S1vAQDxPc8//7z4ZaKb6DINp9MpNGtV8Xg8qKqqwu7du1UPy59EGYWFhUJ5tGWv1ysd0f6efPJJAGHekCVW5LiuXr2K+vp6AJEyLCKoYDCoSgBGVvn5+XrGtGXq1uVyCdVaXZKSkrBlyxY4HA6tuyVLlgAAXn/9dQBhvpr5BiLt1tZWIUau/f/4j/8AEEbVfDZc211dXeLcyf3+7Gc/AxCuLODviF75XYAIWiXPfuPGDTQ1NckOZpLfKyHj8XhEHJNk5cJZtGgRenp6AETC5JycHJHRTM/TKPx+P/bs2QMgUuLgdrvxq1/9CsD0hMzY2Jj+liFMXFycPqu2thZAJLyura2F2+22fLlJKBTCxMQE3G63vntHRweASDLlyJEj2L17N4CIwTQ0NGgBsnyE79+7d6+SBh/96EcBhPVx5swZ/RuIhNcOh0PGx2fIjROIGBZD0vPnz2NyctLyjicxMRF33HEHWltbtRi44ZMSOHXqlPTChb169Wr813/9F4BIfRx1XVtbq8VIyc/P18LnxknKaGJiAr/5zW8ARJxSQ0ODnBcdDMvfrl69qvdZXYLBIHw+H9LT0wVOSBdQX/Hx8dI5gU9tba1CYa7zT37ykwCA5557TiE2kzrnzp3Dxz/+cQCRBDD1d/nyZTkwbrgul0vJNSa8mLycM2cO5syZo31rJjFhtREjRozMILNCjvQQGRkZIqZZ6kBUMTY2JkTHcLC8vFxoiO+jpywqKlI4wzDi5s2bCjMYhhMZ9vX1TWkPAsIkKyE4kQ8918aNGzEyMiKPbFUJBoMYHR1FTk6OkDFRCHXlcDgUMhOBZ2VlyfvSa/NeozuTGI4vX75ciIRlDPz/ixcvCsWzlKenp0dh4Pve9z4AkUTYihUrkJaWZvnwb3x8HFeuXMHKlSt1XyzOJsr2+XyKLpiQ2bVrl3TPBABDtqVLl0rPRCqxsbG6Lmknlp78x3/8h9A418fo6Kiuz7+jnbvdbiEnq4vD4UBycjJOnz6t+45u7QXCXVXUNcv/QqGQdM3SH6L3TZs2KdrkM7v//vulc+4jjKCSk5NV+kQ9Z2dnqwSLURJLec6cOfM7w2qDHI0YMWJkBpkVcnS5XCgsLITP51NhJlEDPV5vb6/ieBLbgUBA5CiLb1kqUlRUJG9Mb9DQ0KCEAzkb8pGbNm0SGmIa3+VyCW2RHCeinTNnDuLi4pSQsKqwxe21114TYqDXI8m/c+dOEfYs9N65c6d6nalHIrwLFy6IGyaSBiK8C5/Bxo0bAYR5ISYSyNEsXLhQpT702tR1c3MzYmJi9LpVxW63w+124+TJkyrlYG81UXNcXBzWr18PAPjOd74DIGxzLAGhHomi29vbhdRpa9evX9f1yMPy2SQmJup35Mva2tqmlK8BkVKTvLw8PQury/j4OK5du4YVK1YIFRMJMiFTUlIiXf7gBz8AAOzZs0fokLrnms7Ly9Pewt9t2LBBpYO0Z9pka2urIlDqMCkpSWVx73rXu6Z85wsXLqCoqOi2Uc+sNsfJyUmMjIwgMzNTC5ehHxfM5s2bFfox9F6wYIFCYYYP3MQKCwu1+HnD999/v0JmLlIa5fHjx/VZ0ZOASOiSoCURnJeXh46OjtvWM1lBPB4PTp8+ja1bt+Lw4cMAIhsgieyTJ08qU8+e0ZqaGrz97W8HEOl9ZmhSWVkpp0I99vb2amMg0c2kS35+vhwb9ZecnKyQ8uWXXwaAKRvGhQsXpiRtrCh0PF6vV8kW6oMOIhAI6Hd33303gPDGyWwoE4JcTHPnztUzoN3evHkTDz/8MIBIuM710dHRoefI6ozc3Fw9MybFeP2enh7L0xUUVlqcPXtWtkGHy7C1rq5OlBerKubPn48jR44AiFRYrFmzBkAYFFGXBFYHDx4UhcFwnJvlo48+in/5l38BEMn8p6Wl6TO57xDU+Xw+JCcnKyyfSUxYbcSIESMzyKyR4/DwMGpra0VsstyEKOTAgQOqYmcocvToUSES1thx5+/u7pbXYIq/sbFRISJDF5aweDweTVShF0hOThZ6YRjORE5jYyNu3LhheXLb6XRi7ty56O3tFWLkvRApJyYmquOAyCQlJUVhBL0qPef+/fuF8Pl3jY2N6pBhYoV1kjU1NQr5GMp4PB6hSH4v0iVAOHyKnopkRZmYmMDNmzexdOlSdV2wJ5d6uXHjhpJcJPSbmppUwnNrh0VLS4v0uHfvXgBh0p+jt6gTotGUlBRdg2hqcHBQemYNJJ9JIBDQ794MEgqF8NBDDyl5cujQIQARm6mvr8c999wDIDKVp7+/X3ridB0i6fLy8mnTo4qLixUBMsHLyCgUCulZkr4YHR1VREl0T1pq4cKF2lt+mxjkaMSIESMzyKyQ4/j4uKaXkHilt6UHiImJkbckIiwoKNBuzt9xR09ISJAXZ7q9pqZG6JOFtiygLS8vV182yd7i4uJpE4jJq5WVlaGhocHyvBj7f0dHR+U9qVPyrxMTE0Jt1JXH45G3JulMpDk8PCwd0WPOnz9fk1PI2RIplZSUiNR+73vfCwB4+umnlSzgbE2i8JycHHg8HsvrNi4uDgsWLMDJkyeV9CMvS33ed999eOGFFwBEUMn4+LgiIuqIr6WlpQkB8v4zMjKUkGGShvr3+/0qnj958iSAsL75fRgJ8Ptcv35d+ra6TExMoL+/Hy0tLUJjLL5mUikvLw/PPfccgEji8OrVq7I3Ro1M0Lz1rW/F448/DiCic5/Pp0QPcxAf+tCHAAA//elPxQuzGNzn8wmZ3xoVTExMoKOj47YzFwxyNGLEiJEZZFbIMTExERs2bEBTU5NQB7lD7sApKSmK79lPnZKSot2fHpLZa5fLJR6IqLKwsFD8GTOC5BfT0tKU4SKqrKqqmtbnGj0Gf9GiRZafBG6325GQkACv16vvSn6FxcmXL18Wn0jepqOjQzriBHHyu93d3UKMLDaObnkjQmV2vKqqCl/84hcBRLKs7373uzUrk0Xj5MXGxsawfv162YJVxev14vXXX4ff7xcHxpIyzgYoLCyU7fDec3Nz1VzAIyUYARUVFUmnzDRfuHBB03hYqcFnkZGRIf6XSNzv9yuLWllZCSCyPt7+9rdPKb+yssTExCA1NRW5ublap7QVRphNTU3SNe/1+9//vuZdvvWtbwUQQXaNjY2aEs5ncPjwYemXz4oIsqioSPbPPMirr74q3pbRDjnjnp6e3xnxzHrwREpKCtavX6/wlWEEQ4Do3moaSHl5ueAtFz6/9AsvvCBimwa7f/9+leQwBOQCzs7OVsjNz9m5c6dq9xiu87Xdu3ejtbXV8gmZiYkJDAwMYGxsTMQ9QwwawPr163Vf7EAoKirSQyaBTQOora1VvSj1d/bsWY3CYhjCUMjn8ykMjD4Sgddl2QQTYiUlJXjllVe0oK0q0WE175VOg0mvQCAgW6bjuX79usJqhsk8cyQzM1N2yM2uublZyQM652i742bHz4yPj1cISKfHNfPaa69N6922qpBu6+3tVdhLB0o9dHd3K7HHvaC8vFybKM/cIaCqrKxUiR/1PDo6ql53lrTxfJnh4WE5KZb+JCUladwZnzuBxNq1axEbG2tKeYwYMWJktjIr5MgwYHBwUCU5PLKAaGV8fFxemUWvXq8XDzzwAIBwIScQ6dJYv369PCl3+YSEBHlxhnREQG63W56dHtvlcsnDM3ShBxseHkZSUpLlp/IwrL569aoG/ZKkpifMyMiQboncBwcHVThL6oKh386dO4UYic6LioqmoXGi+OgEAMOc5557Ttcjqow+njQvL+9NUcrT39+PvLw8EfNMpjA0vnLliuyVqCcmJkbJAw5qJaq/fv26yk+YACsvL59GMXB9ABHbJIpJT08XOrq1zKevr0/fw+pC2+3u7pY+GZUQLSclJSlJw97qpqYmFdyzkYEhd01NjeyU1F1SUpJ0SN2zQyYvL0+om9FVbGys9gp+n+he67i4uNvOXDDI0YgRI0ZmkN9rnuP8+fOFYMgNkpdKT08X6c0SlMbGRiVRSNCyH7e/v1/elkWcW7ZsEVoiKmKhssvlEq9IjxIXFydkGN1vDYTbvPx+v+VnDrpcLpSUlGB4eFjcDMs8SEJHj5XnsOCBgQF5ygMHDgCIFN9fuHBB+iZHc+7cOV2PCR8+i/vvv1/lJizBcDqd4jSpYyL2kZGRN8UxCT6fD3V1dVi1apXulSQ/pxC99NJLQt5sOUtPT5edsxicOnA6nXjooYcARPhfj8cjRM+fLDs7e/as9MZrHThwQL/j+/nsenp6LH+0ByUYDGJsbAypqamKIBn9sGe6trZW3DnLd3bu3Cn9U2h3Ho9Hf8so88aNG0KfjFy5Hj7zmc/oNXLgycnJSviQW4zmL202222Tib9XQqaiokKwloQo6xDnzJmj0ICwODY2VqEEwwdmo4FISE4l9PT0CO7SkCi5ubla8FyUpaWl6vtlaMSGc5vN9qaoxQsGg/B4POjs7JzSQQFMHTjLrHx0WMGwjlO86UiWLVsmp8WQeffu3UqosMeVIfeFCxc0oINdH8XFxXKANDBu2idPnsSiRYtuO/bJCpKQkICVK1ciIyNDmUwm/+hkgsGgMsvRWWjaE4Wk/5YtW/DjH/8YAPDggw8CCDtrXoM2TYDQ1dU1zVGVlJRos6ZOaffRfe5Wl/j4eCxYsAC7du3SOqTjjR41SHvmJva9731PtMUHP/hBAJGKFdJi/FsgDA7orEijcYiF2+3WHsTNcfXq1QJQtzqh5uZmzJ0715whY8SIESOzlVn3Vvf39+PatWsKv4hMGP62tLTIa1La29sFqbmD0wPk5+crfGCNlNPpVHhMVMOwODU1Vf8mGq2qqhJKpSfmT7vdjoqKCsvXOfKck+LiYnlPelWWdNTV1SnsILF8/PhxfPWrXwUQKVMgat6zZ4/CRz6vwcFBeVaiKOomISFBISJpjcnJSSUGGPLRq+/evRvx8fGW1y1PzfT7/Sot4T1RPwsXLhRqY53j29/+diFoouNoRMiEWfRwZ+qCJSrU1b333qvnwgjp3LlzirxIRVHvp0+ftnyiixITE4M5c+bgzJkzshHuCzwl89lnn5XNMklVUFCgsPdWO62urp7WfZeRkTHl/HogUlYWCASEHHleTF1dnSgq7gdEr5WVlcjIyDDHJBgxYsTIbGVWyHFsbAyNjY0oLy+Xh+TOTWQYGxurwlcWtt5xxx16H1+jRz137py8AHmxnJwcEaXRnA3fQ+9C6ezsFCJlxwHJdb/fD4/HY/l5ji6XC6Wlpejv79c9cyYeeZzoshn24K5YsULlEkTg/Dk8PKy+dSZt+vv7px06RLSYnZ0tTo1Iqb29XUkI/o7lEadPn8Y999xj+WQXS9BycnJULsbia6L0vr4+2Rw5Qb/frz5o6pvI88KFC3oWu3btAhBO4PB3tFeWphQWFuqzeC23262IhwX/RJwLFy5UR4nVxe/3o6WlBdu2bRPHxwiH/K3NZlO+IbqBgciRCI8lfNFJX0r0hChen4Odozte7r33XgDAl7/8Ze0H/DvuSSyh4lqbSQxyNGLEiJEZZFbIMT4+HsuWLcPp06fFDzKWZ7vZ2rVr9TuW5ly5ckVoj1OlmW0qKSkR8mFv5Pz58zV1hl6GR4tevHhxynm0QLhAnF5j3759U147ceIEurq6LN8+yEOKqqurVf5Br8sZi+Pj4/KEzOIXFxcL6RDl0EOfO3dOmT6iovT0dHGNLKkg0iwuLlZrIVFiSUmJuDjyNZT169cjLi7uti1YVhC/34/m5mYEg0FFLERxvN/s7GwhYGb7x8bGZJNsRuCzGB0dxYc//GG9DwijSj4DPkNyaefPnxcqIvc4f/58rRWieCKayclJrSmrS2xsLAoKCvDiiy+qWYOImFHGww8/jO9///sAItnku+66S7w61yf1wDkOQHgyFBDOU1Bf3E+i5zjws5999lkAYX6YZX/kkblWVqxY8TvPs5/V5ujz+XDhwgW4XC4tGKbGucBINgORMLm0tFQkLA2EYW5hYaFCHJKjiYmJWuD88jS6UCgkA6UkJCTIoFkiRMMqLCzEzp07FZpaVXhMwoYNGzROixs+DWzRokWqUfyHf/gHAOG6reguASBC7l+/fl3PhXovKCjQM9u2bRuAiDHdcccd6gThMzl+/Lj0ze9DovzKlStoaGjQ97OquN1ubNy4Ea+88oru/dbazfz8fIWEDI2jz0piOMYw2ePxaIPlQnU4HArTuAhJJzGsAyIbR1VVlQAEu5ToxILBoGryrC5erxfnz5/H3Llz5XAJagiCgAjVxUTqSy+9pGQjNz06DrfbrWQObfHatWs61oK2y66Yy5cvKzHGNXLz5k0BAdJu/H55eXlIS0u7beecCauNGDFiZAaZFXKMi4vDokWL0NbWJpRCT0fU19fXh1deeQUARLYeP35cnpGEPxHN4OCgwhJ6ytzcXIXf9NQMJ3NycoSMeLLbqlWr1PfLUJRwemJi4k0xbp6TY65fv65yESI2IpTLly+rRIJoZd68eSoCJ3omShwaGprSpQSEy4MY1hANMUTv7+9X6Ew0uHXrVpUI8TkxlMnMzLQ8XQGEow2i61t7f0n6Nzc3T+tv9ng8KinjMR2ke1auXKkCcdr06dOnFbbzYC7qeMeOHQoPWb6zdu1aUUTUO7/PXXfdpSTam0VKSkpUIsXieiK1hQsXas0zYTI8PKzSH3Yl0dZyc3OnHDEBhPcMvp/X4lrJz8/XNZ5//nkAwJIlS4RIabN8nv39/dixY8dty6UMcjRixIiRGWTWpTwNDQ3YuHGjeANyjOQTLl26pNIGzrZbvXq1uB56UnpwIML/sBzo6tWrugZREdvmTp06JfTEsodgMKhiXaKn6P7rN0MpDxAudwiFQvK+1C05l2AwKBRMxBEXFyfPSt0Shdx1113SMxFJTk6OECmno5CfTUxM1Dw9JiIyMzN1PXpyorCBgYEpk3+sKl6vF+fOncPy5csV6VAv/FlRUSH+ivdjt9vV3EBhtMIjf4FIOcmhQ4fEeZNLI4o/cuSIEDtbPx0Oh6IlInVykD6fT2jf6sJEbUdHh74/mzeI2JYsWaIIhfcc3dTACId6+MlPfqKjOrgH/OAHPxCCZ+KK67qtrU3H7rIt1uPx4JlnngEQ4SH50+FwYHR09Lb7wqwngW/ZsgUej0ewliQoQ92EhAQZIBdhTEyMFjwXdTQRzk2RIUlGRsa0Mz54zcrKSi1q9laPjY3J4AijSYg7nU709vbedjSRFcTlcum8Gzbck6rgok1LS9N9MLzzer0KLUj60zCdTqcSUXQWIyMj2gDZ9M+zU0ZHR5WAYCJh1apVMlwadfTZQJcvX7Z8aB0KhTA2NobXX39dGU3aL++pra1NTiC6No+Og4uRC3zbtm3T6m3nzJkjm2fdIq8fDAaVoCSwWLVqlbK1t55H8/rrryuEtLowmTg+Pj7lvHMgYsNDQ0PqZ2eSdXBwUFQD+9MZVnu9Xumetrh7926dl0RahJvr+vXr9ZncO+bNm6ck262nDeTn52NsbMycIWPEiBEjs5VZIUev14vq6mrk5OQoNc46LiIan8+n3Zj1b6+//rpS+wxT+P8nT57UtYgSW1tbVePELgEi1MHBQXnj6PFGPEebCJVopqOjQ8jByhIMBjE6Ooq4uDjRBaQIiCZSUlJUr8Ww9+bNmwoniEyIgJqbm6VHvuZwODR5h73B0VN/iDT5nBobG/U+lmOxjKKhoQF+v9/ylIXNZoPL5YLD4RCCZtKQSZicnBzpgeFcX1+fqAQiIiZh/H6/bJKhc1FRkegLIiYilZycHCEbIiin0ynqhPWWRPUDAwNCn1aXyclJDAwMaKQhACE82szFixeFuom4a2pqZGdE04xSysvLpyXRoofTEnFTlzU1NXqN0WN7e7s6mhjeR88YmD9/vui+mcQgRyNGjBiZQWY97HZ8fBzBYFDFmvSaTIkXFBQIfbBLYNu2bSKXT5w4ASBC7m/dulXFnuQioqf40DNET1NheQBJ72AwKERF3pIIwe12Y2RkxPITTjweD86cOYONGzcK5RF9kCdrbm6W9yWXsmbNGhXIk4PlM1m9erUIaRLdBw8e1HNh6QPRypUrV6S/u+66S9+LiQden5zvnDlzsGnTJiFRq4rdbpcuqTfOvmT0MXfu3GlF4O3t7eJ/ieyIbKLL2Si5ubnigom2GQXY7XYVRDOps379enHvRD3RPfCMhqwunAuQmZkp5EyUTD3Y7XZ84AMfAIApPCBzCVz7tMXm5mahSXa9HT9+XNdl4od6Ky0tVWKS0UB+fr72FnLptO/8/HwEAoHbdsgY5GjEiBEjM8iskKPdbkdSUhL6+/vFM5Hbowe+ceOGkAt39StXrgj50ROzHKejo0Pekzt/QkKCdn966uhpMZxWQy/tdDp/a7sWS1GsPgkcCCPg3t5eZTyJkKkru90urpa8q81mk/5YTkV+pbu7W0jk4YcfBhDOSLOygNdlgf2qVauUlSV6io2NVYUByyj4zG/cuPE7yyGsILGxscjPz0dXV5dQMCMLRiaxsbFCasyY2u122S2Rd3SGmn9LSUtL0/G1LNuJPsqD3BlRucPhEJq8td89ISHB8tOOKDwmobm5WRwrIzk2cdjtdvH+tEmbzSYEyN+Rx+3p6VFExCj14sWLQpPMddA29+7dK9tlDgOIIFHaOt/T0tKC0tLS22arZx1WBwIBLFy4UGQxa+sYBt95553qVuEXGR8fV3kJiVQu5IMHD2pjY+p+3rx52vj4fo7qP3LkiAhXjt7av3+/rkejZG3UypUrMTY2ZvmwOhQKafFwgXABRw/a4P3xftxut2o8f/WrXwGI9Lu3t7erruvQoUMAwqEwdctxZvwZfWg66/UCgYC+B42bBhwfHz9lQK5VJSYmBjk5ORgdHVXXVnR4DIQTTx/5yEcARMp1mpqaZNdMbP30pz8FEE6wkF5gqNfZ2akFzS4YdmTk5+eLzoiuC/3Rj34EIEKFRJf03G6clpWEx6cMDg7ivvvuAwD88pe/BBAJl0tLSzUEhc577969ckSsY+bAWpfLhfe85z0AIsBo/vz5GgDCa5HaaGtrU73piy++CCBc20uH98QTTwCInHI4OTmJK1eu3NZ2rW3VRowYMfI/JLbZQHebzdYLoP2N+zpvqBSGQqGs3/22/xkxun3j5E2uW8Do942U36rbWW2ORowYMfL/ipiw2ogRI0ZmELM5GjFixMgMYjZHI0aMGJlBzOZoxIgRIzOI2RyNGDFiZAYxm6MRI0aMzCBmczRixIiRGcRsjkaMGDEyg8yqt9rtdodSU1Nhs9k0Aip6yC0QbpjnkAeOuBodHdX72ZfL99hsNvUU8/1paWkap8XRZuw5TUtL0zAA9kUmJCTodX4fFrcHg0E4HA709/fD4/FY9vT5pKSkUFZWFhwOhwY73Hq2ss/n05AH6tPpdKoHl7pib/BMMjY2piEKvBaHdNy8eVOfxd509g0DkaMF+Nkejwculwt9fX0YGRmxrG4TExND7F3mAGbqmPdrs9l0fxyAMjExoffx76IHrtBeOXYrJiZGgxfY/8+RZcnJybo+++IDgYAGL/AZ8nOcTqfs+9q1a31W7pCJj48PJScnT5lfcOugl6SkJL0efc459cPnwwEck5OT0877sdvteh60Wa77YDCo9/Ga1CUwfV9wu90YHx9Hf38/RkdHZ7TdWZ8h8+CDD2Lbtm0614SN9RwCEQwGNc/xbW97GwBMmUsX/YWB8DRpNp3z8KjVq1fje9/73pSb5zSf8vJyzXHkZI4PfvCDmilI5XFiit1uRyAQwN/8zd/M5lb/2yUxMRHvete7kJmZqYXF+2TjfVpamoZLcPLIpUuX1OTPWXoc4BG9gCl9fX0aTMGhHhyIsGDBAmzatGnKa21tbRrqwSnr3Bzj4+PR3d2Nf//3f/+D6OCNkoyMDHz+85/HxMTEtKOE6axbWlpkh//n//wfAGF9c2IR3/d//+//BRA+1+e5554DEFm8O3bs0DR7/o6LvaGhQbbMQSHr1q3T9bOzswFEBmE4HA7N8dy7d6+lW/NycnLwv/7X/4LNZpMN3no4XEZGhn7HjfN//+//Pe1a3Cseeugh2RtnMdbX1+Ohhx4CEJnnyKk7APC1r30NQGTzXbBggaYeHThwAEAEQHR0dGDHjh348z//8996X7PaHJOSkrB582acOHFCY8MOHjwIIDJg8vDhwzKQz3zmMwCAT37yk/KMf/InfwIg4m1ff/11HQdAOX36tDY3KpIb7Ze//GV861vfAoApnoJKItKhl3rppZeQmJh429FEVhC73Y6EhASkpaXp1EYaAB/o5OSkPCYRyunTp3U4Ece1ccrOc889p/dxAwQiBvVP//RPACKb8JNPPilnR323trZq0Ct/clJNY2MjSktL3xRTeTIzM+FwODQFhmPheC8jIyP4i7/4CwCREf8ANGWGv+Piz8vLkyP+4Ac/CCA8iYcTpjiyi1HOCy+8oLFcfE7vec97dEwAh7AStSYmJsopWl3Gx8fR2dmJiooKTcshCOK6nzdvnsbmMSo8f/48fv3rXwMAPvShDwGANqvU1FTZPR2Tx+ORfr/0pS8BiGyEk5OTsuM//uM/BhDem3gcBo8OYaSwd+9etLS0mAO2jBgxYmS2Mivk6PP5cPHiRR0FCkAIkhD40KFD2sE5G3Dz5s2asfZXf/VXACI7+ZkzZzQol17H6/VqsCp/R49fVVWFp556CgDw0Y9+FADw93//95rlxlCEnnjDhg04d+6c5QeHulwuFBUVITY2VodncVAq9VlfX6+QmGjb5XLpnulFGV4sWrRIsxg5n/Hee+8VBUH0xEPMtm7dqkiAyH779u1COvTkDLOJ2Pl7q8ro6CiOHz+OyspKIW+GeKSHFixYoOMLiNgXLlyoUJv3TD2+5z3vERIiGuno6JBOaaPU7b333qvP5KFTly9f1rM9deoUgMizbm5uFu9rdUlISMDKlStx+vRp0TH8yb1ifHxcUQnv8ZVXXtHQZuqQdv2P//iPuv7nPvc5AGEq5I/+6I8AAH/2Z38GAJrP+d73vlfInJ85OjoqZMj9g9RdMBhEY2PjbQ/eM8jRiBEjRmaQWSHHhIQErFmzBvX19Towi6iBO7LP55Nn4PTkq1evKmlAVPPNb34TQHjyL8egP/DAAwCAT33qU/KunHxNr1NeXq5pwJwO/uEPf1hJIE5npvfIyMhAbm6uMlhWFY6ZD4VC8qbkVHm/Y2NjGrEffSgZOT/ys8xWv+Utb8Hf//3fA4hwZ6+88oo4TfJh5IBSUlKwY8cOANBRCs8884y4RpLb5HOXLl2Kl156aUr20YqSmJiIjRs3oq+vT6iQ0+qJrKuqqsQT7tq1C0AYGRNNklMnZw5EkihEPcnJyeIwmdXmMQHnzp3TmmGSLDc3V5EAJ+WTxywrKxPStLp4PB6cPHkSS5YsETfL6IcJvpdfflk6ob2WlZVpsjrXLznd4uJica6MSo8cOSIuk3sG9f2xj31MXDp5xrGxMZw5cwYAcM899wCIHMdQVVWF8vLyaQnLaDHI0YgRI0ZmkFmfIeP3+5Gbm4snn3wSQKTeiLzXmjVr8O53vxtAhEe5du2ayhe4u/N8GbfbrXMgeM5JSkqKSnl4iHr0GTTPP/88gIh3ys/Pn3JuChApdfF6vW+KQ6B4Zkhpaam4pmi+BgifLcPsJrOaH/nIR3TkJVE5Efivf/1rIU1m+a5duyYumDpjOcndd9+N73znOwAiZ3k88MADQphE3zxelwdXWf18nmAwCK/XC5vNJu6Z5SFEOpcuXRJaJgJ/6KGHFMFs2bIFQOQguYSEBCFH2mFvb68iHJaeUTcpKSlC3qwJHhwc1DMgv8yzZ1599VXLRzuUUCiEUCgEl8ulY4X5k/WLiYmJykwzT/H000+r/pB2zfONtm/frmOFmT8YHBwU/7hkyRIAkSqMuLg4fPGLX9TfAmF0ePfddwOAyq74/sHBQfT09Cg6m0l+L+1fuHBBGxoXFsOV++67Twb1jW98A0A4bU7im4fm/OAHPwAQhs9M99PYPvGJT0iRNFgS3bt379b7Was2OjqqzZe1j1ReZmYmmpqaLF9u4na7sWbNGly8eFFhKkltks5AhMagM2poaNAzoAOIPvObzoLPy+/3awFzoTO0DIVC2jQY3gwODiqsoW65aa9atUpEt5UlEAigt7d3iuPhKYIMdWtqarBnzx4AEVuLjY1VyMyDs5ggfPTRR+W0qE+73S4KhBsgy6piY2NVnnbs2DEAYYDATZeJgSNHjgAIbypcU1aXuLg4VFRUoKqqSps7E1ekBo4cOSKKjJt+ZmamKB3aHU8XrKurU5KVlN2+fftEOTHBS7qot7dXm+Pf/u3fAgjXq3IfIFCjI8vOzobH4zEHbBkxYsTIbGVWyHFkZAQvv/wyduzYgVdffRVAJBSmB6ipqRGCIYL84he/qEJOFiUTyTidTnlqhuh+v19oaN26dQAiqMjhcAiqs9j87//+7/Hd734XQOQ8Z5af5OXlobS09LbEqxXEZrMhJiYGubm5QhFMnFBXJSUluP/++wFEvO+hQ4eEoIlS6K3T0tKkI6LRuLg4kf5ETUxsbdiwQd05pCUyMjKmnWVNL+/xeDA5OfmmKJMqLi5Gc3OzCHneHyMTIHKeNMvMnnzySSVUGP7yKOLTp0/jT//0TwFEkmNnzpzRsyAVQeQ5MjKiaxBx2mw2fQ8mGEg7eTweRQ5Wl0AggK6uLrjd7ikRJBBJfK1du1ZNB0Ttjz32mKggIkGu87GxMXW1kFrLzs5WpMLnxgaIZ599VuVnTASnp6crUv3sZz8LIJKQGRgYQFdX123PszfI0YgRI0ZmkFkhR6fTidzcXFRVVQm5VFdXA4j0827atElegIWwdXV1QoI/+9nPAES4xxMnTog/467+uc99TsWeRDf0xMnJyfjwhz8MIDIc4WMf+5i8MkuE+P6UlBScPHlSfIdVxe/3o7W1FVevXhXHQhTB5MtvfvMbcYHsWd2+fbsIa6Jz8ihOp1NIkPefmZmpQluWrJBPGxoaEq9IFH/w4EFs27ZtynUZJRQVFalw3coyOjqKV199FZWVlbIL2hwTi5/5zGekbyLBvLw8cVYsCSHf3dHRIb6Qujp9+jQeffTRKdcl97h7927xcET4RUVFQvm8BnnJkydPWr5EijI+Po729nYkJCSI0+P+EN1AwL5oojWv16uogzbJNb169Wo9K5ZTfeADH1CZINH9008/DSBs+8uWLQMAlQ0+/PDDQvpEn0Smzz33HPLy8qbNeoiWWW2OgUAAN27cwMjIiHpv2aXysY99DEB4ITPZwqxRU1OTslAMU7jI169fj8OHD+sGAWDPnj2C41/+8pcBRCrm165dq/5pGs8Pf/hDZVepUBritm3bkJuba/mMKhAOswKBgDYhhigkk5OTk0X+//M//zOAcJaVYTENkgu4sbFRHRpMJLzvfe+ToTDRwn7W7373u9IjO0Oys7MVwjAZxI32xo0bSExMVOhoVUlMTMTmzZuRnJysjbylpQVAhFJgvR0AOYPnnntOmyI3L+q2srJS/b3UVXV1tRIy0clCIFxFwSEUpH46OjoUAjJrzkU/MjKiRJnVxW63IykpCQ6HQwlAZqEJePbv36+eZ2bmHQ6HbIubKIdRHDlyROv8E5/4xJT3AJGZDtyH/vM//xPvfe97AUQ23+9973v4whe+ACAyR4BJyPnz5yM9Pf22+4IJq40YMWJkBpntPEdUVlYiGAwKpfz1X/81gAjJHAqFcPHiRQDAz3/+cwDhzgpWuzOhQNTy2muvqS6SHvjb3/624DOvReK1sLBQyIpe9h//8R/lcVjD9/jjjwMIo63GxkbL1zkGAgF0dnZi1apVKpUhcmTYXFBQoDIdeuTMzEwlm0hBEJUvXLhQITpRX2pqqpJiDI+ZVLPZbOohji5xIIrv6uoCEHnWHo9H/1lZnE4n5s6di4aGBt0fdfbEE08ACPf7EhUS7b3zne8U2ru1bi81NVUhMKOVj3/84/j+978PAAohiUKffvpp2TcTYix7ASLlQ3x2d955J06fPv0H0sB/j2RkZKhukOuX+jpx4oRQNUv2rl+/rn2B9AUjxcLCQiFGypkzZ5SkYSTKbpjExET86Ec/AhBJbhUVFYkqoe0y2dnd3Q2fzyeEO5MY5GjEiBEjM8iskGMwGMTY2BhiY2NFpJL8ZAIlEAhM4RqBMIdI8p9oj5yPy+VSap/eoKysTEXjRDAc8vqBD3xAaIjc2ec//3khIyIBFoV7vV6MjIzcNmVvBUlKSsKuXbvwwgsvqI+XKIfeb2RkRF0WRBi9vb16FkTj1G1RUZHKgFjSk5WVpdeJfKj/+fPny9OzGHfFihXyxCwtIU/25JNP4s4771QJhVVlaGgIzz//PD784Q+LD2eSi5xXdME8k4clJSWyZXJV0R0tvG+WOpWVlSmSoq6IlubNm6fEQnRhOJ8jIwJylePj40q6WV2SkpKwdetWtLe3K6pg/oAo8a677hKaJGo/ffq0CuLf9a53AYgg7b6+PiFHcol2u138NvVLG16yZImSifxMn88nZEo9sxTo6aefxr/+679qv5hJDHI0YsSIkRlkVsjR7/fj0qVLCAQC8rws0mbG+c/+7M/U28tC0IyMDO3q9BQsJL548aLS+OQrmpubxbdwbhvRTm1trXqJv/KVrwAIZxL37dsHINLixiztihUrsH79evVWWlVYYL97924hF6I3Zkq7u7uFkJnx/Nd//VehQ/ZP06s6HA5lQZm9z8zMVIaOZQ7R52r8x3/8B4BIe+frr7+u6xE1sbg/PT39TdH/Gxsbi6KiIjQ3N2s6C4VIJzY2VjwuEWFTU5N4Qc4XpX0tW7ZMuiVH2dDQICRKhMJs+KuvvqqeYj7DiooK2Sm5ePatj4yMaI1ZXXw+H+rq6nDy5Ell2GkztK3i4mL85V/+JQBMKdHhNB6iaSK52NhYRX+scGloaND1iAS5L9TU1KiihftOX1+fdEhOk+g9EAigqqrqtnz5rEeWrV69GkNDQ0pw0KDY0F1fXy+iPzpNzhCRZ0T827/9G4BwsqahoQFABFJ/4hOfUIM5b54DEY4dOyaF0ogdDoc2EBoqw8/+/n7Ex8fftp7JKhIMBuHz+bQ5kqpgYqu8vFxhF8OFsbEx1cqxvIHn+eTk5OCFF14AEClvOHbsGN761rcCiIThrFXt6+vT33Jjjo2N1b8/8pGPAIC6oy5cuIBXXnnF8jWkNpsNTqcTN27c0Dg32hVpnu7ubtkyHXJtba0WHKkOvj81NVXOIprm4bqIPogLCJdZ0XmxJOXmzZsCCUyS8fu1t7dbvvMoWoLBILZv3z7FcQKRMrR3vOMd0snf/d3fAQg/F26A3CtIL/zjP/6jkiics7Bt2zYlXh955BEAYXAARJwWELH1zMxMJdm4XjjgIi8vD3FxcbfdF0xYbcSIESMzyKyLwDs6OuDz+QRdWT7CEKOkpEQelSR2T0+PijXpGdhB8OlPfxpvectbAGDK4UcsH2GReXRChkiTBO+lS5eUoucEG3qEtLQ0HDx40PLoJjk5Gbt27cL+/ftFxBPR0avu2LFD5D/D2W3btqncickrIsmCggKNeiOSzsnJwdmzZwFEQh5602984xsqXWGYl5SUpNCTiIeeefPmzRgbG7N8aO1wOOB2u9HT06NhyLxnor6rV6+qrIMI7z//8z+VrCICYcmNy+WSzdP2li1bJjqISQc2Kvh8PtkmqYv+/n4lKokYeU0euPZmEHZ3xcXFqVyJ+wIjxpGRERV1M/nys5/9TIlXhuNEi4899pjex7FjX//612Vr/DsKoyEg0qc9b968aUdBswNnYmICZ86cue1RFAY5GjFixMgMMiuXPz4+jo6ODiQmJop3YpkH0V99fb1KT4h8srOzhUjIczFpc/nyZSESjuNfuXKlOEz2TvIwo6amJpHY0RN4mBjitd7+9rcDCBO3aWlplkc3g4ODePbZZzF37lwVtnLqC5NYcXFxKo96xzveASCMbuh1mTRg8filS5fEedEjNzY2CqWQy+HRokuXLtVkGiYGHn30UX0flgMRcf74xz/Gnj17LM/njo6O6ghb9joTXZNLTE1NFT9GBLl06VK9TntndLNv3z4Vht97770Awu2A1AU5N7bKpaaminNjJJCXlzcN0VC3ixcvVuRldbHZbHA4HAiFQopUmJgiX82yOyDC93q9XumQiS8i9a1bt4rjZolfd3e3jvHgHsChwkSqQKRgPzExUckfllTxeTqdzt+ZUPy96hz37ds37Wxlfrjf71c2jgMiMjIyFE4zq80wBYgsdG6cBQUFIlCpLIYi73//+2VQ/A7z5s3TBsFNktmsVatW4erVq5bfHN1uN9auXYvBwUGFH9QZyeazZ88qu8afDzzwAD71qU8BiIw2Y6ZwcnJSi5OOx263a4OgYdGACwoKFCLy71paWrSYmShiwu3OO+9ETEyM5TdHIGy7w8PDomu4SLhJLl++fMrCAcJggHrm4uO92u12bWScTN/S0qLQmfbGbKnNZlM1BkPujRs3ah3QfvkscnJy3jSDJ1JSUrB3717Y7XbZINcyN8nU1FRtaKQSvvKVr0w71ZG637dvnxKR3Oz27t2rfYFrg0lcJmqASIWA2+1WWM19hLSF1+vF2972Ns0SmElMWG3EiBEjM8is4FRsbCzmzp2LF198UR6O3pNE7OTkpJAL65N++MMfql+YnpWe8vDhwzqhjOUMIyMjIrY5Np7919GfScK6tLRUSRoS7kxA1NfXo7Oz87bn01pJMjIy5D2JwBmiJSYmatQWEdADDzwwpewGiJSR3HHHHUrg0JNfvnxZumA3DBHT2bNnRYLTQ69bt04JGU6oYWhz+PBhlJSUWP4ICg67raurE/K79eyWO+64Q5OlaEuVlZXqO4+OaoCwjfLfLAHy+/0qhWLJDxF7cnKy0CQTbocOHRLyoZ0TXVZXV085H97KwmMoioqKhNpIxTCEjq6hJZo8duyYxhqSQmBU8/jjj0tfDLk5e4CfCUSooblz5wpp0nYXLVqk5835BKztbW1txfbt2287UNjaVm3EiBEj/0MyK+Q4MTGB/v5+eDweoTeiG3rM/fv3i5SltLe3y5PSy7Jivbm5WaUo5AuvX7+uVD1T9PQQY2NjKgsiie31enV9DiBl6c/b3va2N8VUnlAohMnJSVRXV6tflH2kRNvl5eVC1/zds88+q9ITIk6WLSUnJyv5wuTYtm3bxDXyhEfyMqmpqdI7eV273T6N943mNFtbW5X0sbLYbDYsWLBA5D6HMxNllJWViX9io0JMTIySi5yHSYJ/6dKlQpPU8Uc/+lEhUSbRiC63bdsmLpk8ZF5env7NDhHOKlizZo2SQFYXh8OBlJQUtLS0aJ2RX2RnWigUUpKVus/JyRGPzX2BSb/U1FS9n6VnmZmZ4iY52JYRZlVVFb761a8CiJShJSQkCLkz6uEhaikpKTh06NBteV2DHI0YMWJkBpkVcuT0jcuXLwtFMJtHXiomJkYcGLmo1NRUIRdyEp///OcBhDOfLKEg6svLy9OOT89O7zw2NiYegXza4OCg2pDozcljXrlyBRkZGZafVk3vu3XrVnldHr7EaTHRmWx6U6/Xq3sj6uO95+XlKTPKgvmRkRFxmLwWM4xer3daxvbSpUtC7fT4LIRubW2Fy+WyvG4Z8bS3t2uSOnlXloM1NzdLV8w+33fffarGIJLmvTY3Nwv10M4zMzP17Giv1FVfX5/WDCOk6EO0iGhZlB4MBrUGrC4+nw8XLlyYUirFiI+89sTEhKIezro8cOCAkCD5XmaT+/v7ZZ/U/cjIiOySJYSMOq9cuaJKGLbRrl+/Xnoll8v9weFwYNu2bYoEZpJZbY4xMTHIyMhAKBRS6MdeSg6Z/c53viPoSmOYO3eujIQPnMNrAWh8Or/4/v379T4aGQdZbtmyRUkXKj4uLk7hCI39F7/4BYDwQxoZGbF8WG232xEfH4+XXnpJNWG3Jk5efvll3QeJ/KSkJFEKLEthTenZs2fVaM/wze12K1yjM3rnO98JAPibv/kbOS8aX09PjzZpbgI0qKGhIRHdVhan04mcnBxkZmaK3OdZJwz7WlpaFEJHb0q8P5bcUI/x8fHa5FhqMjo6KpDA8ismAPbs2aONj9c8fvy4ylr4XLmY6+rq3jQdMpOTkxgeHsZzzz2nzZH3+MwzzwAAPvShD2nPYCh77do11dVyY2M9bkVFhZwynbnb7ZZD4vu47wCR5NqHPvQhAFO7jJgE47Nds2YNnnnmGa2dmcSE1UaMGDEyg8y6t7qnpwdDQ0PygkzEcEfesWOHeiF5CNT4+LgGULLvkeioq6tLJCwTLfPmzZM3vvWc64mJCZHX0VXxJMx5uiG/X1tbGyoqKix/Qh67DJYuXTqtWJhjti5duqSwjugwOztbeiA6Z7IrKytrSkkTXyNSZzkQp6RcuHBBemS5TkNDg0JyenI+87y8PGRnZ1v+8DI2L/j9fiEbFrJTfD6f9BhdisZ7JiphIuz06dNKfEWXQjGJwPCdpSqXL19WmMjopry8XCU8jMCITHNycvQ7q0tKSgr27NmD0dFRjRwjQictUV1drQ4u6u1v//Zv1Z1FJEi6CIjQOEwg/vSnP1WIzWiGz3Hp0qVa49xrVqxYoedFYfRTW1uLuLi425ahGeRoxIgRIzPIrJCjw+FAYmIimpublXQhyuFuzZQ8ECFZu7u7VdDJshGinLq6OiFGlgdFewGWCpEkX758ubwHr9Hc3KyCcxLi5Hw8Ho/l+UYg7E1PnDiB/v5+lX+QJ6FXzc7OFt9KrqS6ulqFsZxzSWQSExOjNkpyXxMTE5qlydmQ73vf+wCEEz+cm0mec2xsTPwcC/f5rG/cuIElS5ZYHjmSE2tra5NuyK0yAbB06VKhctrh+fPnRfjznHWe8b1u3TrZIZOGubm5slM2LZB/37Vrl2yf6DU5OVn2zSlVLCPKzMyU3q0uRObZ2dk64palTJQ/+ZM/kR0TLXu9XnGs1DMjl/Hxcdk4kePExIQiSLYrs/W4pqZGnDH5y4qKCj1Tco9EsrGxscjJybmt7c5qcxwdHcVrr72G2NhYPThmmwhzOREYCA9FAMLh9Wc+8xndNBBZyMXFxfirv/qrKTdQWFgoA6LhEYqXl5crcUNjGx8fV20TkwUk2hcuXIimpibLd8hwIOv69etF/rPujo6ivLxcmT4mu86fP68Qm3QG9QJENkCGaF//+td1fRofDcTlcsl4aGhNTU0ySG4a7GmvqKjAkSNHLN8DHB8fj4ULF+L06dNy5lyMPJ/k2LFjumeS/vPnz5cD/uY3vwkgEkInJSWpioAhdHt7u2pv+Xz49ydOnNDEe4bOiYmJ2hyZ3GHd3s6dOy1vsxSfz4f6+nr4/X5VTFDPpLdSUlJkNwyvf/SjH2mdU2/U0QMPPKBOJUpBQYFskPqi/u69914lGLn2HQ6HXucmSWpjYGAAqampt620MGG1ESNGjMwgsy7lSU1NRUdHh2qJiBhZMrJw4UIlAdi5UVtbqwQL+yrpFUdHRzV4lGPPHn/8cYXCJFz59wUFBUKYnMwRExMjBEtYT7SVkJDwpkgaxMXFYfHixRgaGhKiY+KDOoiJiZH3pUfcvHmzQsTos0+AcBKGHpzo79q1awrbWXvKjqYNGzaIlmAI1NPTI9TO70PvnZSUhFWrVimUtLLY7XakpKQowceQmEmvwsJChXhEz6dPn1aykD+JkqMTB0SCc+bMEWph1xCTO7/61a9UfkWk73a7tX5If7DErbu7W9/V6jIxMYHu7m7Mnz9fVBCPf6BOW1pahCp5z9euXVMUSJtnWH3gwAHRNz/84Q8BhJOE1B1nOdAWvV6vkDyjx9jYWPVsc42w3C0YDGJ8fPy23V0GORoxYsTIDDLrhExycjLWrFkjboCeNHoQKk8FZFLkC1/4gspRWKBJDmfjxo3iyug9d+/erTT+refaRo/+//jHPw4gTPaSCyISYPIgNTUVAwMDlp85yMO1okuO6EWJIC5fvixPzNdWrFihQljqlNyZzWZTMS65rObmZvG+TLowiZaRkaEyIKKhtrY2eXc+a3rfiYkJ9Pb2Wp4bGx0dxfHjx5GSkiL7INfEe3e5XELe1O3ExIQiFtojmxIyMzOFHjl5xmazKblIHbOE6tOf/rSeHRHR7t27FfFw/TAxs3v3bj2fN4PYbDZ0dHSo7IvRCxNgPT09OjDuscceAxCJRIEIT07EuWrVKuUueH71kSNHFAFyvVP3ra2t0hfLChMSEoTcWTrI5//Tn/4UwWDQHJNgxIgRI7OVWSFHu92O5ORknDt3Tl6WJTrktFauXCle4J/+6Z8AhHlG7tCf/OQnAUSmajBzBURajq5fv67SE3oeFpS73W7xCESvJ06cEErl9yJnNjo6itLSUnF0VpWYmBikpKSgs7NTbVbMghKpeTwe8bjsnw4EAipUZmaa915QUCAumJ527dq10j1ROa//yiuv4P3vfz+ACGf2y1/+UuUuzPQRFQWDQdjtdsvzuUlJSdi+fTv8fr/4W7aj/u3f/i2AMIfLIzZYAP/d735XXBhb2Ghz8fHxik4oqampQprRZ40D4dmNzFwz8vnRj34kjo5HjLKd8NixY5pZaHVJSEjAmjVr8Nprr6m6hOU33As2bNiAL3/5ywAihfFA5DgT8uBE7T/84Q/VnMA1fezYMSFxTt1iZcG6deuka6LDe++9V/9mMTi/z6pVq3Dq1KnbHn8767MDQqEQli9fri9C4/n0pz8NIFxOwgQLh0GsX78eX//61wFEeiFZCvS2t71N53IQ+k5MTIjY5gbLspZHHnlE1fcMQV966SWRvDRAwvWamhqUlJRoQVtdXC6XOjVIIrPcJBAI4Mc//jGASELB7/drsCgTJgwvampqpEe+PycnRwZIwzh+/DiAcOkKQ3OGPl/72tc08JXJNBpkeXn5bQ9Ft4owfOrs7JS9sp6QJT2HDx+W02DioKSkRBsUQ2GGbFeuXFEoyI6tZcuWKawkaOD5Mu3t7arNiz7Dnc+a1yCIWLNmjcZsWV18Ph9qamqwZs0a6Y57AAHPwMCAwmNSCI8++qic94EDBwBEeti3b9+Ob33rWwAiZ1RzYwMiCTE6qPr6epX3UJepqalK3Hz7298GEAFxZ8+exYYNG/S5M4kJq40YMWJkBrHdDlZOe7PN1gug/Y37Om+oFIZCIcuOkDG6fePkTa5bwOj3jZTfqttZbY5GjBgx8v+KmLDaiBEjRmYQszkaMWLEyAxiNkcjRowYmUHM5mjEiBEjM4jZHI0YMWJkBjGboxEjRozMIGZzNGLEiJEZZLbnVocyMzMxOTmp9ihO8uXcQIfDoUNr2LLncDjUqsb2IPbpxsfH632co2e323VdtqexVZCfE/1+m8027VAqyujoKILBIEZGRuDz+Sw7msftdodSU1ORkJCgFjLqhfdkt9ulD/ZRx8XFqcWNLYLUtc/nU085+7Tdbrd0xWfInwkJCbo+rzE5Oalnxek7nGjj9/vhdrvR29uLkZERy+o2Pj4+lJycDLfbrelMt95nIBDQHEC+lpWVpbmi1C37yCcnJzWtiLbpdDo1vefWIz/Hxsakt+hW1luPu6XdO51OvXbt2rU+KxeBx8fHh1JSUmC323WPt+rL4XDo3/w5MjIi26aNsbU1JSVFzypaJ7RnTojiM5g/f76mevGZJSYmqsWT+qVOJycn4ff7MTg4CK/XO6PtzmpzzMzMxBe/+MUpB8NzCATPlElLS9PC5YltKSkpGmTAM5N5StvChQvVM8l+zPj4eDWY84Q89rQODg5qcW/YsAFA2Dg5eoo91pTjx49jdHQU//Vf/zWbW/1vl9TUVDzyyCNYvny5jjTgAAmOmo+Li1O/LfuAy8rK1OdOx8PFd+HCBQ2qYA/p2rVrNQLt1jNhli9frmG3NO6BgQGdLcxnwr+7fPky1q1bh7/8y7/8Q6riDy7Jycl48MEHsWHDhmmHwtOxXL9+XfZEm/vIRz6ihUndcmTZyMgI/uEf/gFApF83Ly9PJztyADAXakNDg/r9uS6iz1XeuHEjgIiOs7KytKAfe+wxS3efpKSk4L3vfS+SkpKmnGkORNZtSkrKtP70Y8eOaSAKz9rhQJA9e/Zo4+Qek5eXh/LycgCRc2I4e2H//v14/PHHAUSc28aNG3VcBQe18JrDw8O4dOkSvv/97//W+5rV5jg5OYmhoSHMnTtXnpE7OQc+3Lx5U96Dv3O5XDIgTnjhTb7zne/UWRF8f3Z2tnZ8TqE+ceIEgPDC5GLl3zmdTjWbc+Pk342MjODOO++UMq0qExMT6Ovrw9DQkAZOcGFx0EN6erp0xNcaGxvlMflMqNvMzEzpkQMAYmJipCv+Hf//6tWrmoLEzdFms2mz4AbBZ7lnzx5cuHDB8kM9YmJikJ2djY6ODk2N4QbP2ZebN28WYv/c5z4HIKxvLmQ6LDqqq1evasoMhyDs2bMH3/jGNwCEZ4wCkeEhL774omYLnjp1CkB4GhKfC+2V08KHh4eFoqwubrcbGzZswOuvv66hGpw8xHuuq6vTUA7K+Pi4phDRwb7zne8EAHz1q1/VcBoCnm9/+9s6LYDOh8M8Xn31Vdknp1q1tLRg+/btACJTpng09MKFC5GXl/eHO2CLC7itrU0X5Y5MT+zz+YRM+J6lS5dqhBF/R6TZ2NioxUnIe+7cOS3+z3/+8wAiY7VCoZDQJz2EzWaT0XNjJppKTU3FkSNHpDgri91ux7Vr17S50ctF64f3yZHwBQUFGphKNM/JKE6nU8NDOfGoublZYTQdCYeKOp1OGRERjdPpFFritB/+3f79++F2u7WRWlXi4uJQUVGhyU4AdBgT9djS0qKFSv0Q6QCRYzropJctW6axehy4eurUKf3u3//93wFEJvDcd9992mg5CHbFihVy8JzOw2e5ffv2N83IstHRUbzyyiuIj4+XzngftI3y8nJN04pG1TwJk2E194CUlBQ5XTqtJUuWKLJ59NFHAUTC8Ly8PEVJHEvX3d2toc1cG9Sp2+3GxYsXbzuo2SRkjBgxYmQGmRVyjIuLQ3l5OZqbm+UF6fEYFuTk5Cj0IApZunSpkCX5FnqFQCAgzpFcZSgUUrhDHoFSUFCgQ4jIyR04cEDXI5znSPp9+/bh3LlzQgNWFZfLhcLCQpSWlmrGIFENkXhZWZm8KHXs8XjkFcmPkciuqKjQa0Qoly5d0uBPXpfh+OLFi/V8iPSLi4unoVV626ysLCxatEhH8FpVHA4HkpKSsHHjRumWuiIttHbtWg27pS3V1taKOyPaI/r87Gc/q6QAebPm5maF67RN2nFWVpaSB4xuKisrheyZ1OHPo0ePigO1uthsNsTGxmLevHkKczlHkVFeWVmZ7p+RSHZ2tiJIznSlLa5bt066J1qcP3++ZsSSe+c57rGxsRpSHH1mNiMmJt5IcwwNDaG4uHhaAjdaDHI0YsSIkRlkVnDK5/OhsbERixcvFqd3aynC4sWLlUl9+umn9XecVk3PSGJ1fHxcHoVIs7W1VTs6ERJRZXV1tXgNept58+Yp8UBuiLydw+FAZmam5ZFjIBBAb28vrly5Ii9H1EfOsbq6Wgkt8q0tLS3KdJJrJHqOiYnR+4mQsrOzNdGbGX4+y66uLn029X/u3DnxZuSPKisrAYTH3VdWVsoGrCo8vMzn88nGmGAhZ+73+2XDRCVLly7VPdMOibobGhr0XGhz69atk50TsfCaIyMj4sKi0RSfIxEmo6jJyUmhXKsLUaPX65UtEoUzIRMTEyM9kY8sLy+X7RGZU7/d3d1CgnwGeXl5+lsidCYVh4eHpUvqvq+vT3sEnzuTPPxOtzt4z9pWbcSIESP/QzIrOBUbG4uCggK8/PLLSsszi0xk1tXVhbe+9a0AoPecPn1avAy5SmZIm5qa9BozpHPnzpVnZ5aZKf7k5GTxYuQrHnzwQXzve98DEKmVJK9TU1ODiYmJKcXjVhSec7Jz506VG9ATEvUlJycLNdNjbtmyRbonumGtXHl5ufhC3n9sbKwyhNQ7y066u7vFxTELODIyovex0oCo0mazoaamRn9jVQkGgyr4JcrlPbGU5ujRo9IfM849PT3KbtJuqdvOzk69Rt0ODAyIl2UxONF8eXm5nhPtd2JiQmcq8YC46AO8yI9ZXRwOB1JTU5GYmKj75RG/5LrT09N1sBbPRHrmmWfwkY98BEBEX6yI2LNnj/RLtJeamqoIkXrmMdA/+MEPlP/gZ1+9elVlQCy34vPevHkznn322dva7qw2RyZPEhISFB4zhOYmtmrVKn0RhnRAZOPj+7i47777bpHX0WEEQzmGxzRqdsVEv6e7uxvve9/7AESIcBK1d955J1pbWy1/bjUQJo09Ho8MJfrsbSC8WTIk5sLx+XwKYbiZMpSZnJwUMU6Sev369dowGbazFCUmJkY1lVzwOTk5+OUvfwkA+PCHPwwgciDXvffei7a2Njkiq4rD4YDb7cbAwIDq3rj58/S/np4eOXqGcd3d3SoPufUc5q6uLtXOMnTu7OxUEoHnrDNR+fjjj2vz5cbc1dWlDZN0Bu03KytrSmLByjI2NoaGhgakpKTIsXD9RSe0aKc8YOuxxx6TbfNcdVJxd911lxpFmIRMTU3VqYMEYCz5S0hIEM3BULqkpETXp85J4dlsNixcuPC2p5KasNqIESNGZpBZIcfx8XF0dnZi9+7dKncgec0C2kAgMA2q7t27V2iSrxG19Pf3C9VxF8/NzVVSgWl8Eqof/OAH8cILLwCIIKrVq1cL/dAD04t4PB64XC7LI0ebzQan04nTp09LD/xJlJ2RkaF/E634/X6FDvSiREXZ2dkiwVlk29TUJITJkhyixPHxcXlpevzly5ervILHwvKz6+vrp/R2W1X8fj/a29uxZMkS2RHRLu+3oKBAdAHvt6enR79jpES7f/DBB/GTn/wEQCQkvHHjho60ZfKFR48WFhZOKckCIqE6EHnG7IBqbGxUyYvVxW63IzExEfPmzZN+qEMi45s3bwqZM9QeHByUnmifDz30kK5J6o1r2uFw6H2kIfgc3W63IlUW6peXl6sbifsTw+ojR46gqKjIJGSMGDFiZLYy6yLwiooKHD9+XN6Ph6OTND1+/Lg8JAuJq6qq5CXpidn/ODY2JhKaPdO//vWvRcySqyRfc+XKFb2PXFh+fr4KcvmTXE8oFEJaWprlS3lsNhtiYmKQk5Mj5McCYaK/iYkJ6YH9zoODgyp/oB7JOdbX16tUYvfu3QDCZRTUDREgUXxPT48SX/ToRARAhLckYt+5cydSU1PFE1lVWGrS2dkpPpH3wIjjwIEDQpV333233sPf3X///QAi6LmsrEylJrTt3/zmN0KYjKj4nv7+fvGWRD9er1doikgrehITUa3VhVFPc3OzEoYskXr55ZcBhPVAlMdSpvvuu08lTF//+tcBQJzwjh07ZPeLFy8GEH6O5CvJUfLvu7q6ZOvcd5qbmzXQgokc7lPXrl1DU1PTbdsHZ71j2Gw2DA8PK4RjiMtQbeHChdqgGK7l5eXhueeeAxCpbyQs7u7u1g0zhF67dq3CERoSM1DPPPMMVq1aBSAcYgPAz3/+cymQyQImci5fvgyn02n5jCq7DHp7ewX16SBocBcvXlRtImmHZcuW4dChQwCgZE109prZexrpunXrZCgcIMFrbdy4URUAfJ4XL15Ulnrfvn0AIjqOi4vDtWvXLD94wufzoaamBuXl5dOGnFCCwaCcCx14U1OTEoHR4RuvSZtk+FdYWKhNkc+Cm7Hb7VafLzfcxsZGrRUmfqJDb1JRVhe73Y64uDj09vZq46PeSA24XC4lQ37+858DCG92tB2+j3bX3t4uR8PnEhcXpw2TWX6G0B/5yEfw2c9+FkAkUfvaa68pJOeewbWxdetWHDp0CLc7mtqE1UaMGDEyg8wKOXL+XGJioiA/K+K5u2/dulVek96ju7tbyI6hGd+zePFiocp3vOMdAMLhIJEpf7JzYMOGDYLSJFujEQE9b3SdHtGnlSUUCmF8fBxpaWnyfETlDKWXLVsm1MawIj8/X6ECve66desAhOu8OB+P+vf5fLo+URD70CsqKlQ/xpKesrIyvZ9oiyF3U1MTfD6f5ZGj3W6H2+1GIBBQpxWRIHVWVFSk++Nr0eiFIRuR9aVLl4QEaZtXrlxRdwbRNsP3Bx54QHV4vFZSUpLWDcM7/v2rr756275fK8n4+DiuXbuGYDCo7imuP1I8oVBIdkpaJyMjQ/XJ7COPHtRMvTLUnjdvniig6JInAPjVr36lz2Qic8+ePYqY3v3ud+szgTBq/V3TpAxyNGLEiJEZ5PdCjhUVFdp1uYOTJ+vv71cZCNGlw+GQx6YXIO81PDysgmZ61NHRUaEaJhvoMdrb29WlQP5yYGBACPPWLpHS0lKkp6ffdqilFcThcCAlJUW8TLQQrXi9XqEJeuicnBx1XJA7ZH9veXm5ivTJPf7mN78RMiKaJNo+efKkXiMqGhwcRHV1NYAIX8NnwYSC1ZEjO7v6+vqmIUbyhVVVVUI2vPe1a9eK440eaAuEy1HIm5N77O3tnaY/Pq+xsTEhGiLxzs5OFSdzrTAC8ng8QjlvBgmFQhgcHNR+wFI62vPWrVvVPcPkU05OjnRCG2cSMi8vT/wto6Xs7GxxsywoZyG+3W7X86OdJiUliScnt8vrNzc3Y8eOHeLrZxKDHI0YMWJkBpkVcvT7/Sq0JD9D1EckOTk5qRIRclMbNmzQPMcf/OAHACLp9urqanFkzJ7GxMQo000EyDluq1atmtKXDYQnrPDsDnIY5DTr6upQWVkpZGlVYW919Dkut/Y5L1q0SAXH/Nne3i6einwuUXxxcbE4LcrevXuVUaQnJ6p3u92azUhkmpGRoWfOshfqdtGiRYiJiRFnbFUJBAK4ceMGqqur1ZJG22SWdPPmzULc0a2wRIC8d6K5rq4uRTVE7qWlpWrNpN3SLisqKlSBEV3mQyR0K/qurKy0fOMCJRQKIRAIIBAIKCvMc18Yubz00kvKNN9xxx0Aph55wogvemI3kSBbBoPBoDhMonXa8AMPPCD+kUdV3HXXXUKdnPxOLr20tBQ1NTW3bWCY1ebIszgyMjJEqtIISOQvXbpU4QaN5+jRo9r4mEyhMcTGxqqWkYZ14MABDbRlvzVfi4+PlyJZI1ZRUaEwiSU8/LyUlBSUlpbetofSCuJ2u7F+/XrU1dXpHkg90AHt379fxD3DhcrKymkDarmA6+rq9LfUz8GDB/VcaBgMx3Nzc+W8uDAzMzN1rgm7OLig9+/fj9HRUcsP9WCZVHx8vBYtHTh1dePGDTkhhr3r1q2TI+b7CAoyMzNl81zEPT09qmvkgW5cnBs3blStKe322Wef1ebMvyPF8eqrr1q+Zz1agsEgQqGQ9gMCHiYVP/7xj2vgCYfKLF26FE8++SQA4NOf/jQA6D0rVqxQQpd7xsDAgEJy0hvca772ta8p6cJ++UAgICfIPYb7QENDA8rLy82wWyNGjBiZrcy6t7qtrQ39/f3azY8cOQIgsrtfu3ZNRa48SGf9+vUaX8bQgl0Fq1evFuH/iU98AgCmjJYiUiK6KS8v1yh2lknYbDYlc24dntnb24usrCzLHwIFhCmJ4eFhoXLqjJTEmjVrhHhIOhcVFQkVEmmSFN+3b59KfhiiMzEDRMomiF4cDgc+8IEPAIigm4yMDHld6pDHXWZmZiIpKcnyCGdychKDg4MoKSkR6mWIRyRy/PhxIWkm77q7u5VYIEqkXQ0NDQlNMrrJyspSCMjPYRSQnJws1MrfhUIh2TcjI372nDlzLE8FUTgSbtmyZbIVTsYhRXHq1Cnpl1Fd9Fnh7LIiMl+yZImSVNxPNm/erNIqJmIYJt95550K4dltMzk5qe9DPbPH2uVyYWhoyPRWGzFixMhsZVbIkbxYfX290Ak9JNFHUlKS0vfc1V988UWR/kRD9MA3btyQt4wmXjnwlX8XPUyXfGX0IelErkRI5I1SUlJQW1tr+fbBsbExNDc3IzExUTwKEwTk+kZGRtTnTBRnt9uncV88AvPJJ5/UWHmiuzVr1qgvmzMw6VV9Pp9QKEuu4uPjlcAhCc4SoObmZqxevdryxcossC8oKBAiJgIhggQwDQlu27ZNNkzdMiFw6NAhoZ7oAazkfRn58Fl4PB59JstJsrOzZbe8FpHs4OCg1pbVhQg4Li5Oa5I6Ifc4MjKiRB7nKni9XuUP+LsDBw4ACCNtrnmW+cTFxSn5R5ujnb73ve/Fr3/9awCRYcVtbW1KEDFZw4i1rKwM//zP//yH661mJbzD4ZiSDQYim1dHR4cWD+Fza2ursqsMEdm1smjRImVbCbuDwaA2CIaYfC0zM1PEOa9pt9u1wfJ3VOjg4KAyjlaWyclJDA0NIS0tTbVvXMgcWNvS0qIFzMW0fv16DYsguc3NMiUlRVOXmXEOBoOiKFjDx42zoqJi2jkqV65ckYPis+AmmZqaisuXLyvssarwvPUbN25MGRwMRLKYTz31lOyEm1h5efm0sJfZ1NTUVDlg6pvPAYjolLSG2+3WgNaf/exnAMIbBsNIzhegHfP7vhkkPj4eK1euRHt7u2gtbo68/4yMDOnpm9/8JoDw2dNMzkRP+waAn/zkJ3o2DIX7+vpke+yeoeMuKCjQZ3H9jIyMCMRxUA4rOsbHx5GVlXXbgTQmrDZixIiRGWRWyNHpdCInJwd1dXUKG7gjM6lis9mEeBjKPvzww+qDpudl+HDp0qUpo875d4TN7N3m+xcsWKD3EyqXlJTIQ9yKYtasWQOPx2P5DpmJiQn09/ejsLBQHpDompTFW97yFoXYRD7z5s1TmEwUwtKe1NRUdRIQzRcUFKj7iGQ0n4nf759y0hsQ7uLg+0mJsDyiuLgYZ86csXwpD+02JydH3VvUbfT4MKI3Isi5c+eqg4KImlGLzWaTnok0L168KN1QVyylGhgYUMhIVEUqBYjMECBSnTNnzpsi4gHCyHHJkiVIT0+XflnLyPW4YMECJaIY9vb19akkhyVApCHe8Y53COUx2rzjjjtUVkYhPbd27Vo9I6L8t771rQq1iTj5/Kqrq3H27Fmto5nEIEcjRowYmUFmhRzHxsbQ1NSE8fFxcY0sqiSK8/v9SsETcVy4cEEcDDsNWBl/8OBBeUiiu4GBAXkcchiUgwcPirvga9HEK38yaTM0NITh4WHLl/IkJSVh8+bNKCoqEnqgl+N3/+Uvf4l77rlnymsslwIiySvyrpOTk3ou9JC1tbU6/OhP//RPAUS8dvQ5ykSo69evF1/J58Qi28TERPh8Pn2uVSUUCiEUCiE+Pl5ImjrmPW3cuFEIhaR9XV2deqrZ6cKOo4yMDKFO9kMXFRUJyZAjZ4mK2+1W6RTLW/Lz8xUh8RkzEvP5fOJ4rS5+vx8tLS2oq6tTqRjXL6Oenp4eceIs2XvkkUeU8Pr+978PIGKLsbGx2j/IQx47dkzrmrwikWYgEBDnSx7yhz/8oSIm2jATP7GxsSgsLBTqn0kMcjRixIiRGWRWyJH9vxUVFZpywR2ZxbIDAwNCK0SLgUAABw8eBBCZav3v//7vAMKp/uhUPRBGl8wE3vra6OioPove3263a1Ye30eP3N7ejri4OMv3qfLs387OTiFicnlE2y6XS33X5K3WrFkjL83sMzOAlZWVQpHUy82bN1Uiwow0keDLL78sPTFTWlNTI76G140u6B8cHLR8mdT4+DiuXr2K0tJSoUKiMiKX6IPeiN4aGxvFvbKomXrv6+uT/VH/VVVVOk6BpWj8u49+9KNqpSM31tTUJFTEAmZy8+vWrZvWF29VCQQCuH79Onbu3Ilf/OIXACJHp7K6Yv78+UKFtEkgEuHxebAK5tq1a1r7zCekpqZKr4wQGeHExsYKrbMF813vepeeF3nI6Cnvvb29t620mNXmmJCQgBUrVsDn8ymkYJ0SbyQ2NlZGxgd/5MgRGRkNL3pcFtPyTPKsXLlSpTz88jwv+Ny5c+oN5oYYDAYVlvBa/H4VFRXIzc21/Dkn4+PjaG9vh8fjkf4YOtBwBgYGRD2Q1I6JiZFRMLFCPVZXV+vsa26ca9eu1fkepD/Y5dTc3Ky/pY6jHQvLHn7zm9/oc9LT0y1/Po/T6cScOXNw4MABbUJMDnAYamZm5rS6upycHG1krH2MHupB58LnExcXJ7qD4Rud0tGjRzXMmWH14OCgvg8XNJ1SVVWVrmF1cblcKC0txblz51SWx02I95Odna15CdEOiQmTW8+cCYVC0g31u27dOnzrW98CEEmwRI9EY3KLtajJycnaI0ifRCchN2/erM12JjFhtREjRozMILMeWdbe3o6cnBwR1UytE+FduXJFqINJGyJIIBKGE9EcPXoUDzzwAIBIKU9dXZ28OMN3opz4+HiFnQwPy8rKBJ+JOEm8FxcXvyl6VP1+Py5fvozx8XFV/VNHRIZXr15VmRSR8qFDh+RFiVJI/FMnQCS8Y0kUEEnqRA9mJSVCOXnypCYAMRHGZ71t2za8/PLLb4oi8Bs3bqCsrEz2Rx2Ruujo6JAeqffMzEzpkCUqREYnTpxQIopRUHZ2tigQ6pnof+7cuUKapIyCwaAoCT4Dop7c3FxFQVYXn8+H+vp6bNy4UciP90XUd+PGDd0/mxocDodKn7je+f4NGzbI3lhi9eKLLwpNMnIicrTZbOoo48+ysjKtfdIpLHuLiYlBXl7ebRO1BjkaMWLEyAwyK+TIUf7t7e1Ch5w0wpR9fX29ynRItnq9XpU4kHClRBdos29yxYoV8ij0pPT4a9asUaqeM/B+85vfyOuTG+L/d3Z2orOz0/JnAJO3iYmJ0XncHMxKkt7v92sILT2i1+vVvRHVENFs2rRJ76P3vXr1qhAJuS8W4oZCIXFx1H9aWpo+n/wyUVR7ezsyMjIszznGx8djxYoVuHTp0pTz1YHItKdvfetbikSI4jwez7T30d6vX78uxEh91NfXS8+MYPgzNTVVqIf873e+8x3pnlETn1dBQYGSDhzgbFWx2Wyw2Wy4cOGCkB9tkMisvr5etkVUfe3aNUVJt5bs5eXlqcwmuh+eZVN8Htwf+vv7Zc+MNskXAxF7Ju9bW1uLQCBw26NZZ2XVLpcLRUVFyM/PFzxlRo0b4ebNm/Ul+VpmZqZCbG6qNII5c+ZowCpDGJ5VA0TIW3Z/xMXFCbIze7po0SKFnrduFB6PByUlJZavxQPCRubxePDwww8DmE5LuN1udSLxITNBBkQcCN9fX1+vwQbcwFasWKFaRhpu9ObL7CJrzNLT07VImajgs7t8+fL/X6e4/U+Ly+VCcXExUlJSRAGwkoKJkLvuukt6Iz1RVFQkG2b4Hd21wkUenYxkKMgEA5/X5cuX8YUvfAFAZCNIS0vToqUzp523trZOoaOsLLGxsSguLkZDQ4MGb9CmHnroIQBhmoshMO2vpaUFTzzxBIDIWmY9aWpqqjLffC0zMxObN28GEEl4/eu//iuAcEKGg4n5/pKSErz//e8HEBmVyEEj165dQ05Ozm0pNxNWGzFixMgMMivk6PP5UFtbi6ysLEFWpsYJb9PS0jSSieFhUVGRQhbW6UWfYMdODYbeJ0+e1Ej5W8/1HRoaUhjO8MTlcslTM4zh5/T29mJ4eNjyYTXrHAOBgJJdDNFIWRw/fhxve9vbAESQTHx8vFAz+9DZk/q+971PyCg6lCFyiQ7hgHDpA707keb4+LiIdCbOGNo0NTW9KcJqniFz7tw5IT+WehFtFxcXq36W4Z/H45FdR3dcAWEK4o/+6I8ARGxtaGhI64I2yv9fuXLllJIsIJzAYSTFJEL0qDi+z+oSDAYxMjKCsbExlXlxog71HAgEhNaZHHnuuedkgxyyzETLhQsXVN8YPe2IyTL2tfOaX/rSl/TcvvSlL+kzWcrD9c+1snr1atTU1Nx2XzDI0YgRI0ZmkFm5/FAohMnJSeTk5IhLufWgm8uXL6saPZqfYXExey9Jmvr9fiER8ghVVVXicVg0zp+JiYn6THrbQCAgZEmOktNQGhoa3hSFyg6HA8nJyXj55ZelI+qF3rGxsVEJGaK9pqYmFdizjILlIz//+c91LXLEoVBIuuVr5HerqqrU/UI9Dg4OSpd8ruxy6u/vh8/ns3yplM/nw8WLF7FlyxahEfLiRBu9vb26Z/K0CQkJ6urg74jiGxsbceLECQARXrG4uFgInVwi/z8uLg4f/OAHAUT03t7ernXDKTZMKGZnZ2uNWF1iY2Mxb9489Pb2ah0yOmHkd+7cOXGCRHZtbW2KIJnsY/eMz+fTmieCzsvLUzcXhbMegUg09eUvfxlA+JiQrVu3Aoh00rzrXe8CADzzzDMoKiq67REfBjkaMWLEyAwyKzjFicqHDx8Wl8KeaaK53Nxc8TTM+h05ckQZpPvuuw9ApCSisrJSuzqRyZw5c8Q9EMnQOzc0NKh4OfpIV3oqcma8ZkFBwW1HoVtFxsfH0dnZie3btwvV8CcR9YoVK6Rb6iD6+EneO3ksn88nriy6H5qZUXK2RKE2m02cDvnOjo4OISSWT9C7b9myBb/85S+nnblsNXE6ncjOzsbhw4eFFKkPlslENxJQB0CEM6Q9UWcLFy6UDbNVLhgMytY4LYaZ/M997nO6FgvFL126JGRFTp7IqLi4WJGA1WViYkIImdUA1GW0bTCTz4zxnj17ZM9Eh9G91ix9Yga/ublZCJNHqtDmP/CBDwiZ8uz18+fP45FHHgEAzeVk5JCbm4u5c+feds7rrDbHuLg4zJ8/f8p5GAyp+MCvX7+uTSs6sXD33XcDiGyAvOGCggIt9NraWgDA/fffL2VRCdHKZ30Zuz56enoEr2n8/Jzc3FycPHnytod3W0F4zsnQ0JCcChcYHcuyZcsUarNetKOjQwbDzgOWLRQUFOi5MNS+++67pWcaIp+Xw+GQgXEDraio0GbBMh/2o167du22dWJWEdbnbtq0SQlE2hxLvhYuXCgnwK6v5cuXK3lA+42+JstVqJ/R0VFdnyUt3Pza2tr0HKm/oaEh3HvvvQAiiQLa+fj4uOUHNFN4Kqnf71dSkIkrJmWfeOIJ1Zhy4ywoKFCpD9crEy05OTnSJSmhpKQk7Tuf/OQnAUTCd4fDgaeeegoANPzjl7/8pWqimWwjbZGXl2dGlhkxYsTI7yO22Xh+m83WC6D9jfs6b6gUhkKhrN/9tv8ZMbp94+RNrlvA6PeNlN+q21ltjkaMGDHy/4qYsNqIESNGZhCzORoxYsTIDGI2RyNGjBiZQczmaMSIESMziNkcjRgxYmQGMZujESNGjMwgZnM0YsSIkRlkVu2DiYmJoYyMDLhcrmmTt9nqExsbqwk8/On3+zUVh7/jNAy73a42tuhDj9jWReHnjY+Pq8WK7VVjY2NqY2T/Kntck5KS4Pf70d/fj9HRUcseXh0fHx9KSkpCQkKC+lM5vTz68CHqjxNIAEw7OpW6CoVCugZ1PDExoX5W6pvPInoeIa/hcrl0/Vv1HgqFEBsbi5s3b1petykpKQgGg9IRe545b9Dn86lHmq+NjY3p39Q73z88PCxdcbJ19Pso0dOgaKPUd3TPP98XfT47r9Xd3d1n5SJwt9sdYgsl7Yw2Q51G/456sNlsum/+jjbpcrnUgsj21snJyWn7SLQO+ayoX4fDoevyNf4919jw8DB8Pt+MtjurzTE9PR2f/vSnERMTIyVw5BK/ZEtLiwaycpBofX29BqVybDnlG9/4Bv78z/8cQOTEwFAopHOqqSwqdmJiQn2Yzz//PICwId26SVNBGzduRF1dHb75zW/O5lb/2yU1NRV//Md/jOLi4mlnJXMIh9vt1pEIbLgfGBiQAfAZ8ATBq1evqqeaZ/rm5+drk+OwAx6J0NraqoVO/be2tmp4MX9yCEN9fT1KSkrwN3/zN384RbwBkp2djc9//vNIS0vTeSQcPMujCCYnJ7WA+NqCBQs0LJVjxtgfnZWVpWfBtZCSkqLeYA7z4AJPS0tTPzd74a9fv65BFhzPxXkBqampGijy7W9/29LdJ7m5ufja176Gzs5O9aJz1BvXb/Spl7xXv9+vGQscYMPxe0uXLpWN88yj7u5urWsODuH7W1pa5LT5WfPnz9dsAQ5v4fMOBoPo6enBF7/4xd96X7M+Q6awsBCDg4MyAhoUF9Wjjz6qL/TYY48BAJ566ik14P/4xz+ecgMtLS2a+s2N9uGHH9aX5uLmwv/6178u5bL5/KmnntIsPnoUKu3IkSNYu3btbRvMrSCTk5MYGRnB5OSkviuHSxCtRC9gDtg4evSoBh9wWAdRX2lpqRYkN7u2tjYtWM7b5MY5d+5cfRaf7+rVqzVZnJsGDd7tdqOxsdHyU4+cTidycnIwMDCgYR6cDBM9uZsbE3Xr8/m0mHjP3MyGh4fliNetWwcgbL8czkE75CYZPXOUMwzPnTunCUfchDk3sqenR7ZvdRkdHcXJkyexYsUKTQLnJkZbXL9+vVAb12p/f78GR/C+6dhPnTqluaHU18TEhBw6h9rwOa5evVp2zY3Q7/frevw+3Ic6OztRXFysZziTGM7RiBEjRmaQWSHHwcFBvPDCCyguLpa35Hgshn6pqal461vfCgD44Q9/CAB46aWXNMropZdeAhCZqExPAAB/8Rd/ASDsDciVcUIyZ9slJSXhySefBAA8+OCDAMJnTHCGG7lKhiSxsbFITk6+7cRfK0goFEIgEMC8efMUJhPFEbXExcXpPsiZ5OfnK1S8dVyUw+EQKqQHXbx4sbw5Z9uR5zp79qwQN6dQB4NBTR8nTcLv4/V6sX37dvz617/+Q6riDy4jIyN49dVX4XA4NKeSdkKKYHJyUkiatM3g4KDQO9E8x8L5/X7RQESERKdA5DwkIsnS0lK9j89r3bp1ok747HgezcTEhEZ8WV2CwSD8fj9cLte0yfNE3n19fYooidYKCgqkV653Po+KigpUVVUBgOwvmj7jrEdymtEzJYnuMzMz9Vkc00cUWlRUhIyMDDMJ3IgRI0ZmK7NCjklJSdi0aRNu3rypwaBEOfQQfr9fZ7uQZN27d69I/3e/+91TXluzZo1QEEncz3zmMzqR8L/+678ARBI59fX1mpZMT3T16lUlfIiGSJynpKRgYmLC8kNZeTD6888/Lx6KXCLvd2RkRBPRea4LyWogkkUmOjp69CjuuusuAOEzM4AwYidSJDLlsxkeHlYWnB53//79eMtb3gIggvKJiuLi4t4UurXb7YiLi0NFRYWQB4cGR2dZiZaJZrKyssSPkbsl5xsIBMR909aKi4tlw+TX+CxSUlKUuCHiXLFihbhgIhtGYGlpaeJ6rS6xsbHIz89HTU2NEDBti1Pp7Xa79Mu9IikpSbZD7pFI0Ol06hrcY1avXq2TMJmsufPOOwGE0eKt1RopKSlKIt567nhBQYESar9NDHI0YsSIkRlkVsjR6/Xi/PnzWLZsmc6DIB8QfV7tX//1XwMA3v72twMANm/erLM6yMEwa1RTU6Mzbv/lX/4FQBit0HvTkxAh7du3T5lU8mkchQ5APA09fXp6Oux2+22zUlYQu92OxMRETExMYPPmzQAi5+YQjQwODqoMhDxfXl6e7o3PgMgwOztbf0uONzU1Vd6c3Bqv+corr4ivJMpxu91Ciiy7YJ1kf3//myJbzXq6tra2aWiBVRYVFRV6jfc7f/586YMcF49GSElJUaabmen6+npxbrRzcpA3b95U+Qmv6fV6hWT5O3JulZWVyrpaXVwuF0pKSuBwOBQF0ib4/z6fT2ua3GB/f7+iHf7kPXd1dcnOGEldv35dGXzaPHMLk5OT+lvyiCMjI3qWjLSIXtva2jAwMHBb9DirzZHE69mzZ2VU/MmShRdeeEFQlkmAo0ePYt++fQAiD/+9730vgHD4wI2PYcqhQ4dEmFMxhN0HDhzQwVO8vt/vx89+9jMAkVo8Gml3dzcOHjwo47OquFwuHe3JsIBhLEPAefPmKTzgBhcKhVTWwJCG77l48aI2Si7CrKwsUSIsWeECLiwsVJhJPdbV1Sn5xmQBN4VNmzahpaXF8o7H6XQiLy8PBQUFsjXaUPRioUPdtGkTgHDJCZ9B9IFtQJjS4RG33ABSUlIUtvEZsjwoMTFx2lk8NptNmwgLnbme6uvrdZaN1cXn86Gurg5ABJTQfmgrly5dknOlHmpqarSGGUIfPnxY16V9EiRs2rRJB2WRjuDm1tbWpvfzGQeDQX3W6dOnAUTWQUxMzJQi8ZnEhNVGjBgxMoPMCjk6HA4kJSWhoqJC4e43vvENAMAf/dEfAQh7aSId7tJOp1MlCzzCleiloKBA3punkq1evVrFukzuMHSemJjQSWIsLdm1a5c6YFhMy+v7fD5kZGRMaeOyooyNjaGxsRE7d+7UPTC85hGgN2/eVGEsyxUmJibkHVkiQhRSVlY2rVXt6tWrIv8ZylDXsbGxU4qWgXCJBU914/uJRvPz8+F2u6e1zFlNeLJjMBgUSiby5kmAc+fO1YmLDJ0nJyelGyLH6BI0IkbSE1euXNFz4TG2RPE9PT3SbfSzY+h465GuSUlJSvRYXRj1XL16VaV3TDox2edyubReo6M6NhZQXwyN4+LitA5Y5vP8888rmcPfcW0sW7ZsWmlae3u7UCSjHz7/K1euYPXq1fjFL37xW+/L2lZtxIgRI/9D8nshx9jYWHEJPM+XpSLZ2dn4kz/5EwCRM2u3b98ugp9kLM/1XblyJb70pS8BiBD+27ZtU1KHvZrkLM+fP4/vfve7AKAylS9+8YsqsWB5D4uZa2trsW/fPjzxxBOzudX/dklOTsadd96JxsZGlX+QX2FrVTTPRa4lLi5O/MutTf+xsbHSC7kwm80mz01UTnTjdDpVeMti5+7ubnl1JiXomaurqzE5OTnl4HYryvj4ONrb25GWlibkt3//fgCRBF5vb690S/0tWbJEaIeomWjxypUrinioz87OTiEbok/a9OXLl4VSWYg8NjamdcRyLfK64+PjOtPa6uLz+XDx4kWsX79e556vWrUKQCRZ1dTUJP6WvKLD4RBPyJIeIsH169frLHruGTdu3NBzYKsx9wdg6pnXQJhf5z5AZE40unLlSni93ttyjrPurS4pKcHFixcVyhHe8iBtl8ulG2Vo5vF4tJhpjDSKr371q+p0oYH8+Mc/xh//8R8DgOqaGHJnZWWJ7CWM/uhHPyrFcHOkTE5OYmhoSJuEVcXn86GhoQETExOiI+hAmJVfv3699EhSv6urS//m35Hc7uzs1MPnonvooYcUSpDiYHdRKBTScyUJvmPHDj0DJn6YMVyzZg3cbrf+3qpit9uRkJCA1NRU1S1ykyQFkZaWps2RWdW2tjZVUtDWmP3My8uT02KVQGZmJo4cOaLPBCKJxIKCginhNxDOoPK63Gi5WRQXF1t+HgDFZrMhNjYW58+fn1LDCEScxIIFC0QTMHTu6+vTfTNBxs2stbVVtYx0vsXFxQrXeX06q4KCgmkJxs7OTlEYGzZsAABRJy+++CJWrFihPWomMWG1ESNGjMwgs0KONpsNTqcTXq9XOzxRIhHK66+/rpIcwtv+/v5ps9wef/xxAGGPQo/NEH3z5s3a6elZGMpt2rRJ3oilK9HlQEQ89E6Tk5O4cOHC76yG/5+W+Ph4LFq0CE1NTSqHiL4HIIxGjh8/DiBSUnL+/HkhEUp05ws9K7tcGhsb8cgjjwCIlDd86lOfAhAuk2LYQWT11FNPyduyCym6++NLX/qS5cuk4uPjsXjxYgSDQSUSSUUQZdTU1Mimidj7+/sVsTAhQxRDBApEUOKCBQu0HrgGqKuamhr97T333KNr8rpETtT15cuXRYVYXVwuF0pLS5X4AiJhL5HkxYsXFQJHT/QiAmStLROvS5Ys0fupw+j5pOx5p/5OnDghnXOvKSsrUykVaRSWDi1YsAApKSmmt9qIESNGZiuzLgL3er0YHBwUZ0hehFNzEhISxMWQvzp37pzKe9hT/Z73vAdAOD1PTo0c2PHjx+UZuLOTfzh27JgKw9mlc/r0aV3jvvvuAxApJdi4cSOampqmTM62othsNsTFxSEYDKqTguiEhcFnz55VR9J3vvMd/R2Ja6Jyop3169drOgr5nqVLl07j3ai79PR0vOMd7wAwlQMjCX5r7+rY2BhWr16t/ncri8PhwOnTpxVBMKoh2V9SUiJbZleR3W4XymFihWVkKSkpQkd8Pi+//LLsltwjOzOKi4uV0OJnzpkzR9EBX2MJ2rJlyywf7VC8Xi+qq6uxevVq2RQjOSaf6uvrlRShDTc0NKiUh6V+jJLS0tIUJTHHkJKSorwEEz9EqmvWrNEap522tLRoLbGpJHrOZHNzsyKrmcQgRyNGjBiZQWaFHMfHx3H9+nWUlZUJCXK35g48MDCgchCile9973vyqEznkyd73/vep5ISes8lS5bIa9Czkpvwer3KnrJgdtGiRcoKkoNg8anX68X69etx4MCB2dzqf7tMTk5iYGAAoVBIHCwRSTR3y8woOdm1a9eKh2FmjiintLRUeiYP09LSIlRI9E+kn5iYKM/PcqLq6mp9Pvkj8r82m83yE3mAMEro7OzEyMiIkB8RBHktl8slm6b+RkZGdH9EKEQ46enpeo1229/fL/TJmaZ8Junp6Zp9SiQeGxur6g3OGmD2/NKlS7qu1SU+Ph5LliwRnwdESmd4XytWrNCaJM9bXFwsfUZXtgBhOyU/THTZ0tKiygxWEUTP3owuMQPC9kzOke2NbMns7OzE5OTkbe13VptjXFwcysrKEBMTIwKZoS2NJzU1VTfDBXnvvfdKSUwGUHmPP/64xjTxBnbt2qVuD0Lk6M4DJgBYVhH9b1bCR4+zjyaKrSzBYBBJSUkKe9kNQ5L+5MmTSiTQ8Vy6dEm6pPGx++P48ePa2KjHqqoqhSlM/NCx1dXVKZnD10pLS0VpRIffQHgDPXfunEqIrCoxMTHIzs5GX1+fFh9DYtpFXl6e7pObV0ZGhhwVkzUEA0VFRdJR9Kh+OiM+Hx4Bsn//fm26lEAgIFvmxhJdA2l1KogSCARw/fp1eL1erfNbh0a0t7fLtphoqqqqkn5oQ7Sx2NhY2TWd+PLly+W8WQJFW75+/br0xXVz/fr1KTWr0RIXF6e94beJCauNGDFiZAaZFXL0+Xyora1FcnKyioqZACFR2tTUpIk7H//4xwGEd3Lu+ITU9Ng5OTny2Ax9+/r6NBmF4Q//vrq6Wq8RYicmJuJd73oXgAihTTSam5uLS5cu3ZZ4tYJMTEzg5s2b8Pl8KpmhFyX5nJGRIXRNdHP16lX1YNNLRxcP83dETPfcc8+08JtRgNfrVSTAcCQpKUmIlM+MSZ7s7Gzs3r0bv/rVr/5AWnhjhB0c4+PjCt8Y3ZA+aGtrk62RpmhtbVUXFruViOxCoZAoHF5jZGREiImlViwh2b59u3raSS0NDg7qezBSot0CkajszSB2ux1dXV3SIWmu6EPFiOJIHWzYsEGlO9QbbTcYDAoVMvTetm2bKDUmuojQR0dHRZkQOWZlZekzGX6zA2fXrl2YO3cunn766d9+T7+XJowYMWLk/+MyK+SYmJiITZs24YknnpBnYMkM+ZfKykqNd2d/5ebNm9WyRs9I5PnII48IDdF7xMXFiZdgYSdb1Lq7u1WywmLdEydOyKNEJy+AMALas2ePDvayqvj9frS3t2P37t3qSWdpEz3hoUOHVABP9LFo0SJ5Via9+P4zZ87Ic7PcZ9GiRVMOIAIi/GxKSsq0YaUZGRl6jixBYbG+3+9Hf3+/5VszJycnMTg4iL1798rWSN4T4QWDwWmHl7ndbqHqW+cuvvrqq7Jl8oVdXV1K8LAwnP3RVVVVSi4SEUYPaKVEn70cneCwsrAI/JVXXpE+qS/qKDk5WX3q0Qe6MSJkpEK9ORwORSzcCxobG6UvRi+Mfi5cuKDyqe3bt+sa3FNYVkj9Hjp0CPv27bvtLNJZbY7Dw8N48cUXMTExgQ9+8IMAIllnbljRo8FIzra1tekLMxThTQaDQW2ADP2Gh4cFl5lc4N/fuHFD49GYuNi1a5cUyU2SGdhgMIj8/HzL96nyTPAbN26ILvjRj34EIJJomZiYwN69ewFEKIiMjAwR0TQ+Lr5gMCgims/n8OHD6kagA6GhBQIB7NixA0CExhgZGVEWl8+MBHlWVhaOHj2q7LdVhaPYbDabwlfaAx3DxYsXtelzoWZmZkpvW7duBRCxaeoAiNTmpaenTxmhBUSe04YNG5R95YJ2u92q+yUtxf71Xbt2Wb7ziDI+Po6rV68iKSlJHVa0Merm1KlTSpayEsXr9QpUUV9cv/Pnz1fShbr2eDzq6iJwoH7nzJmjjZW0yOuvv67rsrKFtNuKFStQW1t721pSE1YbMWLEyAwyK+Todruxbt06HDx4UGOH2N/M3bqwsFC7NLtmurq6VBXPqTkkohMSEpTG5zXoPYHIqWxElU6nUwkcQvaGhga9b/HixQAiHn7Lli0YHx+3fD3e5OSk7pHCk9V4Xkx+fr5KUHgaY3d3N5577jkAwCc/+UkAEcpiZGRECQh2CFRWVup3RPkMNVJTU4Ws6PmBiNdliPnjH/8YQDhU8Xg8tx37ZAVxOBxITU3FyZMnRTnQ1mhL0XWLfA6nT59WCQ+RJsOzxYsXC6EzDA+FQop0WLbCurqJiQnpm0j7ypUrigqYMCDS6ujoeNOU8jidTuTm5mJ0dFSIjqV1RIYlJSW6Nz6DxsZGUUF8jSF3T0+PdE179fv9sjUiVEZSbW1timxI+ZWUlEzTOesjjx07ppD7t4lBjkaMGDEyg8wKOcbExCAjIwP33nuvvCYTIPQUra2teo1cVkZGhrhGkvpEeK2treLF6CGqq6vFFXDHp4dpbW1VvyuRjMvlwmc+8xkAEb4tOlkTfW2rCjtkRkZG1N/MziGW6sTHxwtlE3m7XC4V1pPAZ1Kgvb1daIi8YU9Pj3RLxMNuDq/Xq7IheuHXXntNySyWStC7d3d3Y926dZY/X5mlPGvXrhWfy0gjulyGpUq01YyMDPWpR5/LDoQRJ7uU2LW0Zs0a2TcTldFlPtQbI56EhAQhTXbPcF309/dbHpFTeMBWZWWlUBtnK7JbLjs7Wwkv2vDKlSvF+TE5yOfT3d0tnVOHnZ2dui4RKfWdk5OjNc6kWU1NjRA89w8OHJ43bx66u7tvm0w0yNGIESNGZpBZIcfBwUE8++yzyM/PF1dzKwcQ/dqtEzqAiGcgotuxY4d4NCKl8vJyZZ7IGRApXbp0SZwFC8oHBgb0fmZs6Z0LCwtRXV1t+Ra3hIQErF69Gu3t7Sp7IvdFpJeTkyOEzgLZlStXCvHQCxPRXL9+XSiSXNC5c+fEt5GHZNnFli1bpvFc0WiIKIt8ZEdHx5ti5mBCQgIqKyvR0dEhxMhMMEucXnzxRaE3IhWbzaaohvZNVLlu3Tpl+zmf8cqVK+ImWf7E6GbOnDlCNuTdV65cqSJx8rq04yVLlli+woLidDoxZ84cNDQ0aO3zPnhfra2tQo7MP9TV1Wk/IKJj6dOhQ4eE9lhu43A4hASjz74GwqVtXPvkbysqKrSG+DtGUrwO+faZZNa91RUVFRgaGtLC4uZFCLx7926V4bCcprW1VZsbwzve+P79+/XFKePj42oeZ3ImunSCGwRDnJycHMF3hi7RTe5ZWVmWP30wFAohEAggPj5eGxrvhSFHQ0ODjI5hxeHDh7U5slSCJU733HOPKAhuBtH9qdFHLQBhOoPGTEokutebhsRSl9raWpSVlb0pKAvSMxz0wHvmZpednS2HQ6K+v79flAEXNMtwhoeHZdPR8wV4PW6OtOPly5dr8XLtXLt2TXbKxc4wPi8vT105Vhe73Q6Xy4Xu7m6d9cSxb9y8Nm3aJEdBh5CSkiJd0MHQed19993SF533nDlztLlR93yuXq9X48nosFNTUwUYuEnyiI+uri7Mmzfvtg7IhNVGjBgxMoPMCk6NjIzg2LFjsNls8rK3Tn2JiYlRCMLwYc6cOfLG9JREkHPmzFGfJCE5kRIQ8TwsTn7iiSeEVIg+PR6PiGCiIXrilStXYmBgQCStVcXj8eDUqVOYP3+++tVZVkMEOTQ0JBqDXrWsrEy0BcuXopNfDAv5nitXrkyb+kNaY8+ePaI7eA0gUr7C0iwmDQoLC7Fs2TLL69Zms8FutyM1NVU2yXtgQbLf7xe64GmCQGQaD/vcGcFUVFRIp0T6LS0telYME2n3g4ODKgKn/jdt2qSicaIjXqu2tlaF59/73vf+IHp4o4SJ2ry8PK07dsoQ6UUfdkVU3dXVJRqHtkvK5tChQyq7YWR0+vTpaWPi2DE2ODgopM1nNHfuXO1TTJCx4WR4eBjXrl2T3mcSgxyNGDFiZAaZdRF4ZWUlxsfHxX2R9yNp+utf/1rekDzNwYMHVZ4SPcIfCBfX0mtwVuGZM2eEkMiPRZ9L/dhjjwGIHMYzd+5ceXF+Jr3H+fPn4fV6b+shrCAssG9pacFXvvIVAJFCZRLZMTExeOc73wkgUiJRUFAg3oUoh2iltLRUf8syn4mJCXlntn5Gj5XnNcgX+/1+oSYifCLOy5cv4+DBg5Zvc/P7/WhtbYXT6ZRdsA+X0cfQ0JDQJPmslJQU9WITPdPOq6urhXqI8AOBgJKE5G4ZWZ07d05RDVsFDx8+rNIURgLR02aod6uL3+/HlStX4HQ6hRiZRyByCwaD4vcYlZSVlWkuAO2O/7927Vrpl3+3a9cuPQ9Gj+QU77jjDjU/kBeeM2eOuGXqmfmKoqIi1NXV/eF6q202G1wuF/Lz80V60pAYKqSnp8tYHnroIQDhpnBWsj/zzDMApla400BpUE1NTar/4kLkz7KyMkFpLur09HRBdYbf3DyysrKQlpamjcaq4vF4UFVVhZSUFIV1XGi8z+rqai1uhm+NjY1TxmgBkdBsYmJCYSMz2Dk5OXJsJLAZSg8ODorGYIItEAjIUXGhRw8bXrNmjb6LVYWVACMjIyL8ufCYTCksLFSNJ5MEXV1dsknqg8mojo4O7Ny5E0AkibBo0SJtorwu7W5iYkKOism01NRULXLabXSS8dYz2K0qPFsqGAxqE7r1hIBgMCjbYrh88eJFgRnqNXogMPcWbmjRZ9Qw1GbWubm5WRsxqw2i6xgZ0rOKxePxICYm5rabowmrjRgxYmQGmXV9i81mQ2xs7LSKdqbI+/v7Nd2EsmXLFqEb1ttFe09CaULmgoICeRx6CiLJNWvWKCwhKvL5fEKW9DKE0xUVFWhpabH8sFse5VBYWKgwghQEkePY2JhCB+p/8eLFQu30jgyb29ra5Fmje4iJkIgIGV7PnTtXRDc/OxAIqKyHtAdDwAULFmBkZMTyI8vGxsZQX1+PdevWCVWz1IT6iY+Pl06p7+bmZoV7RDZEeosXLxZVE322OlEk0TSf17Jly6RHotDc3FzRHUx2Ucfx8fFaU1YXJmSam5sVVVCHTIRkZmbijjvuABBJvng8HtkZ1zlrnePj4/VsiEZfe+21aeE3B18PDAwI8fNZJSQkCK3S/olUBwcHcePGjduWoRnkaMSIESMzyKyQIz1wZmamCjnZ98syhcHBQaEPJkl+8YtfCE2yLIHEc09Pj/hHcjKXL19WUoHeme8fHR0VcuEI+1OnTslj09vy4J7z58//Tm7BCsIC+5iYGBHWRBjUxfj4uMpuyEtev35dxd/0nPSWXV1d8uDkFR0Oh7oQnn/+eQARXiw9PV2elNxmQkKCkGE0QgLCpPnRo0ffFD3ANpsNNptNRdnkxFiGNDAwIIRCZJ2Xl6d5geQXidLnzp0rXRHtnTp1StfgMyMyzMzMnNaI0N/fr9ISdupwzfh8PhVNW11iY2NRWFiImzdvqlSP0R1lbGxsWldLXl6eoiTOaeQUI4/HI10Q5UfPbKS+eLhZUVGR8gzkiRMSEnQNrpHow/8M52jEiBEjv4fMCjk6HA6kpaUhNTV12mE5zFAvX75c6O3d7343gPDUF3pI8lvkGIaGhsTLEK0UFRXh5z//OQCIp4ge6R/NwQFhz0UEy6JnotDCwkI4HA7L96myfXB8fFxZak7Goa4TExOnlYgkJCSIY6FuqcdAICBOhlzs2NiYSkTo3ZnBi4mJEfdFTjgQCKgMixlYfq+qqiqEQiHLz8q02+2Ij4/HmTNnhDyIvFn43dHRIYRMxN7d3S07ZRsm7aiiokI8NttlN2/erOYGlpjw71wul6YaUf9btmxReyxRKCsIjh49qrIYqwuPZp0/f74mFJH35l6QmJgou6Z9xsbGioelXTMKOXfunPhBIs2rV69K59HHL/BzmKXmFKmysjIhRnKhvD4PXLtd1DOrzTEYDGJ0dBRNTU3qs2VS5KMf/SiA8KJinSNDjO985zsa+f/4448DAL785S+Hv0DUGdhMxaekpKgHliE6S4fWrVun8JHp/IceemjKyYVAhNjt7+/HunXrLD84NBAIoKenB06nUwbFBxpdB8rFxIdKwhuIJMeos+zsbI2dZyiclZWlEhHqnbppbm7WdaPHdjGs4eYYXXeZkZFh+b51t9uNNWvWYP/+/dr4uKFxw4qNjdXGSbsKBoNaoNwUuZnevHlTi506O378uOgfhnO0abfbrZA+eiNgbSWHXkSXXlm9fpRit9sRFxeH3t5eOVw6GiZcWltbp9TOAmF7YukOHTz1vHTpUgEi1kzOnz9ftNKtZ/u89tpr+h2pjEAgILqP1B0Twt3d3UhISNDrM97X76UNI0aMGPn/uNhmExLZbLZeAG+ew3SnSmEoFLLscW5Gt2+cvMl1Cxj9vpHyW3U7q83RiBEjRv5fERNWGzFixMgMYjZHI0aMGJlBzOZoxIgRIzOI2RyNGDFiZAYxm6MRI0aMzCBmczRixIiRGcRsjkaMGDEyg5jN0YgRI0ZmELM5GjFixMgM8v8DHXynAP/+rq8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 16 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Weights with 10000 data points:\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUcAAADuCAYAAACqLcX5AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAACGz0lEQVR4nO29d5ycZdk2fEzbmZ2ZnZ3tvWRTIIUWSMEgAYKAdFBQKYICPuIrAg++Aj4+72NXVGwgCApY6EqVTgDpCAESAgkkIclutvc2uzv9+2Pe47jv2Vnjb/lEJ7/3Ov7ZZHfmnrnP+7yu8zjr5Uin0zAwMDAwyIbz3/0FDAwMDPIRZnM0MDAwmAFmczQwMDCYAWZzNDAwMJgBZnM0MDAwmAFmczQwMDCYAWZzNDAwMJgBZnM0MDAwmAFmczQwMDCYAe7ZvNjv96eLi4szb3Rn3ppIJAAALpcLAJBKpfR6/tvpdGJiYiLr9aFQCAAwNDSEuXPnAgBGR0cBAA6HA/ycjo4OAMDg4CAAIBwOY2pqCgBQWloKAEin03A6nXovAMTjcX3PVCqF4eFhTExMOGZzv/9KULb8/oAlU94LAN0nZZtOp/U6gl1PqVQKHo8HgCUXu6yi0WjW3+yfbe+c4vWTyWTW/51OJxwOBwYHBzE+Pp63sg0Gg+nS0tIsOfL+qMfxeBzBYDDrb6lUCiMjIwCse66trQWQ0Uu+l3Jxu92SLa/h9XoBAH6/X58di8X0Gq4HPk8+Az4bAOjt7e1Pp9MV/7+E8CEiEAikS0pKkE6ndd+UA+/V6XTqbz6fDwBQXFyMgoICAMD4+DivBSBz/9P1MxgMSta8Ft8fi8UQiUT0XiAjU76Oz4/ydjqdcLvdu9XdWW2OoVAIn/3sZ9HQ0KDNipscP3R0dFRK1t/fDwDYvn07Dj30UABAYWEhAOChhx7SzZ1wwgkAgIsuuggAcOONN+LTn/40AKC6uhqApZTd3d1YunQpAGCfffbRa7iZDgwMAACam5sBAMPDw4hGo/jNb34zm1v9l6O0tBRf+cpX4PP5ZEhoLKhgoVBISsdF19raqgdPpeMmUFFRgcnJSQCW8hUVFWmhtra2AgDq6+sBZGTFZ8fP8fl8etZFRUUAgLGxMQBAZWUl4vE4fvGLX/wzRfFPRygUwjnnnINQKKR7ofx6enoAACtWrJCOvPTSS3rvmWeeCcDStVWrVgEA3nnnHbzwwgsAgBtuuAEA8OMf/1iL9fOf/zwA4J577gEArFu3TtcIh8MAMov9jTfeyPoe1HOuKwC4/PLLW///yuDDBPcFO+xGhK/hWubvOjs7cfTRR2e9r7e3FwBwyimn6N+LFi3S3x9//HEAwFFHHQUAePbZZwEAW7ZswRe+8IWsa11++eXYvn07AGDvvfcGABGrZDKJYDCI3/72t3/3vma1OTqdTvj9frS2tmqTszM6ANixY4cs6uLFiwEAd911F1paWgBYm+imTZsAAG1tbTjttNMAALfddhsA4Oqrr9bm9qlPfQqAtYBvvfVWbNiwAYC1WBsaGvSZtDzcUHw+H0KhkP6er3A4HHC73Whvb0dJSQkAS1ZkyN3d3drYdu3aBSCzQQ0PDwOwNkfe+9jYmBYrrzk5OanNl6zSbrV37tyZda3m5mY9W26YfF80GsXIyIiseb7C4XDA6/Wiq6tLMqX8GhoaAAAPPPAAXnvtNQCQJ3PggQfi0UcfBQDJjIbH5/NJDjT0K1euxB//+EcAwB/+8AcAwHXXXQcAuOyyy3DIIYcAAF5++WUAmbXS1NQEwNocyZKcTqeM0J6AVCoFt9st40qdPPzwwwFk1jtl/vDDDwPI3CvXOQ1Hd3c3AOD888/XmuX+sH79etxxxx0AoP2EjPvYY4/VRnf++ecDAM455xz85Cc/0WcBlu6Wl5ejq6trt7prYo4GBgYGM2BWdMrhcMDpdKKxsVHshBaCls/j8Yh93H///QCACy+8EFVVVQAsy0Aa/sQTT4jVXH311QCACy64QNaY7sXq1asBAC+88AIuueQSAMBnPvMZAEBdXR3KysoAWDEIutmRSES/y2ek02nE43FUV1crdkI3pK2tDQDQ2Niov5FBJhIJWT+yG7I+wHLNyaj9fr+uQdkyFFFdXS1LzmeYTCbR19cHwGJZfH9paSncbnfes/J0Oo1oNIqJiQmxX8qFLt6aNWvQ2dkJANKlBx54QCEfuuHvvfceAGCvvfbCl770JQDA22+/DSDD9ijTH/7whwCAb37zmwAyLiT19cknnwQAvPjii3IZ33zzTQDW+ohEIpJzvsPtdqO8vByDg4OS68KFCwFYobXe3l7pMfW1oaFBOk5WyX1kzZo1kjnfd+ihh0rWZKgf/ehHAQD33nuvvAJe/4ILLtDrGA6hGz4wMACfzyd9mAmGORoYGBjMgA8Uc0wkEmIpzAzZM5hkEvaYSWVlJQDguOOOAwBs27YNQMaCd3V1AQDa29sBZBjJQQcdpH8DVkziq1/9ql53+umnA8iwRLJPWhtaj8bGxhkzuvkGh8MBl8uF9vb2nExfY2MjAKCrq0sxL7LhgoICxa2GhoZ0LSAjO7I+Mvx0Oi0GyN/ZM9p8L+O5g4ODYv18nnx/Z2en/pbvcLvdWRnjk08+GQDw6quvAgDmz5+PNWvWAACeeeYZABld22+//QAgJ+vf2NiIOXPmAMgwTCDDRshemGhgtnv58uXS6csuuwxAJoFDVkT9fv311/V9yaryHalUCrFYDH6/X7pL9s149vDwsO7/b3/7G4DMGl2/fj2A7PwBAOU2AOC+++4DkIkFv/LKKwCA448/HoCV89h///2VbGHcNxAIKLFLxs/9JJlMYnJyMqtCYzoMczQwMDCYAbNijqlUCpFIBKFQSDE9MjZmn0dHR/W7r371qwCARx99VBaSPv8BBxwAIJNlYpyLFnvr1q1ik//5n/+Z9fp9991X1piM6f7778eBBx4IwGJZtFx/+9vfUFtbKxaWr0gmkxgbG0M4HBYrJAthHGpkZEQMuLy8HEAmRsN7YwUBXxOLxfQs7OU+ZJ+MjzHj5/F4smpNgZlLs8g4+b3zfZq80+mEz+fDyMiI2AVjV8ymPv744zjxxBMBAO+//z4AYMGCBfjkJz8JAHjssccAWB7PxMSEYoKspDj55JPFZKjv77zzDgDgqaeeEvO+4IILAAAbNmyQnBmjY8wNyK4ZzmcwpltYWCi2S0ZOhj40NCSvkcy5vLxczJHrlZl8p9Opch3KdN68eTjssMMAWJ5TRUWm/HPLli1aLyzpKSoqwhNPPAHAilEyxrtp0yaUlJTsNl4+a7c6FApheHhYSsZNku7s4OAgvvzlLwOwarz22WcfKSHruhiUfvbZZ7H//vsDAD72sY8ByLgpXPDcGLgR3nTTTXj33XcBWIt16dKlUl5utA8++CCAjBIHAoE9ptzE5/PlbDaUdTgclgJwg2psbJTy0PVjQqa1tVXPhSgqKpK7wtdxk+zs7NRnUZkqKyuVtLAX7wIZt8Xuiucr0uk0kskk4vG4NjQuWoaFKisrUVNTA8AKZxx//PEy0izpYRLm0UcfxcEHHwwAeOSRRwBkFt5TTz0FALoW6/3Gx8fxl7/8BQBU0rNy5Uqth7q6OgCW2xcKhWSo9gSkUikEAgFt6NQjyvKwww6Tzu67774AMrK88cYbAWRkAViband3tww7k7G1tbU5BoOEamJiQiEebpz77ruvNl0aK3sysbe3d7cGyLjVBgYGBjNgVswxnU4jFoshmUzKIkwP+Pt8PgWxn3vuOb2PloRlI0zoHHjggbKWmzdvBgAceeSRKuh8+umnAWQC5kAmZc8gLzsalixZgq1btwKwmADLMk455RREIpG8d/2AjBvd29srlmtvmwIyFnF6e9rU1JTumd0wdrCAmxZycnJSLIXXp7tnfz+fidPpVBkQ3Ucy9pGREVRXV+e9bJ1OJ7xeL5qamqSTlCnDDStXrszR5bvvvltu8bJlywBYSa/y8nLs2LEDgFVydc8994gBMQz0+9//HkBGf5cvX551/XA4LHbEhASZfjqdFuvJd5CZu91u6Qa9QHodTz/9NK644goAVlL28MMPV7KF3uDHP/5xAJmwB9c0ZX7YYYcp7ED58vP23Xdf/PWvfwVghZccDoc8SrJPNpCUlZXB4/GYhIyBgYHBbDHr6t1kMonR0VHFt1gewXhUTU2NUvVHHnkkAOAHP/iBrCUt9VtvvQUA2LhxI/77v/8bAHDnnXcCyBTQsvWQfa5kmp/4xCdwyy23ALBY0ejoqHpZyXjWrl0LIFM0Hg6Hsxr58xFOpxPBYBBTU1OKK04vkyFrBywG7vF4lIRi4Tbfl06nFbMluwsEAmKA7F3l3+yfR3be29ur8grGwBjEZgtdvsccGc8dGBhQfJU9t7y32tpaxWKZCGhra1O5zhFHHAEAuOaaawBkio/JVBgPr6qqkl5/5CMfAZBJOAKZ58WED5M6b775ptgqZch+7UsvvVTlJ/kOl8uFoqIidHd3i/ly3dpL8piDIIt76KGH8PWvfx0AcPPNNwOwdHHNmjXyJMm4ly9frnV96623AsjsB0CmXZOskjIdGxvTZ7L8j/vIiSeeiGAwuNsi8FltjsyolpWViepyc6TSPfbYY9oAuSEy+A1YCZN58+YByNBiUmtusBdffLH6IxlIpQvjcrkkEA48qK6uVgaWykW3ac6cOVi9ejU2btw4m1v9lyOVSmFiYgKJREIPjFl5blQul0suLl2HYDCoEAd/cuMMh8PayLiACwoKZCgoM7qFW7dulYvNbHgoFJKyMXHGEMnIyAii0Wjeu9XE888/rwEo1E0ml2pra3HGGWcAsNy+aDQqveGwE2ZCb7rpJrnalF8ymZRs1q1bB8BK1gQCAdXm0Yjts88+qt6gvC+88EIAmbAQn2O+g91dnIAFWOucWXjAure77roLQKZ7iFUDfC4kSE6nU10tTDg++OCD2nyZ1OL7XnjhhZwupjVr1sj9pu4S999/P1avXr1b3TVutYGBgcEMmHVCJpFIIJVKqVSBlpJuCmC503TpPB6PXDhaDVrdbdu2iQXRsl577bWy1CyFIMtpb2/XxJPbb78dQKZu7PnnnwcAnHrqqQAs1vWlL30Jbrdb7CtfkUgkMDg4CJ/PJwZI9ktEo1ElTRjATyaTchnoLvNeKVfAqp+rrKwUmyTr57XsZUT2GY/sbmKAnG58MplEMpnM+3o8zgRYtmyZwhIsJaO7/Pbbb6vEhOz8lVdewec+9zkAFhtnAmDhwoWqh6QX1dnZKZbN3/E1y5cv1/MgW3I4HFo37C1mGcqyZcuk+/kOJmT+78xUAFbpE3W5s7NToQa6yyMjI9oP+LoVK1YAyIQlGD6jTn7sYx+Td8nQxIsvvggg46ozqUP9f/7557Xv0Jvl83G73ejs7Mya8TkdhjkaGBgYzIAPNJWnpqZGjJFFlQzWL1u2TFaQOPXUU2WB6fuT2U1MTKi7hX2TixYtkoVgDJHMZ3x8XNaWVsc+wYQWnrGO//qv/8L4+Hje91a73W6UlpZibGxM8afp06STyaRiZJTHvHnzxDDJisiAduzYoVgwy0JSqZQsKxkkyy0aGxv1XPkMm5ubZX3JMPn+OXPmoK+vL+9lG4/H0dvbi5aWFsmDyReWNd10000qG6PXcsopp6jnmR1gjGGNj48rIcgJUvvvv7+mxDAmxrmDr776qpITTEL09/djyZIlAIA//elPACy9HRoaUmwy38GZC8PDw/Je2PnCBOkzzzyjWDqfwXXXXScmR1ky5n3rrbcqL8GY47PPPqsYI9cGdf3GG28Um+TEL5/Pp+YTxjLpBRUWFv7DRKJhjgYGBgYzYFbM0eVyobi4GDU1NcoIMa5I5lhZWakMM63I2NiYGAl37gULFuiaZJFkQ6tWrVLM5s9//jMAy6Ked955KtthQefbb78ty0NWyQxiWVkZYrFY3pebsNi6sLBQmVRmlWlxa2pqJBeyuE2bNokVkmEyphOPxxVX5DVLSkrUj8rrs8DePmmcr+/v7xeL5+vJBiYnJzE1NZX3MceCggI0NjZiyZIlYoycwEPmEY1G9W/GsquqqiQPyozj9rdt25Y17h/IeFYsw6JMWda2ZcsWscpzzjkHQCZTvmXLFn0+YD3rt956K2uKUD4jkUioTIqxf5ZIMdt/7rnnZskJyHg6ZMfcDyjn/v5+sU9i27Zt8jwpezaLvPzyy5IlcxLPPPOM9hbmOOht1tTUIBwO79brmbVbzeQKFykDsHTtDj/8cG1Q/Nsf/vAH1SrRLWGN2MjIiFzhb3zjGwCASy65RIFWKgs346qqKr2eG+3pp58uxaNAubifeuoprFq1Ku8HssbjcXR1daGyslJlENMH2xYUFEjB6I4tW7ZMCsbaU/6cmJjI6Z7Zvn27Av2UCRMt7e3tWtzs3BgfH5fryUQFDeKiRYsQjUbz3q0mNmzYIN1hTScX1Lvvvqu+afb3Op1O7LXXXgCsxApDDE899ZSMApM6n/jEJ3QEAAP9HGLb09Ojbhm+76abblJyhpsJw0jz58+XS5/vcDqdKCwsRDKZVLKOCRbq3cTEBM466ywA1jkwW7Zs0X1TDtwf/H6/DBGvdfbZZ2tvof5TXkuWLNHQCpKzQCCgkAcTyKxbjcfjxq02MDAw+CCY9ciyWCyGnTt3Km1OC8wkAmCVkJBpPPfccyp2Jb2ly82fgDXe6eCDD8bZZ58NAPjRj34EwCrW7evrU/8qrcD//M//iFHZj2UELFeUjChf4XK5EA6Hs463ZCiCFm5kZCTr2FAgw/amT0DhJJmmpiYxaLLPyspKBb3JosgSi4uL9Tx4fZ/Pp9fRO+C1enp6sgac5iuSySQGBwcxOTkpJkPGyPKxxYsX60gEJl+WL1+eMxmHw2gPO+wwuXa81m9+8xsxH/b00sPq6uqS3PgdPB6PZEf2Tw/rlVde2WOG3TKZmEwmdT/06hj+aWtrw/e//30A1ryE8vJy9WDTe2HxfHNzs8IcDJl1d3fL2+EeQDf57bffVkLyP/7jPwBkTjNlrzZZJZnqtm3bMD4+bqbyGBgYGMwWs24fHBkZQVFRkeJM7KFkUmDp0qWylmQkPp9PAXBaBqbnFy9eLAvBozEvvfRSsUgmXxj7+dOf/qRYJluzDj300JwDipisWbp0KVwuV85cw3wDZTtv3jzFt1jKYGeCBJMuFRUVSpiwLILlIx6PR2yPLB6wklWMTbLwOJ1OqxyCDLW2tlbsicyH8k+n0/B6vXmfkEmlUpicnMScOXPE9sgyqL9er1cyYlJh48aN0msOaGUba2trqxgmde+EE07Qs5h+wNSTTz6p5CJjuO3t7ZohQPA72IvG8x2JRAL9/f1oa2vT96d+Mm560kknKc9g1z96hGzd5POYP3++kjnUu3Q6rZgmy8nYB3/dddfpSF2WYj366KMajju9FbOpqQler3e3ccdZbY7BYBAHH3ww6urqFAjlQqFr99xzz2mxsVbujjvuUP80A9p0LVwulxTvoosuApChvOeeey4Ay9VmBuqrX/2qqu+vvfZaAMAxxxwjmk0KTje7qqoKa9eunXGcVz7B4/GgsrIyq0aURoYbVUdHh8IGXDg9PT0yVHRluCAdDoeeBTdc+xgs/o3ueHt7u65PwzYxMaFnxk2ayup2u7HPPvvkfULG7XajsrIyazEwYcLAfmdnp+TNhNh+++2nBADdab5/fHxcBpehiyeeeEK/48ADe5cLjRFDPMuXL9dmylFqlOXixYv1HPcEOBwOlJWVaR+g+8twwXvvvSfd4/17PB6RJCZkKO+9995bsmQYKBgMyiBxeAWNVWVlpeonmaE+4ogjpPe8LjfvsbExVFZWmt5qAwMDg9niAw27fe655xR4tdfDAcBZZ52F0047DYA1wnx4eFiuBzsHWMX+xz/+USl+nnN96KGHysUmu/nlL38JIOM20wKdd955ADKWiO4SXRZS92uvvRa1tbV5f0wC3epoNCpXmFbYfuIgLR8ZJi0gYJX+kAHZz5RmuMHtdkumlBnLHOrq6vQ71jLaT0MkC6K1Ly4uxqZNm/Q98xXxeBzd3d0YGBgQKySjYBghkUgo+UeUl5frddPlGAqF5KXw+SxevFjMj7Kiq1dWVqYOMI4x27lzp54d6yG5FhwOxx5zTALPre7t7ZXnwX5/6vL27dtVYkMdW7RokfSZOslzp959910xQCZ/Dz30UHmELNljAuu4447T9VkG+Prrr+NTn/oUACsxxJ/d3d3/MBxkmKOBgYHBDJh1EbjP50Nvb69ijWSJLOVZvny50uY8Ze2QQw6Rz88DiBhsnZiYyDpUCMhYIp4FTEvEmGJDQ4OuxaB3aWmpvg8tNWNyJSUl2Lx5c94nZBgf9Pv9sqaM8TEu1dnZKctHthIOh2UBWQ5B9sIhpIDFfLq6usSimcDhtegZABY7nJycVCKIhfhk5xyEm+8xR6/Xi+bmZpSVlSkBMH1YczqdFlOh1/HXv/5V8mNsyh5PJytieciWLVvUP81ECxnUscceq1gbY2gej0f6ze9BNlpSUqLP2hPAvYGJD8qEiZlwOCyWTN065phjVI5HdsjXB4NBJRbpmaTTabFusmpOV7KXtJ1yyikAMmyc+wJzDoxZtrS0IBAI7LYMzTBHAwMDgxnwgY5JOPXUU8UwyPbIEjs7O8V0mGneunWreiLvvfdeAFaP6rp16zTnjRZ7aGhIbOjMM88EAHzlK18BADz88MPqw6QFOv/885VNZKkQLfK6detw5JFHiinkK8gcY7GYmBjlQStZWlqqeCuzpl6vVxaTbJnWcGpqSn9juZTP55MVpdVmWYTf79dnkQHU19eLIbG0iN8vGAyisLAw74vACwsLsc8++6Cvr08eCVkJe6Ddbrf0lYXcd9xxh4qIWd7DeNlxxx0ntkcmXlZWJmZC/WOMs7e3V5lbxspfeOEFTY2h3BnjDAQCYj35jmQyiaGhIQwODkpXGP9mr7TH41HMlXpaWFioOCGz1aySCIfDun8+F34WYDWFkI2GQiGtF3oAZWVlOvqW3hHLfAYGBv7hXIBZbY5jY2N45pln8Oabb2qEOR8+uwTeeustXHLJJQCsLotoNKqRTHTlGLB2uVwSEHtUDzjgAAWoqYxc0O+//37OSXAPPfSQNg0qJTeFp556ChUVFXlfypNKpRCJRBAOh5Vs4UPmIvR4PHLT6CYPDg7K9ePr6JZXVVXp9ZTH6OioNjMqGOUIWMrH1/j9fiXb2D3DJEZ9ff0/7DLIB0xOTmqQA0MxXLwcRNHW1ibd5CLef//9tdmxd59u3+LFi7Wh0f2trKzMGRrChMFrr72mZ8YE2L777pvzjAmHwyESkO/gMQnpdFohF+oWN3un06mSMBKljRs3anPjgGw+g/Hxcf1t0aJF+hv1kqE4PoPe3l59Nrtu3nrrrZzSNL4mnU7/wzPX89vkGxgYGPybMCvm6Ha7UV1djf3220+lOdzJSZUbGhrwu9/9DoCVfCkuLlZ3AN0IJkiGhoZU8M3X77vvvmI8dIf5eevXr5dLRCtzyy23yDWnpSdTPP/881FfXy9XP19B69vV1aXvyuQS2YjT6cwZ615UVCT2wXu2H4DFJACv6fV6lRCgTMkI+/r6FCahezQ6Oiq3myyA1xwfH98jujh4YmYikVCSiy4b3epAICA2wsLioaEhudhMlFBvi4qKJCP+7r777tPhb2SoTB4eeOCBYi32gnKycnpDDFkUFBToOeU7XC4XSktLUV5eru9M1ktZbtmyRR4Ru1aGh4f1O3qNfF9PT49cbXpCgUBAciULZXju5JNPFqunrhcUFGQlhAArDFhZWan94+/BMEcDAwODGTAr5uj1etHY2Ihjjz1WM+/o87Nvct68eSrmZtr9V7/6lcabEzyndv/99xf7ZP+1/djLu+++GwDwk5/8BECmLYuWgWxx7dq1Ob9jXOORRx7B7bffruRNvoKHl4XDYcVBmBSxt2SRrfB+HA6H4irTe6U9Ho9ia2SVPFoXsILltMLV1dVZ03iADFvl68nm+b7x8XG43e68P5qVB0D19vYq7sceXbKTVatWqR+f7OWkk04S8yPjJLxerxIsZHuf//znFf9lnIzPsKysTLMe7QOL+azp8fD7vfzyy0oe5DtisZhKm6g3LNKmbh188ME6qoBne9fX18tr5JwEeiKDg4OSBfXPPoOR8Ujq5OTkpHTXftgf47tcN1wHw8PDCIVC/7ze6snJSdUjMVHCwCtp6/XXX6/FyY3qRz/6ET7zmc8AsDoMeG7M2NiY3BK6G42NjWoop/tD1zEYDErh+Dl//vOfpZRUbH7PwsJCHH300Ur65Cs4rbq9vV0Pkgpgn6zOzY6ZuUgkIqVgrSnd3uHhYbmKVFK760M3j4vV3ovNzTcej2sj4fVZqVBXV4fW1ta8HweXSCTQ09ODj3zkI0ocUv9YGzoyMqJJ9NzsX3zxRS00PgtuhFdddZUMMGv6BgYG9G/2StvPAWdNHpMDzc3N2gCZ4GR3xyGHHJIzCTtfUVBQgIaGBvT19ckFZiUE3eqRkRHpLk8IBazQDpN8TGC1tbWpvpbJ29LSUuka3WVWUNx5550KQ3E/CYfDIl7TQxTl5eVobW01pw8aGBgYzBazrnNkUNtetQ5Yo8/tlJa9qg6HQ+yH7jE7a9ra2vCFL3wBgBUs/clPfoIf/vCHACxGSgsQj8f1Wax/SiaTcks4fJfufnNzM3bs2JH3rh/P/QUsmZLt0Y0dHR0Vo6Ncenp6JBu6GPx/eXm53BReMxAIqGyEz4TMMxKJSG5kSPX19bLWZIy0thMTE1nj6/IZTqcTL774ojwdMmTe2yOPPCJviEmYbdu2Yb/99gNgPYNrrrkGQMZNpDdD13zOnDm6Lnt62Qn229/+Vs+C1/J6vXLz6N6RyTqdTjH2fAcTiS6XSzrMJB5Z3+joqBIg3/rWtwBkxsBxKhJdZ3bXRaNR6RnDF4lEQjKht8O66S1btoiR0jsYHx/X59Pr4TPYvn07iouLd6u7hjkaGBgYzIBZH5PAA5XIOthDSotsB5ldbW2tmCZLHf77v/8bQCbmQCvDsoZLLrlEMR7GMJjkOeaYY5Sk4UzJQCCgIZksS6El6u7uxrx58/Ke3aRSKUxMTKC6ulqyZTDZzuwYa2QSxuFwiAmSMTKQ7ff7ZXUZ57L3BE/vNohEIjnFtW63W/+ePo+vo6MDTU1NeX94GTEwMCCmRrbHDpjf/OY3OludjQ3r16/HVVddBcCKK7KUZHR0VMyRyRp7qRDLnqh3Xq9XxcnU1YKCAjEfxn/5vFatWpU1oDifwQO2hoaGxIC5lskWp6am1OXG5O1VV12lKV30hMg8S0tLlSBj3HL58uVKanE+LE9yrKysVJKYHk46nVZ+gmuJz722thYTExNmnqOBgYHBbDErk+/xeFBVVQWXy6XYHhkhC7MbGxvxxz/+EYBllWtqapQdvPLKKwEA/+f//B8AGXbDDLb9ECiyIMZs7K2IP/3pTwFYDCYUCik7bbfsQCa7PTw8nPfzHAl7lo6z62hVi4qKZFkZOwmFQlnTjACLyTidTllyWtNUKiU2Y5cpr89rMC4WiURUIkQrTNnOnTtXbWN7AhYuXCh9YqkY9aK+vl6xKsbA3W63YtnUTR4UN3fuXBUw8xpXXHGF2mRZecGSnv3220/97cycHnvssWLt1HdmdKempuRR5TscDge8Xi9KSkqkI9QjxmzHx8elRzwCd/PmzdJt6izLfA444ADJi8/qmGOOwW233QbAmmfKGG1RUZGOVKF+vvHGG1ovXEv2+PycOXP+eedWp9NppFIptLa2qmSGSQCWNQwODqpqneU1++23n9wZBlxvuukmAJlgLjdWnhp23XXXSchM1ZNGRyIRlURwUdfX1+szudFSYf1+/z8MvOYLnE4nSktL5a7ZEx8Asu7B3m9NZeBCo/vr8XjkTlP5amtrdV3KnbKOxWI5NZMVFRVazPzJ6+/atQsFBQV5X8pDJJNJ6ROTArz34uLinPNcKisrdQaKvX8ayGyS00ttNm/erO4PlqTQyPT19alU6PLLLweQSQrwWXMdsRzF4XBo/eQ7nE4nvF4votGo7pe6wtDa/PnzZWi42R122GHqimPvOvW1pqZGa5mj8jZs2KB1TnLG53LKKaeofIhGf8OGDdpsKVeG3Xp6ev6hUTdutYGBgcEMcMzGJXI4HH0AWj+8r/OhoimdTlf8u7/E34OR7YeHPVy2gJHvh4m/K9tZbY4GBgYG/6/AuNUGBgYGM8BsjgYGBgYzwGyOBgYGBjPAbI4GBgYGM8BsjgYGBgYzwGyOBgYGBjPAbI4GBgYGM8BsjgYGBgYzYFa91X6/Px0Oh+F2u9WDyyJy9uJywMF08PXsf2QDeGtrq5rU2SMcj8fVw8p+YfZqut1u9U3be47ZY0nw+olEAi6XC0NDQ4hEIn//wIh/MwKBQLq0tDTrDGjeJ2XLe+G/Aavp3w4+k3g8nnOmtMPhkGwI++BXDqrgZycSCf2dv+NrePbv8PBw3suWPbuUB3/y3hwOR44up1IpDRfmoA/2OxcWFmosF/ujKysr1fNOWdnfbx8zB2TkN1229qEg/I5dXV39+dwhU1hYmC4uLobT6dR92OUKZN8Pfzocjhx95jNwOBxZz4E/7X8HrHXudDolX/7O5XJpb5m+DhKJBNLpNEZGRjA5OTmj7s5qcywrK8Nll10Gh8OhTZADDTgj0O/361wITsg54YQT1ODPARJUMq/Xq02R8xkPP/xwzYH7/e9/DwD4+c9/DgA46qijcN111wGwjtV877339PkUDAddlJeXo6ioCFdfffVsbvVfjuLiYpx77rlZ04s5oYWKwP8D1tAI+9kcVDQecsRzMuyvj8fjmhPIhUvD4/V6swZU8PrTB1tQQScmJpBKpXDrrbf+s8TwoaC0tBT/+Z//ifHx8ZwhGZRFJBLRBsjBGiMjI5Ilp4Qfc8wxADJTY6699loA1gzG//W//pfmlFL3OVBheHhYi5cTe+zHBU8/c8Xv92thf+tb38rr1rxwOIzzzjsv60A3zgXlPYdCIekR73tyclJTc7gf2I0+r0F9Gxsb06QiyoZDVAKBgK7F4Rc+n09rgj/5vHfu3Imamhr8+Mc//rv3NavNkTv92NiYvhyFwS9UV1eHJ554AgA0QmhyclKTODjo8gc/+AEA4Jvf/KYO4uJphS0tLXjggQcAQANIOVXjZz/7Gc466ywAGWUEMsNxOSCWxzRweoff70dPT0/eT47hCXlz5szRhBYOQOXm5fP5tIDt46xoCLh52c+55kLnoquoqNCGSYNiP1SLz5VW2+12S3b8HdlRUVFRzql8+Yp0Oq2zwfl/wBpsaz9CgYZ51apVWlQ0JNwQ6+vrxdjPO+88AMANN9ygoaocy//d734XQGaUH3WTer5p0yZNiaEXRIY7MTGh3+0JSCQS8Pl8MqrUGcqot7dXGyA3qHg8rnF7fD3/5vP5dOgWn9Xk5KR0ldfi5tvY2Chdp34CkK7zOZPUuVwubN++Xe+fCSbmaGBgYDADZj3Pkef/cp4afXkyuyuvvBIrVqwAYFlgt9uNN954AwD0t9NPPx1AZhQ9D8ahVX/hhRfEhshITz75ZACZA45OOukkAMDxxx8PIMOsOKiUlohDMBsaGlBSUpL38xzdbjfKysowMDCge+CATsZfo9Go2B4tbTweF5uczjTsg3NpaTs6OuTWkClxbp798C37ca081IiDjfmc+vr6UF5envfHJCSTSQwNDSEQCIiFU6Ycrd/Q0KCjEMheiouL8cwzzwAA9tprLwCQPEOhkO776aefBpCZ4UjZ8FpkLD/60Y+ky1wL4XBY4Sa+nvHLVCo149Ej+QiXy4WysjKMjIzk6C6P9WhpacmZUzo6OqqQERkcvR6PxyNdZ2zX7/dL16nDZNqRSERsnWyxurpaciXzp2sfCoWQTCZz4u92GOZoYGBgMANmZfKTySQGBwfhcDhkgclIuFv7/f6shAAArFmzBnfddRcA6zBuTkz+0Y9+pHHol156KYBMMuBvf/sbAODoo48GAPziF78AAKxevVpxDB6I/uabbypgThZEZhuLxTA1NZX3o/yTySRGR0ezjjFgDIUIh8OKmdgZBkG500oGg8GsrD2QsfJ8L//Ga7jdbrFDZh3dbreuy6Niea3a2lrEYrE9Qrbj4+MoLCzUPTMBQg/F6XSKMTJ59eyzz4rlUQ8p282bN+PQQw8FYOnmpZdeqjgZY96vvPIKgEzsjce6Milp98D4PeyHo/Gz8h08mrWxsVHHF5BhM0ZYUFAgFslDrhobG3WP9FjIFsfHxzUBnImZQCCguDfZJ3Wzvb1dsXr+LhqNZlUGABaTHRkZgdvt1t9ngmGOBgYGBjNgVsyR1tXn8ylbStZB3//UU0+VReVZL++++65eR+tMdvTtb38bjzzyCACrfGTRokU5Zz7w9d/73vdkSVauXAkAOOKII3RcK0so7AdLseQkn5FKpRCJRFBWViZ2yHgUs9aBQEDWkRbZXq5DtkcrmU6nc859GRwcFFuh9aXVjkajYlb8XSwWE6vhs+Cz55ko+X54mdfrxdy5c5FKpfTdGaPmWS9jY2M6EO4Pf/gDgEyMlSVgPGqVZyeNjIzoIDnK+OGHH1ZVBvWNjPOWW27Br371KwBQdUZpaankzfI3Puvh4WGth3yH0+lEQUEBhoeH5dWREXIdhkIhrX2yt8LCwpwYIpFMJnUuFdnk6OioYpL0cLhW6uvr5dFQps3Nzdo3+Pz4XOw1w38Ps9oc4/E4uru74XA4tGAY2KZ7vWzZMqxZswaA5T488cQTKl8gLaZCvfLKK0rccNN75513lLghTb/jjjsAAAceeKAUnPVlJ554ok45oxu0atUqAJmFXFJSslv6nC9wuVyIRqM55xVT0VhoC1gPub29Xco2vWA5Fovp39xUY7GY5GcvayD4PJncCYfDkjP/RgWtr6/X88lnJBIJ9Pf3o7S0VO4eS9B4GuHo6CgOPvhgAFYC4GMf+5gSDKxN5IFR0WhUB2zxBL1ly5bJ8HCzY0Jg//3312FdTOBUVVXJ4PAnNwl7zWm+w+Fw6Nxq3gflS/nFYjGFcxhuGx8fl4Ge3thRVlampCAxMjIio81EF/U7EonosyjDRCIh4sDP5Of4fD5Eo9HdJhONW21gYGAwA2btVhcWFsLtdmvHZ4kDLUVHR4eOQyTNnZqakpXlTs8de/ny5XodXZ0lS5aI6dx5550AkBW4JUvlMZtLlizRedj8nFtuuUWfw7R9PqOgoAANDQ2YmppSuQFBl7itrU3hCzsjtLdjAZZb7fP59HrKz+l05iRd+Pp58+ZlFZwDmZIeunp8HZMZU1NTasPKZ7hcLhQVFSEej+u+qEOPPvoogExIh/dOdvnGG29Ix5iM4vnrTU1NYtCU+8knnyz9YykZy3EGBwd1hjWfybvvvosHH3wQgMUcyX5aWlrEjvIdyWQSIyMjaGpqUkkO2TR1bXJyUnJiuGh8fFxMkPsJvZp0Oq3SJ3uTA5klX899wp7c4Xfw+Xw534OeTklJCZxO52511zBHAwMDgxnwgWKOgNVrSsvKeOHTTz8tJsNdORAI4KijjgIA/PrXvwZgleGMjY2plIfFxoWFhTjxxBMBQEFZfm55ebl6sBlHqKmpUSyIAXDGHL/1rW+hq6tLliZfkUqlMDk5iZ6enpwBD7SuhYWFspz8WV9fr4JbgnFdj8ej+2Zw3y4LMhhev6urSyyVTIkWF7A8ASaDhoeH94gC+1gsho6ODoTDYenkdP1l0gYA/vznPwPIxLlPOeUUAJZufuxjHwOQKUdh/Jcx4u3bt8uD4jP5y1/+AiCTEGCCkux14cKFKvUhw2SSq7S0NCfmlq+g1xOJRBRXpG5RhwsKCpRYYRulz+eTflGHqLsul2u33t50Brl+/fqcYSJLlixRgojfi+yd8cjdfcas3Wqfz5eV8eTNMWC9dOlSCYEB/8HBQSnEQQcdlPUlb7vtNtWV8X1XX321Mt788gcccIBew2QNH8CCBQvw2muvAbDqy772ta8ByHRAnHHGGRJ6PiORSCAQCORMNuFGVVNTIwXgw3a5XDl943TNOjo69DcaL8BK5lBx6MZPTU0p6UJlrays1IKd/lwB5L1LDWRCK6WlpfB4PFpUzDpzg2tqasKGDRsAWBUPgUBAmyhdbYZ+ACspwA2up6dHWVE+M15r/vz5StKceeaZADIuIcMjfOYcptLV1aXPzHfQrQ4EAllTeABrsxsZGZFOMaQ2MTEhnaIcKG+n05mT+fZ6vTluOPeOdDqt93JPqq6uloFh2IKu+vbt2+Hz+UyHjIGBgcFsMSvm6PF4UF1djUgkouA1GQnru0ZHR1WaQ3ZzzDHH4LnnngMATc+hKzI6OipWSKtQV1enhMq7774LwOrd/vrXvy5WeeSRRwLIWC5aZVpxMqBPfOITiEajec9wnE4n/H4/4vF4jutM69rT06N/k2m0tbXpXhlsJrOrqKgQK6SFLC8vl0Vm8JvvHxoaUrCcltk+P5PP3J6I4PfJd6RSKRQVFUk32cnCkE5ra6tKlh5++GEAmXKcc845B4DFduwlYdTNz3zmMwAyyUV6MIccckjW52zcuFGfzb8dcsghKiXi9ZlMmJiYyPtJUnYkk0m0tbXpPqgX3APsJTP8WyAQ0Lqk3lHXysrKxD6pzw6HQzpLr4p6ap/YQ8Y9MDCg61HnySp9Ph9KS0tNKY+BgYHBbDHrqTzpdBolJSUqWbBPnQYyMYBNmzYBsObWDQ8Pq5+SSRQGvcvLyzUjj1N2/uu//ku7P9/HmEEqlcqZTLN582YleBgI50xJl8uFxsbGvGeO8XgcPT09cLvdCs7TSpJRj4+P62+UQUdHR05QmVbY3ivNhFY6nc6ZBEOZeb1exX74vvb2dsWXWRxN1h+Lxf5hf2o+gHrb39+vGBgTfYyDtbS04MUXXwSQCeQDwEknnZQzg5FxxksvvRTnnnsuAIvhr127NmfyNPV9/fr1OP/88wFY8bUtW7aoYYKxTMrW7/fnfVcX4XQ6EQqF4PV6pTeM9dkbDfg33n9PT48aGOiV2JtEqJfU/8HBwawyHfv7pqamlG+gDHt7e5VgnB7TDAQCiEaju5WxYY4GBgYGM2DWU3kGBgawceNGZdVYQvPSSy8ByMRdWDTMOOHnPvc5jYtnOQMtcllZmVgid/Hvfe97+jutDWOWk5OTOSUR++yzj9gN2SSZ5o4dO+B0OvOe3bjdbpSXl2PXrl2yctOnKjc0NGQVsQIZi0jrTPkxIzcyMqJ4IhlNV1eXLDEts30qD9khGU88Hle8ht+D14/FYntEPNflciEYDGJiYiInm8picM4pBaw44dq1a8VCeM+MKUYiEbFxxrl5JAcATZXitPqf/exnkh8ZKtcQYMmWMTqPx6NnkO+Ix+Po7OxEVVWV9I26xQLuQCCgMifKrbKyUl4PmSYbDOzzHO0lOpQv1z6Z//DwcI4X6/V6c67Paw4MDKCvr2+3cd1ZbY5utxsVFRUoKysThaUQ2Dnw5ptvyi0h5e3p6dENsm6MN3LIIYcomfK73/0OAPDpT39aSsiFvHr1agAZl4flEtxA99prLwXHmSCigBobGzE+Pp73HTJAJuBcWVmphAddCAby/X6/jAXdj2AwqB5c3ru9h5VyoBIVFBRk9Z4C2WUXlBN7UktLS7VhTg9nBAKBrKGv+QqHwwG3241UKqUyp+lubDAYVH8+7+fJJ59U0o8LmqUjO3bskFw++9nPAsgce8BnwUQPkwSnnXaaZMoN8+WXX5abTpJhL+3ZUxIyHo8HVVVV6mEHrPVtL5WhXlLmHR0d0j3eN5/P+Pi4XG7KngOL7dciqqurRRgIh8MhXeca4ud1dnZmHdg1E4xbbWBgYDADZmXyE4mExviziJYWmK7XxRdfLPbBQ53OPPNMUWoOu73gggsAZFxi/o1WYeXKlXIf161bB8BKsBx++OGy5t/5zncAZArEaQHWrl0LwCrH8Pl8WL16dd5b4XQ6rWNQGUQmc2TIYGBgIIel2U/NI8umPMfHx3MOiLIP07VPgAEyZVVkpgxL8JkD1qgpO5MtLS3Ne7ea92BPHjFkQWbndruV/KM89tlnHzE73iNlW1VVpcO0WDz+9NNP6/pMdrE5Ip1Oaz3QrY5EImJafK5k/729vTljvPIVLEOzT82hl8EQxcjIiHTKnnyZ3vtvLz3jv7kXBAIByZprgqGH3t5eeVPcf/x+v9bS9CNgi4qKEIvFzLBbAwMDg9liVszR5XKhuLgYyWRSsQEejsUdevPmzZqUwcLwxYsXq/ibcR1aVHs6/9lnn9Xf7r33XgBWgJYWvKSkRMmZT37yk/pMnp3MKUG06pFIBNXV1XnfPsh4TUFBgSwfg9lMcIXDYTEX+0BbxhUZ/6VVHR4eFjNiCVAkEpF1ZwKC729ubhYLJTssLy/PKkIHLMY5Pj6uWF4+w+FwoKCgQKPxAYupTE9mAZb8tm/frmQh5cfYYG1treKKnEK1ePFiPRcO0SUTvPfee3UtsvOnnnpKMwTIYBhLczgcij3nO9LpNGKxGJLJpHSDsVf+PxgM5sz+TCaTkgWn51C/PR5PTunY8PCw9gHuMfSuamtrc86yTiQSilfaf8drFhUV7TZe/oHc6sLCQn05JkV4muDTTz8tJWAtWV9fnzZTuhSc4t3d3a2OFwq0u7sbl19+OQDg9ttvB5DdmcBhoTz7OplManO2n30CZDbvioqKvE8asPuos7NTD5JuCGVdVlYmQ8L7LCwslPsxfcp6dXW1FMx+qDkVhopiP92Q17L3znOD5eu4sXC8Wr4nuziyLBwOqwbXXgEAZDZ+ZkI5lPbBBx/USDMaXS7GHTt25Jz/0tnZKTfRXlcKZIw8N1NuBH19ffpMbtLcTOLx+B4zeCKRSOjMcxpV6jA3u8rKyhyC4nA4FN6wd80AwHvvvaffcVMtLS1V7S9Jk9248bO4XubNmydZ8300OL29vXA4HLsNtxm32sDAwGAGzIpOTUxMYN26dQiHw3IzyA7t9Y5kGHTD7rjjDjFLVqg/8MADAIBvfOMbOfVcd9xxhywIrS0t7ODgoH5HFjoxMaEgOoO9HHl/+umnY/369XnPHB0Oh+ox6faSdVA+fX19Yiu8n4GBAbluZNe0tPa6MFpVno0NWGO4WG7R3d2tf9vHoDFEMb0usqqqStfKZ0SjUezcuTPrHOjpzK66ulq6Yw/a0+0j82ApTyAQ0KBahiKcTqdOzKM+sj7S5XKJvdvlzetND6WUl5fnfbiCcLlcCIVCaG1tlddo780HMuuSLM5+vhOZHxk59bSkpETvZSjO6XSqG4k6SZmm02k908WLFwPIJGSYZLOfSMjXT01N7dbrMczRwMDAYAZ8IDoVDAaVUmd3AOczzpkzR9aPMYO5c+fq4KsvfvGLACxW2d/fj5tuuglAdi82rTKTEXz9ggULtNuzC2H16tUqLufryL62bt0Kv9+/27lt+QCe/VtXV6fpMLwHwul0itWQ5SSTScVRGC9kPGZ4eFgBfg51HR0dVSyTTNA+05DxGjLUdDqta9D68mcymYTL5cr77iPOBrSfqU7ZkuENDAxIRmQgPA3SDrKfiooKyY3F3el0Wl4TY2nU+0AgIJZDnU6lUvo+nM7DCVXJZDJrBmc+I5VKYWJiAs3NzWKC00/Q3HvvvbMYM5DRYcqX69Z+QiF/R7mNj48r3s34Jdn1zp079Xp2HoXD4ZypQDOdtPn3kN87hoGBgcG/CbNmjg6HA/fccw9+/vOfA4BiLGzfO/TQQxVP/PrXvw4g017FzDLZCi3kwMCACsrtmTv2ZbNnm7GfAw88UGUSbDv0er3KOtIq2yfZPPLII4oZ5SscDgc8Hg86OjokB1o9shCn06mMPu+zra1NlQDMKjNDDViz7cgqh4eHJRu7FQUyzIeM0W6R+X2mH8xVVFSEgoKCvGeObrcblZWV6O/vV6yKzIbsYevWrdJD6l44HNYZ0/SG+HPu3Lk5E2iCwaBkw+M6nnrqKQAZPeckK8Z6U6mUStymT1uKRqNi8fkOlkrF43HlFMjIGUO0txZSnxYuXKj9g/rMg8l8Pp9iuYzbzpkzR/sBs+L8WVNTI4+JceRkMikmShZOXS8qKkJvb+9u2eOsh93W1dXhE5/4hL44KS8XqNvtlovNDfSII46Qy8zFTQE5nU4Jja70o48+quMUKFwK8aGHHsJJJ50EwHLlm5qacPfddwOwHgoTMuxHzvfgdjKZRCQSQUlJie6Bmx0TLMFgUG6b/RRHlo8wCG6vD+PC5d/C4bDccJYFER0dHUr+0H2fM2eONml+L3uSp6ioKO/PkOHCbGlp0eLlBkVZBAIBucx//etfAWT0nYuJbhk3123btkmmH//4xwFkurm4YdKd5rNZsmSJ6nj5mV6vV9dn0oGLvbCwMO+NDsFjKLZv355TQ8t7eO+997RBUce2bt0qHWeYiJuky+USKaDse3p6dD3KjRun3+/XmuA1Q6FQjhGkLgeDQRQWFppjEgwMDAxmi1l3yIRCIXz0ox+VuzY98Nzf3y96Sya4YcMGuQ1ke2SCExMTsubHHHMMgEwfNVP2DLyed955AICrrrpKzIXvO+644/Q96KawayGVSsHr9eZ9QgbIuAOTk5Nie2R29nOoaRUZ1A4Gg3KZyfZ4r7FYLKe4trW1NWtCD2BZ4WAwKHnzOyQSCbnaZEr8PlNTUygtLc17huNyuRAOhzE4OCi9YiKEya9du3ZpTgDl6XA4lABgsoau7vPPP6/wDhMB9913Hy677DIAVsiHerh06VIxJn5OS0uL1gGTCGSe8XhcTGhPAI/MoA6yaJ56VFNTI2ZOPQoGg0oO0tvh+ycnJ+Vl2psOyCKpwwxz7LvvvgrZ8RknEgmF0+jZ8ufw8DDi8biZymNgYGAwW8yKOfJsZU7WAayeah5TMDExoZ2bLO7EE0+UhWZPNWOKgBXLYiF5S0uLSn5oWchCzzvvPE3c4a7/05/+FHfccQcAq5+bnzN//nz4/X48//zzs7nVfzm8Xi+am5sxNjam2AxjU0w22UsZ7McZ8BlQxmTWfr9fMmK5icvlUlyIcVgyv3g8LkvL7zAwMJDVj8rP5HfeE8CYI4++BZBzkNiyZctyhi4PDg7qKIRnnnkGgFUMbu+j5s/DDz9cMqGMGUs8+uijxTDJQp988smc0iyy1ubm5qyi9XxGMpnE2NgYent7JV+CDH1iYkKJLrLEWCymf08vK7NPJCI7dLlcislOP9N9fHxczNQ+s3N6YogJs2g0ivLy8t3Gy2e1OXq9XjQ1NWVl2dgIzpt86aWX9DcGqktLS6Vc7MU+44wzAGQSLBQa3ciRkRE88sgjAKyNgRvu+eefr7rIs846C0Cmz5XCpZJxU+UYpXx3/RKJhDZDbj5UFLoS9i4fe1cMFyxDFpTx0NCQZGqf7M3Nk261XSHtysbPpHJOR3FxMXbt2pV1jnU+guGgSCSixcFFQcMyf/586Q77r8PhsMIX04fdnnHGGdrs2KnlcDiUUKBLyM3uzjvv1PNhgm3BggX6fC56LvDCwkKto3wHSVN9fb02e943N6pUKiW50sDX1dXl1Dfy+VRXV0uPeY2ioqKccBv/NjY2pvAQ9bm4uDjHXWdCc2JiAj09PTKOM8G41QYGBgYzYNbMccGCBdi+fbtcD1o61ift3LlTgWpO2znkkEPERL73ve8BgNzg7u5uWU/WmXV1dcmSrFq1CkBmpDyQocN0q1kedO2118qtt5/KB2QsTGFhYd73VnNyjJ0JUsa0fn6/P6cfuqOjQy42LSdZ4sDAgEoXWPIQj8fFmsiUWHbR0tKi15OZ7ty5Uy4OnwmtdzKZ/IdB7XwAx7qVlZXJw6A8mBBpaWmR68XeXIY5AMt9Y0KmpKREDIhs8sEHH9Rz4TNhqGj58uViOddccw2AzHEJ9Bb4TJjA4JEZewIYtggGg/LQqIO8rx07dkiG9AYTiYRez5Inrtv3339fekfPZPv27fKqFi5cCMCSc1tbm8IVZO1TU1NZyUbA0mEydZOQMTAwMJglZkWneMpYIpHQIVrsdGF8oLy8XLEVWojJyUkNqCWDY9xr8+bN+Na3vgXAmvCzadMmHeD1l7/8JetvJ554osoEyGDq6ur0Pdhvze/Q29uL8fHxvGc3nInncDhyprdQVvbZc4zp1NXVKXbIJAMD+RUVFWKfjBv29fWJ7dMKM+brcrl0XVr5RCIhxkNmRTZVVlaG8vLyPYKVFxcXw+l0io1MZ2x9fX3SOcazHnvsMZWlMdlln1NIud1///0AMnEylvBQtuyw2bVrl15HL6e0tFTMlbE2MigmOfYE+Hw+7LXXXhgdHdX353fnups/f37OUQWBQEDPg6+zH8/B9U1Z1tbW5px9TX0tKCiQB8S/pdNpMUe+jz9LS0vh8Xh2OwTbMEcDAwODGTDrIvCioiJs2rRJxxKwyJW91VdeeaWybIcddhiAzFGr9rOOAeDmm28GkGE3jPHQihx33HGy7IyxMYZx9913ixVySvNVV12l8iJaZ8YWYrEYmpub8/6ALbfbjbKyMs2ZA6xYI5mhy+XKmjUIZGTG39FqM+Zit5y0zKlUKuvIBMCKVQ4NDak9jnFPj8cj1jl9arXH49kj2E0sFsOuXbtQVVWle7Af8gRkmIo9FgZk5MheXur5EUccASATE2M8nGVp3d3denZsX73qqqsAZFpcL730UgDQjEGHw6FSE7JzZm+dTqeee77DPoWbcmXslOu2qKhI+wLlPDIyIs+GciBbjMViykWwaL6oqEhyopfEBoVwOKxnybhyUVGR9hGyfPvRxiMjI7ud5zirzTGZTGJ0dBR77bWXzqlmgz37qFOplNxens/7zjvvqF+VC80+SoiuBVFYWKgFzIXLAbeFhYVauMcff7w+m+7OF77wBQDWKYfDw8Po7+/P+1H+PH3QfqQEXVveG4fhApbyjY2N5QTzubh9Pl9OgqC0tFQKOX1IbiKRyFHgyclJPYPpbtH777+PQCCQ933rTqdTLtz0LgrWNr799tu47rrrAECb2P33349bbrlF1wCAF154AUAmfMPzqnmswgUXXCC3kKdm2t0/Egr2WDc0NKgOkovWPopueu97viIej6O7uxvNzc1ZgyYAa/12d3eL/HAPSKfTWsvUQW5w4XBY5IDdbz6fTyEhbo78HPv8BHsXFBMx9gEVQCaBXFNTs9s6R+NWGxgYGMwAx2wSFQ6How9A64f3dT5UNKXT6Yp/95f4ezCy/fCwh8sWMPL9MPF3ZTurzdHAwMDg/xUYt9rAwMBgBpjN0cDAwGAGmM3RwMDAYAaYzdHAwMBgBpjN0cDAwGAGmM3RwMDAYAaYzdHAwMBgBsyqfTAQCKRLSkrgcDjUXzu9/SaVSuVMsAasqSN8H/tG2TZnf73T6cy5Pt+fTqdzDtlxu91/t3ea33V0dBSTk5N5Ow7c7/enw+Ew0um0WqkoR8rK4XCoR5rySSQSObIlvF5vVg82f7LflK1dbLfyer1ZU2Hs38H+evuo+YKCAgwODmJ8fDxvZVtYWJgOhUIoKCiQnlB3+DOdTqv9jDJ2u92SDVs62e6WTCYlG74vmUyqXY2vZ6tcLBbTZ/P6TqdTujx9IrXL5dL129vb+/O5CLywsDDNqUfTdZf35XK5JEu+ZmJiQrLj69gyGwwGJUO2Eo+NjaltkO9j22VBQUHOlHePx5NzJIi9rjudTmN4eBiRSGRG3Z3V5lhSUoKvfOUrcDqdGkrJXkd7jyMVhAstnU6rZ5TDQtmDGovF1BPMXt/CwkKdeULBUCljsZia2fnZZWVl6p2037j9WrfffvtsbvVfjnA4jPPOOw/pdFr9pezLZZ+z0+mUEnHU/tDQkIausm+YijBv3jwNHOawjgULFuCBBx4AYI2Z45it5uZmnHbaaQCsHl/7yY07duwAYI2J2rp1K5qamvCjH/3onymKfzpCoRDOPPNMNDQ0aKwVdY0LMBqNavPnz8rKSp1FxIETr7zyCoDMbIDpwxLGxsY0oownaa5fvx5ARu/ZK8znU1BQIF3mwAluHEVFRXr+//t//++87j4pLi7G2WefnTXsloNfKO/i4mKdUEr9XrdunWTNYSkchXjwwQdjzZo1AIATTjgBQKYn/fe//z0ASz+3bdsGIDO6j/MaOLCivLxcQ0GmjyxzuVyYmprCr3/96797X7MePDE8PIx0Op1zcA136JnOa+ns7NSD5k8KraCgQNfg5J6BgQFtsJymwYnK7e3tYkj2ST/TD+/mz0QigZaWlrw/DCqVSmFqagp1dXXa2Kk4vLdUKqW/kZFMTExoo6Th+fSnPw0gMxfziSeeAGBNWB4cHNTruHC/+tWvAgCuu+46TTo68cQT9d246ZJ9cmJ7R0cH+vr68n7iUUFBARoaGtDW1qZhDvYDyoDMYqbOcPN3u9048sgjAVjnnvDMpFQqpQng3GB37dqFFStWAMicjQRYm159fb3kx7VSWFgoPZ9+TlB/f7/knu9Ip9NIJpMoKSmR7GgAeH+JREIzMTlp67zzzsNvf/tbAMCf//xnANDZUfvuu6+MLnXx0EMP1VAQyvwrX/kKgMygGU5OJ7GqrKzUXsQ1xJ9FRUVZ62kmzHpKaTqdRmFhoZSKGxS/xPj4uKwGGaHL5dL0k+nnBcfjcbFILlr7QUi0JHbaTQZDxlNeXq7rU4ntE21GRkbyfgEDmQ198+bNuhcePUE3eMOGDVi5ciUAazrMpk2bpIgcCHzjjTcCyEyL4XSiu+66CwDw+c9/Xs+ALIisurGxUb+jHJ1Op0Zz8XP4vZqamrB9+/a8HyQMQEOEpx/1wMWbTCa1qDi5ZdGiRVrsHMlH3Vu9erV+R2b/0ksv6e9kMdwIVq9ejWuvvRaANfotHA7r2dILokEsLS3Vd8x3uN1ulJSUIBqNysNjKIj/j0ajOhiPZOib3/wmfvOb3wCwPD3uI+vWrZPR5qFlRx99tP5+4YUXAoCuOTw8rD2Fz+/ZZ5/V9+BPPs+uri40NDTsdlCzScgYGBgYzIAPNN/e7XbLItjHxgMZCzh9IOvcuXNlIafPYSsvL5eLSLZSVVUl2szzqhkzmzt3rtgqWU5DQ4OsONkNYz9DQ0Nwu91ZiYV8RCqVwtjYGHw+n+RBRsL5f4AVXiD7mzNnDg4//HAA1tnKZJB+vx8f/ehHAWRmLwIZdsPXc0YmrXYymdRBZnQ/k8kk7r33XgAWY/zxj38MIMOmYrFY3jNHh8MBp9OJ+vp6Beupv3Sv29vbxdp4QNzvfvc7MToyDnpMmzdvzklalZeX67nwWhs2bACQYaqMQz755JMAkHU42fRx/RMTE/pu+Y5kMonx8XF4PB57EgmANfB6aGhIIYnvfve7AIBXX31VniE9Iq7f6upqXH/99Vmf8/rrr0u3KXvOkfX7/TjllFMAQD/Xrl0ruXLfYZguHo9j27Ztuz1WOL93DAMDA4N/E2bFHJ1OJwoLCxGNRmUFGHAmM1y0aJHiOIz7vfTSS4q9MGbA4KnT6RRLZFxg27Zt+h0TKYwNBAIBsVWOon/11VdzJgSThXZ1dcHhcOR9zJHHJExOTipZRfnZ46i8D8YSL7vsMrGZ6Yzwvvvu0xh5Wubjjz9ez4y/Iys68sgjNZWd8bfFixcr47dx40YAVnJnyZIlcLvdYkv5jpGRkZzYFpMe4XAYhxxyCAArHj45OSldnp7we++99/QsyA7D4bAYKZ8dY5yvvvqqMtN8Pq+88orWAd9n1/vdHf6UT3A6nbrf6cdm2BMh9Dw4/Xz16tW6x4suuggAdBDfypUrlYugDgeDwSw9BqyKmE2bNunkAXpen/zkJ3UU9LJlywBYR48UFRWhoqLCxBwNDAwMZotZMUceCBSJRFRfRMtHq9jW1qYDs2iVHQ5HTmqfMYlt27YpXslDuUOhkHZ0Zm5ZrlJTU6M6JrKbRYsWKTtI8HOKioqQSCTyPubodDoRDAZRVVWFtWvXArDuwc7MKFNm6zZu3JhVaAtAccOjjjpKNY28/9bWVh0Nes899wCwMreDg4OKNTJmu23bNn0P1vqRJcydO1eHReUzWIKWSCTkYfAn43+NjY1i1DxUq7u7W8yEDIcsvbq6WtcgOxwYGBC7pqdE9vPcc8/hscceA2AdyNXX16eKDp7JRDY6Pj6e92fzEDzIzePxaL1yjVKXDzroIDG7Cy64AECGyX/9618HYMXXqVvvv/8+/vu//xuAFVdsbW1VqRnLqOjBlpWV4YorrgCQYelApuSMTJPPg/XQAwMDen5/D7PaHFOpFCYmJhAKhbJqtew3FYlEsGnTJgAWHXa73XLd6H53d3cDyLjSvAZdi8WLFyvgynIde40YN1ZuzA6HQxslHwqp9aJFi/D+++/vEZtjIBDA9u3btUhXr14NwEpsbdq0CZ/85CcBZJIFQMa1pQJwYdKFqKqq0uv5TCoqKvRvyp2b49atWyVvLtY1a9bIradBpJJXVFRg3rx5eV9DCljGh24e9ZV6FgqFdB+Uy4IFC/Q6Gg3qnL1Gjgtu4cKFKjvhtbipMvEAWEZszZo1SrbxGXBzLC0t1RrJd7DLzev1KjTGchoWcNfV1Sm5SgPv9Xp1GB8Tqky8nnbaaVrDrB2dmJjAL37xCwAWAaDxWbZsmfSa7wsEAjj55JMBWBsmn1VlZSUGBgZ2G27L7x3DwMDA4N+EWTHHdDqNWCwGn88nS8cAJylqKBTKKe/xer1yR8gA6UJ3d3crOE4rEolEcs7sJeN0u90K+jIpEY1GRZvJsvi99t57b4yOjub90awulwvsT6U1I9sl+1i1apXYysKFCwFkGAnv7YwzzgBgJRR27twp1/nxxx8HkHGX58yZA8Aq4SHD37hxI84///ysz3S5XCoH4jNnOcQdd9yBOXPm5D0rJ7Ox6xTvhe7cgQceqBY2ejDpdFrMkUzwb3/7G4DsRCKxdetWsc7zzjsPQPb52E899RQAK+myYMECMXSGkfj+jRs35r1c7aCMue64pr/85S8DyIQVuL4Z6rniiitU8kRvhOfOt7a24umnnwZgJV6DwSDOOussANYRzAyLvPDCC7jzzjsBWM901apVKstikpPJRLZn7i7ptedI38DAwOBfiFmX8nDH5Q7Mgmxa2JGREbE3xlbeeOMNWUhaRrKbqqoqxQ3ISNatW4ctW7YAsFoQeXB6Op1WuYm9wJP/pvVnjGhoaGiP6K12Op3w+Xxobm7WvTMZwDjJ3LlzVfjOJMno6KgKwyl3xnTmzZsnOfNvZWVlsqZMctl7XhnvZLxr7733lnXmM2Qf7F577YVwOLzbcoh8QDKZxOjoKAYHB6Vr05MdZWVlkgPvvaGhQSyb76PMWlpa9Dp7svDMM88EYOkri+9/9atfqW+Yf0smk2hrawNgHVx/6623Ash4Pqeffvo/RwAfMtxuNyoqKpBIJOQZMkFLmZaWlsojOuCAAwBkGCF7q7n2GZvdtWuXvFEy+bffflsyYd6BrJK5ECCjs0DGo+Ta4TNiAqevrw8ej+ef11udTqcRj8fR09OjRUQl482999572jBJb91utxYn3WtukpWVldoEXnzxRQAZt5q/Y3CVQu7v75fgmaGeP3++rj89sN3S0oLR0dG83xw9Hg9qamo06QaA3Nm//vWvAIBjjz1WMmKtXDweVzCbi5Wb39tvv63GfCbEqqqqVAtKmdEA7b333nIDP/vZzwLIBLIZsmCygYoWiUTw+OOP6/35ChqeSCQivaNe8Z4AKzlj779mAoe97FxcTU1Nej117vDDD1cigiEfynPnzp3KsNpDPAxB0cCwkqCyslKEI9/hcrkQDocxNjYmt5qJPQ4p6ezsxMc+9jEAVi9/fX29qi5ogJlUaWxslOxvvvlm/Y2bHTdMbo4HH3xwTqK2paVFz4YhCj7HJUuWZA0bmQnGrTYwMDCYAbNijpy+AVguLa0hy0gCgYB2bpaA1NbWislx52eCZevWrXILaXWrqqrEFMn4yHY49QSw2JO9Pm/6JJOqqiqxpnxGKpXC5OQkPB6P2AmtLxleX1+fmA6t6jPPPIPjjjsOgOWS8PV+v19ypowrKirkrjA8wX7Vnp4euXxkhwcccICeHQPkZOxNTU1YtWqVZuzlKzgOrqSkRN+dLhhDGG63W7KijJPJpORw0kknAbAYS11dHW644QYAVvijo6NDsqf3RP2dM2eO5E72+s4776jTgyPOyBzD4XDeM3LC4XDA4/GguLhYa5geIuXn9XrVW86SsLa2tpwwB0Nxzc3NYoLz588HkNkzWOrDv/FZDQ8Pq2ztmmuuAZDRU+4Hzz//PADLG/P7/Whtbd1totYwRwMDA4MZMOtht9OnbzA+Rmtgt8BMn7tcLjFGpu5pbVtbW5X2p1V46aWXZEmmDwE9/PDDxR4ZnwiHw7o+S4Q4QfzVV1/F8PCwPjdf4Xa7UVpain333VclH2QhjFu98sorilGRfdi7EliSw4TM0qVL9SzIkKLRqAqP+Qw4FXvt2rUqEeJn2/uRyZrIvt5++21EIpG8nztI2brdbjETypHySSQS6vahN1RRUaHgPhNgLE/btGkTzjnnHADWs+jq6pJOcpAwdbqurk46zSSN2+1WeREZIyePf+ELX5D3lO9IJBLo7u5GS0uLdIO6RT3asmWL4pHsQHrppZdw+eWXA7Bi4vQ633rrLZX8XHbZZQAy5WvMN7DxgQmtUCik2Dz1FABee+01AFaClrru9XoRDodzjnmxwzBHAwMDgxkw6xoM+uhkZoxH2efiMfbF+IP9YCOWotiLjJk55DzCXbt2KTawbt06AFaMyD7Jg3GdQCCQ9W/AismNjY2hu7s75wCjfAMnm5SWlopJky0z+9zU1JRzwFZtba1YClukWDlQW1urSckssejq6lJMkzJjbHj58uUaSU/29J3vfEfWljEzxnkGBwdzjsTIRzAmNjU1JVZBhsw4lcPhUFyWGdDNmzfnlIhRR6uqqsQAGUv73Oc+p9H+jKVR7qtXrxajZ1WBw+GQ7Lk+2DLq8XjyXmcJl8uF0tJSpNPpnDkJzEVEo1H1ltPTuOSSSxSbpE4yDr5+/Xp8/vOfB2A1KwwODio2+81vfhOA1Sprn0LOn7W1tfK6eN2jjz4aQGZP8ng8u9XfWfdWRyIRVFVV5QScucGtXr1aD5w02p4uZ3kCXYbBwUG5HjMNWOCmyL8FAgENQKArf8ABB8jtmX7qGUcl5fsijsfj6O7uRkdHh747QwU0SH19feop5Tkxq1atUrDZHnoAMkNF2Unwwx/+EABw5pln6qAjunT2EXNUIoZE6urqZOToDvH1w8PDOPnkk/X/fEUqlcL4+DgikYg2+ukHbG3atEmjtOi6fe1rX8PDDz+cdS0mAQcGBnQAFDfJ7373u3pWlC1d9XA4rI2A9aKdnZ1Kpr311lsArJFdhx56qAhIvsPhcMDtdmPr1q3SWe4Pdt2hAeDwiK997Wva7Oh+U96f+tSnFGLgBrhhwwbVSDJ8QfKUTqe137CMqrCwUM/GfpAakNmvysrKzMgyAwMDg9li1h0ywWAQk5OTKq2hG0tLHI1GlTBhAWhxcbEGXdLl4y4fjUYVFGXSYO7cuSoyn84cn3rqKZx66qkArP7NrVu36vPZv0kG+dprr6G3tzfvXZSJiQls2LABa9asUWKFfbx0Q9avX6/SCBbN7tixQ0kDWkG6Feeff77eyw6MX//615L3scceC8Bi8S0tLUoacNSU0+mUq0RGytBIJBLBunXr9Pd8RTweR29vL/bZZx+Vx9DdYvhg7dq1SmzRte3s7FTIgmEMJhomJiZUTkK524/8mH6M7dKlS5UUIEusra0VO+IzITN9//33Vdyf7+DIsrGxMSVUOJaN48YqKip0byxf2rZtm6Zo/eUvfwFg3X91dbVOfiSjf+KJJ3RdPj/uCw6HQ3sFWWtlZaVccrr3TOBEIhEsXLjQFIEbGBgYzBazLgIvLy9HPB4XO6H1ZIwqnU6rLIH+fnt7u+JSZJy0GJs3b9YsRvvB5gxoM3ZDhlpWVqa+brLBiYmJrBgjYBU9Dw4OIpFI5P0hUC6XC6FQCCMjI2InZHa0dn6/XzEtMuTrr79eMV7KgFb44IMPVryGrZzXX3+9mBGtKWOWgUAgpyfY7/crRkarTXZ51VVXwe/37/aQonwA2wcHBwcV96P+kakceeSRYuDUw61bt4pZUkb2PnTGz/n6DRs2KOBPRmIv/WHZEOOQFRUVirdTp8n03333XbGwfMf4+DheeOEFNDc3S6fYWMB7PeaYYyRLHkexadMmMW0yQQ7K/uEPf6jXk0GfeuqpOt6Wsue8xoceeihrehWQeVa8Hsvj6MGuWLECr7/++m69nlltjlNTU9iyZQtisZhcBLoNdHGZWAAsBZmYmFD2iguZoLsHWBkuu6vNm2Om1D7Sy+56c8HydwSVLN83x4KCAtTW1uKZZ55ROIIPkgYlnU5rU+QJa1NTUxoEwY2TStLW1qYs4He+8x0AmQPSucC5UXDqd1tbm9x2bhRLlizBn/70JwDA2WefDcAyiG+//TYuueQS3H333f88QXwICIfDOOmkk9DR0SH94M9vf/vbADKZfuqVvZaULvD0oayVlZUK7vNa++yzj54B6/AYKnrhhRe0cdLwX3PNNdowafyZwOjt7VVGPd9RWFiI/fffH08++aTWLZN31KfBwUGNFKMBnpiY0P7Bvmu+7+Mf/7iy29xwzznnHCWpSCBIuuLxuNx1bsivvvpq1rnkgFUVsHPnTgwMDJgOGQMDA4PZYlbMsaCgAA0NDYjH43Kl6J7Y/28f8AlkrCzdBVpb7ujLly/XgEt7NwKDq7w+U/fRaFSslWw1lUqp9IcgJZ+amkIwGNxtJXw+YHBwEHfddRcKCgrk9tqn6wAZ9sEEAcMGBQUFKj1houS0004DkCnDYe2jXe72/lX7zyeeeEJuHpNBr7/+un7HU/Y4OPeEE05AW1tb3rvVY2NjeO6559Da2qohtHTLyDbmzZsnhk79eu655xTSYAKRsnj99df1N3v5DUeccZIMWdLSpUuVSPzlL38JIBOmmF7Pa6+jJHPKd3BQ87Jly+QJct1SXh6PB3fccQcAYPv27QAyw5npLVImrCdtaGjQYFvKORgMijFSzmR+8+fPz+qWATKyp27S8+S+EwqFUF5erqMbZoJhjgYGBgYz4ANN5RkcHJRl4E5PX350dFSsg4H8SCSiOA7jaCyb6OnpUdKA8YS9995b6Xh7ES2QYZCMZU5P8vA72n9WV1ejpqYm788A5pngxcXFsoaMo7Js57777lM3Bi1tbW2tngFnN5JFU9aA1bO6fft2WV9a2Ntuuw1ARu7sdGKS7IADDtD3Yfzt3nvv1ev333//vI/nxmIxtLa2oqWlRbpJfaUMGhoaFA9nWdhHP/pRJRr5O8YXBwYGlCxkR1hvb69kxQQY2czY2JgG2ZLFDw0NiTGyxIgF+iMjI3rW+Q7OeY1EIopHs8SGurj33ntrvTJ27XA4VPTNyTuMt69cuVJ/Y9JsZGRE8XHqKZl2R0eH4rxM0lx33XWKq1PnuS+kUin09fWZA7YMDAwMZosPNJUnGo2K+TGWyDKP8fFx7erMWsdiMRV/8/W0wFVVVWIeZIRbt27V65jZY8rd4XAo/kNmVV1dLcvLNiRieHgYW7dulQXJVwQCARx88MF48sknZTEZF2OM8IwzzhBL5P2Ojo4qhjN9OvucOXMUj2S2NRqNyprbnwGQyeAxk33YYYcByMQtGWvkMz7qqKP0fdrb2/P+fGWW8rz99tuKAdIzsR+1wUwx2duqVavEGFm6RlmVl5eLjTCWGI/Hpa98ZnbmQpbDa9bW1orJ8PnwWS5btkxHkjImn6+Ix+Po7OxEeXm5DnRjzJ8e5vz58+UBcS4o44aA1evOvWPz5s3yNhm3HB4eVnyXR07w6NeXX35ZzJHMPBqNKgvOg77Y3shyn91h1m51OBxGZ2enAv0sRaBru3z58qxD34FMsoGLjl+cQisuLpbLTFeusbFRC440nS5Pe3u7NkAGf7dv367NgsrL+im/34/BwcG8P33Q6XSioKAAn/3sZ2VUeCQCww69vb2S9x//+EcAwOWXX66FxUQMEyxbt26V28YETnV1tX5HN+fQQw8FANx4443aNO677z4AmYQFXUOCC7qnpyfve9aBjIu3cOFCjI6OaqPnaK2LLroIQGbRcGNiQuahhx7S7AC6idTD4eFhbZiUQTQalVGnbKnv77zzjp4BXce5c+fq3yQGNEoHHnhg3o+CI9g59+STT+p+KUP+LC8vxxVXXAHACjmMjo4q0crnYg+LcS3TqJSVlankicNPeP0VK1aIOFBfly9frufG2ms+91gshoqKCjOyzMDAwGC2mHUR+Pvvv591Xi93elqMdevWybVgSU9tba2sMuk2XbvnnnsuxxoMDAzIYpNFkV02NDTIGtAlKioqkqs93bVkEinfT8iLRqNobW3F1NSUgsQs26ELcOCBB6po9uKLLwaQ6YLhdCIO+/zUpz4FIOOq0erSQg8MDIjBMAFG5rN06VK5zpT3u+++K9eQz4TMNBgMYsmSJbjxxhv/WWL4UMDe6tHRUU114T0xpNPd3a1+aOryjh07cg56I3MsKSmRB0Ovyev1ytOhfnOClP3ZkZ1PTk5KX8lg7INzyYTyHRMTE3jttdcQi8XkNVKHKbeJiQl1xnAA7Zw5c5QopE6y9MZ+ZjjLrcbGxuSaM1lFV/q2226T3rPwfnR0VFOPGE4hQ33iiSdwwAEH6PvNBMMcDQwMDGbArOhUIpFAT08PlixZknXEIWAFrIuLi2X92DpUUFCQdZYsYDHOJUuWKO7CeFpbW5tYJF9nP26Rltp+kBdjmLTE9qGwY2NjeZ804Ey8yclJsTdaQvuMRTIXMvGDDjpI1pZxXf7s6elRfJHB74suukhTTqa3ejU1NenYTMZyGhsbVc4y/bCzzs5OXSuf4fF4UFFRAZ/PJ30iw2MPcGVlpfSPycbh4WHpDe99pgk8ZNJDQ0OSG5k3e9RbW1ul51wDXq8355x1sstNmzbtMb3VLpcrpzibHiLX41tvvSU95e/i8Th+9atfAbDKb+jN3H333ZIh9a27u1vHBLOsjN5VcXGxPCCujZUrV2pfosdARs/Y4+7wgXzNqakpKQkDyfbMKv/NLzJ//nw9fCZOmEzp7OyUe0xh1NbWSiB0QVhJv2vXLmWxmPXr6OjIGV7K6yeTyaw6yHzF1NQUNm/ejFNPPVXUnwpGl/jAAw9U7SiVo7y8XArApBM3002bNmnxc+Hff//9ej5MgNndyOnVAaOjo/odnxMnZXd2duKGG25QBjBfkUgk0N/fj2AwKFlRHnSvh4aGJA9OQ+/q6pKrzZ5qGuFkMqmNj5tYXV2dZE9QZrW1tboWM7P283nsrwOsDPueAIfDAafTCbfbnTPIgQM1wuGw1jLXeSKRUDaZ5IkynTt3rkgCZb5x40YlqU4//XQAwB/+8AcAmeQLZU9ZXn/99ar1pc6znpLn95iEjIGBgcEs8YFGlpWUlIimMt3O9PnQ0JDoM93qXbt2yWKT2TFZYx8jRXfmvffe047O19PS+/1+WRf+jr2dgDVRhRaMpyXme8lJMpnEyMgIXnvtNdUR0sKS/b3++uu48sors95XXV0t5kb3l2GNs846Sx0hvH+PxyP2SQvL0opdu3blMPtQKKRkl30MHJAp99l7773zvkwqnU4jmUzC5/OJ5VHnyCh4pghg3Z/f71edKMNHZNEjIyNKJPJ39fX1Ykq8Ft3kww47TAkzXj+ZTMrjIeiRORyOPaaUx+FwoLCwEM3NzfIkKRv7sRH0/tgz/uCDD4rJsbaUrPruu+/WRC6yv0Qioa4kTuph90wikRC7//jHPw4go9csOWS5G2VaUVGBZDJpmKOBgYHBbDEr5uhwOODz+TA0NKR4CAPVtKylpaVikYzdTE5OygowfkCL6nQ6Fdhm7IsBbvt1+frR0VFZAVrg6upqWSz+juypoKAAyWQy75kjZ+I5HA4dUcASBh6m9cgjj4jpMFb11FNP5cRUKeMNGzaIXTPetWjRIlldJiWIwcFBnWnNgbl33323yqkYF2IsecOGDXoW+QyePtjT0yNGQzbC779jxw7pGFmMfSbAdCaUTqflIVHn7OyTf6M3tHHjRpWT2M9Spp6SyTIBZD8bPt/BcrlUKiV5sgGEjQz19fXSa87/PPPMM7Uu6fnRw1yzZo30jP3Ub775puTKuDeTj21tbfIyn3jiCQAZ+bKUis+BJUZbt27F0NCQ6a02MDAwmC0+UPugy+VS3I/xLmZWASsLSmtYV1en4xLJfFiqMzw8LHbD4s1QKKQdnSyHTCmZTMrCM63f3d0thknLzs+OxWJobGzM+6k8QOa79/f3q3CVsT5mq51Op+K4bDPr7OzUQUTsWeUzsbf3kenvt99+ugbjMczyxeNxxR+vueYafSfGKBmfYdbxnnvuQWdnp46LzVewTMrn80kPGLuiHu7YsUNsknHGBQsWKI7LZ8GYbEFBgcpX+Jp4PC4ZkdEwXjw4OKiyHZZOLViwQNlaPhPGIDs7O1Xhke8gMy8qKhLDps5S71566SV5iJTh3nvvrXjt9BK1fffdVzK85ZZb9HrmOliiRnn7/X6xVH7mnXfeiXPPPReAlRNhKy7Lj3YXc5zV5hiPx9HV1YVgMKhAKuuFuMF1d3drQ+NG1d/fL+WafhpYR0eHXHN2HPT39+tLM7FC2l1XV6egNTfhhQsXyv3hA+CmwFKXPaHOsaCgAKFQSA+eC5cPPRAIyF2hkZmcnFTShe40FWjZsmWSMxfw9ddfr9pEuig8VTASiWgB8+fixYvltnPTZTJt06ZNWef35CuSySTGxsYQCoVkWKkPHGTg8Xg0XJhyLCgo0L1R3/k+uxtn7yfmIuc1qL/19fVaD+xlj8Vi2gxpzBn+cLlcum6+gwlR+8AYhsYooxUrVkg23DN++9vfalAFDS6TKf/zP/+DT3ziEwCAL33pSwAyo/s4U4DDWUiKTj31VA0yZrhor732kptOQ8aOsZKSEhQUFOx23J5xqw0MDAxmgGM2g0odDkcfgNYP7+t8qGhKp9MV/+4v8fdgZPvhYQ+XLWDk+2Hi78p2VpujgYGBwf8rMG61gYGBwQwwm6OBgYHBDDCbo4GBgcEMMJujgYGBwQwwm6OBgYHBDDCbo4GBgcEMMJujgYGBwQyYVftgIBBIh8NhOBwOtUKxV9V+/AFb9+znUbNVjZNG+DeXy6VWQfuEjOlnWdtfP31+oMvlypm6w2ul02l4PB4MDg5ifHw8b0fzFBYWpkOhEJxOp2TK+9xd6+NM924/KnT63+wHjdn71af/bqb3UKZ8Jh6PB7FYDKOjo5iYmMhb2fr9/nQoFFIPMGDdJ9sD0+m0ZGWX2XRdtuuh/RA3/o6vp0zt8ubvKMdEIiHd5/ewz93kZ3V1dfXncxG43+9Ph8NhJJPJnIPs7HKjvNj77PP5cqbIc0ZDW1ubWv8413RycjJrJgNg9Wk3NjZqj+E1fT6fPoufbd9PHA7HbveFWW2OZWVl+OpXv6oRVkD2CCcg0x/N/lV7Yz7P8aUysD/X5/NlDYkAMuOO2M/L97E32P56Nq1PTU1JIOzRpBBSqRTGx8fVvJ6vCIVCOPPMM3VPgGVwqABOp1My5eCEzs5OKST7eXnvw8PDWojs8W1qatLvuFHwWQSDQSkRfzocDj0Lfib73IeHhxGJRHDbbbf9s8TwoaCsrAyXXXZZ1gBZ9uBzUILD4dCILA7EZT82YD0Dvm9gYEC6z8HAS5Ys0TqgzHj9efPmad2wf7qrq0vPk8+MMq6oqNAQhp/97Gd53X1C3bVvgFyH3MTC4TC+//3vA4AGq8yfPx/f+973AACvvPIKAOtMmDfeeEN90xwa0dXVhT/96U8AgJtuuglApqcayMxQWLFiBQDL0AwODmp4LmVu31yrq6tx7bXX/t37mtXmmEqlEI1GkUgkpDR8+FzITqdTi5SCamlpkXJNHzLhdrv1Ny7S7du355yfwQk8U1NTUiQe1FNZWanrcT4eFb24uBjBYHCPmI2XTqfR09OjxUYjwM2vpqZGTfVUAI/HI1lxuAefRX19vWTERedwOCRvbnJ2eXKgBZ9dcXGxpr3TAPE71NfXIxaL5f2sTB4MFwgEsiZTA5bRiMfjWUaFoAGhPOyTxKlznLzT0NAgI05ZcTPt6OjIYYfV1dUy/jSKnF41Ojoqnc93OJ1O+P1+hEIh6R6ncnODu/XWWzX1yH7s8jnnnAMAuOKKKwBYrHpqagrf/OY3AVjDK26++WZN++HEogcffBBA5qC5mpqanO/2uc99DoC1Nrhp19bWYmpqygyeMDAwMJgtPtDIsnQ6nXNoN8cRlZaWyjLQOnd1dYmRcBSW/ewOe8wGyJxLw2twphstflVVlay4/QwZfg+yVrorhYWF6OzszPtzTtxuN8rKyhAMBnUOCVkKv/v4+HjWoedARo6UG2XG13R0dMia8ppDQ0O6HhkMGY39xDvKtr29XQzGPl2d16qrq9P/8xUulwuhUAjpdFr3Svfa7slQp6lf9fX10mvOeKSuLliwQHLk2LH+/n48+uijALLjZECGzVNOHFn22muv6TlylinXRzAY3GOOZk2lUpicnERnZ6dG6tF9pT799Kc/lZf59a9/HUDmXrleeWwtx5lNTk6KVX/jG98AkHkGvMZdd90FwAp3PPTQQ7ouZdrc3KxZpJQvmX1tbS36+/t3uy8Y5mhgYGAwA2bFHF0uF4qKirKCxTzEmxa1v79fMTN7jIF/p1VmrCcUCmXt5kCGIdGC0LLTqg8NDSk2SUYzNjameBHZ0PQ4Wr7HHJ1OJ4LBIGKxmBgDs3W8d5/Pp7MwKINAIKC44vTMfn19fRYTATLPkAxweibP6/WKddrjnRwqPP386u7ubkxMTOT9IOFEIoGhoSGUl5frnikj/j+RSMibYYxrampK+sSkAOOMtbW10m/KZ/HixWKkjOdS/nPnzpXO84yaffbZR4yeDNUe6+X3yHfQkywsLNR35plRnHrudrsVh+U07o6ODtx8880AoInoHNT84osv4otf/CIA4De/+Y0+6+yzzwZgeZ48EbOurk6v47WGh4e1Jjg49xe/+IW+czQaNTFHAwMDg9li1mfIlJaWZmXSyCDJRioqKhSvsv+OOzR/RybZ19cnZkT/v7CwUOyElpoWeWxsLCc76/F4cmolWUoRCoVQVFS027Mi8gmMoQDZFQAAVFMIWCzb6XQqHkYGaa9f5JkwRCKREFvhs+D7amtrJXfGzBKJhFgkZUr5l5SUYHJycrfWNx/g8XhQXV2NqampnCoIe8kX9ZaVDl6vV5l6yoDZ14mJCekaT8fbvHkzVq5cCcA6Q4VHL9jP56FOL1iwQN+H1yBDdblc8hzyHR6PBzU1NZiYmFAMkTpB3Vy0aJFKmf7whz8AyGSYKRPeK5lmJBKR7Bk3v/DCC3WyINf+l7/8ZQCZ84y4pzz00EMAgB//+Mf4yU9+AsDKRbD0Z/v27WhpaZFnOhNmtTny4PlEIqHANm+O5R0dHR3aOPll29vblSChYvBLpVIpbVx215guCDcDCsrv9+t3vOG+vj65MXTR+TnJZDKrjCJfwaM+/X6/DAE3Km6EExMTOUX3PT09kg3dcS5yp9Opa9E97O7uViiEmx0VeHh4WAaKi9bj8WgTnX6MKJCRc76HLFKpFCKRCHw+n/SO8rA3HtAgUJ6JREJuLt04JgT6+/t1VC3lUldXJz1jkoYlJ+Pj4zrrh5/95ptv6lB61vfZD05jKGRPQDKZRCgUUhG3/YhZIHN/3ORY+lRTU4N58+YBsNYtn09XVxcOOuggAFZyNZ1Oa4N94YUXAEAJsFNOOUXfhQYKsIwTnxE/b3BwENFodLchofzWagMDA4N/E2bNHMfGxuD1erWbk/WRcTQ2NuYcFh8KhRRwpuUljZ6amlJhJq1GW1tbVpG4/XMAy7Iz2B0Oh3U9Mir+DAaDe0RChgiFQkqKEPZuIrJmWsKamhqxFXtiBcg8E5aP2NkkXzf9iNtIJKLXk7WMjIzovdMLxHn9fC+TSqVSmJiYgMPhkCyZ8LOXJ1HnmDjwer3SZeoo9XB8fFw698ADDwDIHERPVvT8888DsHT08MMPF2M/9thjAWTK1MhWuZ6OP/54AMDFF1+MpUuX/vOE8CEjnU6jqKgIGzZsAGDJiWy5trZWp14ec8wxADJHCVP3WMpDl7ikpATPPvssACsR2NjYKE/1ox/9KAAr6XvllVfixhtvBAAcfPDB+l48QprF4j//+c8BZMIjf/nLX3brUe4ZO4aBgYHBvxizYo5OpxOFhYWorKxU0JosgnGxtrY2xVu4K09NTSn2wDghY2FTU1N6HeNcXq9XzGV6O2BBQYHiRPzMsbExxRx5LTKgaDSKYDCY9wkZO7thOQSD2oxRDQwM6D7tQxLIhhg7JKOZnJwUM6H8S0tLxWbscUX+jayJSZuamhrFGMm+KePKykokEom8Z+UcODEwMKC4F3+SZdtLlsiQx8bGpEeM59KTWbRokc4T5xnNgCU39g8/+eSTADIsnR4PkzVlZWWKtbGEhSx92bJlec/I7XA6nYhEItJPJvl4zy0tLWroICN88MEH1Zd/9dVXA7DWO2DpM9d5NBrFrl27AFj7x6c//WkAwHvvvYc333wTgNVH/fDDD+u8ca4ltjXOmzcPhYWFu9XdWWerKysrMTo6qk2OykA3NhwOazHRFRkZGcnpvKDraO8X5iJPJBL6N7NS9jo9KjTfV15enpNIYELH4/FgfHw87xXN4XBo0gsz1tO/c1VVle6ZD7u6uloKyUwnN0T77+xJHW6+DHXwWolEQobHntyhTGlg6Nq0tbXB5/PlfZ1jMpnE+Pg45s6dq+oK3gPda8BKNNFA2Ht1uSj50+/3S+6rV68GkFl4NEyvvvoqAGjwQUFBgQw86/Cef/55bawcvEB9Hx0d1WLPd6RSKUxNTcHtdsvw7rXXXgCAv/71rwAySRJ2ulx88cUAMmuU+whDCHSvn3zyST0rGq0XX3xRiZdt27YByNSKAsAXv/hF9VGvXbsWQEbnuY/wGlwbTz/9NGpqakQMZkJ+m3wDAwODfxNmPZWHLIwWd3od3cjIiBgGqTUTOYDVfcAde2hoSL+j1Q0EAnLNyWrINCORiFgqX9PX16eSCVpeWoiuri7MmzcvZ85cvoEhi9HRUfWlkiXaazjpajCZYmeXLKOgrNjXC1hhicbGRsmUzJFyT6VSWckcfjZZOAPj9nmE9vmd+QrW58bjccmPoAvtcDjEHHl/XV1dkvP0e49EIjjyyCMBQAwvFotpPbBMigmBVCqFRYsWAbC8LI/Ho26Z6bWiW7ZskU7nOxwOBxwOB+bPn69198wzzwCw7uvGG2/EueeeC8CSpX0uLHugWaJzxhln4Nvf/jYAS48vvPBCucl0hxna2G+//fRsKbempiZ5qmSo9nGLU1NTppTHwMDAYLb4QHQqlUrJQtinaAAZxsGdm4F7eyKB5RKMq9lT6WSCXq9X1pUBa/5tZGREJRG0HuFwWBaK1p9sqLa2FkNDQ1nFvvkITowZHR3VPVCOvLdoNJozGcdeGMvYIOUyNTWVJTcgu9+UlpbWOxAIiMXbk2SUHWVK1l9RUYHe3t68ly09nampKcltegna8PBwTpIxEomo/IR6y5haIBDImW+5zz776FmcdNJJAKDSlp6enqz1AAAnnniiYmac40im2dbWJraT70gkEujv78eWLVvwyU9+EoAl3/POOw8A8Nhjj0l3KcOzzz47a0g2YHkz5557rgq4qcOPPPKIrksd/OxnPwsgk69gORRlHwqFcMIJJwCwyq24tsrLy7Mmw88EwxwNDAwMZsCsmCOnb4RCIcWhmFKnlYtEImKRtKzFxcWy0IwJ2mNl0wuVU6mUMt36ov83ZlhQUCBrwwxUbW2tPpNTa2iduru7UVhYmPf9v/F4HH19faipqRFDY2aUmeP+/n6xGpZIDA4O6t6mT7e2HylBRhiLxSRLWmm+r6SkRDLlNe1nczB7y+8wMDCA5ubm3fan5gNYCVBWVqbYN+VC1ud0OrPi1EDGg2HZDdk2i7x9Ph8efvhhAFAcrLm5WZ4O4+2U9fHHH69SFs5zHB0d1WdyPbDHenR0dI9qH3S73SgqKsLdd98NwGr24P87OjqUub/jjjsAZIrBf/rTnwKwZMjY4/r163HIIYcAsJoUrr/+enmgLH268sorAWRKczhV/P777weQWfvvvfceAIuR33777QAypVhz587drdcz64TMxMRE1oLklyU9tbu9XET2/l9uYvxSZWVlospc1EVFRVIqLjwqXV1dnRSbZRDt7e36HnQfGfSdmJhAb29v3pebuFwuFBcXY3h4OKdjiOCBVoCVdPH7/TnBZhqsqqqqnDBDOBxWkoHJCP6ts7NTn81NpLOzU+739ASR0+nMchfzGQ6HAzt27NA9U195Lzt37pQR4kIqKCjIqbWjG/z9739ffdG8f5fLpd5oPh/W3LH3GLDK3+xjz1gPyWdXW1urxFy+o6CgAA0NDWhvb9eGzk2fPePz5s2TAeAz2L59uzY5GqE1a9YAAG677Tbce++9ACxDfcABB+i5UfZPPfUUAOCGG27Qv6mnd911F37wgx8AsJ4fv0MsFkNVVZVxqw0MDAxmi1kPuy0uLkY6nZZVIyNjsXF9fb0so/0AJ1oLMjtaaXvBN61NIBCQ+02QHcViMbnfDGxXVFTo9SxnYQFpbW0t5s2bl/eun9PphNfrRWFhYc5hQGTUTqdTwWky8ImJiZwyJYYW4vG4gtRMcNmTXbwWXUafzycmZR+DRmbJv/G5+v1+lfPkMzjG366b1Afq3MaNG8Xa6FbbO1+YHOO4/cMPP1zF3HSvR0dH9V4ydjKVlpaWrPI1IMPO+QzoTvNZNDc3K+yR73C73SgpKcGLL76o3nCGYDjYdsWKFdJZdsj8+Mc/xsaNGwFYw2jvueceAJm1zeJ66l1NTY1GlrE/m8/M6XSqL5uJtRtuuEHF5Sz5+Y//+A8AwL333ovy8vLd6q5hjgYGBgYz4APFHDl7ELASLIyntLe3y7+npaiqqlJyZvrB3iwsB6wEgX08PS389IORAMvK+v1+vd7OsgCLaeZ7XCwej6O3txfNzc05Bz3xnqPRqNgHmYn9EKbp5RButzunb314eFhsabpMp6am5BHYi+kZy2Tc2P4dpqam8r41kwdsrV+/XoF/snImo1auXJnDpFOplA6MYhyS+njggQdKfmRHzz77rLwgFoMzCbN48WKV65DNRyIRfY+Pf/zjAKx45Ouvv67Br/mOWCyGzs5OLFu2TKVLLHgng+zs7NTRCSy/aWtrE2Pmc+C0nc7OTvzxj38EkGHpQMYLpFfE6/N5dnV1ab4mB9o++OCDik2ypIeJsqKiIrS2tuZMwLJj1v5QOp1GdXV11tRuwEqwxONxfQEuSLfbLbea7jEX9+DgoBSEv7OfG8zrUmHLyspyzh4eHBzUZzHZQPT29sLr9eZ9tprTqvv6+nJOBaTsKisrJRu6tolEQr+jPGiogsGgElmUTyAQkJtCWfH1/f39Co/wswsLC6WQ3Ky5AQwPD+8Rsp2amsLmzZtRVFSksIu9AgDIGJ633noLgDU2rLOzU3JjFp91jzt27FDogb28dXV1ui4XHcdntba2KlHwmc98BkCmbo/vZUaW729ra1Nik25ovoIE59Zbb9XAjcceewyAZST6+vqUwWa/dTAYlJ5RF5nQOeqoozSYlmPPuru7ZajpfvP5bN68WdfiBn3ppZdqdBxDH1xLtbW1GBkZMacPGhgYGMwWs57KU15ejnQ6ndMtYR9mO90lHhkZkcvCgDWZoMvlynEVY7GYrOb0Mf/2owLstY9022mB+b54PJ51tnO+gjWk5eXlOfdMdzmVSsn68jWDg4NKFtD68vV2l5dWNZFIiE2SafJZOJ1OydZ+/AXdQFp1vq+6ujrrzJt8hc/nw5IlSxAMBnUGDPWBTGT9+vXSabLLqakpJWCYSLzzzjsBZNxqejxM/hUUFChkQfeYLvcjjzyisp1rrrkGQCYJxO9D95MlLY2NjXvM6YM8lbSurk6lOJxKRJ2JRCJ47rnnAFilNuvWrVNtIkuefv/73wPIDKVlhwyfUSAQkO4x6cIpPo2NjWL+ZN/2kzF5dAKfj8/nw5133pnjadphmKOBgYHBDJgVc4zFYmhvb0d9fX1OsbX9vF0yRlpUj8cjBjP9VMHBwUFZWV6jrq5OZTq0PIzhVFRU6HVkixUVFWI/LFNhbCGZTCISieR9XCyVSmFsbAxTU1OSDdmNfaYlmSPjug6HQ38nkyYD7+3t1et5//F4XExzemdSMpmU3GhRXS6XfseOHV6zoKBgj5jKk0wmMTw8jPnz56sDg2AHxY4dO/Q3MuuJiQnp9/nnnw/AKjr2eDxYt24dACuOHggExHZYOsJhrsuXLxeL5PXLysp0fSZ8uK72pNMHCwsLxYDpudGTpAdSWFgoPVu1apVeM32QMj3G/fffXzMhuZ88/PDDKsXh1B/GFOfNm6dSnssvvxxAZj1wnbCkx961F4vFzLnVBgYGBrPFrI9JYOEvY4jc+cl24vG42BsZSl9fn3Z/ZjrtLWwsv2Fsy+FwyBqzHIif43a7ZZUYPxgdHRXLoiUgG507dy66u7vzvtyEU9YjkUhWuQGQHbu1M0Agwz7IxskEiUAgIDlycrLH4xHTY7yQFQGhUCirAgDIsFXGfPg3vj8ajf7DQtp8QEFBAZqamrBt2zYxP8aqWLr05S9/Ge+88w4Ai70df/zxihMyhmY/onV6a+v69evFoCgTlpoUFhbitddeAwAcffTRAIC3335bWXAWnPP6NTU1e0wR+OjoKJ5++mmcdNJJuh+uc86r3LBhQ84E/1AoJBb5s5/9DIBVildTU6PWwkceeQRApiyKOshqDVZSPProo9J1zoRsbW0VM7/11luzrv/5z38eK1asUHnVTJh1nWMkEoHX69WCmt4cz1pI+98CgYC+FG+AKfienp6cgH97e7sC1Nz07KOmpp897PP5soZWAFYvJQ9yz/dzTohQKJQzjNZ+RASNBReR/azp6V1LqVRKoQ3WKEajUbnh0w3Q0NCQlInyHxsbk7Hj3+z92vF4PO9DFkwYHHLIITIIdIntZ04z+cIFwwHEAHIGHuzYsUMbIMMNtbW16vig8Wcvdjwe10bA2seamho9u+nzCPaE2lyCo7/uvvtunHjiiQCAT33qUwCscFhLS4vkxOHAPT09GkLBEAL1tLGxUZsiB1a4XC5tutRP6u4zzzyjpAvrKEdGRvSMKFcaq1gshqamphxCYceesWMYGBgY/Isxa7c6GAwikUhkDVsFkNULTWZHil1aWqrXMSDKdLvH48k5ObCpqUkWla+3D/6kFeBnVlVV5VgSexLD7krmM9LpNHbs2KEgNhMfZDvpdFqMkUy8oqJCDJNshay+tLRUlpNyLCkpERun204W393dLdbPn6WlpTnj4+wHgPX29uY9w0kkEhgYGEAymczp8rEfzUE5HHXUUQCAl156KWdMG0tt9tprL7nf1O2mpiY9H4adqIclJSXSaT7fyclJyY7Pgq8fGBjYY7wdDhPeunWrTvdj6IAdRq+99ppYNN3eQw89VEyc4PqdnJxUFwx1MRKJKIFGHHHEEQAy7J1J3O985zv6OxMx7LK56667AGSmIJ155pmmCNzAwMBgtpj1sNtoNIpQKCSWwqQLLTKPwQSy2wfJguwj/IEMyyQT4e98Pp8sMBkq/9/f3684EOMZU1NTYk8snCVTHBsb03vzGQ6HA263Gy6XK4e9keUUFxeriN5+oBjZIV9PttPe3q7XkckMDQ3p9bSatNYNDQ0qrmVrnNvtFhPlc5p+sNnuZuLlA1wuF4LBILq6unQoFu+ByYH169dLR6lLLS0tiveylY39u2+88YbilowXLl26VEkXejqM+VZXVyvRxla2Y489VoyUz8A+u5Ax3l/+8pf/NFl8WEin04hEIkqS0rPhcGCXy6UyHyZRtm/frtexB5pM74YbbtBzYAx45cqVKpUiXn75ZQCZ+CJbNbk3LVy4UPFNJryY3ygoKPjn91anUinE4/GshAdgLUyv16vNkYswEolICeznBAOZzZIbGRWxpKREN0hh092Ix+NZHSB8HzddLmB7pnxPQDQaxc6dO1FeXp5zToy9b53gYrLXeDLgTblXV1fPmNSx/xuwFKa7u1vPwp50ocHh6/i3qqqqPWLYrcPhgM/nQ2dnp7LVNAyPP/44gOwzYew9/nQTuYkya2130WmM7rzzThGC6R1kY2Nj0mFuwg6HQ5UAfCZM6PA77wnwer1oaWnB3/72N+kI1+3NN98MILM5sfaTm2NnZ6eSYKwx5dDf22+/XUMoqJMDAwOa8s1Njc9x48aNuO+++7I+89VXX1UlAV1ufvbQ0FDWoO2ZYNxqAwMDgxkwK+bocDjg9XoRDAbFFMlgyB5KS0tFYckWd+3apSQNXVy6bQ6HQ+l5lozs2rVLFpe/Y1IgHo/LCtjPZaY7QmvO7xMOhzEyMrJHHJNQVFSUdVIiZWYf6Dn9aINkMqnXky1TPkVFRVkuMJCRE6/L19nrHCk3yjYQCOQkBsjOy8rKUFdXt9tyiHyAz+fD/Pnz0d/fnzXiDYDctN7eXrl0ZCVVVVXqsybD4N9WrFih6S8MdSSTyZyzqelKt7W1yRviVJqKigrp+fTzld1ut55PvsPn82HBggU444wzxHYZTmDIIZlM4uqrrwZgMW2n04kLL7wQgDVJx35yIJMonBTV2NgoT4iMkW65x+PRmuBA3HfffVefz0lIvFZTUxPq6+t3G3IzzNHAwMBgBnzgDhkyMVpU/j+dTovF0d+vqKhQnJAWlbGWZDIpFsl4ZFVVlSzp9AGuJSUlOTEuJjIA5MQq3W63mGk+gwX2dqbGe6I1LioqyrmXkZERFX9TpnwW8XhcfyM7dLlciklOj93aP5OvTyaTYjdkTfapSHtCKc/Q0BDuvfderFixQtNf7Ie/AZmYNhk1mXBfX590mfdMxnL//fdLHmQs9q4Wsj5ef/78+ZITTx+0Mxden4Xi9u+R7+Cw2xUrVqhM6ec//zkAqxPJ4/FkldgAGb0mw+RcSz6f8vJy/Y1x4XfffRdf/vKXAeQe9tbf36+YI+OSRx99tM7Npny5LwwPD2PHjh3aj2aCYY4GBgYGM+ADtQ+WlZWJ3UyfqGOfdMFdORqNypKyPIGv8Xq9spD22W+0CGSTjMV4PB7F0fi3YDCo6zE+R3bU3d2NgoKCvC8C93g8qKmpgcPhUAyLLJjyicViYhrMaNtjjnwG9h51ZrBZaF9QUCDZktWQ0cyZMyfn2NZwOCwPgGD8t62tDc3NzXlfysNs6tq1a3N0jUzC6/WKtVC3q6urs877BqxnUVFRoSaHgw46CECG7VAnKSO2q/X39+ccudvU1JQzRYrZ8I985CNZE/HzGZxFaj8mgnFV3l8wGFSFC9sCn3nmGc1gZInU7373OwCZCeqMe5NpNzQ0SGeZ8edk9meffRZf+9rXsr7X448/rs+0T84HMgX+J5988m5bX2c97LasrAyDg4ParPjhfJDpdFo3YO9LnV6OwgU1OTmZNXqMN2DvMQUspezo6MhJ6vj9/pxr2IdfdHd3533/L8dqFRcXK2lA5eACHRsb0++4WO01pwx028uqKGe7G8lnRlnRLbSf3cxA9+joqJ4ZP5OK5nK5sGvXrt3WiuUDOMZ/wYIFWQF8wCoTGRkZkUvIMMPU1JQ2Mo4Us/dA84xqyi8ajeaUTtmPDOF7WUva3t6uNUKdJnno6uraI8JBQGa9dnd3o76+Xob94osvBmDp7umnny7XmfK96KKL8L3vfQ+AleTjMQv9/f16Hlz7JSUlqhHlc2GdY11dnYwVB4js2rVL76Usqbu9vb0YHx83HTIGBgYGs4VjNozK4XD0AWj98L7Oh4qmdDpd8Y9f9u+Bke2Hhz1ctoCR74eJvyvbWW2OBgYGBv+vwLjVBgYGBjPAbI4GBgYGM8BsjgYGBgYzwGyOBgYGBjPAbI4GBgYGM8BsjgYGBgYzwGyOBgYGBjPAbI4GBgYGM8BsjgYGBgYz4P8DzHcq8ExwC3sAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 16 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print(\"Weights with 500 data points:\")\n",
    "\n",
    "fig, axes = plt.subplots(4, 4)\n",
    "vmin, vmax = mlp.coefs_[0].min(), mlp.coefs_[0].max()\n",
    "for coef, ax in zip(mlp.coefs_[0].T, axes.ravel()):\n",
    "    ax.matshow(coef.reshape(28, 28), cmap=plt.cm.gray, vmin=.5 * vmin, vmax=.5 * vmax)\n",
    "    ax.set_xticks(())\n",
    "    ax.set_yticks(())\n",
    "\n",
    "plt.show()\n",
    "\n",
    "print(\"Weights with 10000 data points:\")\n",
    "\n",
    "fig, axes = plt.subplots(4, 4)\n",
    "vmin, vmax = best_mlp_large.coefs_[0].min(), best_mlp_large.coefs_[0].max()\n",
    "for coef, ax in zip(best_mlp_large.coefs_[0].T, axes.ravel()):\n",
    "    ax.matshow(coef.reshape(28, 28), cmap=plt.cm.gray, vmin=.5 * vmin, vmax=.5 * vmax)\n",
    "    ax.set_xticks(())\n",
    "    ax.set_yticks(())\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TO DO 9\n",
    "\n",
    "Describe what do you observe by looking at the weights.\n",
    "\n",
    "The images are in 10000 data points are more clear and less noise compared with the 500 data point images. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TO DO 10\n",
    "\n",
    "Pick another classifier among the ones we have seen previously (SVM or something else). Report the training and test error for such classifier with 10000 samples in the training set, if possible; if the classifier cannot run with so many data sample reduce the number of samples.\n",
    "\n",
    "*Note*: if there are parameters to be optimized use cross-validation. If you choose SVM, you can decide if you want to use a single kernel or use the best among many; in the latter case, you need to pick the best kernel using cross-validation (using the functions available in sklearn).\n",
    "\n",
    "CLASSIFIER CHOOSEN: SVM with rbf kernel"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RESULTS FOR OTHER CLASSIFIER\n",
      "\n",
      "Best training error (other model): 0.016700\n",
      "Best test error (other model): 0.122260\n"
     ]
    }
   ],
   "source": [
    "from sklearn.svm import SVC\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "SVM = SVC(kernel='rbf')\n",
    "parameters = {'C': [1, 10, 100]}\n",
    "g = GridSearchCV(SVM,parameters,scoring='accuracy')\n",
    "g.fit(X_train,y_train)\n",
    "\n",
    "training_error_other = 1. - g.score(X_train,y_train)\n",
    "\n",
    "test_error_other = 1. - g.score(X_test,y_test)\n",
    "\n",
    "print ('RESULTS FOR OTHER CLASSIFIER\\n')\n",
    "\n",
    "print (\"Best training error (other model): %f\" % training_error_other)\n",
    "print (\"Best test error (other model): %f\" % test_error_other)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TO DO 11\n",
    "Compare the results of NN and of the other classifier you have chosen above. Which classifier would you preferer? Provide a brief explanation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With minimum test error is the Neural Networks with 10000 samples, 1 hidden layer and 50 neurons for each. So this architecture is good."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Clustering with K-means\n",
    "\n",
    "Clustering is a useful technique for *unsupervised* learning. We are now going to cluster 2000 images in the fashion MNIST dataset, and try to understand if the clusters we obtain correspond to the true labels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "#load the required packages\n",
    "\n",
    "from sklearn import metrics\n",
    "from sklearn.cluster import KMeans"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(Note that the code below assumes that the data has already been transformed as in the NN part of the notebook, so make sure to run the code for the transformation even if you do not complete the part on NN.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "#let's consider only 2000 data points\n",
    "\n",
    "X = X[permutation]\n",
    "y = y[permutation]\n",
    "\n",
    "m_training = 2000\n",
    "\n",
    "X_train, X_test = X[:m_training], X[m_training:]\n",
    "y_train, y_test = y[:m_training], y[m_training:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TO DO 1\n",
    "Cluster the points using the KMeans() and fit() functions (see the userguide for details). For Kmeans, set: n_clusters=10 as number of clusters; n_init=10 as the number of times the algorithm will be run with different centroid seeds; random_state = ID. You can use the default setting for the other parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "KMeans(n_clusters=10, random_state=2041267)"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "kmeans = KMeans(n_clusters=10, n_init=10, random_state = ID)\n",
    "kmeans.fit(X_train,y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Comparison of clusters with true labels"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TO DO 2\n",
    "Now compare the obtained clusters with the true labels, using the function sklearn.metrics.cluster.contingency_matrix() (see the userguide for details). The function prints a matrix $A$ such that entry $A_{i,j}$ is is the number of samples in true class $i$ and in predicted class $j$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[   0 1768 3158   52   24    0  486  147    3  182]\n",
      " [   0  341  218 5016    3    0  117   51    0   48]\n",
      " [   0  136  113    3   24    0  695 3068    1 1777]\n",
      " [   0 1835 1626 1962    4    0  242   31    0   95]\n",
      " [   0  297  904   97   15    0  291 3000    0 1205]\n",
      " [ 452   22    2    0    6  205 3603    0 1497    8]\n",
      " [   0  747  976   15   59    1  917 1618    4 1458]\n",
      " [ 744    0    0    0    3   21  449    0 4571    0]\n",
      " [  49  418   24    3 2571    4  435  271  224 1800]\n",
      " [3043   23    3    0    4 2365  176    0  161   13]]\n"
     ]
    }
   ],
   "source": [
    "# compute and print the contingency matrix for the true labels vs the clustering assignments\n",
    "y_predict = kmeans.predict(X_test)\n",
    "mat = sklearn.metrics.cluster.contingency_matrix(y_test, y_predict)\n",
    "print(mat)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TO DO 3\n",
    "Based on the matrix shown above, comment on the results of clustering in terms of adherence to the true labels.\n",
    "\n",
    "[ADD YOUR ANSWER HERE]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Choice of k with silhoutte coefficient\n",
    "In many real applications it is unclear what is the correct value of $k$ to use. In practice one tries different values of $k$ and then uses some external score to choose a value of $k$. One such score is the silhoutte coefficient, that can be computed with metrics.silhouette_score(). See the definition of the silhoutte coefficient in the userguide."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TO DO 4\n",
    "Compute the clustering for k=2,3,...,15 (other parameters as above) and print the silhoutte coefficient for each such clustering."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Silhoutte coefficient for number of clusters=2: 0.19317571670652278\n",
      "Silhoutte coefficient for number of clusters=3: 0.18433380720956222\n",
      "Silhoutte coefficient for number of clusters=4: 0.18042672896786996\n",
      "Silhoutte coefficient for number of clusters=5: 0.1603201984337485\n",
      "Silhoutte coefficient for number of clusters=6: 0.15622936969582776\n",
      "Silhoutte coefficient for number of clusters=7: 0.1667462399786566\n",
      "Silhoutte coefficient for number of clusters=8: 0.16034511984466143\n",
      "Silhoutte coefficient for number of clusters=9: 0.1536777083218814\n",
      "Silhoutte coefficient for number of clusters=10: 0.13566561580397163\n",
      "Silhoutte coefficient for number of clusters=11: 0.1277787383740401\n",
      "Silhoutte coefficient for number of clusters=12: 0.1349591125810742\n",
      "Silhoutte coefficient for number of clusters=13: 0.1335592010143823\n",
      "Silhoutte coefficient for number of clusters=14: 0.13101268489187756\n",
      "Silhoutte coefficient for number of clusters=15: 0.12965821137618527\n"
     ]
    }
   ],
   "source": [
    "#run k-means with 10 choices of initial centroids for a range of values of n_clusters\n",
    "\n",
    "for i in range(2,16):\n",
    "    kmeans = KMeans(n_clusters=i, n_init=10, random_state = ID)\n",
    "    kmeans.fit(X_train,y_train)\n",
    "    silhouttescore = metrics.silhouette_score(X_test, kmeans.predict(X_test))\n",
    "    print(\"Silhoutte coefficient for number of clusters=\"+str(i)+\": \"+str(silhouttescore))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## TO DO 5\n",
    "\n",
    "Based on the silhoutte score, which $k$ would you pick? Motivate your choice. Does your choice match what you know about the data? If yes, explain why you think this is the case; if no, explain what you think may be the reason."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}