{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Characters Classification with Neural Networks\n", "\n", "In this notebook we are going to use the Neural Networks for image classification. We are going to use the same dataset of the lab on SVM: Kuzushiji-MNIST or K-MNIST for short (https://github.com/rois-codh/kmnist) a dataset of traditional japanese handwritten kana.\n", "\n", "The dataset labels are the following:\n", "\n", "| Label | Hiragana Character | Romanji (Pronunciation) |\n", "| :-: | :-: | :-: |\n", "| 0 | お | o |\n", "| 1 | き | ki |\n", "| 2 | す | su |\n", "| 3 | つ | tsu |\n", "| 4 | な | na |\n", "| 5 | は | ha |\n", "| 6 | ま | ma |\n", "| 7 | や | ya |\n", "| 8 | れ | re |\n", "| 9 | を | wo |" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "scikit-learn version: 0.21.3\n" ] } ], "source": [ "#load the required packages and check Scikit-learn version\n", "\n", "%matplotlib inline \n", "\n", "import numpy as np\n", "import scipy as sp\n", "import matplotlib.pyplot as plt\n", "\n", "import sklearn\n", "print ('scikit-learn version: ', sklearn.__version__)\n", "from sklearn.neural_network import MLPClassifier\n", "from sklearn.model_selection import GridSearchCV\n", "from sklearn.svm import SVC\n", "\n", "# from sklearnex import patch_sklearn\n", "# patch_sklearn()" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "# helper function to load MNIST dataset from disk\n", "def load_mnist(path, kind='train'):\n", " import os\n", " import gzip\n", " import numpy as np\n", " labels_path = os.path.join(path, 'K%s-labels-idx1-ubyte.gz' % kind)\n", " images_path = os.path.join(path, 'K%s-images-idx3-ubyte.gz' % kind)\n", " with gzip.open(labels_path, 'rb') as lbpath:\n", " labels = np.frombuffer(lbpath.read(), dtype=np.uint8,offset=8)\n", " with gzip.open(images_path, 'rb') as imgpath:\n", " images = np.frombuffer(imgpath.read(), dtype=np.uint8,offset=16).reshape(len(labels), 784)\n", " return images, labels" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# TODO \n", "Place a seed for the random generatoryour (you can use your \"numero di matricola\"). Try to change the seed to see the impact of the randomization." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "ID = 1237007\n", "np.random.seed(ID)" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Number of samples in the MNIST dataset: 60000\n" ] } ], "source": [ "#load the MNIST dataset and let's normalize the features so that each value is in [0,1]\n", "X, y = load_mnist(\"data\")\n", "print(\"Number of samples in the MNIST dataset:\", X.shape[0])\n", "# rescale the data\n", "X = X / 255.0" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now split into training and test. We start with a small training set of 600 samples to reduce computation time while 4000 samples will be used for testing. Make sure that each label is present at least 10 times in training frequencies." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Labels in training dataset: [0 1 2 3 4 5 6 7 8 9]\n", "Frequencies in training dataset: [63 63 69 65 53 67 56 60 51 53]\n", "Labels in test set: [0 1 2 3 4 5 6 7 8 9]\n", "Frequencies in test set: [63 63 69 65 53 67 56 60 51 53]\n" ] } ], "source": [ "#random permute the data and split into training and test taking the first 600\n", "#data samples as training and the rests as test\n", "permutation = np.random.permutation(X.shape[0])\n", "\n", "X = X[permutation]\n", "y = y[permutation]\n", "\n", "m_training = 600\n", "m_test = 4000\n", "\n", "X_train, X_test = X[:m_training], X[m_training:m_training+m_test]\n", "y_train, y_test = y[:m_training], y[m_training:m_training+m_test]\n", "\n", "labels, freqs = np.unique(y_train, return_counts=True)\n", "print(\"Labels in training dataset: \", labels)\n", "print(\"Frequencies in training dataset: \", freqs)\n", "\n", "labelsT, freqsT = np.unique(y_test, return_counts=True)\n", "print(\"Labels in test set: \", labels)\n", "print(\"Frequencies in test set: \", freqs)\n" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "#function for plotting a image and printing the corresponding label\n", "def plot_input(X_matrix, labels, index):\n", " print(\"INPUT:\")\n", " plt.imshow(\n", " X_matrix[index].reshape(28,28),\n", " cmap = plt.cm.gray_r,\n", " interpolation = \"nearest\"\n", " )\n", " plt.show()\n", " print(\"LABEL: %i\"%labels[index])\n", " return" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "INPUT:\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAARqUlEQVR4nO3de2xVZboG8OeldKBiuWgLVkAZCBpwlKpbJWAIBzIqJoqKMw5/jBjwdKKSzCjxEo4BIv+gOY4aoxMZJdPROY4TvEDUeLjEeCHhUhQRhCMXq61CW7wADSDQvuePLs6p2PWuzV5r77Xp+/ySpu1++rk/tjys3f3ttT5RVRBR99cj7QkQUWGw7EROsOxETrDsRE6w7ERO9CzknVVUVOiwYcMKeZfutbe3m3nc1Zio8d9++21oVlFRYY4tKSnJaU6e1dfXY9++fdJVFqvsInIdgKcAlAB4XlUXWT8/bNgw1NXVxblLOkWtra1mfvz4cTPv0cN+8nfs2DEzr62tDc3uvPNOc2zfvn3NnH4uk8mEZjk/jReREgDPAJgCYDSA6SIyOtf/HhHlV5zf2a8EsFNVd6vqUQD/BDA1mWkRUdLilH0wgIZO3zcGt/2EiNSISJ2I1LW0tMS4OyKKI07Zu3oR4Gev1qjqYlXNqGqmsrIyxt0RURxxyt4IYGin74cA+CbedIgoX+KUfQOAkSLySxH5BYDfAViezLSIKGk5L72p6nERmQ3gv9Gx9LZEVbcmNrNuJGot+siRI2a+du1aM6+vrw/NHnnkEXNsU1OTmbe1tZl5WVmZme/fvz80s+YNAE8++aSZRy0L0k/FWmdX1bcBvJ3QXIgoj/hPI5ETLDuREyw7kRMsO5ETLDuREyw7kRMFPZ+9mEWdCvr111+HZs8//7w5dvly+71GUecMxDlNNd9XDz569GjOYxsbG81cpMvTsilHPLITOcGyEznBshM5wbITOcGyEznBshM5waW3wO7du8188uTJodm+ffuSnk7B9O7d28yjTr+NUl1dHZpdc8015lguvSWLR3YiJ1h2IidYdiInWHYiJ1h2IidYdiInWHYiJ7jOHhg1apSZz58/PzSbN2+eOfb777/PaU7ZGjt2bGg2a9Ysc+wzzzxj5ps2bcppTtmMHzBgQKz/Np0aHtmJnGDZiZxg2YmcYNmJnGDZiZxg2YmcYNmJnOA6e6C0tNTM77rrrtBsypQp5thly5aZ+Zo1a8x88ODBZj5nzpzQ7LXXXjPHbt682czjqqqqCs3GjRuX1/umn4pVdhGpB3AQQBuA46qaSWJSRJS8JI7s/6aqp++lWoic4O/sRE7ELbsCWCEiG0WkpqsfEJEaEakTkbqobY6IKH/iln28ql4GYAqAe0Rkwsk/oKqLVTWjqpnKysqYd0dEuYpVdlX9JvjcDOB1AFcmMSkiSl7OZReRPiJSfuJrANcA2JLUxIgoWXFejR8E4PXg2t49AfyXqr6TyKyKUElJSWg2YsQIc+x9990XKz927JiZP/3006HZokWLzLHt7e1mHld5eXlo9tJLL5ljL7roIjOfNGmSmZ955plm7k3OZVfV3QDGJDgXIsojLr0ROcGyEznBshM5wbITOcGyEznBU1xPA1FbQn/44YehWdpvUf78889Ds7lz55pj+/bta+aXX365mT/88MOhWdSyXXfEIzuREyw7kRMsO5ETLDuREyw7kRMsO5ETLDuRE1xnL4Co00i3bt1q5jNmzDDzTz755JTndELPnvZfgbKyMjM/ePBgzvcd5cCBA2b+7rvvmvnOnTtDs3fesc/GHj16tJmfjnhkJ3KCZSdygmUncoJlJ3KCZSdygmUncoJlJ3KC6+wJiLrUc21trZkvXLjQzBsaGsw8uJx3To4fP27mcdfRKyoqQrNBgwaZY/v06WPmUdtNq2poNn/+fHPsiy++aOa9e/c282LEIzuREyw7kRMsO5ETLDuREyw7kRMsO5ETLDuRE1xnz9KGDRtCs3nz5pljV61aZeZRa92lpaVmfu2114ZmO3bsMMeOGWNvxDtq1CgzP/fcc8182rRpoVlbW5s5trW11cw//vhjM3/zzTdzygBg/fr1Zj5hwgQzL0aRR3YRWSIizSKypdNtZ4nIShHZEXwekN9pElFc2TyN/xuA60667SEAq1V1JIDVwfdEVMQiy66q7wP47qSbpwI48R7QWgA3JTwvIkpYri/QDVLVPQAQfB4Y9oMiUiMidSJSl/a+Y0Se5f3VeFVdrKoZVc1UVlbm++6IKESuZW8SkSoACD43JzclIsqHXMu+HMCJ6xvPALAsmekQUb5ErrOLyMsAJgKoEJFGAPMBLALwLxGZBeArAL/J5yQLIWo9+vbbbw/Ntm/fnvR0fuKMM84w85qamtCsurraHDt06NCc5lQIAweGvhQEABg+fLiZW3+2qOvGv/rqq2Z+Oq6zR5ZdVaeHRJMTngsR5RHfLkvkBMtO5ATLTuQEy07kBMtO5ES3OcX1yJEjZr5ixQozf+yxx8w8zvLakCFDzHzcuHFmfsEFF5j55MnhCyNRy3bdWSaTCc3uvvtuc+wXX3xh5rt37zbzqGXBNPDITuQEy07kBMtO5ATLTuQEy07kBMtO5ATLTuREt1ln37lzp5nPmDHDzKO2XY4jatvjO+64w8yjTqf0vJZu6dEj/FgW9f9k6dKlZm5dvhvgOjsRpYhlJ3KCZSdygmUncoJlJ3KCZSdygmUncqLbrLP369fPzAcMsDeajTp/uVevXqFZ1LbGc+bMMfOoNVtrvZhy8+CDD5p5RUWFmffu3TvJ6RQE/xYROcGyEznBshM5wbITOcGyEznBshM5wbITOdFt1tmjth6eO3eumd9///1mbl2D/NlnnzXHjhw50sy9irrWf1NTk5mfd955Zi4ioVlZWZk5dtWqVWa+ZcsWM586daqZpyHyyC4iS0SkWUS2dLptgYh8LSKbgo/r8ztNIoorm6fxfwNwXRe3P6Gq1cHH28lOi4iSFll2VX0fwHcFmAsR5VGcF+hmi8jm4Gl+6BvPRaRGROpEpK6lpSXG3RFRHLmW/S8ARgCoBrAHwONhP6iqi1U1o6qZysrKHO+OiOLKqeyq2qSqbaraDuCvAK5MdlpElLScyi4iVZ2+vRmAvQ5BRKmLXGcXkZcBTARQISKNAOYDmCgi1QAUQD2AP+RxjomYNm2amZeXl5u5tQd61LnPnln72t96663m2F27dpn5mDFjzNy6nv7NN99sjm1ubjbzqOvOt7W1mXlJSYmZ50Nk2VV1ehc3v5CHuRBRHvHtskROsOxETrDsRE6w7EROsOxETnSbU1yjRF1K+rbbbivQTArrhx9+MPPGxkYz37Bhg5mvXbvWzLdt2xaaXX311ebYw4cPm/m6devM3PLee++ZeXt7u5lHnVId9bifffbZZp4PPLITOcGyEznBshM5wbITOcGyEznBshM5wbITOeFmnb2YRV1Sef369Wa+cuXK0OyVV14xxzY0NJh51NziiPpz9enTx8zPOeccM+/fv39odujQIXPsV199ZeY9e9rVOXbsmJmngUd2IidYdiInWHYiJ1h2IidYdiInWHYiJ1h2Iie4zl4A9fX1Zj579mwzt9bRAeDo0aOnOqXETJgwwcxnzpwZmkWt4UetVW/cuNHM9+7dG5rFPd/83nvvNfNBgwaZeRp4ZCdygmUncoJlJ3KCZSdygmUncoJlJ3KCZSdyguvsCVBVM3/qqafM/K233kpyOomaPr2rTXz/3wMPPGDmb7zxRmj23HPPmWN//PFHM+/Vq5eZX3rppaFZVVWVOXbNmjVmvn//fjMXETNPQ+SRXUSGisi7IrJNRLaKyB+D288SkZUisiP4bO/CQESpyuZp/HEAc1R1FICxAO4RkdEAHgKwWlVHAlgdfE9ERSqy7Kq6R1U/Cr4+CGAbgMEApgKoDX6sFsBN+ZokEcV3Si/QicgwAJcCWAdgkKruATr+QQAwMGRMjYjUiUhdS0tLvNkSUc6yLruInAngVQB/UtUD2Y5T1cWqmlHVTGVlZS5zJKIEZFV2ESlFR9H/oaqvBTc3iUhVkFcBaM7PFIkoCZFLb9KxhvACgG2q+udO0XIAMwAsCj4vy8sMTwNR2xYvX77czKOWkKKWcfJ5ueeoZcFVq1aZeT5/dSsrKzNz6xTZG264wRw7cuRIM7/44ovNvBhls84+HsDvAXwqIpuC2+aio+T/EpFZAL4C8Jv8TJGIkhBZdlX9EEDYoWVystMhonzh22WJnGDZiZxg2YmcYNmJnGDZiZzgKa5Zsi7XvGDBAnNs1KWko04THTt2rJk/8cQTodn27dvNsU1NTWZ+4EDWb5YsuKFDh5r5LbfcEpoNGTLEHDtixAgzj7qEdjHikZ3ICZadyAmWncgJlp3ICZadyAmWncgJlp3ICa6zZ6lnz/CH6pJLLjHHRm25vGvXLjPv16+fmV922WWhWdS58IcPHzbz1tZWM7/wwgvN3Nq6+LPPPjPHNjfb10P58ssvzdz6s0VdSjpqDb8YLxUdhUd2IidYdiInWHYiJ1h2IidYdiInWHYiJ1h2Iie4zp6lHj3C/12MOh+9vb3dzM8//3wzb2hoMPPhw4eHZldccYU5NmodPera7DfeeKOZW+9PmDRpkjk2ap29tLTUzK1z9aOu1W+9P+B0xSM7kRMsO5ETLDuREyw7kRMsO5ETLDuREyw7kRPZ7M8+FMDfAZwDoB3AYlV9SkQWAPh3ACc24J6rqm/na6LFrLKy0swff/zxAs2k8KLW6a33IKxfvz7WfVv7rwP2Ne+jzmfvjrJ5U81xAHNU9SMRKQewUUROXI3hCVX9z/xNj4iSks3+7HsA7Am+Pigi2wAMzvfEiChZp/Q7u4gMA3ApgHXBTbNFZLOILBGRASFjakSkTkTqWlpauvoRIiqArMsuImcCeBXAn1T1AIC/ABgBoBodR/4ufzFV1cWqmlHVTNTvtkSUP1mVXURK0VH0f6jqawCgqk2q2qaq7QD+CuDK/E2TiOKKLLt0XEbzBQDbVPXPnW7v/HLmzQC2JD89IkpKNq/GjwfwewCfisim4La5AKaLSDUABVAP4A95mSGl6tChQ2Y+c+ZMM1+6dGloNn78eHNs//79zTzqcs+PPvpoaFZeXm6O7Y6yeTX+QwBdXSTb5Zo60emK76AjcoJlJ3KCZSdygmUncoJlJ3KCZSdygpeS7ub27t1r5lGXil6yZImZf/DBB2Z+1VVXhWYLFy40x06cONHM6dTwyE7kBMtO5ATLTuQEy07kBMtO5ATLTuQEy07khKhq4e5MpAXAl51uqgCwr2ATODXFOrdinRfAueUqybmdr6pdXv+toGX/2Z2L1KlqJrUJGIp1bsU6L4Bzy1Wh5san8UROsOxETqRd9sUp37+lWOdWrPMCOLdcFWRuqf7OTkSFk/aRnYgKhGUnciKVsovIdSLyPyKyU0QeSmMOYUSkXkQ+FZFNIlKX8lyWiEiziGzpdNtZIrJSRHYEn7vcYy+luS0Qka+Dx26TiFyf0tyGisi7IrJNRLaKyB+D21N97Ix5FeRxK/jv7CJSAuBzAL8G0AhgA4DpqvpZQScSQkTqAWRUNfU3YIjIBACtAP6uqr8KbnsMwHequij4h3KAqj5YJHNbAKA17W28g92KqjpvMw7gJgB3IMXHzpjXb1GAxy2NI/uVAHaq6m5VPQrgnwCmpjCPoqeq7wP47qSbpwKoDb6uRcdfloILmVtRUNU9qvpR8PVBACe2GU/1sTPmVRBplH0wgIZO3zeiuPZ7VwArRGSjiNSkPZkuDFLVPUDHXx4AA1Oez8kit/EupJO2GS+axy6X7c/jSqPsXW0lVUzrf+NV9TIAUwDcEzxdpexktY13oXSxzXhRyHX787jSKHsjgM478g0B8E0K8+iSqn4TfG4G8DqKbyvqphM76Aafm1Oez/8ppm28u9pmHEXw2KW5/XkaZd8AYKSI/FJEfgHgdwCWpzCPnxGRPsELJxCRPgCuQfFtRb0cwIzg6xkAlqU4l58olm28w7YZR8qPXerbn6tqwT8AXI+OV+R3AfiPNOYQMq/hAD4JPramPTcAL6Pjad0xdDwjmgXgbACrAewIPp9VRHN7EcCnADajo1hVKc3tanT8argZwKbg4/q0HztjXgV53Ph2WSIn+A46IidYdiInWHYiJ1h2IidYdiInWHYiJ1h2Iif+F3vBY/85GRIAAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "LABEL: 4\n", "INPUT:\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAATqUlEQVR4nO3da3CUVZ4G8OdPCGCFu4ncRDKOCQpeELqQKrYsdHDECyolYw0UwlIi84GLKB9UFhT8pOuKOKBcBBRQnBocWRCvEUZRipqisVBQ2AWpcJEIAUruyCX//ZBmK2rO/8R+u/ttPc+vikrST073seWhkz7v+x5RVRDRb1+juCdARLnBshMFgmUnCgTLThQIlp0oEI1z+WDFxcVaWlqay4ckCkplZSUOHjwo9WWRyi4iAwC8AKAAwHxVfdr6/tLSUiSTySgPSUSGRCLhzNL+MV5ECgC8COA2AN0ADBGRbuneHxFlV5Tf2XsD2KGqO1X1DIC/Abg7M9MiokyLUvZOAPbU+Xpv6rYfEZHRIpIUkWR1dXWEhyOiKKKUvb43AX527K2qzlPVhKomSkpKIjwcEUURpex7AXSu8/WlAPZFmw4RZUuUsm8AUCYivxORJgD+DGBlZqZFRJmW9tKbqp4TkbEAPkDt0ttCVf0qYzPLsKqqKjN/9NFHzbysrMyZjR8/3hzbqlUrM6f6ff3112Y+efJkM9+zZ48zu+WWW8yxY8aMMfNOnX729tSPHDt2zMxbtGhh5tkQaZ1dVd8F8G6G5kJEWcTDZYkCwbITBYJlJwoEy04UCJadKBAsO1Egcno+e5z27bMP7lu2bJmZ19TUOLP33nvPHPv++++becuWLc08VN262SdR+o5vGDZsmDN7+mnzbGxs2LDBzOfPn2/m7dq1M/M48JWdKBAsO1EgWHaiQLDsRIFg2YkCwbITBUJyubFjIpHQfL267JIlS8x81KhRzuzcuXPmWN+pmL68sLDQzKPw/f8XqfeqxA22cqX7Egfl5eXm2K5du5q5b27W8tlNN91kjv3hhx/M/KqrrjLzadOmmfmgQYPMPF2JRALJZLLeJ4av7ESBYNmJAsGyEwWCZScKBMtOFAiWnSgQLDtRILjOnuJ7Hvr37+/M1qxZY45t2rSpmS9evNjM77vvPjPPZ9YxCI0a2a81vjwKa/0fAO6//34zP3r0qJkXFxeb+UcffeTMrrvuOnOshevsRMSyE4WCZScKBMtOFAiWnSgQLDtRIFh2okAEcylpH9+50TfeeKMz862z+86NHjdunJmfP3/ezO+66y5nVlRUZI7NNmvr4oKCAnNsNi+xbT1nAPDiiy+a+YMPPmjmBw8eTPvxKyoqzLG+6wC4RCq7iFQCOAbgPIBzqpqIcn9ElD2ZeGW/SVXtf8aIKHb8nZ0oEFHLrgA+FJGNIjK6vm8QkdEikhSRZHV1dcSHI6J0RS17X1XtCeA2AGNE5GfvYqnqPFVNqGqipKQk4sMRUboilV1V96U+HgCwHEDvTEyKiDIv7bKLSJGItLjwOYA/AtiSqYkRUWZFeTe+HYDlqfXpxgCWqqq9N/GvWK9evdIe27p1azPv0KGDmQ8fPtzMBw4c6MzGjh1rjr355pvNPCrrfPbjx4+bY7O5zu47dmHIkCFm7ju2YtGiRWb+/fffO7MVK1aYYydMmODMrOsypF12Vd0JIP2z7Ikop7j0RhQIlp0oECw7USBYdqJAsOxEgcjpKa7nz5/HkSNHnPnMmTPN8Y8//rgz850uGVVNTU3aY3v06GHmb775ppk//PDDZv766687sy1b7EMf3nnnHTMvKyszc5+2bds6s2z/P7NEfewZM2aYeffu3c180qRJzsy3hbe13fSpU6ecGV/ZiQLBshMFgmUnCgTLThQIlp0oECw7USBYdqJA5HSdvaamBidOnHDmc+bMMcdbp4I+8MADac+rIXbv3p322JMnT5p5mzZtzHz69Olmvm7dOmfmm/cLL7xg5rNmzTJzH2vdt3nz5pHuO06+y4P7jo2wLrG9YMECc+yHH37ozKzjWPjKThQIlp0oECw7USBYdqJAsOxEgWDZiQLBshMFIqfr7IWFhejYsaMz910yeeLEic7sm2++McdeffXVZt63b18zP3v2rJlbCgsLzdx32eHnnnvOzCsrK53ZpZdeao4dPHiwmUfVtGnTrN5/XKLubvTkk086M9/xBwsXLnRmhw8fdmZ8ZScKBMtOFAiWnSgQLDtRIFh2okCw7ESBYNmJAiHWFq+ZlkgkNJlMOvPt27eb46+//npnZp0n3xAtWrQwc+t58m09fNFFF5m5dc434J/blClTnJnvPH/ruu7kdujQITNv1aqVmTdu7D7E5fTp0+bY8ePHO7Ply5ejurpa6su8r+wislBEDojIljq3tRWRChHZnvpoX32BiGLXkB/jXwUw4Ce3PQZgtaqWAVid+pqI8pi37Kq6FsBPj8G7G8CFYzwXAbgnw/MiogxL9w26dqpaBQCpj5e4vlFERotIUkSS1dXVaT4cEUWV9XfjVXWeqiZUNRH15AEiSl+6Zd8vIh0AIPXxQOamRETZkG7ZVwIYkfp8BIAVmZkOEWWLd51dRN4A0A9AMYD9AJ4E8N8A/g7gMgC7AfxJVd0n0qb41tl9rOtpjxo1Ku37zTbfry+jR482c99/W5cuXZyZSL1LrhRRTU2Nmfue9yj/X86dO+fMbrjhBmzcuLHeO/devEJVhziiPzRsakSUD3i4LFEgWHaiQLDsRIFg2YkCwbITBSKnl5KOauTIkc7Md8nip556ysx9l6K2llp8yyhLly418/79+5v5b5Xv8ty+ZeGCgoJIeRSNGsX3OmmdHmv9XeQrO1EgWHaiQLDsRIFg2YkCwbITBYJlJwoEy04UiF/VOru1tjls2DBz7L333mvm1ha6APDss886M9+2yH369DHzOFmnSwLAgQP2dUmWLFli5ps2bUorA/zr7MXFxWZ+7bXXOrOePXuaYy+++GIzb9++vZlfc801Zu7bljkb+MpOFAiWnSgQLDtRIFh2okCw7ESBYNmJAsGyEwXiV7XOHoVv22TferOle/fukR47m3zbSVdVVZm573nxHb8waNAgM7ecOXPGzJs0aWLm1jEA8+fPN8du27bNzI8cOWLm5eXlZj527Fhn5rt0eLp/n/jKThQIlp0oECw7USBYdqJAsOxEgWDZiQLBshMFIph1dp9PPvkk7bF9+/Y182xev9zHd950WVlZjmaSe9OmTXNmU6dONcfu2bPHzP/617+a+axZs8z8oYcecmZbt241xz7zzDPOzNrfwPvKLiILReSAiGypc9tUEflWRDal/tzuux8iildDfox/FcCAem5/XlV7pP68m9lpEVGmecuuqmsBHM7BXIgoi6K8QTdWRL5M/ZjfxvVNIjJaRJIikqyuro7wcEQURbplnw3g9wB6AKgC8JzrG1V1nqomVDVRUlKS5sMRUVRplV1V96vqeVWtAfAygN6ZnRYRZVpaZReRDnW+HARgi+t7iSg/eNfZReQNAP0AFIvIXgBPAugnIj0AKIBKAH/J4hwz4ttvvzVz3/7s1rnTAwcONMf6zst+9dVXzfzQoUNmfvDgQWfm+9VpwoQJZt6sWTMzz2dR9lAvLS01c2utGwDuvPNOMx8+fLgzmz17tjn29OnTzmz37t3OzFt2VR1Sz80LfOOIKL/wcFmiQLDsRIFg2YkCwbITBYJlJwpEMKe4rlq1ysx9lwa+8sornVnbtm3NsdZlgwHg5ZdfNvNs+uKLL8x87ty5Zt6yZctMTudXw7esJyJmbp1avH//fnPs3r17nZm1zMtXdqJAsOxEgWDZiQLBshMFgmUnCgTLThQIlp0oEL+ZdXbf1sJvv/12pPu3tj4eOnSoOXbdunWRHtt3mmm3bt2cWceOHc2xy5cvN/PGje2/IosWLTLzKKeZRmX9nfBdKvrTTz8186VLl5r5+vXrzfzo0aNmbrniiiuc2aZNm5wZX9mJAsGyEwWCZScKBMtOFAiWnSgQLDtRIFh2okD8ZtbZP/74YzOvqKiIdP/WOcRW1hA9e/Y0c9/2wv369XNmvi2bR4wYYebLli0z8ylTpph5eXm5M/MdG7Fjxw4znzFjhplb23Bbl98GgOLiYjP38V3jwNpa2cc6rmPNmjXOjK/sRIFg2YkCwbITBYJlJwoEy04UCJadKBAsO1EgflXr7JWVlc5s3Lhx5ljftslRtG7d2sx9c3vkkUfMvFWrVr94ThcsWbLEzN966y0zt7YHBoDp06ebuTV3ax28IXr37m3mixcvdmadO3c2x/q2uvZdF953DMGuXbuc2caNG82xffr0cWZFRUXOzPvKLiKdReSfIrJVRL4SkYdSt7cVkQoR2Z762MZ3X0QUn4b8GH8OwERVvQpAHwBjRKQbgMcArFbVMgCrU18TUZ7yll1Vq1T189TnxwBsBdAJwN0ALlyTaBGAe7I1SSKK7he9QScipQCuB/AvAO1UtQqo/QcBwCWOMaNFJCkiyerq6mizJaK0NbjsItIcwD8ATFDVBl8tT1XnqWpCVRO+Nz2IKHsaVHYRKURt0V9X1Qtv3+4XkQ6pvAOAA9mZIhFlgnfpTWrXGBYA2KqqdddZVgIYAeDp1McVUSezc+dOM7/jjjuc2bZt26I+vMnamviVV14xx95zj/12hu+yxtZyCgB88MEHzuyxx+z3TU+cOGHm1tbCgH+r69LSUmdmLY0BwOWXX27mvstcx6lJkyZmbj2vvufcYi0JNuTZ6gvgfgCbReTCRaknobbkfxeRBwDsBvCntGdIRFnnLbuqfgbA9c/FHzI7HSLKFh4uSxQIlp0oECw7USBYdqJAsOxEgcjpQmVNTY25rus71TPba+mWiRMnOjPfOrrvdEfr8r8A8Nprr5n56tWrnZmqmmMHDx5s5jNnzjTz9u3bmznlD76yEwWCZScKBMtOFAiWnSgQLDtRIFh2okCw7ESByOk6+6FDh8xzmH3rzY0auf9tirIFbkN89tlnzmz48OHm2M2bN5v5li1bzNy3Tm8ZOXKkmc+ZM8fMfedl+9bxfZdcptzhKztRIFh2okCw7ESBYNmJAsGyEwWCZScKBMtOFIicrrOfOXMGu3fvduZnz541xxcWFqb92NYaPQCcOnXKzCsqKtJ+7Gy79dZbndlLL71kjvWto/ucPHky7bG+6+Fnk3WtfQDo1auXmRcXF2dyOjnBV3aiQLDsRIFg2YkCwbITBYJlJwoEy04UCJadKBAN2Z+9M4DFANoDqAEwT1VfEJGpAB4EUJ361kmq+q51X4WFhejUqZMzP336tDmXAQMGOLPnn3/eHOvby3vy5Mlmvn79emd2+PBhc6xvD3TfOeHNmjUz8xkzZqQ9Nirf/fv+26KM3bp1q5nPnj3bmc2dO9cc27VrVzNftWqVmXfp0sXM49CQg2rOAZioqp+LSAsAG0XkwhEmz6vqf2VvekSUKQ3Zn70KQFXq82MishWA++WZiPLSL/qdXURKAVwP4F+pm8aKyJcislBE2jjGjBaRpIgkjx8/HmmyRJS+BpddRJoD+AeACap6FMBsAL8H0AO1r/zP1TdOVeepakJVE82bN8/AlIkoHQ0qu4gUorbor6vqWwCgqvtV9byq1gB4GUDv7E2TiKLyll1qLw+6AMBWVZ1e5/YOdb5tEAD7EqlEFKuGvBvfF8D9ADaLyKbUbZMADBGRHgAUQCWAv/juqKioCL17u38AsJaQAGDo0KHOrKSkxPfwpqVLl5q5dTnn7777zhy7du1aM9+5c6eZX3bZZWZeXl5u5tlUUFCQtfuOetqxtfz1xBNPmGN9lybftWtX2o8dl4a8G/8ZgPou/m2uqRNRfuERdESBYNmJAsGyEwWCZScKBMtOFAiWnSgQEuUUxF8qkUhoMpnM2eMRhSaRSCCZTNa7TzZf2YkCwbITBYJlJwoEy04UCJadKBAsO1EgWHaiQOR0nV1EqgHUPRG4GMDBnE3gl8nXueXrvADOLV2ZnFsXVa334g45LfvPHlwkqaqJ2CZgyNe55eu8AM4tXbmaG3+MJwoEy04UiLjLPi/mx7fk69zydV4A55aunMwt1t/ZiSh34n5lJ6IcYdmJAhFL2UVkgIj8j4jsEJHH4piDi4hUishmEdkkIrGefJ/aQ++AiGypc1tbEakQke2pj/XusRfT3KaKyLep526TiNwe09w6i8g/RWSriHwlIg+lbo/1uTPmlZPnLee/s4tIAYD/BXALgL0ANgAYoqpf53QiDiJSCSChqrEfgCEiNwI4DmCxql6duu0/ARxW1adT/1C2UdVH82RuUwEcj3sb79RuRR3qbjMO4B4A/44YnztjXvchB89bHK/svQHsUNWdqnoGwN8A3B3DPPKeqq4FcPgnN98NYFHq80Wo/cuSc4655QVVrVLVz1OfHwNwYZvxWJ87Y145EUfZOwHYU+frvciv/d4VwIcislFERsc9mXq0U9UqoPYvD4BLYp7PT3m38c6ln2wznjfPXTrbn0cVR9nruz5WPq3/9VXVngBuAzAm9eMqNUyDtvHOlXq2Gc8L6W5/HlUcZd8LoHOdry8FsC+GedRLVfelPh4AsBz5txX1/gs76KY+Hoh5Pv8vn7bxrm+bceTBcxfn9udxlH0DgDIR+Z2INAHwZwArY5jHz4hIUeqNE4hIEYA/Iv+2ol4JYETq8xEAVsQ4lx/Jl228XduMI+bnLvbtz1U1538A3I7ad+S/AfAfcczBMa/LAXyR+vNV3HMD8AZqf6w7i9qfiB4AcDGA1QC2pz62zaO5LQGwGcCXqC1Wh5jm9m+o/dXwSwCbUn9uj/u5M+aVk+eNh8sSBYJH0BEFgmUnCgTLThQIlp0oECw7USBYdqJAsOxEgfg/Xs0uJZ3zkkoAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "LABEL: 0\n", "INPUT:\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAQu0lEQVR4nO3de4xUZZoG8OelaYIwGBBap2VguWlY2QQZSlSY4GWQCCa2xLAZ/ljYqAsmaBguRqLE8Q9HWyKDKJshja3DjKOTQWaEeGEhZBKciBMKZLkMURxoh6ahqfaGgNhc3v2jDpsW+7ynqVOnTjXv80tIdddTX5/PkodTXafO+URVQUSXvi5pT4CISoNlJ3KCZSdygmUncoJlJ3Kiayk31q9fPx00aFApN1kSzc3NZt7Y2Bjr54uImQ8bNiw0u/zyy2Nt+9y5c2be0tJi5qdPnw7NevToYY6NmntFRYWZe9TQ0ICWlpZ2/8LEKruI3AlgGYAKAC+paq31+EGDBiGbzcbZZFl67rnnzPyRRx6J9fMrKyvNfPny5aHZxIkTY2375MmTZl5fX2/mTU1Nodno0aPNsXfccYeZ9+rVy8y7dPH3wjWTyYRmBT8bIlIB4L8BTAJwHYBpInJdoT+PiJIV55++MQA+UdX9qtoK4A8AaoozLSIqtjhl7w/gYJvvG4P7vkNEZopIVkSyuVwuxuaIKI44ZW/vTYDvffZWVetUNaOqmaqqqhibI6I44pS9EcCANt//CED4uzFElKo4Zd8K4BoRGSwi3QD8DMC64kyLiIqt4ENvqnpGRB4C8D/IH3p7WVX3FG1mnUjcX0+ijqPPnj3bzG+77bbQ7MSJE+bYtWvXmrl1WA8AtmzZYuZxRB1ae+ONN8w8zmHHuGeDRv0/TUOs4+yq+g6Ad4o0FyJKkL9PHRA5xbITOcGyEznBshM5wbITOcGyEzlR0vPZL1VRx7KjjBs3zsznzp1b8M9esGCBma9YsaLgn520gQMHmrl1Omdc5XicPC7u2YmcYNmJnGDZiZxg2YmcYNmJnGDZiZzgobciuPfee8086lLS8+fPN/O+ffua+ZEjR0Kz1157zRzbvXt3Mz916pSZR10afNGiRaHZZ599Zo594oknzPzZZ58186eeeio0i7pi76WIe3YiJ1h2IidYdiInWHYiJ1h2IidYdiInWHYiJyTuJXMvRiaT0UtxFde0tba2hmabN282x/bv/70Vu75j1qxZBW87avtRy0FHnfr74YcfmvmDDz4Ymi1btswc21mPw2cyGWSz2XbPz+WencgJlp3ICZadyAmWncgJlp3ICZadyAmWncgJns9+CejWrVtoNmHChFg/e+zYsWa+ZMkSM9+2bVtodvPNN5tjb7zxRjPfvn27mVuXybaWuQaAqVOnmnlnFKvsItIA4GsAZwGcUdXkLuRNRLEUY89+m6q2FOHnEFGC+Ds7kRNxy64ANojINhGZ2d4DRGSmiGRFJJvL5WJujogKFbfs41T1xwAmAZgtIuMvfICq1qlqRlUzVVVVMTdHRIWKVXZVbQpujwL4M4AxxZgUERVfwWUXkZ4i0uv81wAmAthdrIkRUXEVfD67iAxBfm8O5N/Vf01Vf2mN4fnsnU9DQ4OZDx8+3MxvuOGG0Gz9+vXm2E2bNpl5TU2NmVuGDRtm5jt37jTzyy67rOBtJ8k6n73gQ2+quh/AyIJnRUQlxUNvRE6w7EROsOxETrDsRE6w7ERO8BRXMg0cONDMp0+fbuYrV64MzebNm2eOXbhwoZn37t3bzL/88svQbP/+/ebYqEtw33LLLWYetRR2GrhnJ3KCZSdygmUncoJlJ3KCZSdygmUncoJlJ3KCSzZTLLt27TLzMWPCr2dy6tQpc+zgwYPN/NNPPzXzqCWhLSNGjDDzjRs3mnl1dXXB246DSzYTEctO5AXLTuQEy07kBMtO5ATLTuQEy07kBM9nd662ttbMu3Sx9wcffPCBmbe2tl70nM47cOBAwWOjRB0HX7x4cazx5Yh7diInWHYiJ1h2IidYdiInWHYiJ1h2IidYdiIneJzduajro0edr37ixAkzt5Y2jhqbpPr6ejOfNGlSiWZSOpF7dhF5WUSOisjuNvddISIbRWRfcNsn2WkSUVwdeRn/GwB3XnDfQgCbVPUaAJuC74mojEWWXVU3A/j8grtrAKwKvl4F4J4iz4uIiqzQN+iuUtXDABDcXhn2QBGZKSJZEcnmcrkCN0dEcSX+bryq1qlqRlUzVVVVSW+OiEIUWvZmEakGgOD2aPGmRERJKLTs6wDMCL6eAWBtcaZDREmJPM4uIq8DuBVAPxFpBPALALUA/igi9wP4J4CpSU6SkrN69Wozb2lpMfOoa7MfPHgwNHv33XfNsUuWLDHz06dPm7nl0KFDBY/trCLLrqrTQqKfFnkuRJQgflyWyAmWncgJlp3ICZadyAmWncgJnuLqXM+ePWPlUaxll8eOHWuObW5uNvNXXnmloDkBwOOPP27m48ePN/Nrr7224G2nhXt2IidYdiInWHYiJ1h2IidYdiInWHYiJ1h2Iid4nJ1S07Wr/ddvzpw5Zv7222+b+dGj4ddUsTIAWL58uZkvXbrUzCsqKsw8DdyzEznBshM5wbITOcGyEznBshM5wbITOcGyEznB4+xUtkaOHGnmUcfho85Zt9TV1Zn59OnTzTyTyRS87aRwz07kBMtO5ATLTuQEy07kBMtO5ATLTuQEy07kBI+zU6fVp0+fxH521HLQTU1NiW07KZF7dhF5WUSOisjuNvc9KSKHRGRH8GdystMkorg68jL+NwDubOf+pap6ffDnneJOi4iKLbLsqroZwOclmAsRJSjOG3QPicjO4GV+6C9PIjJTRLIiks3lcjE2R0RxFFr2XwMYCuB6AIcBLAl7oKrWqWpGVTNVVVUFbo6I4iqo7KrarKpnVfUcgJUAxhR3WkRUbAWVXUSq23w7BcDusMcSUXmIPM4uIq8DuBVAPxFpBPALALeKyPUAFEADgFkJzpGoXcOGDUvsZ6uqmX/xxReJbTspkWVX1Wnt3F2fwFyIKEH8uCyREyw7kRMsO5ETLDuREyw7kRM8xTVw9uxZM9+yZUto9tZbb5ljo5YHjjJ5sn1S4aRJk0Kznj17mmPPnTtn5l26lO/+4NSpU4n97NGjR5t5TU1NYttOSvn+nySiomLZiZxg2YmcYNmJnGDZiZxg2YmcYNmJnHBznL21tdXMZ8+ebeZvvvlmaLZmzRpz7NatW818z549Zv7CCy+Y+erVq0OzqOPox48fN/OoZZP79+9v5n379g3NTp48aY796quvzPz999838zgmTJhg5r17905s20nhnp3ICZadyAmWncgJlp3ICZadyAmWncgJlp3ICTfH2T/++GMzf/XVV818ypQpodn48ePNsVF5lKhj5d98801oFvXfvWHDBjP/9ttvzTybzZr57t3hSwpEfb4gattxdO1q/9Wvrq42886Ie3YiJ1h2IidYdiInWHYiJ1h2IidYdiInWHYiJ9wcZ6+srDTziooKMz906FAxp3NRoq7dbl0bftSoUebYqDwu6zoCjY2N5tiGhgYzj7pOwIEDB0KzESNGmGPvu+8+M++MIvfsIjJARP4iIntFZI+IzAnuv0JENorIvuC2T/LTJaJCdeRl/BkA81X1XwHcBGC2iFwHYCGATap6DYBNwfdEVKYiy66qh1V1e/D11wD2AugPoAbAquBhqwDck9QkiSi+i3qDTkQGARgF4G8ArlLVw0D+HwQAV4aMmSkiWRHJ5nK5eLMlooJ1uOwi8gMAawD8XFWPdXScqtapakZVM1VVVYXMkYiKoENlF5FK5Iv+e1X9U3B3s4hUB3k1gHhLlRJRoiIPvYmIAKgHsFdVf9UmWgdgBoDa4HZtIjMskqjTJaMuNb158+bQbPHixebYBQsWmHk5L4scV7du3UKzIUOGmGOj8ttvv72gOXnVkePs4wD8B4BdIrIjuO8x5Ev+RxG5H8A/AUxNZopEVAyRZVfVvwKQkPinxZ0OESXl0n39SETfwbITOcGyEznBshM5wbITOeHmFNfhw4eb+U033WTm7733Xmj26KOPmmObmprM/OmnnzbzHj16mDlRR3DPTuQEy07kBMtO5ATLTuQEy07kBMtO5ATLTuSEm+Ps1nnVADBv3jwz37JlS2h25swZc+yyZcvM/KOPPjLzl156ycyvvvrq0Cx/OQIi7tmJ3GDZiZxg2YmcYNmJnGDZiZxg2YmcYNmJnHBznD3K3XffbeYPP/xwaPbiiy+aY6OOw69fv97MM5mMmVtzi/r8QPfu3c2cLh3csxM5wbITOcGyEznBshM5wbITOcGyEznBshM50ZH12QcA+C2AHwI4B6BOVZeJyJMA/gtALnjoY6r6TlITTVrUGunPPPNMaHbXXXeZYx944AEzb2hoMPMjR46Y+aJFi0KzyspKc+z8+fPN/FJeO96bjnyo5gyA+aq6XUR6AdgmIhuDbKmqPpfc9IioWDqyPvthAIeDr78Wkb0A+ic9MSIqrot6jSYigwCMAvC34K6HRGSniLwsIn1CxswUkayIZHO5XHsPIaIS6HDZReQHANYA+LmqHgPwawBDAVyP/J5/SXvjVLVOVTOqmqmqqirClImoEB0qu4hUIl/036vqnwBAVZtV9ayqngOwEsCY5KZJRHFFll3ylyetB7BXVX/V5v7qNg+bAmB38adHRMUiqmo/QOQnAN4DsAv5Q28A8BiAaci/hFcADQBmBW/mhcpkMprNZmNOufM5fNh8WvD888+b+YoVK8z82LFjoVnUobPt27eb+ciRI82cyksmk0E2m233+uEdeTf+rwDaG9xpj6kTecRPTBA5wbITOcGyEznBshM5wbITOcGyEznBS0mXQHV1tZnX1taa+dy5c828rq4uNNu3b585dujQoWZOlw7u2YmcYNmJnGDZiZxg2YmcYNmJnGDZiZxg2YmciDyfvagbE8kB+LTNXf0AtJRsAhenXOdWrvMCOLdCFXNu/6Kq7V7/raRl/97GRbKqai8+npJynVu5zgvg3ApVqrnxZTyREyw7kRNplz38Q93pK9e5leu8AM6tUCWZW6q/sxNR6aS9ZyeiEmHZiZxIpewicqeIfCQin4jIwjTmEEZEGkRkl4jsEJFUL3IfrKF3VER2t7nvChHZKCL7gtt219hLaW5Pisih4LnbISKTU5rbABH5i4jsFZE9IjInuD/V586YV0met5L/zi4iFQA+BnAHgEYAWwFMU9W/l3QiIUSkAUBGVVP/AIaIjAdwHMBvVfXfgvsWA/hcVWuDfyj7qOqjZTK3JwEcT3sZ72C1ouq2y4wDuAfAfyLF586Y17+jBM9bGnv2MQA+UdX9qtoK4A8AalKYR9lT1c0APr/g7hoAq4KvVyH/l6XkQuZWFlT1sKpuD77+GsD5ZcZTfe6MeZVEGmXvD+Bgm+8bUV7rvSuADSKyTURmpj2Zdlx1fpmt4PbKlOdzochlvEvpgmXGy+a5K2T587jSKHt7S0mV0/G/car6YwCTAMwOXq5Sx3RoGe9SaWeZ8bJQ6PLncaVR9kYAA9p8/yMATSnMo12q2hTcHgXwZ5TfUtTN51fQDW6Ppjyf/1dOy3i3t8w4yuC5S3P58zTKvhXANSIyWES6AfgZgHUpzON7RKRn8MYJRKQngIkov6Wo1wGYEXw9A8DaFOfyHeWyjHfYMuNI+blLfflzVS35HwCTkX9H/h8AHk9jDiHzGgLgf4M/e9KeG4DXkX9Zdxr5V0T3A+gLYBOAfcHtFWU0t98hv7T3TuSLVZ3S3H6C/K+GOwHsCP5MTvu5M+ZVkueNH5clcoKfoCNygmUncoJlJ3KCZSdygmUncoJlJ3KCZSdy4v8APjwZarON7DQAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "LABEL: 1\n" ] } ], "source": [ "#let's try the plotting function\n", "plot_input(X_train,y_train,10)\n", "plot_input(X_test,y_test,100)\n", "plot_input(X_test,y_test,1000)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## TO DO 1\n", "\n", "Now use a feed-forward Neural Network for prediction. Use the multi-layer perceptron classifier, with the following parameters: max_iter=200, alpha=1e-4, solver='sgd', tol=1e-4, learning_rate_init=.1, random_state=ID (this last parameter ensures the run is the same even if you run it more than once). The alpha parameter is the regularization term.\n", "\n", "Then, using the default activation function, pick four or five architectures to consider, with different numbers of hidden layers and different sizes. It is not necessary to create huge neural networks, you can limit to 3 layers and, for each layer, its maximum size can be of 50. Evaluate the architectures you chose using GridSearchCV with cv=5.\n", "\n", "You can reduce the number of iterations if the running time is too long on your computer.\n" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "C:\\Users\\Anay Deshpande\\Anaconda3\\lib\\site-packages\\sklearn\\neural_network\\multilayer_perceptron.py:566: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", " % self.max_iter, ConvergenceWarning)\n", "C:\\Users\\Anay Deshpande\\Anaconda3\\lib\\site-packages\\sklearn\\neural_network\\multilayer_perceptron.py:566: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", " % self.max_iter, ConvergenceWarning)\n", "C:\\Users\\Anay Deshpande\\Anaconda3\\lib\\site-packages\\sklearn\\neural_network\\multilayer_perceptron.py:566: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", " % self.max_iter, ConvergenceWarning)\n", "C:\\Users\\Anay Deshpande\\Anaconda3\\lib\\site-packages\\sklearn\\neural_network\\multilayer_perceptron.py:566: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", " % self.max_iter, ConvergenceWarning)\n", "C:\\Users\\Anay Deshpande\\Anaconda3\\lib\\site-packages\\sklearn\\neural_network\\multilayer_perceptron.py:566: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", " % self.max_iter, ConvergenceWarning)\n", "C:\\Users\\Anay Deshpande\\Anaconda3\\lib\\site-packages\\sklearn\\neural_network\\multilayer_perceptron.py:566: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", " % self.max_iter, ConvergenceWarning)\n", "C:\\Users\\Anay Deshpande\\Anaconda3\\lib\\site-packages\\sklearn\\model_selection\\_search.py:814: DeprecationWarning: The default of the `iid` parameter will change from True to False in version 0.22 and will be removed in 0.24. This will change numeric results when test-set sizes are unequal.\n", " DeprecationWarning)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "RESULTS FOR NN\n", "\n", "Best parameters set found:\n", "{'hidden_layer_sizes': (40,)}\n", "Score with best parameters:\n", "0.7316666666666667\n", "\n", "All scores on the grid:\n", "[0.66666667 0.69666667 0.73166667 0.675 0.55166667]\n" ] } ], "source": [ "# these are sample values but feel free to change them as you like, try to experiment with different sizes!!\n", "parameters = {'hidden_layer_sizes': [(10,), (20,), (40,), (20,20,), (40,20,10) ]}\n", "\n", "mlp = MLPClassifier(max_iter=100, alpha=1e-4, solver='sgd',\n", " tol=1e-4, random_state=ID,\n", " learning_rate_init=.1)\n", "\n", "mlp_arch_CV = GridSearchCV(mlp, parameters, cv=5)\n", "\n", "mlp_arch_CV.fit(X_train, y_train)\n", "\n", "print ('RESULTS FOR NN\\n')\n", "\n", "print(\"Best parameters set found:\")\n", "print(mlp_arch_CV.best_params_)\n", "\n", "print(\"Score with best parameters:\")\n", "print(mlp_arch_CV.best_score_)\n", "\n", "print(\"\\nAll scores on the grid:\")\n", "print(mlp_arch_CV.cv_results_['mean_test_score'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### TO DO 2\n", "\n", "Now try also different batch sizes, while keeping the best NN architecture you have found above. Remember that the batch size was previously set to the default value, i.e., min(200, n_samples). \n", "Recall that a batch size of 1 corresponds to baseline SGD, while using all the 480 training samples (there are 600 samples but in cross validation with 5 folders we use 1/5 of them for validation at each round) corresponds to standard GD and using a different mini-batch size lies in the middle between the two extreme cases." ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "C:\\Users\\Anay Deshpande\\Anaconda3\\lib\\site-packages\\sklearn\\neural_network\\multilayer_perceptron.py:566: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", " % self.max_iter, ConvergenceWarning)\n", "C:\\Users\\Anay Deshpande\\Anaconda3\\lib\\site-packages\\sklearn\\neural_network\\multilayer_perceptron.py:566: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", " % self.max_iter, ConvergenceWarning)\n", "C:\\Users\\Anay Deshpande\\Anaconda3\\lib\\site-packages\\sklearn\\neural_network\\multilayer_perceptron.py:566: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", " % self.max_iter, ConvergenceWarning)\n", "C:\\Users\\Anay Deshpande\\Anaconda3\\lib\\site-packages\\sklearn\\neural_network\\multilayer_perceptron.py:566: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", " % self.max_iter, ConvergenceWarning)\n", "C:\\Users\\Anay Deshpande\\Anaconda3\\lib\\site-packages\\sklearn\\neural_network\\multilayer_perceptron.py:566: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", " % self.max_iter, ConvergenceWarning)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "RESULTS FOR NN\n", "\n", "Best parameters set found:\n", "{'batch_size': 32}\n", "Score with best parameters:\n", "0.75\n", "\n", "All scores on the grid:\n", "[0.08 0.75 0.72833333]\n" ] } ], "source": [ "# these are sample values corresponding to baseline SGD, a reasonable mini-batch size and standard GD\n", "# again feel free to change them as you like, try to experiment with different batch sizes!!\n", "parameters = {'batch_size': [1, 32, 480]}\n", "\n", "# need to specify that you would like to use the standard k-fold split otherwise sklearn create splits of different sizes\n", "kf = sklearn.model_selection.KFold(n_splits=5)\n", "best_hidden_layer_sizes = mlp_arch_CV.best_params_['hidden_layer_sizes']\n", "mlp = MLPClassifier(best_hidden_layer_sizes, \n", " max_iter=100, alpha=1e-4, solver='sgd',\n", " tol=1e-4, random_state=ID,\n", " learning_rate_init=.1)\n", "\n", "# recall to use cv=kf to use the k-fold subdivision seen in the lectures\n", "mlp_batch_CV = GridSearchCV(mlp, parameters, cv=kf)\n", "\n", "mlp_batch_CV.fit(X_train, y_train)\n", "\n", "print ('RESULTS FOR NN\\n')\n", "\n", "print(\"Best parameters set found:\")\n", "print(mlp_batch_CV.best_params_)\n", "\n", "print(\"Score with best parameters:\")\n", "print(mlp_batch_CV.best_score_)\n", "\n", "print(\"\\nAll scores on the grid:\")\n", "print(mlp_batch_CV.cv_results_['mean_test_score'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### QUESTION 1\n", "\n", "What do you observe for different architectures and batch sizes? How do the number of layers and their sizes affect the performances? What do you observe for different batch sizes, in particular what happens to the training convergence for different batch sizes (notice that the algorithm could not converge for some batch sizes)?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## [ANSWER TO QUESTION 1]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### TODO 3:\n", "\n", "Plot the train and test accuracies as a function of the numbero of neurons in your neural network. Print also the computation time for the various configuration you try." ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Training MLP of size (10,) ...\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "C:\\Users\\Anay Deshpande\\Anaconda3\\lib\\site-packages\\sklearn\\neural_network\\multilayer_perceptron.py:566: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", " % self.max_iter, ConvergenceWarning)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Done, training time: 0.35 sec\n", "\n", "Training MLP of size (20,) ...\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "C:\\Users\\Anay Deshpande\\Anaconda3\\lib\\site-packages\\sklearn\\neural_network\\multilayer_perceptron.py:566: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", " % self.max_iter, ConvergenceWarning)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Done, training time: 0.39 sec\n", "\n", "Training MLP of size (40,) ...\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "C:\\Users\\Anay Deshpande\\Anaconda3\\lib\\site-packages\\sklearn\\neural_network\\multilayer_perceptron.py:566: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", " % self.max_iter, ConvergenceWarning)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Done, training time: 0.56 sec\n", "\n", "Training MLP of size (20, 20) ...\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "C:\\Users\\Anay Deshpande\\Anaconda3\\lib\\site-packages\\sklearn\\neural_network\\multilayer_perceptron.py:566: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", " % self.max_iter, ConvergenceWarning)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Done, training time: 0.43 sec\n", "\n", "Training MLP of size (30, 30, 20) ...\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "C:\\Users\\Anay Deshpande\\Anaconda3\\lib\\site-packages\\sklearn\\neural_network\\multilayer_perceptron.py:566: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", " % self.max_iter, ConvergenceWarning)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Done, training time: 0.59 sec\n", "\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA3AAAAFNCAYAAABBkY2NAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3gVZfrG8e+TBiRA6AFCD116lRp0rVixY68o1tV1d3XXbbru/tbdtWBBsaBiJ3ZFXVvoNfTeSyDUQAoh/f39cQ4aESRAkjnl/lxXLpOZOXPuM4bMec688z7mnENEREREREQCX4TXAURERERERKR8VMCJiIiIiIgECRVwIiIiIiIiQUIFnIiIiIiISJBQASciIiIiIhIkVMCJiIiIiIgECRVwIuVgZpFmlmtmLbzOIiIiIiLhSwWchCR/sXXwq9TMDpT5+apj3Z9zrsQ5V9M5t7ky8oqIiFS1ij5XltnvLDO7uiKzisiPorwOIFIZnHM1D35vZhuBm51z3xxpezOLcs4VV0W2qhbKr01ERI7fsZ4rQ4XOixLsdAVOwpKZ/d3M3jWzt80sB7jazAb4PzXcZ2YZZjbGzKL920eZmTOzVv6f3/Cv/8LMcsxsppm1PsJzRZhZiplt9+871cw6lVkfa2ZPmNlmM8sysylmVs2/bqg/U5aZbTGza/zLp5nZ9WX2cbOZpR6S9XYzWwus9C9/xszSzSzbzOaa2cAyj48ysz+Z2Tr/+nlm1tTMXjCzfx3yer4wsztP/P+CiIgEMv/tA38ys/VmttvM3jSzOv51cWb2jpll+s9ts82srpn9F+gLvOS/kvffw+w3yszeN7Md/sd+b2YdyqyP859jt/jPf5PNLMq/bliZ8+JmM7vSv/wnV/3M7DYz+8b/fXX/eXG0ma0DlvqXjy1zXpxjZicfkvEv/td+8LzZ2MxeNrNHD3k9X5vZbRV46EV+kQo4CWcjgLeAeOBdoBi4B2gADALOAm79hcdfCfwJqAdsBh75hW0/A9oBjfGdOCaUWfcE0A3o79/XH4BSf0H4OfA4UB/oCSw5htd3Pr6TaFf/z7P9z1MPSAEmHiwUgd8Cl+B7zXWAm4F84DXgSjOLADCzBCAZeOcYcoiISHD6LXAGMBhoBhThO2eB7zwRBSTiO2/eCRQ6534DzMV3Na+m/+fD+QRIwndeXInvfHPQGKAjvnNYPeAhwJlZW3zn03/jOy/2BpYdw+s51/+Ynv6fZ+I7R9YHPsZ3Xoz2r3sQuND/+usAo/jpedEAzKwpvvcM7x1DDpETogJOwtk059ynzrlS59wB59xc59xs51yxc249MA5fsXIkKc65ec65IuBNoMfhNvLv/1XnXI5zLh/4K9Db/wljJHA9cLdzLsN/r900/z6vBr50zr3nz7TbObfwGF7fP5xze51zB/w5JjjnMv3DRh4DagNt/dveDPzBObfGn3ehf9sZwIEyx2Ek8I1zbvcx5BARkeB0K/CAc26b//z1N+Byf/FSBDQEkvznqLnOuf3l2al/+9ecc7ll9tvPf6UsGrgWuMs5t91/XpzqnCsBrgE+dc6979/HLufcomN4PY865/aVOS++7j9PFgH/wFfItfFve7P/ta/1nxcXOOf2AVMBh6+oBd+HuV865zKPIYfICVEBJ+FsS9kfzKyjmX3uH+qYDTyM71PFI9le5vs8oObhNvIPQXns4DAMYK1/VQMgAYgB1h3moc2PsLy8Dn19vzOzlWaWBewF4vjx9f3Sc72Or5jE/98JR9hORERChL9Iaw5M8g9z3AcswPfesT7wMjAZSPEPQ/yH/0PJ8uw7ysz+U+a8uBIw/36b4Luyt/4wD63o8+KDZraqzHmxOtDA/9oTD/dczjmHzoviMRVwEs7cIT+/gG94Y1vnXG3gz/hOKCfqWmA4cCq+4ZoHr3oZsAMoxDeM5FBbjrAcYD8QW+bnxofZ5ofXZ2anAPcBF+MbClIXyOXH1/dLzzUBuMjMevq3+fQI24mISIjwFypbgVOdc3XKfFX3jwgpcM792TnXERgKXApccfDhR9n9DfiGJp6C77zY0b/cgAx8tzS0OczjKvK8eDpwF77bKergG6p5ALAyr/1Iz/U6cImZ9cZXVH5+hO1EKoUKOJEf1QKygP3mm2Tkl+5/O9b9FgB78J1cfrj52T8k5FXgSf/N0ZFmNsg/hOQN4Cwzu9j/aWUDM+vuf+hC4GIzq2Fm7YEby5GhGNgNROMbxhlXZv1LwN/NLMl8ephZPX/GTf7new2Y6B/uIiIioe954P/MrDmAmTUys/P8359mZp3990hn4zvHlPgft4PDF2AH1cJ3P9kefOeivx9c4R/O+DrwlJkl+M+Lg/1X914HzjWzEf7lDc2sm/+hC/EVVdXNrCO+2xN+SS18w0B34RsJ8zC+K3AHvQT8w8za+M+LPc0/gYv/NovlwHjgXedc4VGeS6RCqYAT+dFvgOuAHHxX496toP2OB7b5v5YBMw5Zfy+wAkgDMvGNwzfn3AbgPOD3/uXz+XFCkv/g+yRxJ/AKvmLvl0wCvgHWABvxnWwzyqz/N/AR8K1/3Th+eiJ7zf/cGiYiIhI+HsN37vjOfDM2zwB6+dcl4pv4Iwff6JVJ/DiRxxPAtWa218weO8x+X8ZXOG3HNznXtEPW341v+OICfEXeI/jOi+uAC/BN9rUXmAecVCZrlH+/4zj6efFTYIr/edbj+4BzV5n1/4fvytp3+M6LzwPVyqzXeVE8Y76rxCIiR2Zmp+I74bZx+qMhIiJhzszOAJ5zzrU96sYiFUxX4ETkF5lZDL72Ci+qeBMRkXDnPy/eje9Kn0iVUwEnIkdkZl3xDVOph68vj4iISNgysx74zou1gGc9jiNhSkMoRUREREREgoSuwImIiIiIiASJoxZwZvaKme00s6VHWG9mNsbM1prZYjPrVWbddWa2xv91XUUGFxERERERCTdHHUJpZkPxNfx93TnX5TDrh+NrhDgc6A885Zzr7+8hNQ/og2+68zSgt3Nu7y89X4MGDVyrVq2O46X81P79+4mLizv6hnJMdFwrj45t5dBxrTwVcWzT0tJ2O+caVlCkkFcR50j9m6g8OraVQ8e18ujYVo7KPj9GHe3BzrkpZtbqFza5AF9x54BZZlbHzJoAw4CvnXOZAGb2NXAW8PYvPV+rVq2YN2/e0WIdVWpqKsOGDTvh/chP6bhWHh3byqHjWnkq4tia2aaKSRMeKuIcqX8TlUfHtnLouFYeHdvKUdnnx4q4By4R2FLm53T/siMtFxERERERkeNw1Ctw5WCHWeZ+YfnPd2A2ChgFkJCQQGpq6gmHys3NrZD9yE/puFYeHdvKoeNaeXRsRUREql5FFHDpQPMyPzcDtvmXDztkeerhduCcG4e/GWKfPn1cRVzK1SXhyqHjWnl0bCuHjmvl0bEVERGpehUxhPIT4Fr/bJQnA1nOuQzgK+AMM6trZnWBM/zLREREQoKZnWVmq/wzMT9wmPVPmNlC/9dqM9tXZl1JmXWfVG1yEREJVke9Amdmb+O7ktbAzNKBvwDRAM6554FJ+GagXAvkATf412Wa2SPAXP+uHj44oYmIiEiwM7NI4FngdHyjTuaa2SfOueUHt3HO3Vtm+7uAnmV2ccA516Oq8oqISGgozyyUI4+y3gF3HGHdK8ArxxdNREQkoPUD1jrn1gOY2Tv4ZmZefoTtR+L7EFREROS4VcQQShERkXBU7tmWzawl0Br4rszi6mY2z8xmmdmFlRdTRERCSUVMYiIiIhKOyj3bMnAFkOKcKymzrIVzbpuZtQG+M7Mlzrl1P3uSCp6pWbOHVh4d28qh41p5dGwrR2UfVxVwIiIix+dIszAfzhUccruBc26b/7/rzSwV3/1xPyvgKnqmZs0eWnl0bCuHjmvl0bGtHJV9XDWEUkREjolzjhenrOdA8ZEuNoWNuUA7M2ttZjH4irSfzSZpZh2AusDMMsvqmlk1//cNgEEc+d45kbCUub+QKat3sWJPydE3FgkjugInIiLH5Ilv1jDm2zVcf1IMZ3sdxkPOuWIzuxNfi5xI4BXn3DIzexiY55w7WMyNBN7xT/p1UCfgBTMrxfdh6v+Vnb1SJNzsyilg6bYslqZn+f67NZut+w4AvrHKw4ftp2X9OG9DigQIFXAiIlJu76elM+bbNVzWpxnJ9dUZxjk3CV87nbLL/nzIz389zONmAF0rNZxIAHLOsSO7gKVbs1iyNYtl23z/3ZFd8MM2rRvE0atlXa4d0JJWDeK44400xk1Zz6Mj9E9GBFTAiYhIOc1ct4cHPljMoLb1eXREV6ZPneJ1JBEJYM45tmXlsyT9x0Jt6dZsduf6ijUzSGpYkwFt6tMlMZ4uifGc1LQ2tapH/2Q/gxKjmJiWzj2ntaNRrepevBSRgKICTkREjmrtzlxunTCPlvXjeO6q3kRH6hZqEfmRc44tmQdY+kOhlsWybdlk7i8EIDLCaNeoJsntG9I1sTZdEuPp1KQ2cdWO/lZ0eOtopm49wCvTNvLA2R0r+6WIBDwVcCIi8ov25BZw46tziYmKYPz1fYmvEX30B4lIyCotdWzcs5+l27JZ6i/Wlm7NIju/GICoCKN9Qi1O75RAF3+x1rFxbWrERB7X8yXERXB21ya8OWsTt5+SRO3q+hsk4U0FnIiIHFF+UQmjJqSxIzufd0adTPN6sV5HEpEqVFLqWL8r94eJRZZszWL5tmxyC3zFWkxkBB2b1OLc7k3p0jSeLom16dC4FtWijq9YO5LRyUl8vjiDCTM3cccpbSt03yLBRgWciIgcVmmp4/6Ji0jbtJexV/WiZ4u6XkcSkUpUXFLK2l25/nvWfizWDhT5pvGvFhVB56a1GdEzka6J8ZyUWJv2CbWqZEh1l8R4hrZvyPjpG7hpcGuqR1dsgSgSTFTAiYjIYT3+9Wo+W5zBA2d35OyuTbyOIyIVqLC4lNU7cn4yuciKjGwKiksBiI2J5KSmtbm8b3O6JMbTNTGepIZxRHl4/+vtw5K4YtwsJs7bwjUDWnmWQ8RrKuBERORn3pu3hWe+X8vIfs25dWgbr+OIyAnILyph1facH4ZBLt2axartORSW+Iq1WtWi6Ny0Ntec3PKH2SBbN4gjMsI8Tv5T/VvXo2eLOrwwZT0j+7XwtJgU8ZIKOBER+YkZa3fzhw+WMKRdAx6+oAtmgfUmTkSO7EBhCSu2/zi5yJKt2azZkUNxqa+PfHyNaLok1uaGwa3896zF07JeLBEBVqwdjplx+7C23PL6PD5bnMGFPRO9jiTiCRVwIiLyg7U7c7j1jTTaNIzj2at6qV2ASADbX1DM8ozsH5tib81mzc4c/LUa9eJi6JIYzykdGtLVf2WtWd0aQf2hzK86NqJdo5qMTV3HBT2aBvVrETleKuBERASAXTkFXD9+LtWiInnl+r6aqlskgGTnF7Fsa3aZe9ayWL97P85frDWoWY2uibU586QETvLfs9YkvnrIFTgREcZtyUn8ZuIivl+1k1M7JngdSaTKqYATERHyi0q45fV57M4t4N1RA2hWV+0CRLyyL6/Qd6/ath97rG3ck/fD+ibx1TmpaTznd0/8oc9aQu3qHiauWuf3aMrjX6/mue/XqYCTsKQCTkQkzJWWOu57byGL0vcx9qredG9ex+tIImFjT27BTxpiL9maRfreAz+sT6xTg66J8VzSuxldEuM5qWk8DWtV8zCx96IjI7hlSGv++uly5m7MpG+rel5HEqlSKuBERMLcY1+tYtKS7Tx0TifO6tLY6zgiIWtnTr6/UMv237OWxbas/B/Wt6wfS/fmdbiqf0vflbWm8dSNi/EwceC6vG8Lxny3lue+X8v4G/p5HUekSqmAExEJY+/M2czzk9dxVf8W3DS4tddxREKCc47t2fksSc9i6bZslvmvrO3MKQDADFo3iKNPq3o/NMQ+qWk88TV032l51YiJ5IaBrfjv16tZkZFNpya1vY4kUmVUwImIhKmpa3bxx4+Wkty+IX87/6SQm+xApCrN37yXlNWFjF8/h6Vbs9izvxCACIO2jWoyuG2DH3qsdW5am5rV9BbsRF07oBXPT17H2NR1jBnZ0+s4IlVGfz1ERMLQ6h053P7GfNo1qskzV/ZUQ1yRE7AjO5/Lnp9JqXO0T8jn1I6NfijWOjWpRWyM3m5VhvjYaK46uSUvTV3P/Wd0oEV9Tb4k4UF/UUREwszOnHxuGD+XGjG+dgG11C5A5IR8uGArxaWOfw6uwchzh3odJ6zcNLg1r07fyAtT1vHoiK5exxGpEvrIVUQkjBwoLOGW1+aRub+Ql6/rS9M6NbyOJBLUnHOkpKXTp2VdmtTU26qqllC7Ohf3TmRiWjo7c/KP/gCREKC/NCIiYaK01PHrdxeweGsWY0b2pGuzeK8jiQS9hVv2sXZnLpf0buZ1lLA1amgSxSWljJ++0esoIlVCBZyISJj4vy9X8tWyHfzpnM6c3lnNb0UqQkpaOtWjIzinWxOvo4St1g3iOLtrE96YuYns/CKv44hUOhVwIiJh4M3Zmxg3ZT3XDWjJDYNaeR1HJCTkF5XwyaJtnN2lie4l9djo5CRyCop5Y9Ymr6OIVDoVcCIiIW7y6l38+eNlnNKhIX86t7PaBYhUkK+X7yAnv1jDJwNAl8R4hrZvyCvTNpBfVOJ1HJFKpQJORCSErdyezR1vzqd9Qi2evrKX2gWIVKCUtHSaxldnQJv6XkcRfFfhducWMjEt3esoIpVKZ3IRkRC1MzufG8fPJa5aJK9c30eNg0Uq0PasfKau2cXFvZsREaGr2oHg5Db16NmiDuOmrKO4pNTrOCKVRgWciEgIyiss5qbX5rHvQBEvX9eXJvFqFyBSkT5YkE6pg4t7afhkoDAzRicnsSXzAJ8vyfA6jkilUQEnIhJiSkod97yzkGXbsnh6ZE+6JKpdgEhFOtj7rV+rerRqEOd1HCnjtE4JtGtUk7Gp63DOeR1HpFKogBMRCTH/mLSCr5fv4C/nncSvOqldgEhFW7BlH+t37dfkJQEoIsK4LTmJldtz+H7VTq/jiFQKFXAiIiHk9ZkbeXnaBm4Y1IrrBrbyOo5ISJo4L50a0ZEMV++3gHR+j6Yk1qnB2NR1XkcRqRQq4EREQsT3K3fy10+WcVqnRjx0Tmev44iEpPyiEj5btI2zuzTWxEABKjoygluGtGbuxr3M3ZjpdRyRCqcCTkQkBCzfls2db82nU5PaPHVFTyI1K55Ipfhq2XZyCoq5pI+GTwayy/u2oF5cjK7CSUhSASciEuS2Z+Vz46tzqV0jmleu70ucrgqIVJqUtHQS69Tg5Nbq/RbIasREcsPAVny3cicrMrK9jiNSoVTAiYgEsf0Fxdz02lxy8n3tAhJqV/c6kkjIysg6wLS1u9X7LUhcO6AVcTGRPD9ZV+EktKiAExEJUiWljrvfXsCKjGyeuaoXnZvW9jqSSEj7YP5WnIOLeyV6HUXKIT42miv7t+DTRdvYvCfP6zgiFUYFnIhIkHrks+V8u3Inf7ugC6d0aOR1HJGQ9kPvt9b1aFlfvd+Cxc1D2hAVEcG4qboKJ6FDBZyIx5akZ3Hza3OZvrXI6ygSRMZP38CrMzZy8+DWXHNyS6/jiIS8+Zv3smH3fi5V77egklC7Ohf1SuS9eensyinwOo5IhVABJ+KRvMJi/v7Zci54dhqTV+/ixSWF3D9xEXmFxV5HkwD3zfIdPPLZcs7onMCDwzt5HUckLKSkpRMbE8nwrur9FmxuTU6iqKSUV6Zv8DqKSIVQASfigdRVOzn98Sm8NG0DV/RrwZw/nMYFSdG8Pz+d856ephmz5IiWbs3irrcX0CUxniev6KF2ASJV4EBhCZ8uyuDsLk00y2sQat0gjuFdmvDGzE1k52u0iwQ/FXAiVWh3bgH3vLOA68fPpXp0BO/dOoB/jOhK3bgYRrSL4c2b+pOdX8yFz07nrdmbcc55HVkCyLZ9B7jx1bnUjY3mpWv7EBujN5IiVeGrZdvJLSjmEg2fDFqjhyWRU1DMG7M2eR1F5ISpgBOpAs45Js7bwmmPT2bSkgzu+VU7Jt0zhH6t6/1ku4FtGzDpbt/yP3y4hDvfXqBPCwWA3IJibnx1LnmFJbxyQ18aqV2ASJVJSUuneb0a9D/kb7YEjy6J8Qxp14BXpm0kv6jE6zgiJ0QFnEgl27h7P1e9NJvfpiymbcOaTLp7CPee3p5qUZGH3b5hrWq8dkM/fndWB75cup1zx0xjcfq+Kk4tgaS4pJS73prPmp25PHtVLzo2VrsAkaqydd8Bpq/bzcW91Pst2N0+rC27cwuYmJbudRSRE1KuAs7MzjKzVWa21sweOMz6lmb2rZktNrNUM2tWZt2/zGyp/+vyigwvEsiKSkp5LnUtZz45hSXpWfz9wi68d+sA2iXUOupjIyKM24e15d1RJ1NcUsrFY2fw8rQNGlIZhpxz/O3T5Xy/ahePXNCF5PYNvY4kZZTj/PiEmS30f602s32HrK9tZlvN7JmqSy3H4sP56f7ebxo+GexOblOPHs3rMG7KOopLSr2OI3LcjlrAmVkk8CxwNtAZGGlmnQ/Z7D/A6865bsDDwD/9jz0H6AX0APoDvzUzfXQsIW/hln2c9/Q0HvtyFad0aMTX9yVz9cktj/nT2z6t6jHpniEkt2/EI58t55bX09i7v7CSUksgemX6RibM2sStQ9twZf8WXseRMspzfnTO3euc6+Gc6wE8DXxwyG4eASZXRV45dgd7v53cph7N68V6HUdOkJkxelgSWzIP8PmSDK/jiBy38lyB6wesdc6td84VAu8AFxyyTWfgW//335dZ3xmY7Jwrds7tBxYBZ514bJHAlFtQzN8+XcaI56azN6+QF67pzfPX9KZx/PHfr1QnNoYXr+3Nn8/tzOTVOxk+ZirzNmZWYGoJVP9btp2/f76cs7s05vdndfQ6jvxcec6PZY0E3j74g5n1BhKA/1VqSjlu8zbtZeOePC7p3dzrKFJBTu+UQNtGNRmbuk6jWiRolWcKs0RgS5mf0/FdTStrEXAx8BQwAqhlZvX9y/9iZo8DscApwPITDS0SiL5buYOHPlxKRnY+V/dvyW/P6kDt6tEVsm8z48bBrenTqi53vrWAy8fN4r7T2zM6OUn3ZISoxen7uOedhXRrVofHL+uh/8+BqTznR8B3qwHQGvjO/3ME8F/gGuBXv/QkZjYKGAWQkJBAamrqCYXOzc094X2Ei1eWFlA9EmrtXUNq6tqjbq9jWzkq+rgOSyjipSWFjEn5lu4Nw3s2X/3OVo7KPq7l+a093LuGQz+yuB94xsyuB6YAW4Fi59z/zKwvMAPYBcwEftaluKJPTqBfyMqi4/pz+wpKeWtFIXO2l5BY0/hDv+q0q7Ob+bN2H9N+yntsH+gJry6L4N9frWLSvDWM6lad+Gp6c38kwfg7u+dAKQ/Pyic2Em5sW8jsGVO9jnRYwXhsK1h5zo8HXQGkOOcOTn93OzDJObfF7Jf//TrnxgHjAPr06eOGDRt2fGn9UlNTOdF9hIO8wmLu/P5bzuvRjDNP616ux+jYVo6KPq6DSkqZtPl7pu2pwT2XDqyw/QYj/c5Wjso+ruUp4NKBsmMHmgHbym7gnNsGXARgZjWBi51zWf51jwKP+te9Baw59Akq+uQE+oWsLDquP3LO8d68LTyauoL8IsdvTm/PrclJxEQd3+Sux3Jszz7N8c7cLfz1k2X8fV4JT17eg0FtGxzX84a6YPudzckv4pKxMyklkndGDyzXpDdeCbZjWwmOen4s4wrgjjI/DwCGmNntQE0gxsxynXM/mwhFvPHlUvV+C1XRkRHcMrQNf/t0OXM3ZtK3ldpDSHApzzvNuUA7M2ttZjH4TkKflN3AzBr4h4MAPAi84l8e6R9KiZl1A7qhsf4SAtbtyuWKcbP4/ftL6NikNl/8egh3/ardcRdvx8rMGNmvBR/fOYj4GtFc/fJsHv/fKs2qFeSKS0q5460FrNuVy9irewd08SZAOc6PAGbWAaiLbxQKAM65q5xzLZxzrfCNYnldxVtgSUlLp0W92J/165TQcEXfFtSLi2Fs6jqvo4gcs6O+23TOFQN3Al8BK4D3nHPLzOxhMzvfv9kwYJWZrcZ3Q/aj/uXRwFQzW47vCtvV/v2JBKXC4lKe/nYNZz81lRUZ2fzfRV1555aTSWpY05M8HRvX5pM7B3FJr2aM+W4tV744m4ysA55kkRPjnOPPnyxjyupd/P3CLgxupyuqga6c50fwTV7yjtOMCUEjfW8eM9bt4ZLezTjaEFcJTjViIrl+YCu+W7mTFRnZXscROSblunPTOTcJmHTIsj+X+T4FSDnM4/LxzUQpEvTSNu3lwQ8Ws3pHLud0a8JfzutMo1rHP7tkRYmNieLfl3ZnYNv6/PHDpQx/air/vaw7p3ZM8DqaHIOXpm7grdmbGT0siSv6qV1AsDja+dH/81+Pso9XgVcrOJqcgA/mbwXgol6JHieRynTtgJa8MHkdL0xex5NX9PQ6jki5Vc14L5EglpNfxJ8/Xsolz88gJ7+Yl67tw7NX9gqI4q2sET2b8eldg2kcX4MbX53Ho58vp7BYQyqDwZdLM/jHFys4p2sTfntGB6/jiIS1g73fBibVp1ld9X4LZXViY7iyfws+XZzBlsw8r+OIlJsKOJFf8L9l2zn98SlMmLWJ6wa04uv7kjmtc+Be2UpqWJMPbx/INSe35MWpG7j0hZk6KQW4hVv28et3F9KjeR3+e1l3tQsQ8dicDZlszszT5CVh4qbBbYgwGDdlvddRRMpNBZzIYezIzmf0G2mMmpBGndhoPhg9kL+efxI1qwV+v5jq0ZE8cmEXxl7Vi/W7chk+ZiqTlmR4HUsOY0tmHje/NpeGtarx4rV9qB4d6XUkkbCXkpZOzWpRnNWlsddRpAo0jq/Oxb2a8d68LezKKfA6jki5qIATKaO01PHm7E2c9vhkvl25k9+e2YFP7xpMzxZ1vY52zM7u2oRJdw+hTcOa3P7mfB76aAn5RSVHf6BUiawDRdz46lwKi0sZf31fGtSs5nUkkbC3v6CYz5dkcE7XJsTGBP4HdlIxRg1tQ2FJKeOnb/A6iki5qIAT8Vu7M4fLx2ryKC8AACAASURBVM3kjx8upWtiPF/9eih3nNKW6Mjg/WfSvF4sE28dwC1DWvPGrM2MeG4G63bleh0r7BWVlHLHm/PZsHs/z1/Tm7aN1C5AJBB8uXQ7eYUlXNJHwyfDSZuGNRnepQkTZm4iO7/I6zgiRxW870xFKkhBcQlPfrOa4U9NY/WOXP59STfevLk/rRvEeR2tQsRERfDHczrzyvV92J51gPOensYH89O9jhW2nHM89OFSpq3dzT8v6srAJLULEAkUE9O20LJ+LH1aBt+oCzkxo4clkVNQzJuzNnsdReSoVMBJWJu7MZPhT03lyW/WcFaXxnz7m2Qu7dM8JPv+nNoxgUn3DKFL03jue28R909cRF6h2jJWtecnr+fdeVu485S2XNqnuddxRMRvS2Yes9Znckkv9X4LR10S4xnSrgEvT9ug2w0k4KmAk7CUdaCIP3y4hEufn0l+USnjb+jLmJE9Q/4+pCbxNXjrlv7cfWpb3p+fzvnPTGfldjUwrSqTlmTwry9Xcl73ptx3enuv44hIGe/PT8cMLtLsk2Fr9LAkducWkJKmUSoS2FTASVhxzvHFkgxOf3wy78zZzM2DW/O/e4dySodGXkerMlGREdx3RgfeuKk/WQeKuOCZ6bw9ZzPOOa+jhbT5m/dy77sL6d2yLv++pJvaBYgEkNJSx/vz0xmU1IDEOjW8jiMeGdCmPt2b12HclPUUl6iPqgQuFXASNjKyDjBqQhqj35xPg5rV+OiOQTx0bmfigqA1QGUY1LYBk+4eQr/W9XjwgyXc9fYCcnTzdqXYkpnHLa/No3F8dcZd01vtAkQCzJyNmWzJPKDeb2HOzLh9WBKbM/P4XO13JICpgJOQV1rqeH3mRk5/fApT1+ziwbM78vGdg+jWrI7X0TzXsFY1XruhH787qwNfLN3OuU9PY0l6ltexQkpWXhHXj59Dcanjlev7Uj/Eh+mKBKOJ83y93848Sb3fwt3pnRJo26gmY1PXaWSKBCwVcBLSVm3P4ZLnZ/Dnj5fRs0Ud/vfrZG5NTgrq1gAVLSLCuH1YW94ddTJFxaVcNHY6r0zboBNXBSgsLmX0m2lszszjhWt6k9SwpteRROQQ+wuK+WJpBud2a0KNGF0dD3cREcZtyUms3J5D6qpdXscROSy9i5WQlF9Uwn//t4pzn57Kht37efyy7rx+Yz9a1I/1OlrA6tOqHp/fPYTk9g15+LPl3PJ6GvvyCr2OFbScc/zxwyXMWLeHf13cjZPb1Pc6kogcxqQlGeQVlnCper+J3/ndm9I0vjpjU9d5HUXksFTASciZtX4Pw5+aytPfreW8bk359jfDuEjTQpdL3bgYXry2D386tzOTV+9k+FNTmbcx0+tYQem51HVMTEvn7l+146JeemMoEqhS0tJp3SCOXi3U+018YqIiuGVoG+ZszNQ5UAKSCjgJGVl5Rfw+ZTFXjJtFUWkpE27qx+OX96BeXIzX0YKKmXHT4Na8P3ogUZERXD5uFs9+v5bSUg2pLK9PFm3j31+t4oIeTbn3tHZexxGRI9i8J4/ZGzK5pLc+5JOfurxvc+rGRusqnAQkFXAS9JxzfLpoG796fDIp89O5dWgb/vfrZIa0a+h1tKDWrVkdPrt7MGd1acy/v1rFdePnsCunwOtYAS9tUyb3T1xE31Z1eeySbnpTKBLAUvy930b0TPQ6igSY2Jgorh/Ymm9X7lS/VAk4KuAkqG3dd4CbXpvHXW8voEl8dT6+YxAPDu+kG9ErSO3q0Twzsif/GNGVORsyGT5mKjPW7vY6VsDatGc/t7yeRmKdGoy7pg/VovR7KBKoSksd76elM7htA5qq95scxnUDWxIbE8nzugonAUYFnASlklLHK9M2cPrjk5m5bg8PndOJD28fSJfEeK+jhRwz48r+Lfj4zkHUrh7FVS/P5vH/rVKT00PsyyvkhlfnUup87QLqauiuSECbtWEPW/ep95scWZ3YGK7s14JPF2ewJTPP6zgiP1ABJ0FnRUY2F42dwcOfLadvq3r8796h3DykDVFqDVCpOjauzad3DebiXs0Y891arnxpNtuz8r2OFRAKi0u5dUIa6ZkHGHdNH1o3iPM6kogcRUpaOrXU+02O4uYhbYgwGDdlvddRRH6gd7wSNPKLSvjXlys57+lppGfm8dQVPXj1hr40r6fWAFUlNiaK/1zanccv687SrVmc/dQUvl+50+tYnnLO8cAHi5m9IZN/X9qNfq3reR1JRI4it6CYL5Zs59zuTakeraHOcmSN46tzUc9mvDdvi+4Dl4ChAk6CwvS1uznzySmMTV3HiJ6JfPubZC7okagJIjxyUa9mfHrXYBJqV+eGV+fyj0krKCwOzyGVT3+3lg/mb+W+09tzQQ9NhCASDCYtzuBAUYmGT0q53JrchsKSUsZP3+B1FBFABZwEuL37C7l/4iKuemk2Brx1c3/+fWl36sTq/iKvJTWsyUd3DOKak1sybsp6LnthZtjdI/DRgq08/vVqLuqVyF2ntvU6joiUU0paOm0axtGrRR2vo0gQaNOwJmd3acyEmZvIzi/yOo6ICjgJTM45Pl64ldMen8xHC7Zy+7Akvvz1UAa2beB1NCmjenQkj1zYheeu6sW6nbkMHzOVL5ZkeB2rSszZkMnvUhbTv3U9/nlRV10NFgkSG3fvZ85G9X6TYzM6uS05BcW8NXuz11FEVMBJ4NmSmcf14+dyzzsLaVYvlk/vGszvzuqo+xQC2PCuTfj87iG0aRDH6Dfn86ePlpJfVOJ1rEqzYfd+Rk2YR7O6NXjhmt5qFyASRN6fn06EwUU9NXxSyq9rs3iGtGvAy9M2hPT5TYKDCjgJGMUlpbw4ZT1nPDGFuRsz+ct5nflg9EA6NantdTQphxb1Y5l420BuGdKaCbM2MeK5Gazflet1rAq3d38hN4yfQ4QZ42/oq+G8IkHkh95v7RrSOL6613EkyIxOTmJXTgHvz0/3OoqEORVwEhCWbs1ixHMzeHTSCgYm1efr+5K5YVBrIiM0vCWYxERF8MdzOvPydX3IyDrAuU9P48MFoXOiKygu4dYJaWzLymfcNb1pWV/tAkSCycz1e9iWlc+lmrxEjsOApPp0b16HFyavVy9U8ZQKOPHUgcIS/jFpBRc8O52MrHyevbIXL13Xh8Q6NbyOJifgV50S+OKeIXRpGs+97y7itxMXkVdY7HWsE+Kc4/cpi5mzMZP/XNqdPq3ULkAk2KSkpVOrehSnd07wOooEITNjdHISmzPzmLR0u9dxJIypgBPPTFm9izOenOybwbBPM769L5lzujXRTeUhokl8Dd66pT93ndqWlPnpnP/MdFZtz/E61nF78ps1fLRwG789swPnd2/qdRwROUY5+UV8sTSD89X7TU7AGZ0TSGoYx9jUdTjnvI4jYUoFnFS5PbkF3PvuQq59ZQ7RERG8M+pk/nlRN+Jjo72OJhUsKjKC35zRgQk39mdfXhHnPzONt+dsDrqT3vtp6Tz17Rou7d2M24cleR1HRI7D54szyC8qVe83OSEREcZtyUmsyMgmdfUur+NImFIBJ1XGOd/N46c9PpnPFm/j7lPbMumeIZzcpr7X0aSSDW7XgC/uGUK/1vV48IMl3P3OQnKCpJfOrPV7eOCDxQxoU59HR6hdgEiwSklLJ6lhHD2aq/ebnJgLeiTSNL46Y79f53UUCVMq4KRKbN6Tx7WvzOE3ExfRukEcn989hPvO6KBhLGGkYa1qvHZDP357ZgcmLcng3KensSQ9y+tYv2jdrlxunZBGi3qxPH91b2Ki9CdTJBht2L2feZv2cmmf5voQRk5YTFQENw9pw5yNmczbmOl1HAlDejcilaqopJTnJ6/jjCcns2DzPh654CRSbhtI+4RaXkcTD0REGHec0pZ3Rp1MYXEpF42dzvjpGwJySOWe3AJuGD+XqAhj/PX9NMRXJIi9n+br/TaiZ6LXUSREXNGvOXVjo3l+sq7CSdVTASeVZnH6Ps5/Zjr/98VKhrRryNf3DeWaAa2IUGuAsNe3VT0m3T2E5PYN+dunyxk1IY19eYVex/pBflEJoyaksSM7nxev60OL+rFeRxKR41RS6nh/fjpD2zckobZ6v0nFiI2J4vqBrflmxc6gnqBLgpMKOKlw+wuKeeSz5Vz47HT25Bbw/NW9ePHaPjSJV2sA+VHduBhevLYPfzq3M6mrdnLOmGmkbfJ+KEppqeO3KYtJ27SXxy/rQa8Wdb2OJCInYMa63WRk5WvyEqlw1w1sSWxMpK7CSZVTAScV6vtVOznjiSm8PG0DV/ZvwTe/SeasLk28jiUBysy4aXBr3h89kMgI47IXZvFc6lpKS70bUvnEN6v5dNE2fn9WR87ppt9dkWCXkpZOfI1oTuuk3m9SserExnBlvxZ8smgbWzLzvI4jYUQFnFSIXTkF3PX2Am4YP5caMZGk3DaAv1/YldrVdd+QHF23ZnX47O7BnNWlMY99uYrrxs9hd25BleeYOG8LT3+3liv6Nue25DZV/vwiUrGy84v4cul29X6TSnPTkNZEGLw4db3XUSSMqICTE+Kc4715Wzjt8cl8tXQ7957Wns/vHkyfVvW8jiZBpnb1aJ4Z2ZNHR3Rh9oZMzn5qKjPW7q6y55+xdjcPfrCEwW0b8MiFXTRTnUgI+HxxBgXF6v0mladJfA0u6tmMd+duYVdO1X/wKOFJBZwctw2793Pli7P5Xcpi2ifUZNI9g7nntHZUi9KnnHJ8zIyr+rfk4zsGUbt6FFe9PJvHv15NSSUPqVy7M4fb3kijdYM4nru6F9GR+tMoEgomzttCu0Y16dYs3usoEsJGJbehsKSUV2ds8DqKhAm9S5FjVlRSyrPfr+XMJ6ewdFsW/xjRlXdHDaBtI7UGkIrRqUltPrlzMBf1bMaYb9dw5Yuz2J6VXynPtTu3gBtenUtMVASvXN9Xw35FQsS6XbnM37yPS3o30xV1qVRJDWtydpfGvD5zEzn5RV7HkTCgAk6Oybp9JZz39DT+/dUqTuvUiG/vS+bK/i3UGkAqXFy1KP57WXf+e2l3lmzNYviYqXy/cmeFPkd+UQm3vD6PndkFvHRdX5rXU7sAOTZmdpaZrTKztWb2wGHWP2FmC/1fq81sn395SzNL8y9fZma3VX360PZ+WjqREabeb1IlbktOIie/mDdnb/Y6ioQBFXBSbu/O3czfZ+WzL6+IF6/tw3NX9aaReupIJbu4dzM+uXMwjWpV44ZX5/LPSSsoKik94f2Wljp+894iFm7Zx5OX96BH8zoVkFbCiZlFAs8CZwOdgZFm1rnsNs65e51zPZxzPYCngQ/8qzKAgf7l/YEHzKxp1aUPbSWljg/mbyW5fUOdp6RKdGtWh8FtG/DytA3kF5V4HUdCnAo4KZd9eYU8+vkK2teN4Ov7hnJ6Z03HLFWnbaOafHTHIK4+uQUvTFnPpc/PPOEpm//9v1V8viSDB8/uyNld1S5Ajks/YK1zbr1zrhB4B7jgF7YfCbwN4JwrdM4dnPGgGjofV6hpa3ezPVu936Rq3T4siV05Bbw/P93rKBLidMKQchnz7VpyC4q5pnM1aukeIfFA9ehI/n5hV569shfrduZyzpipfLk047j29e7czYxNXceV/VtwyxC1C5DjlghsKfNzun/Zz5hZS6A18F2ZZc3NbLF/H/9yzm2rxKxhJSUtnTqx0fyqUyOvo0gYGZBUn+7N4nlh8nqKK2CkiMiRRHkdQALfxt37mTBrI5f1aU6zWplex5Ewd063JnRNjOeut+dz2xvzuW5ASx4c3qncPZ6mrdnNHz9cytD2DXn4/JM0uYGciMP98hxpytQrgBTn3A9jq5xzW4Bu/qGTH5lZinNux8+exGwUMAogISGB1NTUEwqdm5t7wvsIZPuLHF8sySO5WRQzp02t0ucO9WPrlWA6rkMbFvP0ggL+8953nNwk8N9mB9OxDSaVfVwD/zdLPPfYVyuJiojgvtPbs3z+LK/jiNCifiwTbxvIY1+u5KVpG5i7cS/PXNmTNg1r/uLjVu/IYfQbabRtVJNnr+xJlNoFyIlJB5qX+bkZcKSraFcAdxxuhXNum5ktA4YAKYdZPw4YB9CnTx83bNiwE4gMqampnOg+AtmbszdRXLqUX59/Ml2ruH1AqB9brwTTcR1a6piUPpnJOyP5/RWDA/5DwmA6tsGkso+r3r3IL0rblMmkJdu5NbmNbgSXgBITFcFD53bm5ev6sC3rAOc9PY2PFmw94va7cgq4YfxcqsdE8vL1fTUUWCrCXKCdmbU2sxh8Rdonh25kZh2AusDMMsuamVkN//d1gUHAqipJHeJS0tLpkFCLLom1vY4iYSgiwrgtOYkVGdlMXr3L6zgSospVwJVjmuSWZvatmS02s1Qza1Zm3WP+KZJXmNkYC/SPIuQHzjn+/vkKGtWqxqihuk9IAtOvOiUw6e4hdG5am1+/u5DfpSwir7D4J9scKCzh5tfnsWd/AS9f14fEOjU8SiuhxDlXDNwJfAWsAN5zzi0zs4fN7Pwym44E3nHOlR1e2QmYbWaLgMnAf5xzS6oqe6hauzOHBer9Jh67oEciTeKr81zqOq+jSIg6agFXnmmSgf8ArzvnugEPA//0P3Ygvk8VuwFdgL5AcoWll0r1+ZIMFmzex/1ndCA2RqNtJXA1rVODt285mTtPacvEtHQueGY6q7bnAFDqHPe+u5DF6ft46oqedGumdgFScZxzk5xz7Z1zSc65R/3L/uyc+6TMNn91zj1wyOO+ds51c8519/93XFVnD0UpaVuJjDAu6KmODOKdmKgIbh7ShjkbMknbpLkDpOKV5wpceaZJ7gx86//++zLrHVAdiME3TXI08LMbtCXwFBSX8K8vV9KxcS0u1jTMEgSiIiO4/8wOTLixP3vzijj/mWm8M2cz760q4stl2/nj8E6ceVJjr2OKSCUpKXV8uCCdUzo0pFEtDfkXb43s15w6sdGM1VU4qQTlKeDKM03yIuBi//cjgFpmVt85NxNfQZfh//rKObfixCJLVXh9xia2ZB7gj+d0IjJCw1AkeAxu14BJ9wymb6t6PPDBEr7cWMQ1J7fkpsGtvY4mIpVo6ppd7MguUO83CQixMVFcP7AV36zY+cOIEJGKUp5xceWZJvl+4Bkzux6YAmwFis2sLb5x/gf/mn5tZkOdc1N+8gQVPEUyaFrUE5Fb6HhiSh5dG0RSsnUZqWXmhdBxrTw6thXrxiRH08hotmcXMqz2LiZPnux1pJCj31kJJBPT0qkbG82pHRO8jiICwHUDWjFuynqen7yOJy7v4XUcCSHlKeCOOk2yv/noRQBmVhO42DmX5S/MZjnncv3rvgBOxlfklX18hU6RDJoW9UQ8/Oly8ks28J+rB9Ghca2frNNxrTw6thXvVHRcK5OOrQSKrLwivl62gyv7tyAmShNsS2CoGxfDyH4teHXGRu47vT3N68V6HUlCRHn+yh11mmQza2BmB/f1IPCK//vNQLKZRZlZNL4JTDSEMoCVbdp9aPEmIiISiD5ZvI3CklINn5SAc/OQ1kQYvDh1vddRJIQctYAr5zTJw4BVZrYaSAAe9S9PAdYBS/DdJ7fIOfdpxb4EqUj/+nIl0ZG+pt0iIiLBICUtnY6Na3FSU/V+k8DSJL4GI3om8u7cLezOLfA6joSIcs0N75ybBEw6ZNmfy3yfgq9YO/RxJcCtJ5hRqsi8jZl8sXQ7957WXk27RUQkKKzZkcOiLft46JxO6v0mAenW5CQmpqXz6vSN3H9mB6/jSAjQQHEBftq0+5ahmq1PRESCQ0paOlERxoU9D50gWyQwJDWsyVknNea1mRvJyS/yOo6EABVwAsBnizNYuEVNu0VEJHgUl5TywYKtnNKxEQ1qVvM6jsgRjR6WRE5+MW/N3ux1FAkBKuBETbtFRCQoTV2zm1056v0mga9bszoMbtuAl6ZtIL+oxOs4EuRUwAmvz9hE+l417RYRkeCSkpZOvbgYTunQyOsoIkc1elgSu3IK+GD+1qNvLPILVMCFub37C3n6uzUkt2/IkHYNvY4jIiJSLvvyCvl6+Q4u6NFUvd8kKAxMqk/3ZvG8MGUdxSWlXseRIKa/eGFuzHdryC0o5g/DO3kdRUREpNw+WaTebxJczIzRw5LYtCePL5Zu9zqOBDEVcGFsw+79TJi5icv7qmm3iIgEl5S0dDo3qc1JTeO9jiJSbmd0bkybhnGMTV2Hc87rOBKkVMCFsce+XElMVAT3nqam3SIiEjxWbc9hcXqWrr5J0ImIMG5LTmJ5RjaTV+/yOo4EKRVwYWquv2n3rUOT1LRbRESCyvvzfb3fLujR1OsoIsfswh6JNImvztjUdV5HkSClAi4MHWzanVBbTbtFRCS4FJWU8sH8rZzasRH11ftNglBMVAQ3D2nD7A2ZpG3a63UcCUIq4MLQZ4szWLRlH79R024REQkyU1bvYnduAZf2ae51FJHjdkXf5tSJjdZVODkuKuDCzE+advfSvQMiIhJcUtLSqR8Xw7AOan0jwSuuWhTXD2zFNyt2sGp7jtdxJMiogAszr83YSPreAzx0Tmc17RYRkaCyd38h36zYwYU9E4mO1FsYCW7XDWhFbEwkL0zWVTg5NvrrF0Z8TbvXMqxDQwa3a+B1HBERkWPyyaJtFJU4zT4pIaFuXAwj+7Xg40XbSN+b53UcCSIq4MLImO/WsL+gmAfPVtNuEREJPhPTtnBS09p0alLb6ygiFeLmIa2JMHhxynqvo0gQUQEXJtS0W0REgtmKjGyWbs3mUl19kxDSJL4GI3om8s7cLezOLfA6jgQJFXBh4l9f+Jt2n66m3SIiEnzeT0snOtI4v0ei11FEKtSooUkUlpTy6vSNXkeRIKECLgzM3ZjJl8u2c1tyEo1qqWm3iIgEl6KSUj5auJVfdUygXlyM13FEKlTbRjU5s3NjXp+5kZz8Iq/jSBBQARfiyjbtvnmImnaLiEjwSV21i925hZq8RELW6GFJZOcX89bszV5HkSCgAi7Efepv2n2/mnaLiEiQSknbQoOa1UhW7zcJUd2b12FQ2/q8NG0D+UUlXseRAKcCLoTlF5Xwry9W0qlJbS5S024REQlCe3IL+HbFTkb0bKrebxLSbh/Wll05BXwwf6vXUSTA6S9hCHt95ka27jvAH4d3UtNuEREJSp8s2kZxqeNiDZ+UEDcwqT7dmsXzwpR1lJQ6r+NIAFMBF6LUtFtEREJBSlo6XRPj6dhYvd8ktJkZtw9LYtOePL5YmuF1HAlgKuBC1FPf+pp2/2G4mnaLiEhwWr4tm2XbsjV5iYSNMzo3pk3DOJ77fh3O6SqcHJ4KuBC0Yfd+3pi1icv7tqB9gpp2i4hIcEpJSycmMoLzuzf1OopIlYiIMG4bmsTyjGymrNntdRwJUCrgQtC/vlhJtagI7j29nddRREREjkthsa/322mdG1FXvd8kjFzYM5HGtavz3PdrvY4iAUoFXIiZs0FNu0VEJPilrtpJ5n71fpPwExMVwc1DWjN7QyZpm/Z6HUcCkAq4EFJa6nj08+X+pt1tvI4jIiJy3CampdOwVjWGtlPvNwk/I/u1oE5sNGNT13kdRQKQCrgQ8tmSDBalZ3H/GR2oERPpdRwREZHjsju3gO9X7uSinolEqfebhKG4alFcN6AV36zYweodOV7HkQCjv4ohQk27RUQkVHy8UL3fRK4f2Ioa0ZE8P1lX4eSnVMCFiNdm+Jp2P3SOmnaLiEhwS0lLp3uzeM2kLGGtblwMI/u14JOF20jfm+d1HAkgKuBCQOb+Qp75fi2ndGjIoLZq2i0iIsFr2bYsVmSo95sIwM1DWmMGL03d4HUUCSAq4ELAGH/T7gfVtFtERILcxHm+3m/nqfebCE3r1ODCHom8M3cze3ILvI4jAUIFXJBbvyuXN2Zt4op+atotIlLVzOwsM1tlZmvN7IHDrH/CzBb6v1ab2T7/8h5mNtPMlpnZYjO7vOrTB57C4lI+XriV009KoE6ser+JANyanERBcSmvztjodRQJECrggty/vvQ17f71aWraLSJSlcwsEngWOBvoDIw0s85lt3HO3euc6+Gc6wE8DXzgX5UHXOucOwk4C3jSzOpUXfrA9N3KnezNK9LwSZEy2jaqyZmdG/PajI3k5Bd5HUcCgAq4IDZnQyZfLduhpt0iIt7oB6x1zq13zhUC7wAX/ML2I4G3AZxzq51za/zfbwN2AmHf8CwlLZ1GtaoxRPdzi/zE6GFJZOcX8/aczV5HkQCgAi5IHWza3bh2dTXtFhHxRiKwpczP6f5lP2NmLYHWwHeHWdcPiAHCeq7wXTkFfL9qJyN6qfebyKG6N6/DoLb1eWnqBgqKS7yOIx6L8jqAHJ9PF29jUXoW/7m0u5p2i4h443A9W9wRtr0CSHHO/eSdl5k1ASYA1znnSg/7JGajgFEACQkJpKamHndggNzc3BPeR2X4ckMRJaWOliUZpKbu8DrOcQnUYxvsdFx9BtYpYfraAv759ncMax5dIfvUsa0clX1cVcAFofyiEh77chWdm9RmRM/DftgrIiKVLx1oXubnZsC2I2x7BXBH2QVmVhv4HHjIOTfrSE/inBsHjAPo06ePGzZs2AlEhtTUVE50HxXNOcc/F0ylR/NYrjx3kNdxjlsgHttQoOPqk+wcX22fTur2Iv50VXKF9P3Vsa0clX1cNUYhCB1s2v1HNe0WEfHSXKCdmbU2sxh8Rdonh25kZh2AusDMMstigA+B151zE6sob8Bati2bVTtyNHmJyC8wM0YnJ7FxTx5fLM3wOo54SAVckDnYtPvUjo3UtFtExEPOuWLgTuArYAXwnnNumZk9bGbnl9l0JPCOc67s8MrLgKHA9WXaDPSosvABJiUtnZioCM7rpt5vIr/kjJMa06ZBHGNT1/HTPykSTjSEMsj80LT77I5eRxERCXvOuUnApEOW/fmQn/96mMe9zy3eUgAAIABJREFUAbxRqeGCREFxCR8t3MoZnROIj62Y+3pEQlVkhHFbchK/e38xU9bsJrl92E9eG5Z0BS6IlG3a3U5Nu0VEJAR8t2In+/KKuLRP86NvLCJc2DORxrWrMzZ1rddRxCMq4ILIwabd957W3usoIiIiFSIlLZ2E2tUYrNsCRMolJiqCm4e0Ztb6TOZv3ut1HPGACrggMXv9Hr5atoPRw5JoWKua13FERERO2M6cfFJX7+KiXs00KZfIMRjZrwV1YqMZmxrW7SPDlgq4IFBa6vjHpBU0rl2dmwarabeIiISGjxZspaTUafZJkWMUVy2K6wa04uvlO1izI8frOFLFylXAmdlZZrbKzNaa2QOHWd/SzL41s8X/396dh0dVnv8ff98kQFjDHiFsYRUQWUVZ1CgqS1txwVa62dLW1qXV9qsV++3V+rVVqdpq+6vaqkVttVqNVamCKEtckDXIjksIgQwgyE5YQpbn98c5qWNMQoCZOZmZz+u65sqZM885c597hnl4znlmbjPLNbPO/voLwn5da6WZHTWzyyJ9EImusmj3LeP6qmi3iIgkBOccOXkhhnZtRc/2zYMORyTuXDOqO00apvDIW7oKl2yOO4AzsxTgIWAC0B+YYmb9qzS7H6+WzZnAncA9AM65Bc65wc65wcCFwGHgjQjGn/DCi3ZfoaLdIiKSINZs3c9HO4qZPEw/XiJyMto0a8TVI7owc+U2QnsPBx2OxFBdrsCNAPKdcwXOuWPAc8CkKm36A/P85QXVPA4wGZjtnNM77AQ86Rft/uWX+tFA3w8QEZEEkZMXonFqA750ZsegQxGJWz841/tqzePvbAo4EomlugzgMoGisPshf124VcCV/vLlQAsza1ulzdXAsycTZLLac+gYD833inaP0q9ziYhIgigpK+eVldsYN+A00puo9pvIyerUqgmXDcnkuWVb2F1cEnQ4EiN1KeRd3WWfqqXfbwH+bGbfAd4GtgJl/92BWUdgIDCn2icwuxa4FiAjI4Pc3Nw6hFW74uLiiOwnSE+vL+HQsTLGtjtYb44lEfJaXym30aG8Ro9yKydr7vqd7D9Sqh8vEYmAH53fgxdXhHjyvUL+55K+QYcjMVCXAVwICJ+g3hnYFt7AObcNuALAzJoDVzrn9oc1+SrwknOutLoncM49CjwKMHz4cJednV3X+GuUm5tLJPYTlIJPi8l9422mjOjKN748MOhw/ive81qfKbfRobxGj3IrJysnr4iO6WmM1uwSkVPWq0MLLumfwVPvFfLD83vSvHFd/nsv8awuUyiXAb3NLMvMGuFNhZwZ3sDM2plZ5b5uB2ZU2ccUNH3yhEyf7RXtvllFu0VEJIHsPHCUtz76lCuGZqr2m0iEXJfdiwNHy3h2yZagQ5EYOO4AzjlXBtyIN/1xA/C8c26dmd1pZpf6zbKBD83sIyADuKtyezPrjncF762IRp7AlhTs5o31KtotIiKJ56X3t1Lh4Mqhmj4pEimDu7RiVM+2PP5uASVl5UGHI1FWpzpwzrlZzrk+zrmezrm7/HW/cs7N9JdznHO9/Tbfd86VhG1b6JzLdM5VROcQEouKdouISKJyzvFCXohh3VrTQ7XfRCLq+uxe7DhQwksrtgYdikRZnQZwEjuVRbtvVdFuERFJMKtC+8nfWcxV+vESkYgb3astAzPT+evbBZRXVP29QUkkGsDVI+FFuy9X0W4REUkwOXlFpDVswETVfhOJODPjuuyebNp1iNfXfhJ0OBJFGsDVIyraLSIiiepoaTkzV25j/IDTaJmm2m8i0TBuwGn0aNeMh3PzcU5X4RKVBnD1xO7iEh6an89YFe0WEZEENHfDDg4cLWPysC7HbywiJyWlgfHD83uwbtsB3vl4V9DhSJRoAFdP/GnexxwuLef2iacHHYqIiEjEvbA8RKf0NEb2bBt0KCIJ7bIhmZzWMo1HcjcGHYpEiQZw9cDGT4t5ZskWrj6rC706tAg6HBERkYj6ZP9R3vn4U64c1lm130SirHFqCt8/N4tFBbt5f8veoMORKNAArh74nYp2i4hIAlPtN5HYmjKiK+lNGuoqXILSAC5gi/2i3ddf0EtFu0VEJOE458jJK+Ks7q3p3q5Z0OGIJIVmjVO5ZlR33li/g493HAw6HIkwDeACVFm0u2N6GlNHZwUdjoiISMS9X7SPjZ8eYrJqv4nE1HdGdadJwxT+8lZB0KFIhGkAF6D/rN7G6tB+brlERbtFRCQx5eSFaNIwhS+d2SnoUESSSptmjbh6RBdeWbmVrfuOBB2ORJAGcAGpLNo9oJOKdouISGI6WlrOf1ZtY8IZp9G8cWrQ4Ygkne+f2wOAx97WVbhEogFcQJ5Y6BXt/t+JKtotIiKJ6Y31Ozh4tEzTJ0UCktmqCZcNyeS5ZVvYXVwSdDgSIRrABWB3cQkPL1DRbhERSWw5eSEyWzXhnB6q/SYSlB+d34OSsgqeeq8w6FAkQjSAC4CKdouISKLbvv+IV/ttaKZmmogEqFeHFlzSP4OnFm2muKQs6HAkAjSAi7HKot1TRqhot4iIJK5/r9iKc3Clpk+KBO667F7sP1LKs0u2BB2KRIAGcDE2ffYHpDVMUdFuERFJWM45XswLMSKrDd3aqvabSNAGd2nFqJ5tefzdAkrKyoMOR06RBnAxtLhgN2+u38F12T1p11xFu0VEJDGt2LKPgl2q/SZSn1yX3ZMdB0p4acXWoEORU6QBXIyEF+3+3hgV7RYRkcSVk1dEk4YpTBzYMehQRMQ3plc7zshsyV/fLqC8wgUdjpwCDeBiZOYqr2j3reP6ktZQRbtFRCQxHTlWzqurtjNxYEfVfhOpR8yM67N7sWnXIV5f+0nQ4cgp0AAuBo6WlnPfHK9o92WDVbRbREQS1xvrP+FgiWq/idRH4wacRo92zXjkrXyc01W4eKUBXAz8t2j3l1S0W0REEltOXojOrZtwdlaboEMRkSpSGhg/PL8Ha7ce4N38XUGHIydJA7goqyzafVG/DozqqaLdIiKSuLbtO8K7+bu4cmhnnbAUqacuG5JJRsvGPLxgY9ChyEnSAC7K/ugX7Z42QUW7RUQksf17RQjn0PRJkXqscWoKPzi3B4sKdrNwa6l+0CQOaQAXRSraLSIiycI5R05eiHN6tKFLm6ZBhyMitZgyoiu9OjTnsTXHyL5/AY+/U8CBo6VBhyV1pAFcFE2f/QFNVLRbRESSQN7mvRTuPszkYV2CDkVEjqNZ41Rev+lcbhjcmNNapvHb1zYw8u553DFzHYW7DgUdnhyHft83SiqLdt86rq+KdouISMLLyQvRtFEKE844LehQRKQOUlMacNZpqdx69SjWhPbzxMJNPLNkM08tKmTs6R347ugsRvVsi5m+z1rfaAAXBRUVjrteU9FuERFJDoePlfHqaq/2WzPVfhOJOwM7p/OHrw1m2oTTeXrJFp5ZvJm5G5bQN6MFU8d0Z9LgTNUxrkc0hTIKZq7axpqtKtotIpLozGy8mX1oZvlmNq2axx8ws5X+7SMz2xf22Otmts/MXo1t1JE3Z90nFJeUcZV+vEQkrnVomcbPLu7DwmkXcu/kMzGD215cw6jp87l/zofsOHA06BAFXYGLuKOl5dz7+geckami3SIiiczMUoCHgIuBELDMzGY659ZXtnHO/TSs/Y+BIWG7uA9oCvwwNhFHT05eiK5tmnJWd9V+E0kEaQ1T+OrwLlw1rDOLC/YwY+EmHsrN5y9vbeTLZ3bku6OzGNSlVdBhJi0N4CJsxsJNbNt/lPu/Okg1cEREEtsIIN85VwBgZs8Bk4D1NbSfAvy68o5zbp6ZZUc7yGgL7T3Mext3c/PYPur3RBKMmTGyZ1tG9mzL5t2HeOq9zTy/vIiXV25jWLfWTB2dxbgBGaSmaFJfLGkAF0Fe0e6NKtotIpIcMoGisPsh4OzqGppZNyALmH+iT2Jm1wLXAmRkZJCbm3vCgYYrLi4+5X2Em7nxGM5Bp2NF5OZujdh+41Gkcyse5TV6TjS357WAs85txDuhBry5eR83/HMFbdKMi7qmcn6XhjRrqJM4EP33rAZwEfTHeR9zpLScaRP6BR2KiIhEX3X/U6mpIu7VQI5zrvxEn8Q59yjwKMDw4cNddnb2ie7ic3JzcznVfVRyzvHrZbmM6tmSqyaeE5F9xrNI5lY+o7xGz8nmdgLwmwrH/A92MuPdTTz/0W7+s6mCK4dl8p1RWfTq0DziscaTaL9nNYCLkPydXtHur/uFEUVEJOGFgPCiZ52BbTW0vRq4IeoRxdiywr1s3n2Ym8b2DjoUEYmxlAbGxf0zuLh/Bhu2H+CJhZt4fnmIpxdv4fw+7Zk6JovzerdTGYIo0ITVCKks2n3TRerERESSxDKgt5llmVkjvEHazKqNzKwv0BpYFOP4oi4nr4hmjVIYr9pvIkmtX8eW3Dt5EO9Nu5CfXdyH9dsPcM2MpVz0h7d4evFmDh8rCzrEhKIBXAQs2ribuRt2cF12TxXtFhFJEs65MuBGYA6wAXjeObfOzO40s0vDmk4BnnPOfW56pZm9A7wAjDWzkJmNi1XskXD4WBmvrd7Ol87sSNNGmtAjItCueWN+MrY3C2+7kAe+NoimjVL55ctrGXnPfKbP/oBt+44EHWJC0CfuKaqocNw9awOdVLRbRCTpOOdmAbOqrPtVlft31LDtudGLLPpmr/mEQ8fKmTysy/Ebi0hSaZTagMuHdOaywZnkbd7LjIWbePTtjTz2TgHjzziNqaOzGNq1laZXniQN4E7RK6u2smbrfh742iAV7RYRkaSRkxeiW9umnNW9ddChiEg9ZWYM796G4d3bENp7mL8v2syzS7fw2urtDOqcztQxWUw4oyONUjUp8EQoW6fgaGk5973+IQMz05k0SEW7RUQkORTtOcyigt1MHtpZZ9BFpE46t27KLyb2Y/HtY/nNpAEcPFrGTc+t5Nx75/PQgnz2HDoWdIhxQwO4U1BZtPsXE/upeKmIiCSNf6/YihlcMaxz0KGISJxp1jiVb43sztyfnc8T3z2LPhktuG/Oh4y8Zx7TXlzNh58cDDrEek9TKE/Srv8W7c5gZM+2QYcjIiISExUVjpwVRYzq2ZbMVk2CDkdE4lSDBsYFfTtwQd8OfLTjIE8sLOSl90M8t6yI0b3aMnV0Fhf07aCLJNXQFbiT9Me5lUW7Tw86FBERkZhZWriHoj1HuEo/XiIiEdInowX3XDGQRdPG8vPxfdm48xDfe2o5Y//wFk+9V0hxicoQhNMA7iTk7yzmn0tVtFtERJJPTl6I5o1TGTdAtd9EJLJaN2vE9dm9eOe2C/jTlCG0atqQX89cx8i75/HbV9dTtOdw0CHWC5pCeRIqi3bfrKLdIiKSRA6VlDFrzXYuHdSJJo30y8siEh0NUxpw6aBOXDqoEyu27OWJhYU8+V4hMxZu4uL+GUwdncWIrDZJ+yNKGsCdoMqi3T8f35e2KtotIiJJZPbaTzh8rJzJ+vESEYmRoV1bM7Rra7ZPPJ1/LNrMP5duYc66HQzo1JKpo7P48qCONE5NrhNKmkJ5AioqHHfNWk+n9DSmjlbRbhERSS45eUVktWvGsG6q/SYisdUxvQk/H386i6aN5e7LB3KsrIL/eWEVo6cv4MG5H/HpwZKgQ4wZDeBOwCurtrJ26wFuHd9XRbtFRCSpbNl9mMUFe5g8TLXfRCQ4TRql8PWzu/LGT8/jH98bwcDMljw492NGT5/P/zy/inXb9gcdYtTVaQqlmY0H/gikAI8756ZXebwbMANoD+wBvumcC/mPdQUeB7oADpjonCuM1AHEiop2i4hIMntxRQgzuHyI+kARCZ6ZcW7v9pzbuz0bPy3mqfcKeWF5iBdXhDg7qw1Tx2RxUb8MUhKwDMFxr8CZWQrwEDAB6A9MMbP+VZrdD/zdOXcmcCdwT9hjfwfuc871A0YAOyMReKz97V0V7RYRkeRUUeF4cUWIMb3a0Um130SknunZvjl3TjqDxbeP5RcTTye09wg//Ece2fcv4PF3CjhwtDToECOqLlMoRwD5zrkC59wx4DlgUpU2/YF5/vKCysf9gV6qc+5NAOdcsXMu7n7/c1dxCY/kqmi3iIgkpyWb9hDae0Q/XiIi9Vp604Zce15P3ro1m0e+MZTTWqbx29c2MPLuedwxcx2Fuw4FHWJE1GUKZSZQFHY/BJxdpc0q4Eq8aZaXAy3MrC3QB9hnZv8GsoC5wDTnXPmpBh5LlUW7b5+oot0iIpJ8XsgrooVqv4lInEhNacCEgR2ZMLAja0L7eWLhJp5ZspmnFhUy9vQOfHd0FqN6to3b7/PWZQBX3ZG5KvdvAf5sZt8B3ga2AmX+/s8FhgBbgH8B3wH+9rknMLsWuBYgIyOD3NzcusZfo+Li4ojsZ1txBc8sOcIFXVIpWrf8cyPZZBSpvMoXKbfRobxGj3KbHIpLypi95hMuG5KpH/ASkbgzsHM6f/jaYKZNOJ2nF2/mmSVbmLthCX0zWjB1THcmDY6/z7a6DOBCeD9AUqkzsC28gXNuG3AFgJk1B650zu03sxDwvnOuwH/sZeAcqgzgnHOPAo8CDB8+3GVnZ5/UwYTLzc0lEvv5/lPLaNqolHu/na26b0Qur/JFym10KK/Ro9wmh1lrtnOkVLXfRCS+dWiZxs8u6cv1F/Ri5qptzHh3E7e9uIbfvf4hXx/RlW+N7EZGy7Sgw6yTunwHbhnQ28yyzKwRcDUwM7yBmbUzs8p93Y73i5SV27Y2s/b+/QuB9acedmy8t3EXczfs5PoLemrwJiIiSSknL0SPds0Y2rVV0KGIiJyytIYpfHV4F2bfdC7P/uAchnVrzUO5+YyePp+bn3ufVUX7gg7xuI57Bc45V2ZmNwJz8MoIzHDOrTOzO4HlzrmZQDZwj5k5vCmUN/jblpvZLcA88yaZ5gGPRedQIquiwnH3rA1ktmqiot0iIpKUNu8+xNJNe7h1XN+4/a6IiEh1zIyRPdsysmdbNu8+xJN+GYKXV25jWLfWTB2dxbgBGaSm1L+y2XWqA+ecmwXMqrLuV2HLOUBODdu+CZx5CjEG4uWVXtHuB782OO7mxYqIiETCi3khGhhcOVTTJ0UkcXVr24xff2UAP7u4Dy8sD/Hke4Xc8M8VdEpP49ujujPlrK6kN20YdJj/Vf+GlPXA0dJy7pvzIWd2TufSQZ2CDkdERCTmvNpvWxnTuz2npcfH90JERE5Fi7SGTB2TxYJbsnns28Pp1rYZ02d/wDn3zOOXL68hf2dx0CECdbwCl2z+9u4mtu8/ygNfG6yi3SIikpQWF+xm674j3DZBJXREJLmkNDAu7p/Bxf0zWL/tAE8s3MTzy0M8vXgL5/dpz9QxWZzXu11gU8t1Ba6KyqLdF/fP4JweKtotIiLJKScvRIu0VC7pnxF0KCIigenfqSX3XTWI96ZdyM8u7sP67Qe4ZsZSLn7gbZ5evJnDx8piHpMGcFU8OPcjjpSWM01nHEVEJEkdPFrKrLXbuXRQJ30PXEQEaNe8MT8Z25uFt13IA18bRFrDBvzy5bWMvGc+02d/wLZ9R2IWi6ZQhsnfeZBnlxbxjbO70rN986DDERERCcSsNds5Wlqh2m8iIlU0Sm3A5UM6c9ngTJZv3suMdzfx6NsbeeydAsafcRpTR2fhnItqDBrAhZk++wOaNkzhprG9gw5FREQkMDl5IXq2b8bgLqr9JiJSHTPjrO5tOKt7G0J7D/P3RZt5dukWXlu9nR7pDRg5pjxqMxg0hdL3WdHuXiraLSIiSatw1yGWFe5l8rAuqv0mIlIHnVs35RcT+7H49rH8ZtIAstIbRHX6ua7A4f1U8l2veUW7vzu6e9DhiIiIBObFFV7ttyuGZgYdiohIXGnWOJVvjexOl5LCqD6PrsDhFe1et+0APx/fV1/WFhGRpFVe4XgxL8R5fdqT0VK130RE6qOkH8AdOfZZ0e6vnKmi3SIikrwWbdzNtv1H9eMlIiL1WNIP4GYs9Ip2/2JiPxXtFhGRpJaTV0TLtFQu6qfabyIi9VVSD+A+PVjCwwvyVbRbRESS3oGjpby+7hMuHazabyIi9VlSD+D+OO8jSsoquF1Fu0VEJMm9ttqr/XbVsC5BhyIiIrVI2gFceNHuHiraLSIiSS4nL0TvDs05s3N60KGIiEgtknYAd88sr2j3T1S0W0REklzBp8Xkbd7L5GGdVftNRKSeS8oB3Hv5u5j3wU5uuFBFu0VERCprv10+RLXfRETqu6QbwFVUOO6a5RXt/s6o7kGHIyIiEqjyCse/V2zl/D7t6aDabyIi9V7SDeBeel9Fu0VERCotzN/F9v1HuWq4frxERCQeJNUA7sixcu5/Q0W7RUREKuXkhUhv0pCx/ToEHYqIiNRBUg3gKot2/6+KdouIiLD/SClz1n3CpMGdaJyqWSkiIvEgaQZwlUW7L+mfwdkq2i0iIhFgZuPN7EMzyzezadU8/oCZrfRvH5nZvrDHrjGzj/3bNbGN3PPa6u2UlFUweVjnIJ5eREROQmrQAcTKg3O9ot3TVLRbREQiwMxSgIeAi4EQsMzMZjrn1le2cc79NKz9j4Eh/nIb4NfAcMABef62e2N4CLyQV0TfjBYMzFTtNxGReJEUV+A+3nGQ55YV8c1zuqlot4iIRMoIIN85V+CcOwY8B0yqpf0U4Fl/eRzwpnNujz9oexMYH9Voq8jfWcz7W/ap9puISJxJigHc9Nkq2i0iIhGXCRSF3Q/5677AzLoBWcD8E902Wl5cESKlgTFpiH7US0QkniT8FMrKot3TJpxOm2aNgg5HREQSR3WXrVwNba8Gcpxz5Se6rZldC1wLkJGRQW5u7gmG+XnFxcXMX7CAZxcd4Yy2DVift5j1x99M6qC4uPiUXx/5IuU1epTb6Ih2XhN6AFdR4fjtayraLSIiURECwoundQa21dD2auCGKttmV9k2t7oNnXOPAo8CDB8+3GVnZ1fXrM5yc3OxTgPYV7KUey4ZRPbAjqe0P/lMbm4up/r6yBcpr9Gj3EZHtPOa0FMoX3p/K+u3q2i3iIhExTKgt5llmVkjvEHazKqNzKwv0BpYFLZ6DnCJmbU2s9bAJf66mHhheRGtmzZkbL+MWD2liIhESMJegSspd9w350MGqWi3iIhEgXOuzMxuxBt4pQAznHPrzOxOYLlzrnIwNwV4zjnnwrbdY2a/wRsEAtzpnNsTi7gPlTreWL+Dr4/oSqPUhD6PKyKSkBJ2ADensJRPDpTypylDVLRbRESiwjk3C5hVZd2vqty/o4ZtZwAzohZcDZZsL+OYar+JiMSthDz19unBEmYVlDJuQAYjstoEHY6IiEi98e7WMk4/rQUDOrUMOhQRETkJCTmAe3DuR5RWwG3jVbRbRESkUv7OgxTsr1DtNxGROJZwA7idB4/y/PIiLuiSqqLdIiIiYV5+fxspBpcNiWnJORERiaCE+w5chxZpzLxxDPlr8oIORUREpF758dhepB8O0a5546BDERGRk5RwV+AA+nVsSYtGmhoiIiISrnFqCr1bq6yOiEg8S8gBnIiIiIiISCLSAE5ERERERCROaAAnIiIiIiISJzSAExERERERiRMawImIiIiIiMQJDeBERERERETihAZwIiIiIiIicUIDOBERERERkTihAZyIiIiIiEic0ABOREREREQkTphzLugYPsfMPgU2R2BX7YBdEdiPfJ7yGj3KbXQor9ETidx2c861j0QwySBCfaT+TUSPchsdymv0KLfREdX+sd4N4CLFzJY754YHHUeiUV6jR7mNDuU1epTb+KTXLXqU2+hQXqNHuY2OaOdVUyhFRERERETihAZwIiIiIiIicSKRB3CPBh1AglJeo0e5jQ7lNXqU2/ik1y16lNvoUF6jR7mNjqjmNWG/AyciIiIiIpJoEvkKnIiIiIiISEKJmwGcmfU1s5VhtwNmdrOZDTKzRWa2xsz+Y2Ytw7Y5039snf94mr9+mH8/38z+ZGYW3JHFnpnNMLOdZrY2bN0dZrY1LL8T/fWNzOwJP1+rzCw7bJu7zKzIzIqr7L+xmf3Lz+8SM+seo0MLnJmlmdlSP1frzOz//PU3+vlwZtYurH26/76tbP/dsMdeN7N9ZvZqlefI8vP6sZ/nRrE7wuBU97711//YzD7083evv25E2Ht5lZld7q+v9vXxH0vWvHYxswVmtsHPyU3++po+E6rNrf9YTa9RGzN708/tm2bWOrZHmdjUP0aW+sjoUP8YPeofo6Ne94/Oubi7ASnAJ0A3YBlwvr9+KvAbfzkVWA0M8u+3BVL85aXASMCA2cCEoI8pxvk7DxgKrA1bdwdwSzVtbwCe8Jc7AHlAA//+OUBHoLjKNtcDf/GXrwb+FfQxxzC3BjT3lxsCS/w8DQG6A4VAu7D2vwB+5y+3B/YAjfz7Y4GvAK9WeY7ngav95b8A1wV93DHKbXXv2wuAuUBj/34H/29TINVf7gjs9D8Tqn19kjyvHYGh/nIL4COgfy2fCdXmtqbXyF9/LzDNX55W+Z7XLSqvp/rHU8+h+sjo5FX9Y/Ryq/4xOnmtt/1j3FyBq2IssNE5txnoC7ztr38TuNJfvgRY7ZxbBeCc2+2cKzezjkBL59wi52Xr78BlsQ0/WM65t/E+COuiPzDP324nsA8Y7t9f7JzbXs02k4Cn/OUcYGyynMV1nsqzrQ39m3POve+cK6xuE6CFn5/meK9Lmb+vecDB8MZ+uwvx8gpenpPi/VvD+/Y6YLpzrsRvs9P/e9g5V+a3ScPLc42vT5LndbtzboW/fBDYAGTW0r7a3PqP1fTZEv6ZkDS5DYj6x1OkPjI61D9Gj/rH6KjP/WO8DuCuBp71l9cCl/rLVwFd/OU+eG+8OWa2wsx+7q/PBEJh+wpRy4uRZG40s9X+Zd7KS7irgElmlmpmWcAwPstxTTKBIgD/jbwf7wxvUjCzFDNbiXexo+bEAAAJLUlEQVTm5U3n3JJamv8Z6AdsA9YANznnKmpp3xbYF/YBkezv3z7Auf7UjrfM7KzKB8zsbDNbh5fXH1XmrIbXR3kF/KlcQ/DOvEL1nwk15rYWGZX/kfX/doh48FJJ/WP0qI88ReofY0r9YwTVt/4x7gZw/rzbS4EX/FVTgRvMLA/v8uYxf30qMAb4hv/3cjMbi3eJuCr9FCc8AvQEBgPbgd/762fg/WNdDjwIvId/BqwWSZ1j51y5c24w0BkYYWZn1NJ8HLAS6ISX+z9b2PdUqpHUua1GKtAabxrOrcDzlWeynXNLnHMDgLOA283/jk8Nr0/S59XMmgMvAjc75w5Q82dCjbmVYKl/jCr1kRGg/jGm1D9GSH3sH+NuAAdMAFY453YAOOc+cM5d4pwbhnfWcaPfLgS85Zzb5Zw7DMzCm3sawntjVuqMd3YnqTnndvj/cCuAx4AR/voy59xPnXODnXOTgFbAx8fZXQj/DKSZpQLp1H06SsJwzu0DcoHxtTT7LvBvf+pCPrAJOL2W9ruAVn5eQe/fEJ/lbylQAbQLb+Cc2wAcAs6osj789UnqvJpZQ7zO6Rnn3L+h5s+EcDXltho7/Ol5+H93RjJ++S/1j1GiPjKy1D/GhPrHCKiv/WM8DuCm8Nn0EMysg/+3AfBLvC9XAswBzjSzpv6b7nxgvX958qCZneOfifg28EosD6A+qnzz+C7Hm3qDn79m/vLFQJlzbv1xdjcTuMZfngzMd84lxdkaM2tvZq385SbARcAHtWyyBe87K5hZBt53VgpqauzncQFeXsHLczK/f1/Gm5uPmfUBGgG7zPvFrFR/fTe8vBbW9Pokc179z8G/ARucc38IW1/TZ0K1uT3O04R/JiRNbgOg/jFK1EeeOvWPMaf+8RTV6/7R1YNfeanrDe/XXXYD6WHrbsL7VZiPgOn4xcn9x74JrPMTe2/Y+uH+uo14c6wtVsdQH254Hfx2oBTvDM33gH/gzddd7b+ZOvptuwMf4n1xcy7QLWw/9/rbV/h/7/DXp+FN4cnH+0WzHkEfcwxzeybwvp/HtcCv/PU/8XNUhnfm6nF/fSfgDT/3a4Fvhu3rHeBT4Ii/7Th/fQ8/r/l+nhsHfdwBvm8bAU/7uVsBXOi3/Zb/b3+lv/6y2l6fJM/rGLzpMKv9fK0EJtbymVBtbmt6jfz1bfF+6OFj/2+boI870W6of4xkLtVHRiev6h9j+55V/3jqea23/aP5G4uIiIiIiEg9F49TKEVERERERJKSBnAiIiIiIiJxQgM4ERERERGROKEBnIiIiIiISJzQAE5ERERERCROaAAnMWVmzsx+H3b/FjO7I0L7ftLMJh+/5Sk/z1VmtsHMFlRZ393M1kb7+WuJ6w4zu6Wa9YHGJSIix6f+MapxqX+UhKIBnMRaCXCFmbULOpBwZpZyAs2/B1zvnLsgSrGYX3g3LlUWsRQRkROi/vH4sah/FEEDOIm9MuBR4KdVH6h6htDMiv2/2Wb2lpk9b2Yfmdl0M/uGmS01szVm1jNsNxeZ2Tt+uy/726eY2X1mtszMVpvZD8P2u8DM/olXkLFqPFP8/a81s9/5636FV9jxL2Z2X00HWctzNjezeWa2wt/3JH99d/+s5cN4xR+7mFmxmd1lZqvMbLGZZfhtv2JmS8zsfTObW7neN8jM5pvZx2b2g7rGVaVNdzP7wMye8tvkmFnTyuP3t11rZo+amfnrc83sbjN7C7ipphj9s6BPmdkbZlZoZleY2b1+Ll43s4Z+u+lmtt5//vtryrOISAJR/6j+Uf2j1E3QVc51S64bUAy0BAqBdOAW4A7/sSeByeFt/b/ZwD6gI9AY2Ar8n//YTcCDYdu/jndiojdepfs04Frgl36bxsByIMvf7yEgq5o4OwFbgPZAKjAfuMx/LBcYXs023YG1/nJNz5kKtPTXtwPyAfO3rQDOCdufA77iL98btr/WgPnL3wd+7y/fAawCmvj7LvKP47hxVXMcDhjt358B3OIvtwlr94+w+HKBh8Meqy3Gd4GGwCDgMDDBf+wl4DKgDfBh2Patgn7f6qabbrpF+4b6R/WP6h91q+NNV+Ak5pxzB4C/Az85gc2WOee2O+dKgI3AG/76NXgfqJWed85VOOc+BgqA04FLgG+b2UpgCdAWrwMDWOqc21TN850F5DrnPnXOlQHPAOedQLw1PacBd5vZamAukAlUniHc7JxbHLaPY8Cr/nJe2HF2BuaY2RrgVmBA2DavOOeOOOd2AQuAEXWMq6oi59xCf/lpvLOqABf4Zw7XABdWee5/hS3XFuNs51wp3muXgvefCvjstTwAHAUeN7Mr8DoxEZGEp/5R/aP6R6kLzcWVoDyINxXiibB1ZfjTev2pB43CHisJW64Iu1/B59/HrsrzOLxO4cfOuTnhD5hZNt4ZxurYcY+gdjU953fwzloOc86Vmlkh3llQqoml1DlXeTzlfHac/w/4g3Nupn8Md4RtU93xHzeuanxhP2aWBjyMd3a1yLwv16eFtQmPv7YYSwCccxVmFn6MFUCqc67MzEYAY4GrgRvxOkMRkWSg/lH9o/pHqZWuwEkgnHN7gOfxvvBcqRAY5i9PwptGcKKuMrMG/rz/HnhTDeYA14XNH+9jZs2Os58lwPlm1s68L3BPAd46gThqes50YKffOV0AdDuho/Ok402TAbimymOTzCzNzNriTYFZVse4qupqZiP95Sl40zoqO6NdZtYcqO0XzWqLsVb+vtOdc7OAm4HBJ7K9iEg8U/+o/rEm6h+lkq7ASZB+j3f2qNJjwCtmthSYR81n/2rzIV5HkgH8yDl31Mwex5t6sMI/c/kp3lzyGjnntpvZ7XjTLAyY5Zx75QTiqOk5nwH+Y2bLgZXAByewz0p3AC+Y2VZgMd53ByotBV4DugK/cc5tM7PudYirqg3ANWb2V+Bj4BHn3GEzewxvKkchX+z86hrj8bTAex+k4eX+C1/oFxFJcOof1T9WR/2jAJ99CVJEBPB+ZQt41Tl3RsChiIiI1BvqH6W+0BRKERERERGROKErcCIiIiIiInFCV+BERERERETihAZwIiIiIiIicUIDOBERERERkTihAZyIiIiIiEic0ABOREREREQkTmgAJyIiIiIiEif+P12DQ23vy9q/AAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "import time\n", "from functools import reduce\n", "# def count(iterable):\n", "# return reduce(lambda x,y: x*y, iterable)\n", "\n", "# Function to compute the number of learnable parameters of a mlp given the size of its hidden layers\n", "def param_count(hl_size):\n", " tot = 0\n", " input_size, output_size = X_train.shape[1], len(labels)\n", " tot += (input_size+1)*hl_size[0]\n", " for i in range(1,len(hl_size)):\n", " tot += (hl_size[i-1]+1)*hl_size[i]\n", " tot += (hl_size[-1]+1)*output_size\n", " return tot\n", "\n", "hl_sizes = [(10,), (20,), (40,), (20,20,), (30,30,20)]\n", "hl_labels = [param_count(t) for t in hl_sizes]\n", "\n", "ti = time.time()\n", "train_acc_list, test_acc_list = [], []\n", "for hl_size in hl_sizes:\n", " print('Training MLP of size {} ...'.format(hl_size))\n", " mlp = MLPClassifier(hidden_layer_sizes=hl_size,\n", " max_iter=100, alpha=1e-4, solver='sgd',\n", " tol=1e-4, random_state=ID,\n", " learning_rate_init=.01)\n", " mlp.fit(X_train, y_train)\n", " train_acc_list.append(mlp.score(X_train, y_train))\n", " test_acc_list.append(mlp.score(X_test, y_test))\n", " print('Done, training time: {:.2f} sec\\n'.format(time.time()-ti))\n", " ti = time.time()\n", "\n", "fig, ax = plt.subplots(1,2, figsize=(15,5))\n", "\n", "\n", "ax[0].plot(train_acc_list)\n", "ax[0].set_xlabel('Number of learnable params')\n", "ax[0].set_title('Train accuracy')\n", "ax[0].set_xticks(np.arange(0,len(hl_labels)))\n", "ax[0].set_xticklabels(hl_labels)\n", "ax[0].grid(True)\n", "\n", "ax[1].plot(test_acc_list)\n", "ax[1].set_xlabel('Number of learnable params')\n", "ax[1].set_title('Test accuracy')\n", "ax[1].set_xticks(np.arange(0,len(hl_labels)))\n", "ax[1].set_xticklabels(hl_labels)\n", "ax[1].grid(True)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Question 2:\n", "\n", "Comment about the training and test accuracies referring to the discussion on underfitting and overfitting we did in the course" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## [ANSWER TO QUESTION 2]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### TO DO 4\n", "\n", "Now try also to use different learning rates, while keeping the best NN architecture and batch size you have found above. Plot the learning curves (i.e., the variation of the loss over the steps, you can get it from the loss_curve_ object of sklearn) for the different values of the learning rate. Try to run each training for 600 iterations. " ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "C:\\Users\\Anay Deshpande\\Anaconda3\\lib\\site-packages\\sklearn\\neural_network\\multilayer_perceptron.py:566: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", " % self.max_iter, ConvergenceWarning)\n", "C:\\Users\\Anay Deshpande\\Anaconda3\\lib\\site-packages\\sklearn\\neural_network\\multilayer_perceptron.py:566: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", " % self.max_iter, ConvergenceWarning)\n", "C:\\Users\\Anay Deshpande\\Anaconda3\\lib\\site-packages\\sklearn\\neural_network\\multilayer_perceptron.py:566: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", " % self.max_iter, ConvergenceWarning)\n", "C:\\Users\\Anay Deshpande\\Anaconda3\\lib\\site-packages\\sklearn\\neural_network\\multilayer_perceptron.py:566: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", " % self.max_iter, ConvergenceWarning)\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEJCAYAAACZjSCSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3xb1fn48c/RsizvHe+R2BlOjLMdsgkhJOxRKFBCKSFltBAo8AVaSumPTUvZlBEgAcoqNEB2yCSb7J04ZHjEGd7bsqTz++PawolHbEfyiM779dLLsXR173PdokdnPUdIKVEURVE8l66zA1AURVE6l0oEiqIoHk4lAkVRFA+nEoGiKIqHU4lAURTFw6lEoCiK4uHclgiEELFCiOVCiL1CiN1CiPubOGacEKJECLGt7vFXd8WjKIqiNM3gxnPbgD9JKbcIIfyAzUKIJVLKPWcc96OU8nI3xqEoiqK0wG2JQEqZB+TV/btMCLEXiAbOTARtEhoaKhMSEs49QEVRFA+yefPmfCllWFOvubNF4CSESAAGAhuaeHmEEGI7cAx4SEq5u6VzJSQksGnTJpfHqCiKcj4TQhxt7jW3JwIhhC/wNTBDSll6xstbgHgpZbkQYgowB0hu4hzTgekAcXFxbo5YURTFs7h11pAQwoiWBD6VUn5z5utSylIpZXndv+cDRiFEaBPHvSulHCKlHBIW1mTLRlEURWknd84aEsBMYK+U8uVmjulRdxxCiGF18RS4KyZFURSlMXd2DY0EbgV2CiG21T33OBAHIKX8N3A9cLcQwgZUAb+Wqhyqonic2tpacnJyqK6u7uxQuj2z2UxMTAxGo7HV73HnrKHVgDjLMW8Ab7grBkVRuoecnBz8/PxISEigrpNAaQcpJQUFBeTk5JCYmNjq96mVxYqidLrq6mpCQkJUEjhHQghCQkLa3LJSiUBRlC5BJQHXaM/f0WMSQWGFlae+302l1dbZoSiKonQpHpMI1hzM56O1R7j+7XXkFFV2djiKonQxvr6+bX5PYWEhEydOJDk5mYkTJ1JUVNTkcbNmzSI5OZnk5GRmzZrlfH7z5s0MGDCAXr16cd9991E/V6a583766aekpaWRlpbGhRdeyPbt29txp415TCK44oIoPrhtKNlFlVz5xhrWH1KzVBVFaZndbm/x9eeff54JEyaQmZnJhAkTeP755xsdU1hYyFNPPcWGDRvYuHEjTz31lPOD/e677+bdd98lMzOTzMxMFi5c2OJ5ExMTWblyJTt27OCJJ55g+vTpLrlPj0kEAOP7hPPtvSMJshj5zfsb+HRDsyuuFUXxUCtWrGD8+PHcfPPNDBgwoMVjv/32W2677TYAbrvtNubMmdPomEWLFjFx4kSCg4MJCgpi4sSJLFy4kLy8PEpLSxkxYgRCCKZOnep8f3PnvfDCCwkKCgIgIyODnJwcl9xzh9Qa6jIcdpJCfZhz70ju+2wrf/7fLg6dquDxKX3R69RAlaJ0BU99v5s9x86sRnNu+kX58+QVqa0+fuPGjezatcs5BXPKlCm8//77REVFnXbciRMniIyMBCAyMpKTJ082Oldubi6xsbHO32NiYsjNzSU3N5eYmJhGz7f2vDNnzmTy5MmtvqeWeE6LYM+38Gw0lGTjZzby3tQh3D4ygZmrD3Pn7E2UVdd2doSKonQRw4YNO20e/vz58xslgdZqao2sEKLZ51tj+fLlzJw5kxdeeKFdMZ3Jc1oEflFgq4LjOyEwDoNex5NXpJIU5svfvtvNlW+s4a1bBtE30r+zI1UUj9aWb+7u4uPj06rjIiIiyMvLIzIykry8PMLDwxsdExMTw4oVK5y/5+TkMG7cOGJiYk7r2snJyXEmm5bOu2PHDqZNm8aCBQsICQlp5x2eznNaBBH9AKElggZuzYjnP9OGU1Fj4+o31/DVpuzOiU9RlG7nyiuvdM4CmjVrFldddVWjYyZNmsTixYspKiqiqKiIxYsXM2nSJCIjI/Hz82P9+vVIKZk9e7bz/c2dNysri2uvvZaPP/6YlJQUl92H5yQCkw+E9GyUCACGJ4Uw777RDIoL4uH/7uDBL7apriJFUZymTJnCsWPHGj3/6KOPsmTJEpKTk1myZAmPPvooAJs2bWLatGkABAcH88QTTzB06FCGDh3KX//6V4KDgwF4++23mTZtGr169aJnz57OPv/mzvv3v/+dgoIC7rnnHtLT0xkyZIhL7k90txpvQ4YMke3emOar30LuFpixo8mX7Q7Ja0szeX1ZJtFB3rxy40AGxwe1P1hFUVpl79699O3bt7PDOG809fcUQmyWUjaZOTynRQDQYwAUH4Wq4iZf1usED0xM4cvfj0BKuOGddfxj0X5qbC3PJVYURenOPCwRpGk/T7S4GyZDEoKZf/9orkqP4o3lB7ny9TXszCnpgAAVRVE6nmclgoj+2s8mxgnO5G828vIN6cy8bQjFVVaufmsNzy/YR5VVtQ4URTm/eFYi8OsBllA4cfZEUG9C3wgWzxjLtQOj+ffKn5n0yipWHTjlxiAVRVE6lmclAiG0cYJWtAgaCrAYeelXF/CfO4ej1wmmfrCR+z/fyskytZuSoijdn2clAtASwcm9YG/79NALe4ay4P7R3DchmQU7jzPhnyv5eP1R7I7uNfNKURSlIc9MBHYr5Ge26+1mo54HJ6awcMZo0mICeGLOLq5+cw2bjxa6OFBFUTpSVypD/fDDD9OnTx/S0tK45pprKC5ueqajq3hmIoA2dw+dKSnMl0/uGM6rv07nVFkN1729jhmfb+V4ieouUpTzRWeVoZ44cSK7du1ix44dpKSk8Nxzz7n+5hrwvEQQkgx6Lzje9KKythBCcFV6NEv/NJY/jO/F/F3HueifK3hrxUG19kBRuqmuUIb6kksuwWDQSsG5stx0czyn6Fw9vUGrO3SOLYKGfLwMPDSpNzcMieXpeXt4ceF+vvgpmycu68eEvuFqL1ZFaYsFj7r0v09A6wmY3PjbenM6uwx1Qx988AE33nhjq2NvD89rEYC2nuDELnBxeY24EAvvTh3C7N8Nw6ATTJu9ids+/ImDJ8tdeh1FUdyrq5ShfuaZZzAYDNxyyy3tunZreV6LALQVxls/hrI88G/f/7gtGZMSxsIZY5i97iiv/HCAS19ZxdQRCdw/IZkAi9Hl11OU80obvrm7S2eXoQZtgHnu3LksXbrU7b0KntkiqB8wznPNxs9NMep13DEqkRUPjeNXQ2L5cO1hxv5jObPWHqHW7nDbdRVF6TjuKkO9cOFCXnjhBb777jssFovb78NzE4HQwbFtbr9UiK8Xz107gHl/HE2/SH+e/G43l76yiiV7TjTZNFQUpevp6DLUf/jDHygrK2PixImkp6dz1113ufX+PKsMdUNvZkBgHNzy5bmfq5WklPyw9yTPzd/LofwKhiUE89iUPgyMU6WuFc+mylC7lipD3VpRA+HYVpcPGLdECMHEfhEsemAMT1/dn0P55Vzz1lru/XQLRwsqOiwORVGUhjw4EaRDxUltwLiDGfU6fpMRz4qHx3PfhGSW7TvJxS+v5G/f7Sa/vKbD41EUxbN5TCI4XnGchUcWUm6tm8oZNVD7eWxrp8Xk62XgwYkprHx4HNcPjmX2uiOMfXE5Ly85oLbKVBSlw3hMIth+ajsPr3yYvIq6FkBEfxD6Tk0E9cL9zTx37QAWPzCWsb3DeG1pJmNeXM77Px5SK5QVRXE7j0kE3gZvACptldoTJguE9+0SiaBer3Bf3rplMN/9YST9ovx5et5eLvrHSv67OUdVOFUUxW3clgiEELFCiOVCiL1CiN1CiPubOEYIIV4TQhwUQuwQQgxyVzz1iaDKVvXLk1HpHT5g3BppMYF8Oi2DT+4YTrCPiYe+2s7kV9WUU0VR3MOdLQIb8CcpZV8gA7hXCNHvjGMmA8l1j+nA2+4KxmLQFmVU1TZMBAOhsgBKst112XMyKjmUb+8dyRs3D6TWLrlz9iaue3st634u6OzQFOW809FlqCsrK7nsssvo06cPqampzjUIncFtiUBKmSel3FL37zJgLxB9xmFXAbOlZj0QKISIdEc8TbcIOn/A+Gx0OsHlaVEsfmAMz107gNziKm56bz23ztzA9mz31ihXFE/n7jLUDz30EPv27WPr1q2sWbOGBQsWuOU+zqZDxgiEEAnAQGDDGS9FAw2/jufQOFm4RJOJIDwVdIYOWWF8rox6HTcNi2Plw+P5y2V92X2slKveXMOdszexN6+0s8NTlPNGR5WhtlgsjB8/HgCTycSgQYPcXm66OW4vOieE8AW+BmZIKc/8xGqqklKjTnAhxHS0riPi4uLaFUeTicBohvB+XbpFcCazUc+00Un8elgcH6w+zHurDjFl749cnhbFAxcnkxTW9uatonQlL2x8gX2F+1x6zj7Bffi/Yf/X6uM7ogx1Q8XFxXz//ffcf3+jodQO4dYWgRDCiJYEPpVSftPEITlAbIPfY4BGBT2klO9KKYdIKYeEhYW1KxZvYxOJADplhbEr+HoZuG9CMj/+33juGdeTpXtPcPHLK3n4q+3kFFV2dniK0q11RBnqejabjZtuuon77ruPpKSkdl3jXLmtRSC0O50J7JVSvtzMYd8BfxBCfA4MB0qklG5Z6mvSmdAJXdOJYMssKDoCwYlNvrcrC7SYeHhSH24fmchby3/mkw1HmbMtl18PjePe8b3oEWDu7BAVpU3a8s3dXTqiDHW96dOnk5yczIwZM8417HZzZ4tgJHArcJEQYlvdY4oQ4i4hRH0pvfnAIeAg8B5wj7uCEUJgMViaTgQAx7a469IdItTXi79e0c+5SvmzjVmMeWk5T32/m5Nlah9lRXGHcylDDfCXv/yFkpISXnnllQ6N+0zunDW0WkoppJRpUsr0usd8KeW/pZT/rjtGSinvlVL2lFIOkFK6oKxo87wN3o0TQUQqGMyQs9mdl+4wkQHePHftAJY/NI6r06OYve4oY15czjPz9qg6RorSTu4oQ52Tk8MzzzzDnj17GDRoEOnp6bz//vsdel/1PKoM9WXfXEZqaCovjnnx9BdmTgLpgGlLXBBh13Ikv4LXlmUyZ2suXgY9Uy+M5/djehLsY+rs0BTFSZWhdi1VhroF3gbv0xeU1YsZou1WZrN2fFBulhDqw8s3pLPkwbFckhrBu6sOMfqFZfxj0X5KKlVhO0VRPDERnNk1BFoisNfAiZ0dH1QH6Rnmy6u/HsiiGWMY1zucN5YfZNSLy/jXkgOUVKmEoCieTCUCgOi61tJ5Mk7QkpQIP968ZRDz7xvNhT1DeHVpJqNfWMZrSzMpVaWvFcUjeVwicFYfbSggBnx7QK5bx6q7lH5R/rxz6xDm/nEUw5NCeHnJAUY9v4xXflAtBEXxNJ6VCIzNtAiE0LqHcn7q+KA6Wf/oAN6bqiWEjKQQXvkhk1HPL+Ofi/dTVHH+jZkoitKYZyWC5rqGQEsEhYegsrBjg+oi+kcH8O7UIcy7bxQje4Xy+rKDjHphGc8v2MepMjXtVFHOZyoR1HOOE3hO91BTUqMC+Petg1k0YwwX9Y3gnVU/M+qFZfz1212qdIVyXnNXGept27YxYsQIUlNTSUtL44svvnBFuC7lcYmg2lbd9OYuUQNB6DxqnKAlvXv48fpNA1n64FiuTo/ms41ZjHtpBY/8dztH8is6OzxF6RCuKENtsViYPXs2u3fvZuHChcyYMYPi4q5VQt7jEoFEUm1vouSCl69WidQDxwlakhTmywvXp7HqkfH8JiOeb7cd46J/ruDBL7Zx4ERZZ4enKC7n6jLUKSkpJCcnAxAVFUV4eDinTp1yfeDnwO1lqLsS5y5ltipnWerTRA+GPXPA4QCdR+XIs4oM8OZvV6Zyz/ievLfqEJ+sz+Kbrblc3DeCu8clMTg+uLNDVM4Tx599lpq9ri1D7dW3Dz0ef7zVx7uyDPWZ57VarfTs2bONd+BeHvVp1+SeBA3FDIXqEig42IFRdS/hfmb+fFk/1j56ETMuTmbT0UKue3sdN7yzjpUHTqk9lZXzgivLUNfLy8vj1ltv5cMPP0TXxb5oelSLwLknQVNlJkCbOQRa91BYSgdF1T0F+ZiYcXEK08ck8dnGbN5bdYjbPthI/2h/7hrbk8n9I9Hrmtp3SFFa1pZv7u7iyjLUAKWlpVx22WU8/fTTZGRkuDJUl+haacnN6ruGmlxUBhDaG8wBkH3mjppKcywmA3eMSmTVI+N54boBVNTY+cN/tnLRP1fwyfqjVNe2PNimKN1Za8pQW61WrrnmGqZOncqvfvWrjg6xVTwqEZy1a0ing5hhKhG0g8mg48ahcfzw4Fj+/ZtBBFpM/GXOLka9sJy3V/xMmSpfoXRj51KG+ssvv2TVqlV89NFHpKenk56ezrZtXWufdI8qQ70rfxc3zbuJ1y96nXGx45o+aOVLsPxp+L8j4B3U7jg9nZSS9YcKeWvFQX7MzMfPbODWjHhuH5lImJ9XZ4endDGqDLVrtbUMtWeNEZytRQAQN1z7mf0TpFzSAVGdn4QQjOgZwoieIezMKeGtFQd5e+XPvL/6MNcNimH6mCQSQ1vXD6soinupRHCm6MEg9JC9XiUCFxkQE8DbvxnM4fwK3vvxEP/dnMPnP2UxsW8E08ckMTg+6LTNvBVF6VgqEZzJ5AORaZClxglcLTHUh2evGcADF6fw8bojzF5/lMV7TpAeG8jvxyRxSWoPNdPIg0kp1RcCF2hPd7/HDBbbi4sRuzMx1sqWEwFA7HDI3Qx2NcDpDmF+Xjx4SW/WPnoRf78qlaJKK3d/uoXx/1jBrLVHqKixdXaISgczm80UFBSodSjnSEpJQUEBZrO5Te/zmBZBxbp15D3wIBF3GqisPUvxtNjhsOHfcHyH1lWkuIXFZGDqiARuGR7Pkj3HeWfVIZ78bjf/XLyfm4bFcduFCUQFNrECXDnvxMTEkJOT0+VKL3RHZrOZmJiYNr3HYxKBzqKtIQiwe529RRBXt+Aja4NKBB1ArxNc2j+SS/tHsvloER+sPsx7Px5i5urDXHFBFNNGJ5IaFdDZYSpuZDQaT1vJq3Qsj0kEwlv7ZhngMJ09EfhHQUCctp5gxD0dEJ1Sb3B8EIPjg8gpquTDNUf4fGMW/9uay4U9Q5g6Ip6L+0Zg0HtMj6aidAiPSQQ6izZV0c/eikQA2jTSI6tBSm0HM6VDxQRZeOLyftw3IZn/bMjik/VHueuTLUQGmLlleBw3D48n2MfU2WEqynnBY75a6Sxai8DXbmhdIogdDmV5UJzl5siUlgR4G7l7XE9WPjyOd28dTK9wX/6x+AAjnlvKY9/sUKWwFcUFPKhFoI0R+NoM5DRXa6ih2PqFZRsgKN6NkSmtYdDruCS1B5ek9iDzRBkfrDnCN1ty+GxjNiN7hXDbiAQm9I1Q008VpR08p0VQN0Zgsela1yKISAWvAK17SOlSkiP8eO7aAax7bAKPXNqbQ6cqmP7xZsa+pNU1KihXeywrSlt4XiKobWUi0OkhYSQcXuXmyJT2CvYxcc+4Xvz4yHjevmUQ0YHevLBwHyOeW8YDX2xj89FCNS9dUVrBY7qGhMkERiPeNl3z+xGcKXEs7J+vjRMExrk3QKXdDHodkwdEMnlAJJknyvhk/VG+3pLL/7bm0qeHH7dkxHN1ehR+ZmNnh6ooXZLHtAhAGycwW89SYqKhxDHaT9Uq6DaSI/x46qr+bHh8As9dOwC9TvDEnF0Mf1YbXN6ZU9LZISpKl+MxLQLQuoe8atuQCML7gk8YHFoJA3/j3uAUl/LxMnDTsDh+PTSW7Tkl/GfDUf63NZfPNmbTP9qfG4fGcVV6FP6qlaAoHpYILBa8rA6qbFWtK3AlhNYqOLxKrSfopoQQpMcGkh4byF8u78ecumTwxJxdPDNvD5cNiOKmYbGqAqri0TwrEXh7Y7JWIZHU2GswG1pRmClxLOz6GvIPQFhv9wepuI2/2cjUEQncmhHPztwSPtuYzffbj/H1lhx6hftyw5AYrhkYozbOUTyO28YIhBAfCCFOCiF2NfP6OCFEiRBiW93jr+6KpZ7OYsFYV9lSjRN4LiEEaTGBPHftADY8PoEXr0/D32zg2fn7yHhuKdNm/cSi3ceptTs6O1RF6RDubBF8BLwBzG7hmB+llJe7MYbTCIs3hqJfEkEQrdiKMjhRmzF0eCUMu9PNESodzcfLwA1DYrlhSCwHT5bz3805fL0lhx/2niTU18Q1A6P51ZBYUiL8OjtURXEbtyUCKeUqIUSCu87fHjqLBUNbWwSgtQr2zgWHXVtfoJyXeoX78ujkPjx0SQorD5ziy03ZfLjmCO/9eJgLYgK4fnAMV1wQRaBF1ThSzi+dPX10hBBiuxBigRAitbmDhBDThRCbhBCbzqVeuc7bgq5a22zmrHsSNJQ4DqqL4fjOdl9b6T4Meh0T+kbwzq1DWP/4BJ64vB81NgdPfLuboc/8wO8/3sTCXcepsdk7O1RFcYnOHCzeAsRLKcuFEFOAOUByUwdKKd8F3gUYMmRIu5eK6iwWdNVWoK0tgtHaz0MrICq9vZdXuqFQXy/uGJXIHaMS2ZVbwv+25vLttmMs2n2CAG8jl6VFcs3AaAbHBaFTdY6UbqrTEoGUsrTBv+cLId4SQoRKKfPddU2dtzdUVQOibYnArwdE9IcDi2DUDHeFp3Rx/aMD6B8dwGOT+7D6YD5ztubyvy25/GdDFtGB3lyWFsnlaZEMiA5QU1GVbqXTEoEQogdwQkophRDD0LqpCtx5TZ2PBWGzo7fr25YIAPpcBqtegop88Al1T4BKt2DQ6xjXO5xxvcOpqLGxaPdxvt9+jA9WH+bdVYeID7Fw1QVRXJkeTa9w384OV1HOym2JQAjxGTAOCBVC5ABPAkYAKeW/geuBu4UQNqAK+LV0c4Ww+sJzbSozUa/PZbDyBTiwUK0yVpx8vAxcOyiGawfFUFxprUsKebyx/CCvLTtI/2h/Lk+L4rIBkcQGWzo7XEVpkjtnDd10ltffQJte2mFE3Z4EXrVQ2Zo9CRrqkQYBsbBvvkoESpMCLSZuHBrHjUPjOFlazXfbj/H99mM8v2Afzy/Yx8C4QC4bEMmUAZFEBXp3driK4uRhK4u1RGBuS72hekJA7ymwZTZYK8Gkvt0pzQv3NzNtdBLTRieRVVDJ3J3HmLs9j6fn7eXpeXsZFBfIlLqKqdEqKSidrLOnj3ao+l3KzNY2DhbX6zMFbFVwaLmLI1POZ3EhFu4Z14v5949m+UPjeHhSb6pqHTw9by8jn1/GVW+u4Z2VP5NV0MZWqqK4iGe1COr2LfZ3GNuXCOJHgjkA9s3TxgwUpY0SQ324d3wv7h3fiyP5FSzYdZz5O/N4bsE+nluwj36R/kzu34NL+/cgWa1mVjqIhyUCrUXgbze1LxHojZA8CfYvALsN9B7151NcLCHUh7vH9eTucT3JLqxk0e7jLNh1nH8uOcA/lxygZ5gPk1J7MCm1B2kxakqq4j4e9UlWP2vIz97OFgFoLYGdX2qb2ieMdGF0iieLDbY4xxROlFazuC4pvLPqEG+t+JnIADMT+0Vwcd8IMpJCMBk8qldXcTPPSgR1LQJfu5FTrd2u8ky9JoDeBHu/V4lAcYsIfzO3jkjg1hEJFFVYWbrvJIt2H+fLTdnMXncUPy8DY3qHMbFvBON6h6naR8o586hEUD991Mem52hbp4/W8/KD5Etg9zcw6RlVhE5xqyAfE9cPjuH6wTFUWe2sOZjPkj0nWLrvJPN25KHXCQbHBzGhTzgX9QmnV7iv6kJS2syjEoGuQSJod9cQQNoNsG+uVpq650Uuik5RWuZt0nNxvwgu7heBwyHZkVvCD3tOsGzfSedgc2ywNxP6RDCxXwRDE4JVF5LSKh6VCITJBDod3rXtnD5aL3kSeAXA9i9UIlA6hU73yxacD03qTV5JFcv2nWTZ3pN8tjGLj9YewdfLwKheoVzUJ5xxvcMI92/FjnyKR/KsRCAEOovl3BOB0QypV8PO/4L1ZTD5uC5IRWmHyABvbhkezy3D46m02lidmc/y/SdZvu8UC3cfB6BfpD/jeocxrnc4A+MCMepVa0HRtCoRCCF6AjlSyhohxDggDZgtpSx2Z3DuoPP2xlwrqGrvGEG9tBthyyyt5ETar1wTnKK4gMVk4JLUHlyS2gMpJXvzylhx4CQr9p9yzkLy8zIwomcIY3uHMTYljJggtVLek7W2RfA1MEQI0QuYCXwH/AeY4q7A3EVnseBllefWIgCIG6HVHtrxuUoESpclhKBflD/9ovy5Z1wvSqtrWXuwgJUHTrHqwCkW7zkBaLuzjU0JY1RyKMMTg7GYPKqzwOO19n9th5TSJoS4BnhFSvm6EGKrOwNzF2Gx4GWtpMpWhZSy/TMsdDoY8CtY8wqUnwTfcNcGqihu4G82cmndymUpJT+fKmfF/lOsPHCKj9cfZebqwxj1gkFxQYxJCWNMchipUf5q053zXGsTQa0Q4ibgNuCKuueM7gnJvXQWC8aachzSgdVhxUvv1f6Tpd0Iq1+GXV9Dxt2uC1JROoAQgl7hfvQK92Pa6CSqa+38dKSQ1Zn5/JiZz0uL9vPSov2E+JjI6BnCyJ6hXNgzhPgQi5qiep5pbSK4HbgLeEZKeVgIkQh84r6w3Efn7Y2xVNtrtqq26twSQXgfiEzXKpIOv0urUKoo3ZTZqGd0chijk8N4DDhVVsPqg6f48UA+a38uYN6OPACiA70Z1SuUkcmhZCQFE+6nZiN1d61KBFLKPcB9AEKIIMBPSvm8OwNzF53Fgr7GBmilqAMJPLcTDr0DvvsjHF2rVhor55UwPy+uGRjDNQNjkFJyKL+CtQfzWX0wn/m78vhiUzagjS9kJAVzYc9QMpJCCPZRK527m9bOGloBXFl3/DbglBBipZTyQTfG5hY6b2/01bVAO/YkaEr/62HxE/DTeyoRKOctIQQ9w3zpGebLrSMSsDskO3NLWH+ogPWHCvjfllw+WZ8FaNNUM5JCGJYYzPDEYIJUYujyWts1FCClLBVCTAM+lFI+KYTY4c7A3EXnY0Ff48JEYLJoO5Zt+DeUHgP/qHM/p6J0cfoGC9ruGtuTWruDHTnFrD1YwECy9DAAACAASURBVJqf8/l0w1E+WHMYgN4RfgxLDHYmBrWwretpbSIwCCEigRuAP7sxHrcT3t6IqhoAymvLXXPSoXfAujdh80cw/nHXnFNRuhGjXsfg+GAGxwfzxwnJ1Njs7MwpYcPhQjYcLuSbLTl8vP4oAAkhFoYlBjM0QXuowefO19pE8HdgEbBGSvmTECIJyHRfWO6js1gQ1lqEQ09RTZFrThqcBMkTtUQw+iEwqKaw4tm8DHqGJAQzJCGYe8eDze5g97FSfjqiJYbFe07w5aYcAEJ9TQyJD2ZoYjDDEoLpG+mHQa167lCtHSz+Cviqwe+HgOvcFZQ7Ndy3uLCq0HUnHjYdPr0e9n4HA6533XkV5Txg0Ou4IDaQC2IDmTY6CYdDcvBUOZuOFPHTkUJ+OlLoLIXhY9KTHhdY18IIIj0mkABLt5yt3m20drA4BngdGAlIYDVwv5Qyx42xuUV9BVJvq6Cw2oWJoOcECErUxgr6X6emkipKC3Q6QUqEHykRftw8PA6AvJIqNh4uZPPRIn46UsQbyzJxSO34xFAfBsYGMrSuS6lnmI/qTnKh1nYNfYhWUqK+lsJv6p6b6I6g3Kl+3+Iw4UdRtYu6hkBbaTziXpj/EGStg/gLXXduRfEAkQHeXJUezVXp0QCUVdeyM6eEbTnFbMsqZlXmKb7ZmgtAsI+J9NhALogJJD0uULUazlFrE0GYlPLDBr9/JISY4Y6A3K2+RRCGn2tbBADpt8CK52H1v1QiUJRz5Gc2cmGvUC7sFQqAlJLD+RX8dKSQTUeK2JZdzPL9J5F1rYakUB/S4wIZGBfEwNhA+vRQYw2t1dpEkC+E+A3wWd3vNwEF7gnJvUTdvsUhwpdsVycCkwUy7oJlT8PxXdCjv2vPrygeTAhBUpgvSWG+3DhU604qrW81ZBezNauYVQfy+WaL1mowG3UMiA7QWg51rYeYIG/VpdSE1iaC3wFvAP9CGyNYi1Z2otupbxEESwvbXZ0IAIZOg9WvaMXornvf9edXFMXJ32xkZK9QRjZoNeQUVbE1u5gtR4vYnlPMrHVHsf6orWkI8TFxQWwgaTEBpMUEMCA6kDC/cygzc55o7ayhLLSVxU51XUOvuCMod9JZtE1kAh3eFNUcdv0FvINgyO3auoLxf4bgRNdfQ1GUJgkhiA22EBts4coLtMWdVpuDfcdL2Z5TwvbsYraf0aUUGWBmQHSA9ojRfob4elZyOJei4w/SLROB1jUU6DBTUlNCraMWo87Fg0wZ98KGd2Dt63D5y649t6IobWIy6EiLCSQtJpBbM+IBqKixsftYKTtyitmZW8LOnBLn3gwAUQFm+tclh/51ySH0PE4O55IIumVHW33XkJ9d+/Avri4mzBLm2ov4R0L6zbD1Yxg1AwLjXHt+RVHOiY+XwVn2ol5pdS27c0vZmVvMztxSduc2nxxSo/1JjQog3M/rvBhzOJdEIF0WRQfS1Q0W+9q1Wy+sLnR9IgAY8whs/xyWPwvX/Nv151cUxaX8zUZG9AxhRM8Q53Nl1bXsPlbKrtwSduSUsPOM5BDq60X/aH9So7TE0DfSn/hgS7fbyKfFRCCEKKPpD3wBeLslIjcTZjMIgcWmTStz+RTSegHR2mrjta/DhX+EiFT3XEdRFLfxMxvJSAohI+n05LA3r4zdx0qcSeLHzHzsdavfLCY9vXv4kRrlT7/IAPpF+dM7wg9vk76zbuOsWkwEUkq/jgqkowidDuHtjbdVy9guXVR2plEPaBvcL/073PyF+66jKEqH8TMbG3UrVdfaOXCijH15ZezJK2VvXinfbjvmLM2tE9rq6L6R2v7R/SK1R1gX6Vpy2w7VQogPgMuBk1LKRhPqhXb3rwJTgErgt1LKLe6KpyGdxYJXrZa93dYiALAEw8gZsPQpOLoO4ke471qKonQas1HvHJCuJ6Uku7CKPXkl7MkrY29eKduyi5lbt9MbaCukUyJ86dPDn5QIP3r30B6+Xm77aG6SO6/2Edrag9nNvD4ZSK57DAfervvpdjpvb/Q1NvRC795EANoWlhvfhSV/hd8t0kpRKIpy3hNCEBdiIS7EwqX9I53Pl1TVsjevlD3HSjlwooz9J8r4alM2FVa785iYIG/69PCjTw9/Z3JIDPXB6KaV0m5LBFLKVUKIhBYOuQqYLaWUwHohRKAQIlJKmdfCe1xCZ7Egq6oI9Ap0fyIwWeCiJ+Dbe2Dzh9reBYqieKwA78bjDg6HJLe4in3Hy9h/vJR9x8vYd7yM5ftPOcceTHod94zvyYyLU1weU8e2P04XDWQ3+D2n7jn3JwJvb2RlJUHmIPcnAtCmku78EpY8CSmTICDG/ddUFKXb0Ol+WQg3sV+E8/nqWjuHTlWw/0Qp+4+X0z8qwC3X78xE0NQISZNTUoUQ04HpAHFx5z4nX2ex4KisJMQc4t7B4npCwBWvwlsjYO4DcPOXqky1oihnZTbqtcHlKH+3XqczO6xzgNgGv8cAx5o6UEr5rpRyiJRySFjYuc/5FxZvHB3ZIgAIStC6iDIXw86vznq4oihKR+nMRPAdMFVoMoCSjhgfgLoWQVUVwebgjmkR1Bv+e4geAgse0Ta6VxRF6QLclgiEEJ8B64DeQogcIcQdQoi7hBB31R0yHzgEHATeA+5xVyxn0nlrXUPB5mDKasuw2q0ddGG9tsrYVgPfTAeH/ezvURRFcTN3zhq66SyvS+Bed12/JfUtgiBzEKCtJejh06NjLh6aDFNegm/vhR9fhrEPd8x1FUVRmuGRk9rrZw2FmLRE0KHdQ6DtZNb/eljxHGSt79hrK4qinMEzE4GPVoE0SGh7E3TYgHE9IeDyf0FgLHw9DSo7+PqKoigNeGQiqN+uMsih/ezwRABg9ofrP4Cy4zDnbnA4Oj4GRVEUPDQR1O9J4C+1jSY6JREARA+GSc/CgYWw9jW3XKL22DHs5eVuObeiKOcHz0wE3loisNTqMAhDx48RNDTsTuh3tVah9Mgal5xS2myULl7M0dtv5+BFEzjx7HMuOa+iKOenzlxZ3GnqWwSyUltL0GktAtDGC658HY7vhP/+DqYvB/+oNp1CSsnRm26mJjMTabcjbTaw2TBERmLo0YOagwfdFLyiKOcDj2wRGCLCAa3bpENXFzfH7A83fgzWCvjkOqhqWwvFUV5O1bZtmFNTCbr5ZkJu/y0xb7xOryWL8R09mtqcHDcFrijK+cAjWwSmhATQ6bAeOkRwnw5eXdyciFT49adaIvjsZrj1GzC2bhM4e0EBAIHXXUvAVVed9poxJgZ7YSGOykpnS0hRFKUhj2wR6EwmjLEx1Bw+RJA5iILqgs4OSZM0Fq59B7LWadNKW7ny2FaoJTJ9cHCj14zR0QDU5ua6Lk5FUc4rHpkIALwSk7AeOtzx9YbOpv91cOnzsG+utvq4FdNK7YVaImsqEZhitERgVYlAUZRmeGwiMCUlYT1yhBBTEJW2Sqpt1Z0d0i8y7oLxf4btn8G8B0A2WZ3byVagjXEYQkIaveZsEeSoRKAoStM8cowAwCspEVlTQ3jpL5vYR/pGnuVdHWjMw2Crhh//CQaz1kpoZg8De5GWCJpqEehDQxFeXqprSFGUZnlsIjAlJQEQcrIG0BaVdalEIIS2f0FtNax/E/RGmPj/mkwGtoJCdL6+6EymJk4jMEZHq5lDiqI0y3MTQWIiAP55JeBP1xkwbkgImPQM2K2w9nXQGWDCk42Sgb2gAH1I49ZAPWNMtGoRKIrSLI9NBIagIPRBQXgd0xJBdln22d/UGYSAyS+Cwwar/wU6I1z059MOsRUVYghuPD5QzxgdTdX2He6OVFGUbspjEwHUdQ9lHcMvzY9DxYc6O5zm6XRw2ctaMlj1IljL4ZKntY1uAHtBIcbY2GbfboqJwVFSgr2sDL2fX0dFrShKN+Gxs4ZAGzC2Hj5MUkASh0q6cCIALRlc8RoMvwvWvwWf3ww1ZQDYCgsxNDFQXE+tJVAUpSUenQhMiUnYCwvpa4jp+okAtGQw+QWY8g/IXAwfTEYWZWEvKmp5jCA6BuC8HjCWDge2QrWvg6K0h2cngiRtwLhPmR+F1YVda2FZS4bdCTd/BUVHsL9xEdjtLbcIYs7/FkHxF19wcMLF2MsrOjsURel2PDoReNVNIY0t1P4M3aJVUC/5YrhzKXa7LwD64uYHg/WBgegsFqzn8aKy0oWLkFVV2E4c7+xQFKXb8ehEYIyORhiNhJzUVhV3q0QAENYb24SXATDsmQXf3Qe2mkaHOdcSNGgRVO3cScXGjR0WqjvZS0up3LwZANupU50cjaJ0Px6dCIRejykhAWPOSbwN3l175lAz7BW1AOhH/Aa2zIIPp0BJ42/+xpgY5xiBdDjI/dND5P3liQ6N1V0q1qwBmw0A26n8To5GUbofj04EoE0hrT10mAT/hO7XIgBsdQXnDFMehxs+hlP74N2xcHjVacfVtwiklFSsWUttVha1WVk4Krp/n3r5ihXOEtu2fJUIFKWtVCJISsSak0Mvn/humQjsdQXn9EFB0O9KmLYUzIEw60r44W9gswLagLGjogJ7cTFFn3/ufH9NZmZnhO0y0m6nfOUqfC+egDCZsOWrriFFaSuPTwReSUlgt9OvOoTjFcepqO1e35DtRYXoAwMRhrq1geF94PcrYdBUbSXyzImQn+lcS1C5aRPly5fjP2UKANUHDrg9Rmm3k33PvVSsW+fyc1dt34G9uBi/8eMxhIaqMQJFaQeVCHr2BCDxlFa/53DJ4c4Mp81sBYWNq46afODK1+DGT6D4KLw9EtPJFQDkv/YaSEnYgw+gs1io2e/+RFCbd5zyZcsoW7rM5ecuX7EC9Hp8Ro7EEBaG3Y1dQ7UnTmAvLnbb+RWls6hEkJKCzseHsH0ngO43c8heUND8GoK+V8A9GyBlEsbdbwJQk3kQnzGjMcXE4JWSQk0HtAisR4/U/Tx6zucqW7qUEy+9hK1IW/NRvmIFlsGD0fv7ow8Ldetgcfa0aRx/5lm3nV9ROovHJwJhMGAZOhTd1r0YdAZ+Lv65s0NqE1thIfomNqRx8ouAGz9G/5tZ6EzaBjdBKbVQU45XSgrVBw4gz7LxDUDFxo0cuuKKdi3Yqs3KAs49ERTPmUPOH++jcOYHHLp0Mvn/foeaAwfwHTcOwK1dQ/byCmoyD3b7MRVFaYrHJwIAy/Dh1B45Qpojuvu1CAoL0QcHnf3Afldh6tkHY5AZ35Kv4c1heAVYcZSUYDt58qxvr9ywkZrMg1Tv3t3mGK1HtARQm5uLrK1t8/sBir74krzHHscyfBgJX36BV0oKp155BeCXRBAWhr24GGm1tusaLalvOVmzslqVOBWlO/Ho6qP1fDKGAzA8z4d5ft0nEUibDXtxcYslqBvq8ff/Bzodwq8c5s7AvOt9IJSajcsxXvHrFt9bm6OV6a7Ztxef4cPaFKe1rkWA3Y41Jwevur0gWqv4f3M4/uST+IwdQ8yrr6Izm4mbPYvSufOwZmdhSkwAwBAaBmitJGOPHm26hrTZQK9HNLMLXM2B/dpxlZVad1xoaJvOryhdmWoRAF69e6MPCKDPYSs55TnU2Buvzu2K7HX95C0VnGvIe8AAvFNTIW44/H4VXtc+BkDNpw/Dkr86q5k2pb48RfW+/W2O03r0KIawMOe/20JKSf7bb2NOSyPm9dfRmc2Atlo64IrLCbvnHueHd/012to9ZC8vJ3PkKEq+/bbZY6r37fvlfuoTm6KcJ1QiAIROh2X4cML2nsDhsHO09NwHNTuCrVBLBC0VnGuW3oj+4gcxRIRRTU9Y8yq8Phi2fQYOR6PD61clN/xAbA1pt1ObnY3PmNHaedqYCKp37qQ2K4ugG29ocivOhgxh2rf0lgaMm1pAV7V9O/aSEkrnz2/2fTX79v+SzFQiUM4zbk0EQohLhRD7hRAHhRCPNvH6b4UQp4QQ2+oe09wZT0ssw4dhOFlEeDEcLDrYWWG0ib1uVXFTm9a3llefvtRUBWoL0QJiYM5dMPNiOLAY6vrCHTU12E6cQBiN1Bw82KY++Nq848jaWrwvuACdv3+bWwQlc+cijEb8Jk4867H13TXNLSqrOXiQ/cMzqPzpp9Oer9q2DdDGQRxVVY3eJx0Oqg8cwHf8eNDpnIPfinK+cFsiEELogTeByUA/4CYhRL8mDv1CSple93jfXfGcjU9GBgCDc0xsPN49irHZ6lYVG1qaNXQW5pQUag4dQoanwR0/wNVvQ/kp+M+v4N1xsHeuszXgc+GFUFtLzaFfxlEcFRWceu31ZktV1GZpH/ym+ARM8fHOgePWkHY7pQsW4DN2DHp//7MeX/93aK7MRMXadWCzUbZs+WnPV23bDkYjsqaGyiYK8dVmZyMrKzEP6I8xMhJr1unbmtYcOkTVzl2tvS1F6XLc2SIYBhyUUh6SUlqBz4Gr3Hi9c2JKSkIfFsroE0GsylmFQzbuHulq7HUbsZxTiyClt/bhfviItvFN+s1w3xa48g2oLoEvbqH27esA8LtoLADVe3/pHiqZN4/8t96idOHCJs9f3wIwxcdpiaANLYLKjRuxn8on4PLLW3W8MJnQBwY2O0ZQtW0rABUb1jufkw4HVdu3EzBlMsLbm/KVqxq9r35cxNynD8a4WKzZp7cIjv/tKY491qjBqyjdhjsTQTTQ8KtTTt1zZ7pOCLFDCPFfIUTzG++6mRACn+EZJBws41TlSfYW7u2sUFrNVlgAej36gIB2n8MrJQXg9IVleiMMuhX+sAmum4m10giAz56/IkwGanbvdB5aXvftuvzH1U2e33o0C2E2YwgPxxQfT+2xYzhqfhmML1+9hvIfVzc5JbNk7lx0FotzemhrGMJCm11dXLl1GwhBzd59zgVp1sOHcZSWYhk2DJ+MDMpXrmwUS83+/aDT4ZWcjCkuntqjvyQCabdTtWsXtTm5alqp0m25MxE0NQ/vzP9SvgcSpJRpwA/ArCZPJMR0IcQmIcSmU26sJWMZPgxDURkxBYJV2Y2/GXY19oJC9EFBCF37/2f0SkwAg8E5PfI0egMMuJ7a+OsRJiOGxD54+VVSvfRTWP82juJ8rX6QEFSsXatNwTyD9ehRTLGxCJ0OU0I8SElttvb9wFFVRc4f/0j2nXdy+NrrKF24EGm3a69ZrZQtXoLfxInOmUKtYQgLa3KwuPb4cWx5efhdOgmkpHKjNk5QPz7gnZ6O79gx1ObmYj18epmR6v37MSUkoDObMcXFYi8uxl5aCmjjDrKyElldrcpPKN2WOxNBDtDwG34McKzhAVLKAill/dfD94DBTZ1ISvmulHKIlHJIWN3MDXfwGTECgBsOBLMyZ6XbruMqZ9u0vjWEyYRXUlKLxedqc3IxxsQifvs95mEXU12kRy54lIrHhyFragi89gocpaVU7Wi8S5o1K0tLAIApXvtZ3z1UsW4dsqqKoKm3IquqyJ3xAAcnTuTky/+i6NP/4Cgrw//yy9p0P/pmVhfXf+AH3zoVncVCxfp1zud1/v6YEhPxHa3NbCpfdfqXgJp9+/DqrbWcjHFxdfelJbOq7dt/+TsdO+3/3orSbbgzEfwEJAshEoUQJuDXwHcNDxBCRDb49UqgU/tjTDExBN5wA8NXnaJq1y7yq7p2bXttVfG5JQLQ1lFU79nTbNeGNTcHY2wMAOZh43BUO7Bd8RllRVHojA7C5Aegg/LvP3XONIK6qaNZWc4PT2ciqBswLvthKTo/PyIeeoikeXOJfvVVvHr1omDmTE6+8AL64GDnIH5rGULDsOXnN7qXqq3bEF5eePdPxXvoECrXb9Ce37YN7wsuQOh0GKOj8UruRUWDRGAvK6M2Nxdz7z7aPdTdS23dOEH1zl+6yWx5eW2KVVG6CrclAimlDfgDsAjtA/5LKeVuIcTfhRBX1h12nxBitxBiO3Af8Ft3xdNa4Q8/hAgJ4q75dn48sqKzw2mRrbCFgnNtYBk6BPupfKyHml5VXZudgylaSwRefbQPxKoiI+VHJb7jxmMYfQfeoXYqFs6B1wdpaxIq8rEd16aO1icAfUAA+sBArEePavsILF+O79ixCJMJodfjP+kS4t59l+SVK4j485+Jeu5ZhNHYpnsxhIUha2pwlJef9nzltq2YB/RHmEz4DM/AevgwNT//TM3Bn/FOv8B5nM/oMVT8tMk5C6pm//66++4NgClWa+Ra68YJqrbvwJyaqv2dVItA6abcuo5ASjlfSpkipewppXym7rm/Sim/q/v3Y1LKVCnlBVLK8VLKtq1WcgO9nx/Rf3uKhJNQ+uHszg6nRfaCsxSca6X6LrGKdesbvWYvKcFRVoax7gPQnJICQlD85VfYCwvxvfQKmPw8vjf+keoiEzYRpq1Sfrkv1o/vA35pCdT/23r0KFVbt2IvKsJvwkWNrmkIDSX41t/gO3Zsm+/FuZagQfeQo6aG6j17saSna/dbV1Kk4N33QEq8654H8B0zBmprnYPfDWcMAegsFvRhoViztd3dag4e1JKZ2UztMdUiULontbK4Cf4XX0z20DgGzsukPLPtJRU6gsNqxVFejqE1BefOwhQbizEmpsmNY6zZ2hoCY4w24Uvn44MpPp6KH38Eg8HZr+4zVvtAr4i7Vyt9PeR3WPdp0zVNy++BdW9C2QlMCVoiKFu6DGE04lP3fldpanVx9e7dUFuL98CBgNaq0QcEUDJ3LgiB9wW/tAgsgwZiiIjg2COPkP/ue1Tv3o0+IABDRITzGFNsHLVHs6javRscDrwvSMMYGUmt6hpSuimVCJrh/cgfqTbC4T/d75Zqli2pPXGSvKeeanE/YeuRIwDOncfOlc+IDCo3bmw086d+MVl9lwj80j1kGTrEudDL3K8v+pAQ7Zt0eB+Y/ALWlN8hTAYMIUGw6HH4Z2+MBauwHT9O6fx5WDIy0Pv6uiT+es56Qw1WF1dt1RJS/Tf/+pIi2O149ep1WgzCZCLhqy/xHTeOUy+/TMn//odX796nFaMzxcVhzc6mum5w3JymEoHSvalE0IyhfS/mwyvMGA4c5dTrr7vtOk0N0J565RWKP/ucivWNu2rqVdetZDX37++SOHxGjMBRVtaozHR91VFjTIzzufpuEr/xv3TrCJ0O31EjqVi92jkF1JqThyk+EXHnD3DPehj3KCZfrQy17cRJ/AKOwMEfmqxt1F71XUMN1xJUbt2KMS7utBXYPiO0QeiG3UL1jOHhxLz2KjFvvoExLq5RF5UxLhbbiRNUbNyonTcoCENUpBojULotlQiaYTaY8Z94CasGeVHw/kwq6maZuFLh7I/5edKlzsqeADU//+ysglm9e0+z763evQudxYIpIcElsVjqZuecOU5gzclBHxh42rdmn9GjMMbF4TfpktOO9Rk9BntxMYUfzcJRU6OtIaibOkp4Xy0RTPtlqYiv70H45Dr4Vz+Y9xAcXgX2xmsR2kLn769tYl83RiClpGrbdiwDT//A9xk5UtvickTzs5L8Jkyg1+JFhNzxu9OeN8Vp91Sxdh3eaWkAGCMjsefnn7ZYTlG6C5UIWjAlcQrvjbdhiw7n2P/9n0sXDJWvWcOJ55+nNiuLY3/6k3PDllOvvobObMYQFUn13uZn01bt2o05NfWcFpM1ZAgOxqtPn0bjBNoagpjTnvNOTaXX4kUYG/Sbg7ZBjPfAgZx86SUOTrgY69Gjzqmj9eoTg/mCNIx/2QvXfwAxQ2DrJzDrCvhnb5j7gJYUHPY234cQom6nsvy6+HOw5+c7xwecccTF0euHJfhNntzma5ji6rrJbDa80wYAYIzSuuhsx4+3+XyK0tlUImjBhVEXYvYLZP5tKdgKC8mZ8QAOF4wXWHNyOPbgn/DqmUTks89StX07p159laqduyhbvJjg22/HMmQI1XuabhFIq5WavXsxDxhwzrE05JORQdWWLadV4KzNzm6UCJqj9/Uh/j+fEvfRh5h799Y+KM/outL7+uI/ZTLBU6eCwQv6Xwc3fgKP/Aw3zIbEMbD9cy0pvJgEX9wKP70Pha3fMEgfFuosPFcwcyYIoY0JnMEYGdnsRjQtMTVIbg1bBIAaJ1C6JbVDWQuMeiOXxF/C14fmcvvf/kLBn5/k2J8eIvpfLyMM7fvTaWUV7kNKScwbb2CKj6dq+3YK3p9J2Q9L0QcEEHz7byn+738p/e57bPn5jXbDqs7M1Eo79091xW06+Vw4gsKPPqJyyxZ8R47UFoQdO4bfJWcvAV1PCIFPRgY+GRnafspBjWc1Rb/8cuM3mnyg31Xaw1oJmYsg8wc4tAL21q1DDOsLvSdrj+jBoNM3GYMhNIza7Gwq1q+n+PMvCP7tb9u8K1pL9IGB6Pz9cVRV4dW3LwDGqLpEkKvGCZTuR7UIzmJK4hSqbFX8NNCHiMcfo2zJEvKefLLdBcYK3nuPmn37iH7pRef8+ojHHsUrORnrkSOETJ+O3tcXcz+tYndT3UPVu7QBXVcNFNezDB4MBgOVdYPUtpMnkbW1GKNb1yI4kyE4uF3fuDFZIPUauPpNeGAX/GEzTHoOfEK1xWozJ8KLifDlVNg8C0pP//A1hIVSe/w4eX95AmN8HGH339eu+FsMMTEBc9++6Ly8tGv26AFCqBaB0i2pFsFZDIoYRIQlgvmH53PZ1DexF5eQ/9Zb6H19CX/00TZ90EkpKfl+Lj4XXnjaTBSd2UzMG69T/PU3BN1yMwDmum+a1bv3OOfq16vetQtdQIBzkZer6Hx88E6/gLKlywj81a+wnTgB4Cwv0SmEgNBe2mPEPVBZCIeWw8Fl8PNS2FO3vWTEAOg1ARLHYAgKwFFaiqOsjPiPZ6Pz9nZ5WFHPPKOV7a6jM5kwhIZSm6daBEr3oxLBWeiEjsmJk/lkzycUVxcT+sc/YC8vo3DWbNDpCX/k4VYng+pdu6jNzib0rrsavWaKjyf8wQecv+v9/DDGxzU5TlC1axfeqant+7Z9FoHXX0/eY4/zyDCyRQAAHMtJREFU8yWTMNT1e5taOUbQISzB2rhC/+u0ukYn90DmEu2x7g1Y8wqGn30Bf4LG9MISWAw15eDl2vUKXr16NXrOEBXZJesN5T3xBI6aGqJffLGzQ1G6KJUIWmFK4hQ+2v0Ri44s4sY+NxLx2GNgs1P44YegE4Q/9FCrPpRL5y8AoxG/iye06rrmvv0azet3VFdTk5mJ7+9+18y7zk3g1Vfjk5FByXffUzJnDoawMOdAaJcjBESkao9RM8BaAdkb8N26mGDHUsLC18InK0HoIWogJIyE+JEQeQH4RmjvdyFjZBQ1Lcz0kg6Hy2Z5tVZtbi7FX3+D0OtxPPkkOh+fDr2+0j2oRNAKfYL70De4L7P3zOa6lOsw6AxEPPEXpHRQOPMD9H5+TX7Lb0g6HJQuWIDvqFGt3kjG3K8fZQsXYi8pcb6nZv9+sNkwD3Dt+EBDxh49CJ1+JyF3TgMpO/zDq91MPtDzIow9LyLiepyJgSOr4cgaWPeWNsYA4B2sJZDY4VqCiB2uvf8cGKOiKF+2DCnlaV8MpNVK3t+eouLHH0mc879z2lq0rYo++wwcDqTDQcXGjfiNH++W69Tm5WHo0cMtrVTF/brJf+GdSwjB79N+T1ZZFgsOL3A+1+OJJ/CfMplTb77lLPnQnKpt27AdP47/lNbPW/9lwPiXWnz1e+OeOS3THYQQ3ScJNKUuMTDhr3DHIng0C347Dya/CH0vh5oyWP0v+PgaeD4O3hmrrWHY+gmc2NPmdQzGyEik1ercQhS0on1Zd06n5JtvsOXna4XuOoijqoqir/6L77hxCLOZitVr/n97Zx5fVXH3//f3brnZNwKELSyCQn0qCloXquKCa5W64mNVXKuglqp1afvo4777VNG2glasWisKv8ojFVyK+tSFRXBBBTeWQEACCdluctfv748593ITEkhCQiB33q/XeZ1z5szMmbmTnM+ZmTPfb6fcp/7z5Xx77HFUPvd8p+Rv6Xxsj6CVjB0wlmH5w5j22TROHnQybpcbcbnodcst1L77Hj/cex/9n/xzi+mr5/4TSUsja+z21jZbwj/CmTD+8suExcyG5ctxFxaar1QsbcOXAQPHmC1OsAbWLoQ178P6j+HzV2DJX8w1b4YZRupzoNmKR0LhPo0miZNJfEJatgFPYSHh9etZe/kVhEpL6fPA/dR9+BGVL75IwcUT8XZC+1XPm487Ly/xt1L12mvEqqoovPQSNBal7t/NuxPdVcqnPgaxGFumTyfv3HMSX1JZ9h724te93YtLXFx5wJWsrl7N66tfT4R7ioroMXkyte++S8077zSbVqNRqufPJ+uoo3BntX74wVNQgKd370YTxg1fLCd9//1tF7yjSMuGocfBcbfBRXPgpjUweTH8/Ek48AKIRYwwzL4cnjgY7u0L04+FOdfCoumw5gOoNyvOE4vKysrQWIz1N/yGSHk5A55+itzTTqPH5MmoKpv/+KcOr0YsGKTspptYe8klVL78MqpK5XPPk7bffqSPHk3WmDGE1qwh5BgR7CgCy5ZR997/kTV2LJHycrbOmtWh+Vt2D7ZH0AaOHXAs++Ttw7TPpnHSwJNwOwuaCn5xPltffplN995H5uGH4/L5GqULLF5MdPPmNg0LxfGPGJEQguD3qwh+9z3Z407Y9cpYmsflgqJhZjtgggmLRmDzSij7BH5YDhs/N4vclia52M7qhddnzEyE33uWrSveoX7ZMorvvovMQw4BwNevL/lnn03lzJkUXnZpoxXKu0r90qVoMIivpISN/3UrgY8WEvz6a4rvutMs8htjekF1//43vgkTOuy+5Y89hruggL4PP8TaSy9jy/SnyD/rLKTJ/wBgjPT17t2h9bZ0DLZH0AZc4uKXB/ySVVWrmLd6XiJcfD56/fYWQmvWUPGXZxql0UiELdOm4crIaJejFf+IEYRWrWLjnXfx/Wmn4fL7yT7+uF2ui6UNuD1mYvnA8+HEe2Hia3DjKrhuBZz/Chx7Gww9HldOFuKBhiUfsOnpWWQUBcn96mqYNhb+MRk+mErhCcMRt5vyqY93aBHr3n8fvF4GznyJ3NNPo3ruXNy5ueSceioAvkGDzGR20vCQRiKESkvRdlp/rVu0iMCHH1F4+eW4MjLoMekqIhs2UDVnznZxA0uXsXbixaw6+5yE/2jLnoPtEbSRcSXjeLrgae5fdD+jeo2id6YZ68366U/JHjeO8kcfxdunmNzTjDfOTQ8+RN0HH9L7zjvatbDJP2IEqFL54ovknXkmRddek7C5b+lCRCCn2GxDjQkOAbwvn0r1d9+Bx03vG69EPBth01fwzRvwyfN4gfxBOVT87xyKei7Ct++PoecIKBwCBYMhtz94mn+bTv/Rj1r8/LP2gw/IGDkSd24uxffeS9p+w/EWF+Py+53iml5B9dy5aDiMhkKsveRS6j/9FFdODukjDyB77FjyJkxocdixet58yh+fiq//ADJGHUTNm2/hKSoi/zzTw8gcMwb//vuz+clp5I4fnzDDEq2tpezGG828iNfDmksupf/jU8k8/PBdbARLR2GFoI24xMUDRz7AeXPP4/p3rmfGiTPwuo1f3T7330dpdTVlN98CImgwSMWzz5J/4QXkn312u+6XOeYIin51LVnHHGMMuVn2aLzFxYS++47Cyy4j7WdTGl+s2wKbV1L49RIqpvyZik/q6R1rMsQkLsgbYEShYDDklVC/Cdbe8ieyjjyCfk9O3+5BHamoIPjlVxRN+ZWThYvCiyduV7bMMUewdeZMAosXs3n6dOqXL6fH1VcT+WEjgSUfs/H2O4hUVlI0aVKjdKrKlulPUf7II6QN3YfQqlXULlgAQK/f/76R2PS46krWTb6ajXfeRc8brsednc0Pd95FuKyMkuefw9e/P2svvYzSX15JwcSJ+AYPxtu3D/4RIzrcSZGl9Uh7beZ0FaNHj9YlS5Z0dTF4c82bXPfOdUzYdwK/O/R3ifBYfT2lV15FYPFiY+/+kEPo/+Sf222kzrJ3Uf74E9TMn8/Al2cmHpDNUXbTTdS8+Rb7vLMAt6vBWFet+B62fAeVq8y+YhUEqyhblEvV96Yn0OsnEQqOHAwFgyB/IOSVULVsPWX3P83A558hfXTL/hWiNTV8fehhuDIyiNXUUHzfveSNHw+Yh/2Gm2+h6tVXKb7nHvLO+DkAsbo6Nt5zD1WzZpNzyikU33M3rrQ0Ilu2EPzuOzJGj270ibGq8sOdd1H54ou4CwrIOeEEKv/2N3pMuoqia43Np2hVFeumTCHw0UKzOhxwFxZSfPddZB999K78/JYdICIfq+roZq9ZIWg/Dy95mBlfzOCeMffwsyE/S4THAgHWXX01kfLNlLzwfMKdoyU1aM0K4vrlX7D6rLPodcvNFFx0UYvxolvK+ObYk8k54sdEt5RT9/kaBl5QjN+3wTG2p5QtzKNmvZ9h4zciGbmmR5FXYvY5fSG3r9nn9GX1Vb+hfukyev32txRceEHjcodClF55FXULF9L7tlsJrlhB1atziNXV0WPSJHpcc3Wrv1arX/4FP9x9N/XLluE/4McMfP55xOttFCcWChHZsIHgqlWUP/I/BL/+mrxzz6Xnb27AlZlpv4zrYKwQdBKRWIQr3ryCTzZ9wpPHP8nBvQ9OXFNViMUQd/Omki2W1f95PpHycobMe73Fv5PKl2ay8bbbGPjS3/EOGMCq08fjyshg0KxXcKV50Mq1fDv+QtKH9qHfZT+FqnWwdQ1UrjbH4UCj/Boq0whFe5BzYH/I7eeIRHzfh6hmsmbyjQRXrER8PnJOOpH8885r1qXnzlBV6v79Pv4Rw3e6mjoWClH+h0eN2RZVEEG8Xrx9+pA7/nRyx4/vlLUXqYQVgk6kKljFRa9fxKbAJmacNINh+cO6ukiWvYTqefNYP+XX9PvjE2Qf0/xCw1VnnoWGwwx69R+ICHUfLWTtxReTefjh9Ht8KuGyMr4/5VR633E7+eec0zixKtRXQvV603uoXg9V641AVK+HqlJzHgs3ShZpcFFbnk/WiCI8PftAdrExAZ7RAzKLnEnyvia8g435BZYtI7BwIRoKEQsGaVj+BYGFC0GE9IMOIm3IEHwlJXj79sGVkYH4/bjS03FlZuHOzsKVk2MXtLWAFYJOZkPtBn7xz1+AwAsnv5D4kshi2REaifDt8ePwlZRQMuOZ7a43fPklq844k16/+x0FF/wiEb511mw2/P73pI86iKwjjqD80ccY8tZb+Pr1bXshYjGoKzfCULMBajeZ89ofoGajEZCajRDYDNFmvPP5sow4ZPUyYpFZ5Gw9tolHRqGzFRivdG0kVFrK1tmzqfvgA8Jr1u7UZaynVy8jFgP6G7HwercNS8XUCKTbhbg94HaZ85iisSgiLvC4EY/XfAammOuq5iT+vHS5EbfLTO67XIjLDGNpJIpGo+Z3dbm2xREx+ZGUJ5jrLrcpjzMUpqoQjUEsikZjJo7bg3g8+EcMT3jFaytWCHYDKytWMnHeRHJ8OTx01EP8R1HHupG0dE82T59O+cOPkDV2LPn/eR6ZRxyRmF/YcPvtVM3+fwx9793tDBVWzZ1L2Y03QTSKr6SEIfPnNZd9x6FqzHHUlRvBiPcwajc52w8Q2GKuB7aAtrA2wZeVJAyOOKQXOMf528LjgpKev50numhVFeGNG9H6emINQWKBALG6WmK1tUQqKwmvLSW0ejWh9evQhiAaChmf4CLbtlgMok1sSYlse9DvoRRefjk9r7+uXWl3JAT2U5YOYt+CfZl2/DRuePcGLnz9QqaMmsKFIy60E16WHVJw0UXEAgG2znyZ2gUL8BQX4+vXD1d2NoGPPiL7hHHNWqvNPeUUXH4/66f8mqxOsijaCBHw55itcMiO48aixuxGXbnpSQS2OCKxBeoroG6zCa8rNyu2AxUQqm05P2+mGYJKy4GMQtyZPXCn50N6HvjzIDsPeuSCvxD8gyH9WCMg/rxm12TEMW/eUVM3l3kjj4dpJNK47iLmhT7+1h6LGauu0Rhg5gNRNV8Hut1mhbojNqpJPQln/iNhAj0WMz2ISKSxWXS3sWWG223iRCJoJNIpTpbA9gg6nKpgFbe+fyv/Kv0XhxUfxnWjr2O/gv26uliWPRwNhah+801q5s0jUllJrKYWDYfp8+ADpP+oZd/UkfJyXLm525k12euIBM18RqDCEQlnC2wxIhGsgYaqbaISqICGrRBp2HG+3oxtouDPMbal0rKNqPhzzRYP82UlXY/HyQGPv8N9V3QFdmhoN6Oq/H3l35m6bCo1oRrGlYxj0shJDMnbyZuUxWJpG+EGIxDBarNv2Gp6I/WVZh8/b9jqxKnetm+o2m6ivFlc3iYCkSQavkxz7MswouPNAG+6CW8UL8v0anxZ5rpr939NaIWgi6gOVfPsF8/y3JfPUR+pZ1SvUZw59EyOLzkev6flxUYWi2U3oArhetPbiPc64vtgTWPhCNYYl6fx8ES8WvOJbqgOtA3+K9w+RzCytomEJx28fhMeFxRv5jaR8WVB34OgX7PP8p1ihaCLqWioYPY3s5n9zWxKa0rJ9mVz8qCTOXPomQwvHN7VxbNYLLuKqhneitQbcQkFIFSTJCq15jxUZ3ox8Xjx8GCtGeYK1zv7wLZ8wnXbJt/HXGdMprcDKwR7CDGNsWTjEmZ9M4u31rxFKBZiWP4wxvQdw6HFh3JQr4NIc9tvoC0WSxKqRhxCdeD2mnmNdmCFYA+kKljF3O/n8saaN/h006dENILX5WWfvH0YXjicffP3ZVDuIAbmDKRXZi9cYi2GWyyW9mOFYA8nEA6w5IclLNm4hBUVK1hRsYLKYGXiepo7jf7Z/RNbUXoR+f58CvwF5PvzyUvLI9+fT4Ynw36uarFYmqXL1hGIyInAo4AbeEpV72tyPQ34KzAK2AKcq6qrO7NMeyIZ3gyO7HckR/Y7EjBfHZXXl7Omeg2rq1ezumo1pTWllNaU8mHZhzREm/9kzuPykOvLJTfN2Xy55KTlkOXNIsObQYYng3RPemLze/xmc/sTx+nudNI8aaS50/C5fXjEY8XFYunmdJoQiIgbeAI4HlgHLBaROar6ZVK0S4FKVd1HRCYA9wPndlaZ9hZEhJ4ZPemZ0bORITswIhGIBKior2BLwxa2BrdS2VDJ1uBWqoJVVIWqqApWUR2sZmNgIysrV1IXriMQDhDRSAt3bBmXuPC6vI03t9l7XJ7EuUc8eFwe3OLG7XLjcXkSYfHw+N4lLjwuDy5x4Xa5t4XJtjCXuBLhLnEhiDl3mXBBcLvMPh4nsWEWByWnFZFEeKPjpPTx396Fcy40ipfYO2kQEufJ1xLhSfcBGsdrLiwpPF6Wpvknh293zDYzBomyOHGS72WxNKUzewSHAN+q6vcAIvJ34HQgWQhOB/7bOX4FeFxERPe28ardiIiQ6c0k05tJ/5z+bUobioaoj9RTH6knEAlQH6knGAnSEG2gIeJs0QbqI/WEo2GC0SDBaJCIRghHw4RjYSKxCOFYmHA0TEQjhKIhwrEw0ViUcCxMgzYQjUWJxCJEYhGi6hxrhGgsSlTNFovFEmExjRHVKIpt9t1Je8Skabqd5ZccHhfIRHhzebaQ93Z5tZC+6XGjurQmzxbyb085GsVvY5yW7nXG0DO46Ectmy1vL50pBH2B0qTzdcBPWoqjqhERqQIKgc3JkUTkCuAKgAHW8XW78bl9+Nw+ctPa99VBZxMXhJjGEgIRI4aqNhse021bPI6izR+romgibvxaPCymsUZhqpq4D8q2tCSlVW10nlyPpnk1jdc0LC6CievNhG+XR5JwNhe3URylUVjT+yTy2VEeTj47TNckPPl6c/Gb1nm7dC2laSZ9c/dsWpfW5t9SnF0ph3Oy03x2dO/C9B2b824vnSkEzclb09q2Jg6qOg2YBmayeNeLZtkTSR6iwbpxsFh2G535TeI6IHnsoh9Q1lIcEfEAuUBFJ5bJYrFYLE3oTCFYDAwVkUEi4gMmAHOaxJkDxAe8zgL+ZecHLBaLZffSaUNDzpj/1cB8TEf/L6r6hYjcASxR1TnA08BzIvItpicwobPKY7FYLJbm6dR1BKr6T+CfTcJuTTpuAM7uzDJYLBaLZcdYuwUWi8WS4lghsFgslhTHCoHFYrGkOFYILBaLJcXZ66yPikg5sKadyXvQZNVyipCK9U7FOkNq1jsV6wxtr3eJqhY1d2GvE4JdQUSWtGSGtTuTivVOxTpDatY7FesMHVtvOzRksVgsKY4VAovFYklxUk0IpnV1AbqIVKx3KtYZUrPeqVhn6MB6p9QcgcVisVi2J9V6BBaLxWJpQsoIgYicKCIrReRbEbm5q8vTGYhIfxFZICJficgXIvIrJ7xARN4UkW+cfX5Xl7UzEBG3iCwTkdec80EistCp90uOFdxug4jkicgrIrLCafPDUqGtReTXzt/3chF5UUT83bGtReQvIrJJRJYnhTXbvmJ4zHm+fSYiB7XlXikhBEn+k08CRgDniciIri1VpxABrlfV4cChwGSnnjcDb6vqUOBt57w78ivgq6Tz+4H/cepdifGR3Z14FJinqvsBB2Dq3q3bWkT6AtcCo1V1f4xl47i/8+7W1jOAE5uEtdS+JwFDne0K4E9tuVFKCAFJ/pNVNQTE/Sd3K1R1g6oudY5rMA+Gvpi6PutEexYY3zUl7DxEpB9wCvCUcy7AMRhf2NDN6i0iOcCRGFPuqGpIVbeSAm2NsZqc7jizygA20A3bWlXfY3tHXS217+nAX9XwEZAnIsWtvVeqCEFz/pP7dlFZdgsiMhA4EFgI9FLVDWDEAujZdSXrNP4A3AjEnPNCYKuqRpzz7tbmg4Fy4BlnOOwpEcmkm7e1qq4HHgLWYgSgCviY7t3WybTUvrv0jEsVIWiVb+TugohkAbOAKapa3dXl6WxE5FRgk6p+nBzcTNTu1OYe4CDgT6p6IFBHNxsGag5nTPx0YBDQB8jEDIs0pTu1dWvYpb/3VBGC1vhP7haIiBcjAi+o6mwn+Id4N9HZb+qq8nUSRwCnichqzLDfMZgeQp4zfADdr83XAetUdaFz/gpGGLp7Wx8HrFLVclUNA7OBw+nebZ1MS+27S8+4VBGC1vhP3utxxsWfBr5S1UeSLiX7hr4IeHV3l60zUdVbVLWfqg7EtO2/VPV8YAHGFzZ0s3qr6kagVET2dYKOBb6km7c1ZkjoUBHJcP7e4/Xutm3dhJbadw5wofP10KFAVXwIqVWoakpswMnA18B3wO+6ujydVMcxmO7gZ8AnznYyZrz8beAbZ1/Q1WXtxN/gaOA153gwsAj4FngZSOvq8nVwXUcCS5z2/geQnwptDdwOrACWA88Bad2xrYEXMfMgYcwb/6UttS9maOgJ5/n2Oearqlbfy64stlgslhQnVYaGLBaLxdICVggsFoslxbFCYLFYLCmOFQKLxWJJcawQWCwWS4pjhcDS5YiIisjDSec3iMh/d1DeM0TkrJ3H3OX7nO1YAF3QJLyPiLziHI8UkZM78J55IjKpuXtZLG3BCoFlTyAInCEiPbq6IMk4Vmtby6XAJFUdmxyoqmWqGheikZh1HW0pg2cHl/OAhBA0uZfF0mqsEFj2BCIYt3u/bnqh6Ru9iNQ6+6NF5F0RmSkiX4vIfSJyvogsEpHPRWRIUjbHicj/OfFOddK7ReRBEVns2G//ZVK+C0Tkb5iFOU3Lc56T/3IRud8JuxWzmO/PIvJgk/gDnbg+4A7gXBH5RETOFZFMx+b8Ysdw3OlOmoki8rKI/C/whohkicjbIrLUuXfccu59wBAnvwfj93Ly8IvIM078ZSIyNinv2SIyT4xN+weSfo8ZTlk/F5Ht2sLSfdnR24bFsjt5Avgs/mBqJQcAwzGmer8HnlLVQ8Q45LkGmOLEGwgcBQwBFojIPsCFmGX4B4tIGvC+iLzhxD8E2F9VVyXfTET6YOzej8LYvH9DRMar6h0icgxwg6ouaa6gqhpyBGO0ql7t5HcPxhzGJSKSBywSkbecJIcBP1bVCqdX8HNVrXZ6TR+JyByMkbn9VXWkk9/ApFtOdu77HyKyn1PWYc61kRjLtEFgpYhMxVix7KvGxj9OeSwpgu0RWPYI1FhJ/SvG6UhrWazGB0MQs7Q+/iD/HPPwjzNTVWOq+g1GMPYDxmFss3yCMdVdiHHqAbCoqQg4HAy8o8bgWQR4AeMToL2MA252yvAO4AcGONfeVNW4LXoB7hGRz4C3MOaFe+0k7zEY8wuo6gpgDRAXgrdVtUpVGzB2ekowv8tgEZkqIicC3d5qrWUbtkdg2ZP4A7AUeCYpLILzwuIYGUt2QRhMOo4lncdo/Lfd1I6KYh6u16jq/OQLInI0xqRzczRn6ndXEOBMVV3ZpAw/aVKG84EiYJSqhsVYWfW3Iu+WSP7dooBHVStF5ADgBExv4hzgklbVwrLXY3sElj0G5w14Jo3dDK7GDMWAsUPvbUfWZ4uIy5k3GAysBOYDV4kx242IDBPj2GVHLASOEpEezkTyecC7bShHDZCddD4fuMYROETkwBbS5WL8LYSdsf6SFvJL5j2MgOAMCQ3A1LtZnCEnl6rOAv4LY9LakiJYIbDsaTwMJH89NB3z8F0ENH1Tbi0rMQ/s14ErnSGRpzDDIkudCdYn2UkPWY1Z31swJo8/BZaqalvMHS8ARsQni4E7McL2mVOGO1tI9wIwWkSWYB7uK5zybMHMbSxvOkkN/BFwi8jnwEvARGcIrSX6Au84w1QznHpaUgRrfdRisVhSHNsjsFgslhTHCoHFYrGkOFYILBaLJcWxQmCxWCwpjhUCi8ViSXGsEFgsFkuKY4XAYrFYUhwrBBaLxZLi/H81l2lx8zvnDAAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "RESULTS FOR NN\n", "\n", "Best parameters set found:\n", "0.02\n", "Score with best parameters:\n", "0.76525\n" ] } ], "source": [ "import matplotlib.pyplot as plt\n", "import operator\n", "\n", "lr_list = [0.0002, 0.002, 0.02, 0.2]\n", "#lr_list = [10**exp for exp in range(-3,1)]\n", "scores = {}\n", "\n", "best_hidden_layer_sizes = mlp_arch_CV.best_params_['hidden_layer_sizes']\n", "best_batch_size = mlp_batch_CV.best_params_['batch_size']\n", "\n", "for lr in lr_list: \n", " mlp = MLPClassifier(best_hidden_layer_sizes, batch_size = best_batch_size,\n", " max_iter=100, alpha=1e-4, solver='sgd',\n", " tol=1e-4, random_state=ID, learning_rate_init=lr, n_iter_no_change=200)\n", " mlp.fit(X_train, y_train)\n", " scores[lr] = mlp.score(X_test, y_test)\n", " plt.plot(mlp.loss_curve_, label='lr: ' + str(lr))\n", "\n", "plt.legend(loc = 1)\n", "plt.xlabel('Number of iterations')\n", "plt.ylabel('Loss')\n", "plt.show()\n", "\n", "\n", "print ('RESULTS FOR NN\\n')\n", "\n", "print(\"Best parameters set found:\")\n", "print(max(scores.items(), key=operator.itemgetter(1))[0])\n", "\n", "print(\"Score with best parameters:\")\n", "print(max(scores.values()))\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### QUESTION 3\n", "\n", "Comment about the learning curves (i.e. the variation of the loss over the steps). How does the curve changes for different learning rates in terms of stability and speed of convergence ?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## [ANSWER TO QUESTION 3]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### TO DO 5\n", "\n", "Now get training and test error for a NN with best parameters (architecture, batch size and learning rate)from above. Plot the learning curve also for this case." ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "scrolled": true }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "C:\\Users\\Anay Deshpande\\Anaconda3\\lib\\site-packages\\sklearn\\neural_network\\multilayer_perceptron.py:566: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (500) reached and the optimization hasn't converged yet.\n", " % self.max_iter, ConvergenceWarning)\n", "No handles with labels found to put in legend.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "RESULTS FOR BEST NN\n", "\n", "Best NN training error: 0.000000\n", "Best NN test error: 0.248000\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEJCAYAAACZjSCSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXRcZ33/8fd3RiONdsmSLNvyIm8hJCFxbOEQQtPAAZqkKSkNgaRpgdL+3BYKpKULtL/Slv7aQ0vZE0gCJCmUsiXQJjQ0GHAWciCJ7The48R24kSRLdvyIsuLpNF8f3/cK2csy7Jk6/pKcz+vc+bMzL2P7nyvLOuj5y7PY+6OiIgkVyruAkREJF4KAhGRhFMQiIgknIJARCThFAQiIgmnIBARSbjIgsDMZpnZCjPbZGYbzOzDw7S5wswOmNma8PHxqOoREZHhlUS47RzwEXdfbWbVwCozW+7uG4e0e9Tdr4mwDhERGUFkQeDuO4Ad4euDZrYJaAGGBsGYNDY2emtr65kXKCKSIKtWrdrj7k3DrYuyR3CMmbUCFwOPD7P6UjN7GugA/tzdN4y0rdbWVlauXDnuNYqIFDMz236ydZEHgZlVAfcCN7t795DVq4E57t5jZlcD/wUsHGYby4BlALNnz464YhGRZIn0qiEzyxCEwDfd/ftD17t7t7v3hK8fADJm1jhMuzvcvc3d25qahu3ZiIjIaYryqiEDvgZscvfPnKTNtLAdZrY0rKcrqppEROREUR4augz4XWCdma0Jl/01MBvA3W8D3gH8sZnlgCPADa7hUEVERtTf3097eztHjx49YV02m2XmzJlkMplRby/Kq4Z+Dtgp2twC3BJVDSIixai9vZ3q6mpaW1sJD6oA4O50dXXR3t7O3LlzR7093VksIjLJHD16lIaGhuNCAMDMaGhoGLanMBIFgYjIJDQ0BE61fCSJCYLNOw/yyR89Q/fR/rhLERGZUBITBC/uPcxtD29l666euEsREZlQEhME85sqAdiiIBCRInCyCyxP58LLxATB7CkVZNLG1t2H4i5FROSMZLNZurq6TvilP3jVUDabHdP2zspYQxNBSTpFa0MlW3erRyAik9vMmTNpb29n9+7dJ6wbvI9gLBITBABzGyvZ3nU47jJERM5IJpMZ030Cp5KYQ0MAM+rK6ThwJO4yREQmlEQFwfTaLAeP5jioS0hFRI5JVhDUlQOw48DY7roTESlmiQqClrrgTHrHfh0eEhEZlKggmF4b9Ag69qtHICIyKFFBMLW6jJTBDp0wFhE5JlFBUJJO0VyTVY9ARKRAooIAwktIdY5AROSYxAXB9NqsDg2JiBRIXBDMqCtnx4GjpzUwk4hIMUpcEEyvzdKby7P3UF/cpYiITAiJC4IZdbqEVESkUPKCYPBeAp0nEBEBEhgE08O7i3foyiERESCBQdBQWUppSYoOjTckIgIkMAjMLLyEVEEgIgIJDAKA5posnQoCEREgyUFwUEEgIgIJDYJpNWXs1E1lIiJAQoOguSa4qezAEc1UJiKS2CAA6OzujbkSEZH4JTIIptUGQbCzW+cJRESSGQSDPQJdOSQikswgaKouA6BTPQIRkWQGQTaTpr4io0NDIiJEGARmNsvMVpjZJjPbYGYfHqaNmdkXzGyLma01s8VR1TNUc01WPQIREaAkwm3ngI+4+2ozqwZWmdlyd99Y0OYqYGH4uAT4cvgcuSAIdNWQiEhkPQJ33+Huq8PXB4FNQMuQZtcCX/fAL4E6M5seVU2FptVkdWhIRISzdI7AzFqBi4HHh6xqAV4qeN/OiWERiebaLHt6eukfyJ+NjxMRmbAiDwIzqwLuBW529+6hq4f5khPGfTCzZWa20sxW7t69e1zqaq4pwx329OjwkIgkW6RBYGYZghD4prt/f5gm7cCsgvczgY6hjdz9Dndvc/e2pqamcalt8F6CnbqXQEQSLsqrhgz4GrDJ3T9zkmb3Ae8Orx56HXDA3XdEVVOhV4aZUBCISLJFedXQZcDvAuvMbE247K+B2QDufhvwAHA1sAU4DPxehPUcZ3CYCV05JCJJF1kQuPvPGf4cQGEbBz4QVQ0jmVJRSiZtunJIRBIvkXcWA6RSxtRqzVQmIpLYIIDgyiHNVCYiSZfwIMjqqiERSbzEB4FOFotI0iU6CKbVZunpzdHTm4u7FBGR2CQ6CJprgnkJdunKIRFJsGQHQbWmrBQRSXQQTA3vLt6l8wQikmCJDoJX7i5Wj0BEkivRQVBVVkJlaVpXDolIoiU6CCC8hFQ3lYlIgiU+CKbWlOmqIRFJtMQHQbOmrBSRhFMQhHcXBwOhiogkj4KgJktfLs+BI/1xlyIiEgsFQXh3sa4cEpGkUhBoykoRSTgFgYaZEJGES3wQTNXAcyKScIkPgmwmTW15RucIRCSxEh8EANNqsjpHICKJpSAgODzUeVA9AhFJJgUBwZVDOkcgIkmlICC4l2DXwV4G8rq7WESSR0FA0CMYyDtdh3R4SESSR0EATK3WTGUiklwKAjRTmYgkm4IAjTckIsmmIAAaq8owU49ARJJJQQBk0ikaKssUBCKSSAqCUHONgkBEkklBEBqcqUxEJGkiCwIzu9PMdpnZ+pOsv8LMDpjZmvDx8ahqGY3mmiy7DqpHICLJUxLhtu8GbgG+PkKbR939mghrGLXmmjL29PTRP5Ank1ZHSUSSI7LfeO7+CLA3qu2Pt2maqUxEEiruP30vNbOnzexHZnZ+nIXMqCsH4OV9R+IsQ0TkrIvy0NCprAbmuHuPmV0N/BewcLiGZrYMWAYwe/bsSIppqQ+DYL+CQESSJbYegbt3u3tP+PoBIGNmjSdpe4e7t7l7W1NTUyT1zKhVj0BEkim2IDCzaWZm4eulYS1dcdVTXpqmobKUjgMKAhFJlsgODZnZt4ArgEYzawf+DsgAuPttwDuAPzazHHAEuMHdY50QoKW+nHb1CEQkYSILAne/8RTrbyG4vHTCaKkrZ3PnwbjLEBE5q+K+amhCaakrp2P/EWLumIiInFUKggIt9eUc7c/Tdagv7lJERM4aBUGBFt1LICIJpCAooHsJRCSJFAQFBnsEHQoCEUkQBUGB2vIMlaVpXUIqIomiIChgZrTUl+vQkIgkioJgiJa6cp0sFpFEURAMMbO+gvZ9h+MuQ0TkrFEQDDGnoYLuozn2H9a9BCKSDKMKAjObb2Zl4esrzOxDZlYXbWnxmNNQCcDzew7FXImIyNkx2h7BvcCAmS0AvgbMBf4zsqpiNLexAoDtXTo8JCLJMNogyLt7Dng78Dl3/1NgenRlxWdmfQVm6hGISHKMNgj6zexG4D3AD8NlmWhKilc2k2ZGbTnbuxQEIpIMow2C3wMuBf7J3Z83s7nAf0RXVrxaGyt4QYeGRCQhRjUfgbtvBD4EYGb1QLW7fzLKwuLU2lDJ/6zbEXcZIiJnxWivGnrIzGrMbArwNHCXmX0m2tLi09pQyf7D/bqEVEQSYbSHhmrdvRv4LeAud18CvDm6suI1rym4hHSbThiLSAKMNghKzGw68E5eOVlctBZOrQZgS2dPzJWIiERvtEHwCeBBYKu7P2lm84DnoisrXi315WQzKZ7bpfmLRaT4jfZk8feA7xW83wZcF1VRcUunjPlNVTy3Sz0CESl+oz1ZPNPMfmBmu8ys08zuNbOZURcXp4VTq3hOh4ZEJAFGe2joLuA+YAbQAtwfLitaC5ureXn/EXp6c3GXIiISqdEGQZO73+XuufBxN9AUYV2xWzC1CoCtOjwkIkVutEGwx8x+x8zS4eN3gK4oC4vbwjAINnfqhLGIFLfRBsH7CC4d3QnsAN5BMOxE0ZrTUEl5Js2mHd1xlyIiEqlRBYG7v+jub3P3Jnef6u6/SXBzWdFKp4xzp1ezoUNBICLF7UxmKPuzcatigjp/Rg2bOrpx97hLERGJzJkEgY1bFRPUedNrOdibo12T2YtIETuTICj6P5PPn1EDwIaOAzFXIiISnRGDwMwOmln3MI+DBPcUFLVXTasmZeg8gYgUtRGHmHD36rNVyESUzaQ5p7maNS/tj7sUEZHInMmhoRGZ2Z3hkBTrT7LezOwLZrbFzNaa2eKoajkTba31PPXifgbyRX8kTEQSKrIgAO4Grhxh/VXAwvCxDPhyhLWctiVz6unpzbF5p24sE5HiFFkQuPsjwN4RmlwLfN0DvwTqwjkPJpS2OVMAWPXivpgrERGJRpQ9glNpAV4qeN8eLptQZtaX01RdxqoXRso0EZHJK84gGO4+hGEPxJvZMjNbaWYrd+/eHXFZJ3w2bXPqWbldPQIRKU5xBkE7MKvg/UygY7iG7n6Hu7e5e1tT09kf9HTJnHra9x2hs/voWf9sEZGoxRkE9wHvDq8eeh1wwN13xFjPSS2ZUw/AKvUKRKQIjWqqytNhZt8CrgAazawd+DsgA+DutwEPAFcDW4DDTODRTC9oqaWiNM0vtnZx9Wsm3PlsEZEzElkQuPuNp1jvwAei+vzxlEmnuGTuFB7bsifuUkRExl2ch4YmlcsWNLJtzyFe3q8B6ESkuCgIRukNCxsBeOw59QpEpLgoCEbpVc3VNFWX8fBzZ/fyVRGRqCkIRsnMeOOrmnhk8276cvm4yxERGTcKgjF4y3nTONib44nndZexiBQPBcEYvGFBI2UlKX6yqTPuUkRExo2CYAzKS9P8ysJGlm/s1DzGIlI0FARj9Jbzmnl5/xE27dCw1CJSHBQEY/Smc5sxgwc37Iy7FBGRcaEgGKOm6jJeN7eB+5/u0OEhESkKCoLTcO2iGWzbc4j1L2tSexGZ/BQEp+GqC6aTSRv/vebluEsRETljCoLTUFuR4YpXTeX+tR2a1F5EJj0FwWm6dtEMOrt7+eW2rrhLERE5IwqC0/TmVzdTW57h20++dOrGIiITmILgNGUzaX5rcQsPrt9JV09v3OWIiJw2BcEZuHHpbPoG8nxvVXvcpYiInDYFwRk4p7ma189v4K7Hnqc3NxB3OSIip0VBcIbef8UCOrt7+f5qXUoqIpOTguAMXbaggQtn1nL7w1t1KamITEoKgjNkZrz/ivm80HWYB9btiLscEZExUxCMg7eeN435TZV86aGtGn9IRCYdBcE4SKWMP/rV+Wza0c1DmzWnsYhMLgqCcXLtohZm1Gb50kNb4i5FRGRMFATjpLQkxf+5fB5PvrCPx7bsibscEZFRUxCMoxuXzqalrpz/9z+bdAWRiEwaCoJxlM2k+aurzmXTjm7uXa27jUVkclAQjLPfuHA6i2bV8W8PbuZwXy7uckRETklBMM7MjL+95tXsOtjLHY9si7scEZFTUhBEYMmcKfz6a6Zz+8Pb6Nh/JO5yRERGpCCIyEevOheAv/nBOt1kJiITmoIgIrOmVPAXv/YqVmzerQHpRGRCUxBE6L2vb6VtTj3/cP8GdnUfjbscEZFhRRoEZnalmW02sy1m9tFh1r/XzHab2Zrw8QdR1nO2pVLGv77jQnpzef7inrXkdW+BiExAkQWBmaWBW4GrgPOAG83svGGafsfdF4WPr0ZVT1zmNVXxt9ecx8PP7uZzP3k27nJERE4QZY9gKbDF3be5ex/wbeDaCD9vwrrpktlcv2QmX/jZFpZv7Iy7HBGR40QZBC3ASwXv28NlQ11nZmvN7B4zmxVhPbExM/7xNy/gNS21/Nl31rBtd0/cJYmIHBNlENgwy4YeJL8faHX3C4GfAP8+7IbMlpnZSjNbuXv35BzmOZtJc9vvLiFTkuIPv7GKnl7ddSwiE0OUQdAOFP6FPxPoKGzg7l3u3hu+/QqwZLgNufsd7t7m7m1NTU2RFHs2tNSV88UbL2br7h7+8p6ndX+BiEwIUQbBk8BCM5trZqXADcB9hQ3MbHrB27cBmyKsZ0K4bEEjf3XluTywbie3/ExzF4hI/Eqi2rC758zsT4AHgTRwp7tvMLNPACvd/T7gQ2b2NiAH7AXeG1U9E8myy+fxzM6DfHr5s0ypKuWmS+bEXZKIJJhNtsMTbW1tvnLlyrjLOGP9A3n+8BurWLF5F1+88WKuuXBG3CWJSBEzs1Xu3jbcOt1ZHJNMOsWtv72Ytjn13PztNfxwbcepv0hEJAIKghiVl6a5872vZfHsej70raf47sqXTv1FIiLjTEEQs+pshn9/31IuW9DIX96zlrsfez7ukkQkYRQEE0B5aZqvvqeNXzu/mb+/fyO3rtDVRCJy9igIJoiykjS3/vZi3n5xC596cDP/97/W0T+Qj7ssEUmAyC4flbErSaf49PUXMbWmjNsf3sbWXYf40k2Lqa8sjbs0ESli6hFMMKmU8bGrXs2nr7+IVdv38Ru3/JynXtwXd1kiUsQUBBPUdUtm8t0/uhR3uP62X/Clh7YwoPkMRCQCCoIJbNGsOh740K/w1vOb+df/3cw7b/8F27sOxV2WiBQZBcEEV1uR4dbfXsxn33URz3Ye5KrPP8pdjz1PTieSRWScKAgmATPj7RfP5MGbL2fJnHr+4f6NXPPFn/PE83vjLk1EioCCYBKZUVfO19+3lC/ftJjuI/288/ZfcPO3n2JX99G4SxORSUxBMMmYGVe9Zjo//cgVfPBNC3hg3U7e+G8P8ZVHtum+AxE5LQqCSaq8NM1H3voqfvynl7N07hT+6YFNXPX5R/mftTt0dZGIjImCYJJrbazkrt9bytfe00Y+73zgP1fzls8+zPdWvqQegoiMiuYjKCIDeedH63dw64qtbNrRTUtdOcsun8f1bTOpKNVN5CJJNtJ8BAqCIuTuPLR5N7es2MKq7fuoq8hw0yWzefelrTTXZOMuT0RioCBIKHdn5fZ9fPXRbfx4YydpM9547lSuWzyTN507ldISHRkUSYqRgkDHC4qYmfHa1im8tnUK27sO8R+/3M4Pnupg+cZO6isyXLuohesWz+SClhrMLO5yRSQm6hEkTG4gz6PP7eGe1e0s39hJXy7POc1VXLd4JtdcNIOWuvK4SxSRCOjQkAzrwOF+friug3tXtbP6xf0AnD+jhje/upm3nNfM+TPUUxApFgoCOaXn9xzixxt2snxjJ6te3Ic7zKjNcvk5Tbx+QSOvn99AY1VZ3GWKyGlSEMiY7Onp5WfP7OInGzv5xbYuDh7NAXDutGouW9DIZQsaWDq3gaoynWISmSwUBHLacgN51nd089iWPTy2ZQ8rt++jL5enJGWc31LLxbPquHh2HRfPqmfWlHIdShKZoBQEMm6O9g+wavu+Y6Gwrv0AR/oHAGioLGXRrDoumlXH+TNqOH9GLc01ZQoHkQlAl4/KuMlm0uHhoUYg6DFs7jzImpf289SL+1nz0n5+tnkXg39fNFaVct6MWs6dVs05zdWcO62aBVOryGbSMe6FiBRSj0DGXU9vjmd2dLP+5QNs6OhmQ0c3W3b30JcLxj5KGbQ2VDKvqYr5TZXMa6pkflMVcxoqaawqVQ9CJALqEchZVVVWQlvrFNpapxxblhvI80LXYTbvPMjmnd08t6uHbbsP8cizu+krGByvsjTNnIZKWhsrgueG4HlOQwVNVWWUpHU3tMh4UxDIWVGSTrFgahULplbx6xdOP7Z8IO+8vO8IW/f08GLXYV7oOsQLew7xzI6DLN/YSf/AKz3WlEFTdRnTarI012SZVhs+F76uzepqJpEx0v8YiVU6ZcxuqGB2Q8UJ63IDeXYcOMr2rsNs33uIzgNH2dl9lJ3dvWzvOszjz+/lwJH+E76uqqyE5poyptVmaagso74iQ11FKfUVGeorS195XVFKXUWGqrISHY6SRFMQyIRVkk4xa0oFs6ZU8AYah21zpG+Azu4gIDq7j7LzwPGvn963n32H+ugO74UY9nNSRl1BWBwfFMcvG3yuzmbIZlIKECkKCgKZ1MpL07Q2VtLaWDliu9xAngNH+tl3uJ/9h/vYf7iffQXPg8v3He7jpb2HWdseLBs8wT0cM6gsLaGyLE1laQkVZWkqSkuoLE1TURY+h+uPXx60rSwtoaI0TXlpmmwmTbYkFTxn0qRTChg5eyINAjO7Evg8kAa+6u6fHLK+DPg6sAToAt7l7i9EWZMkU0k6RUNVGQ1jGCbD3TnSP8C+w/3sO1QYHn309A5wuC9HT2+Ow70DHOrLcbhvgEO9Ofb09HFo7+Fjyw/15hjr7KGZtJEtSVOWSVNWkiKbeSUkspkU2ZLgdVkmRVlJipJUikw6RabEyJzw2siUhMvSFj6P9HrkduoFFZ/IgsDM0sCtwFuAduBJM7vP3TcWNPt9YJ+7LzCzG4B/Ad4VVU0iY2FmVJSWUFFackajsro7vbn8saA43BcGR+8APb05enMDHO0f4Gh//pXngmW9/QPh+2D9kb4B9h3q52hugN7+PP0DwSM34PQN5OkbyBPlVeElKTsxIIYGUDoVvC8xSlIpSlJGKmXHntNmJywrSRkpM9InWTb4GG5ZenCb6SHrzEinw+ch7VNmpIzw2UilOHGZGWaQSgXL0xYEYWEbSwXLj7UN16dTkyc0o+wRLAW2uPs2ADP7NnAtUBgE1wJ/H76+B7jFzMwn280NIiMws2N/zU+pLD0rnzmQ92MB0T/g5MKAGPq6sE1/Lk8un6dvmNf9A3lyeadvuNfhdvqGBFL/QJ7e/jw9AzkG3MkNOHl3cnknn3/lecCdgXzwGG7ZWHtTE81xoREGxMlC41Rtb3jtLP7gV+aNe41RBkEL8FLB+3bgkpO1cfecmR0AGoA9EdYlUvSCv3rTRXEHtw+GQkE4HPcYbvnJ2obrcnkHJwyaIGw8fB5wD187+Tzk3fFw+XFtw5DKFyw/1vaE7Ybr86ffNu8e2QjAUQbBcH2iodk+mjaY2TJgGcDs2bPPvDIRmTQsPOSjK1uiE+Vtmu3ArIL3M4GOk7UxsxKgFtg7dEPufoe7t7l7W1NTU0TliogkU5RB8CSw0MzmmlkpcANw35A29wHvCV+/A/iZzg+IiJxdkfW2wmP+fwI8SHD56J3uvsHMPgGsdPf7gK8B3zCzLQQ9gRuiqkdERIYX6WE3d38AeGDIso8XvD4KXB9lDSIiMjIN5SgiknAKAhGRhFMQiIgknIJARCThJt1UlWa2G9h+ml/eSPLuWtY+J4P2ORnOZJ/nuPuwN2JNuiA4E2a28mRzdhYr7XMyaJ+TIap91qEhEZGEUxCIiCRc0oLgjrgLiIH2ORm0z8kQyT4n6hyBiIicKGk9AhERGSIxQWBmV5rZZjPbYmYfjbue8WJmd5rZLjNbX7BsipktN7Pnwuf6cLmZ2RfC78FaM1scX+Wnz8xmmdkKM9tkZhvM7MPh8qLdbzPLmtkTZvZ0uM//EC6fa2aPh/v8nXCkX8ysLHy/JVzfGmf9p8vM0mb2lJn9MHxf1PsLYGYvmNk6M1tjZivDZZH+bCciCArmT74KOA+40czOi7eqcXM3cOWQZR8FfuruC4Gfhu8h2P+F4WMZ8OWzVON4ywEfcfdXA68DPhD+exbzfvcCb3L3i4BFwJVm9jqCeb4/G+7zPoJ5wKFgPnDgs2G7yejDwKaC98W+v4Pe6O6LCi4VjfZn28Np2Yr5AVwKPFjw/mPAx+Kuaxz3rxVYX/B+MzA9fD0d2By+vh24cbh2k/kB/DfwlqTsN1ABrCaY+nUPUBIuP/ZzTjD8+6Xh65KwncVd+xj3c2b4S+9NwA8JZjQs2v0t2O8XgMYhyyL92U5Ej4Dh509uiamWs6HZ3XcAhM9Tw+VF930IDwFcDDxOke93eJhkDbALWA5sBfa7ey5sUrhfx80HDgzOBz6ZfA74SyAfvm+guPd3kAM/NrNV4TS9EPHPdlKmAR3V3MgJUFTfBzOrAu4Fbnb3brPhdi9oOsyySbff7j4ALDKzOuAHwKuHaxY+T+p9NrNrgF3uvsrMrhhcPEzTotjfIS5z9w4zmwosN7NnRmg7LvudlB7BaOZPLiadZjYdIHzeFS4vmu+DmWUIQuCb7v79cHHR7zeAu+8HHiI4P1IXzvcNx+/XqOYDn8AuA95mZi8A3yY4PPQ5ind/j3H3jvB5F0HgLyXin+2kBMFo5k8uJoVzQb+H4Bj64PJ3h1cavA44MNjdnEws+NP/a8Amd/9Mwaqi3W8zawp7AphZOfBmgpOoKwjm+4YT93nSzgfu7h9z95nu3krw//Vn7n4TRbq/g8ys0syqB18DbwXWE/XPdtwnRs7iCZirgWcJjqv+Tdz1jON+fQvYAfQT/HXw+wTHRn8KPBc+TwnbGsHVU1uBdUBb3PWf5j6/gaD7uxZYEz6uLub9Bi4Engr3eT3w8XD5POAJYAvwPaAsXJ4N328J18+Lex/OYN+vAH6YhP0N9+/p8LFh8HdV1D/burNYRCThknJoSERETkJBICKScAoCEZGEUxCIiCScgkBEJOEUBBI7M3Mz+3TB+z83s78fp23fbWbvOHXLM/6c68PRUFcMWT7DzO4JXy8ys6vH8TPrzOz9w32WyFgoCGQi6AV+y8wa4y6kUDhq7Wj9PvB+d39j4UJ373D3wSBaRHC/w1hqGGkYmDrgWBAM+SyRUVMQyESQI5iC70+Hrhj6F72Z9YTPV5jZw2b2XTN71sw+aWY3WTBm/zozm1+wmTeb2aNhu2vCr0+b2afM7MlwHPc/LNjuCjP7T4IbdIbWc2O4/fVm9i/hso8T3OR2m5l9akj71rBtKfAJ4F3hOPPvCu8ivTOs4Skzuzb8mvea2ffM7H6CwceqzOynZrY6/Oxrw81/Epgfbu9Tg58VbiNrZneF7Z8yszcWbPv7Zva/Foxt/68F34+7w1rXmdkJ/xZSvJIy6JxMfLcCawd/MY3SRQQDr+0FtgFfdfelFkxU80Hg5rBdK/CrwHxghZktAN5NcDv+a82sDHjMzH4ctl8KXODuzxd+mJnNIBjnfgnBWPg/NrPfdPdPmNmbgD9395XDFerufWFgtLn7n4Tb+2eCoRDeFw4f8YSZ/ST8kkuBC919b9greLsHA+s1Ar80s/sIxqS/wN0XhdtrLfjID4Sf+xozOzes9Zxw3SKCEVt7gc1m9kWC0Sxb3P2CcFt1I3/rpZioRyATgrt3A18HPjSGL3vS3Xe4ey/BLfaDv8jXEfzyH/Rdd8+7+3MEgXEuwRgu77ZgWOfHCW7hXxi2f1W7+3MAAAIBSURBVGJoCIReCzzk7rs9GOr4m8DlY6h3qLcCHw1reIhgmITZ4brl7j44aJoB/2xma4GfEAwz3HyKbb8B+AaAuz8DbAcGg+Cn7n7A3Y8CG4E5BN+XeWb2RTO7Eug+g/2SSUY9AplIPkcw4cpdBctyhH+wmJkBpQXregte5wve5zn+Z3voOCpO8Mv1g+7+YOEKC4Y8PnSS+k46zvVpMuA6d988pIZLhtRwE9AELHH3fgtG5MyOYtsnU/h9GyCY6GWfmV0E/BpBb+KdwPtGtRcy6alHIBNG+Bfwd3ll+kEIZmtaEr6+FsicxqavN7NUeN5gHsEsTg8Cf2zBcNaY2TnhaI8jeRz4VTNrDE8k3wg8PIY6DgLVBe8fBD4YBhxmdvFJvq6WYGz+/vBY/5yTbK/QIwQBQnhIaDbBfg8rPOSUcvd7gb8FJt28znL6FAQy0XwaKLx66CsEv3yfIJia8WR/rY9kM8Ev7B8BfxQeEvkqwWGR1eEJ1ts5RQ/Zg+F9P0YwFPLTwGp3/++RvmaIFcB5gyeLgX8kCLa1YQ3/eJKv+ybQZsFE5jcBz4T1dBGc21g/9CQ18CUgbWbrgO8A7w0PoZ1MC/BQeJjq7nA/JSE0+qiISMKpRyAiknAKAhGRhFMQiIgknIJARCThFAQiIgmnIBARSTgFgYhIwikIREQS7v8D8IEIzwnnmpIAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "#get training and test error for the best NN model from CV\n", "mlp = MLPClassifier(hidden_layer_sizes=(40,), max_iter=500, alpha=1e-4, batch_size=480,\n", " solver='sgd', verbose=False, tol=1e-4, random_state=ID,\n", " learning_rate_init=.01, n_iter_no_change=5000)\n", "mlp.fit(X_train, y_train)\n", "training_error = 1. - mlp.score(X_train,y_train)\n", "test_error = 1. - mlp.score(X_test,y_test)\n", "\n", "print ('\\nRESULTS FOR BEST NN\\n')\n", "\n", "print (\"Best NN training error: %f\" % training_error)\n", "print (\"Best NN test error: %f\" % test_error)\n", "\n", "plt.plot(mlp.loss_curve_)\n", "plt.legend(loc = 1)\n", "plt.xlabel('Number of iterations')\n", "plt.ylabel('Loss')\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## More data \n", "Now let's do the same but using 4000 (or less if it takes too long on your machine) data points for training. Use the same NN architecture as before, but you can try more if you like and have a powerful computer!!" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Labels in training dataset: [0 1 2 3 4 5 6 7 8 9]\n", "Frequencies in training dataset: [373 407 391 428 412 424 403 351 390 421]\n" ] } ], "source": [ "X = X[permutation]\n", "y = y[permutation]\n", "\n", "m_training = 4000\n", "\n", "X_train, X_test = X[:m_training], X[m_training:]\n", "y_train, y_test = y[:m_training], y[m_training:]\n", "\n", "labels, freqs = np.unique(y_train, return_counts=True)\n", "print(\"Labels in training dataset: \", labels)\n", "print(\"Frequencies in training dataset: \", freqs)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## TO DO 6\n", "\n", "Now train the NNs with the added data points using the optimum parameters found above. Eventually, feel free to try different architectures if you like. We suggest that you use 'verbose=True' so have an idea of how long it takes to run 1 iteration (eventually reduce also the number of iterations to 50)." ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Iteration 1, loss = 2.14266735\n", "Iteration 2, loss = 1.51350587\n", "Iteration 3, loss = 1.12918023\n", "Iteration 4, loss = 0.94114462\n", "Iteration 5, loss = 0.82673938\n", "Iteration 6, loss = 0.74933779\n", "Iteration 7, loss = 0.69200063\n", "Iteration 8, loss = 0.64779316\n", "Iteration 9, loss = 0.61184241\n", "Iteration 10, loss = 0.58057975\n", "Iteration 11, loss = 0.55405990\n", "Iteration 12, loss = 0.53110954\n", "Iteration 13, loss = 0.50966604\n", "Iteration 14, loss = 0.49366454\n", "Iteration 15, loss = 0.47414323\n", "Iteration 16, loss = 0.45776128\n", "Iteration 17, loss = 0.44294163\n", "Iteration 18, loss = 0.42953290\n", "Iteration 19, loss = 0.41659606\n", "Iteration 20, loss = 0.40319847\n", "Iteration 21, loss = 0.39299637\n", "Iteration 22, loss = 0.38161172\n", "Iteration 23, loss = 0.37138473\n", "Iteration 24, loss = 0.36152693\n", "Iteration 25, loss = 0.35311397\n", "Iteration 26, loss = 0.34231147\n", "Iteration 27, loss = 0.33439405\n", "Iteration 28, loss = 0.32638349\n", "Iteration 29, loss = 0.31854382\n", "Iteration 30, loss = 0.31099924\n", "Iteration 31, loss = 0.30155978\n", "Iteration 32, loss = 0.29511490\n", "Iteration 33, loss = 0.28922119\n", "Iteration 34, loss = 0.28177023\n", "Iteration 35, loss = 0.27600635\n", "Iteration 36, loss = 0.26982855\n", "Iteration 37, loss = 0.26422327\n", "Iteration 38, loss = 0.25768772\n", "Iteration 39, loss = 0.25242771\n", "Iteration 40, loss = 0.24619470\n", "Iteration 41, loss = 0.24085778\n", "Iteration 42, loss = 0.23599865\n", "Iteration 43, loss = 0.23048444\n", "Iteration 44, loss = 0.22594700\n", "Iteration 45, loss = 0.22137536\n", "Iteration 46, loss = 0.21642141\n", "Iteration 47, loss = 0.21185816\n", "Iteration 48, loss = 0.20663508\n", "Iteration 49, loss = 0.20274334\n", "Iteration 50, loss = 0.19867472\n", "Iteration 51, loss = 0.19329944\n", "Iteration 52, loss = 0.19030252\n", "Iteration 53, loss = 0.18585535\n", "Iteration 54, loss = 0.18272922\n", "Iteration 55, loss = 0.17856397\n", "Iteration 56, loss = 0.17530242\n", "Iteration 57, loss = 0.17120118\n", "Iteration 58, loss = 0.16775246\n", "Iteration 59, loss = 0.16401747\n", "Iteration 60, loss = 0.16081124\n", "Iteration 61, loss = 0.15709794\n", "Iteration 62, loss = 0.15474646\n", "Iteration 63, loss = 0.15127930\n", "Iteration 64, loss = 0.14783126\n", "Iteration 65, loss = 0.14579118\n", "Iteration 66, loss = 0.14252026\n", "Iteration 67, loss = 0.13929694\n", "Iteration 68, loss = 0.13665077\n", "Iteration 69, loss = 0.13412805\n", "Iteration 70, loss = 0.13144808\n", "Iteration 71, loss = 0.12951204\n", "Iteration 72, loss = 0.12609818\n", "Iteration 73, loss = 0.12395740\n", "Iteration 74, loss = 0.12155964\n", "Iteration 75, loss = 0.11867529\n", "Iteration 76, loss = 0.11659668\n", "Iteration 77, loss = 0.11438473\n", "Iteration 78, loss = 0.11236484\n", "Iteration 79, loss = 0.11030000\n", "Iteration 80, loss = 0.10783509\n", "Iteration 81, loss = 0.10565650\n", "Iteration 82, loss = 0.10370305\n", "Iteration 83, loss = 0.10185361\n", "Iteration 84, loss = 0.10025789\n", "Iteration 85, loss = 0.09849387\n", "Iteration 86, loss = 0.09623808\n", "Iteration 87, loss = 0.09491886\n", "Iteration 88, loss = 0.09256020\n", "Iteration 89, loss = 0.09121802\n", "Iteration 90, loss = 0.08942062\n", "Iteration 91, loss = 0.08747106\n", "Iteration 92, loss = 0.08583215\n", "Iteration 93, loss = 0.08421011\n", "Iteration 94, loss = 0.08274373\n", "Iteration 95, loss = 0.08115535\n", "Iteration 96, loss = 0.07951320\n", "Iteration 97, loss = 0.07814988\n", "Iteration 98, loss = 0.07705146\n", "Iteration 99, loss = 0.07577825\n", "Iteration 100, loss = 0.07419023\n", "\n", "RESULTS FOR NN\n", "\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "C:\\Users\\Anay Deshpande\\Anaconda3\\lib\\site-packages\\sklearn\\neural_network\\multilayer_perceptron.py:566: ConvergenceWarning: Stochastic Optimizer: Maximum iterations (100) reached and the optimization hasn't converged yet.\n", " % self.max_iter, ConvergenceWarning)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "NN training error: 0.007250\n", "NN test error: 0.154625\n" ] } ], "source": [ "#for NN we try the same architectures as before\n", "parameters = {'hidden_layer_sizes': [ (40,) ]}\n", "\n", "mlp_large = MLPClassifier(max_iter=100, alpha=1e-4, hidden_layer_sizes = (40,),\n", " solver='sgd', tol=1e-4, random_state=ID,\n", " learning_rate_init=.01, verbose=True)\n", "mlp_large.fit(X_train, y_train)\n", "\n", "print ('\\nRESULTS FOR NN\\n')\n", "\n", "#get training and test error for the NN\n", "training_error = 1. - mlp_large.score(X_train,y_train)\n", "test_error = 1. - mlp_large.score(X_test,y_test)\n", "print (\"NN training error: %f\" % training_error)\n", "print (\"NN test error: %f\" % test_error)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## QUESTION 4\n", "Compare the train and test error you got with a large number of samples with the best one you obtained with only 600 data points. Comment about the results you obtained." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### [ANSWER TO QUESTION 4]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### TO DO 7\n", "\n", "Plot an example that was missclassified by NN with m=600 training data points and it is now instead correctly classified by NN with m=4000 training data points." ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "INPUT:\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAPf0lEQVR4nO3db4xUZZbH8d9ZVFQQAbtlse3YKmgWN1k0FbIGYzDjEiUmOhoNmIwajT2JkowGkzVujLwwEdedIWo2Y3BBUWYlY2ZUTHRWJSY4bwgFIuICKtAq0kIjEhgVBTz7oq+bFrueaure+tOc7yfpVPU9deueFP3j3qrn3nrM3QXg+Pd3zW4AQGMQdiAIwg4EQdiBIAg7EMQJjdxYW1ubd3V1NXKT4R08eDBZ37ZtW7J+3nnnJesnn3zyMfeE+unp6dGePXtssFqusJvZVZIelzRC0n+5+4LU47u6ulQul/NsEsdo06ZNyfrNN9+crC9fvjxZv/DCC4+5J9RPqVSqWKv5MN7MRkj6T0lXS5oiaY6ZTan1+QDUV5737NMkfezu29z9e0nLJV1bTFsAipYn7B2SPhvw+45s2U+YWbeZlc2s3NfXl2NzAPLIE/bBPgT42bm37r7I3UvuXmpvb8+xOQB55An7DkmdA34/W9LOfO0AqJc8YV8jabKZnWtmJ0maLWlFMW0BKFrNQ2/uftjM5kr6H/UPvS1x9w8K6wxDlrpy8a677kquO3PmzGSdobXjR65xdnd/TdJrBfUCoI44XRYIgrADQRB2IAjCDgRB2IEgCDsQREOvZ0d97N+/v2Jt7dq1yXWfe+65ottBi2LPDgRB2IEgCDsQBGEHgiDsQBCEHQiCobfjQE9PT8XalVdemVy3s7MzWW+mQ4cOJevVJiU96aSTimxn2GPPDgRB2IEgCDsQBGEHgiDsQBCEHQiCsANBMM5+HHjmmWcq1qZPn97ATo7Nu+++m6xv3749WX/00UeT9WXLllWsTZ48Obnu8Yg9OxAEYQeCIOxAEIQdCIKwA0EQdiAIwg4EwTj7MHD48OFkfcWKFRVrixcvLrqdY7JmzZqKtdWrVyfXnTt3brK+c+fOZH3WrFkVa6lzEyTpsssuS9aHo1xhN7MeSQckHZF02N1LRTQFoHhF7NmvcPc9BTwPgDriPTsQRN6wu6Q3zGytmXUP9gAz6zazspmV+/r6cm4OQK3yhn26u18i6WpJd5vZ5Uc/wN0XuXvJ3Uvt7e05NwegVrnC7u47s9vdkl6SNK2IpgAUr+awm9koMzvtx/uSZkraWFRjAIqV59P4CZJeMrMfn+e/3f0vhXSFn9i8eXOy/vnnn1esdXR0FN3OT6xbty5Zf/311yvWHnzwwVzbnjRpUrK+devWirW33347uS7j7AO4+zZJ/1RgLwDqiKE3IAjCDgRB2IEgCDsQBGEHguAS12Fg1apVyXpqauKzzz4717bfe++9ZP2+++5L1l9++eWKtWzYtmbVzshMTel86qmn5tr2cMSeHQiCsANBEHYgCMIOBEHYgSAIOxAEYQeCYJx9GHjrrbeS9TPOOKNirdp48oYNG5L122+/PVl/+umnk/UxY8Yk63ns27ev5nUnTJhQYCfDA3t2IAjCDgRB2IEgCDsQBGEHgiDsQBCEHQiCcfYWUG1K5nfeeSdZv/jiiyvWUl+nLEm33HJLsr5kyZJk/ZJLLknW62nPntrnEz399NML7GR4YM8OBEHYgSAIOxAEYQeCIOxAEIQdCIKwA0Ewzt4C1q5dm6xXG0/u6+urWJszZ05y3XvuuSdZb+Y4ejV79+6ted2zzjqrwE6Gh6p7djNbYma7zWzjgGXjzexNM/soux1X3zYB5DWUw/hnJV111LL7Ja1098mSVma/A2hhVcPu7qskHX28dK2kpdn9pZKuK7gvAAWr9QO6Ce7eK0nZ7ZmVHmhm3WZWNrNy6r0lgPqq+6fx7r7I3UvuXqo2ER+A+qk17LvMbKIkZbe7i2sJQD3UGvYVkm7N7t8q6ZVi2gFQL1XH2c3sBUkzJLWZ2Q5JD0laIOmPZnaHpE8l3VjPJo93y5Yty7X++vXrK9YeeeSR5Lq33XZbrm03U57r2ceOHVtgJ8ND1bC7e6WzMn5RcC8A6ojTZYEgCDsQBGEHgiDsQBCEHQiCS1wb4JtvvknWly9fnuv5U5eh3nvvvbmeu5Xt378/WR8/fnzFWkdHR9HttDz27EAQhB0IgrADQRB2IAjCDgRB2IEgCDsQBOPsDVBtyuU8l2pK6ctYR44cmeu5W9lnn32WrKcuYz3xxBOLbqflsWcHgiDsQBCEHQiCsANBEHYgCMIOBEHYgSAYZ2+AZ599Ntf655xzTrJ+xRVX5Hr+4eqrr75K1idOnFixNmLEiKLbaXns2YEgCDsQBGEHgiDsQBCEHQiCsANBEHYgCMbZC/DJJ58k66+++mqu5x83blyyHnHMWJIOHDiQrKe+Nz6iqnt2M1tiZrvNbOOAZfPN7HMzW5/9zKpvmwDyGsph/LOSrhpk+UJ3n5r9vFZsWwCKVjXs7r5K0t4G9AKgjvJ8QDfXzDZkh/kV31SaWbeZlc2s3NfXl2NzAPKoNey/l3S+pKmSeiX9ttID3X2Ru5fcvdTe3l7j5gDkVVPY3X2Xux9x9x8kPS1pWrFtAShaTWE3s4HXDv5S0sZKjwXQGqqOs5vZC5JmSGozsx2SHpI0w8ymSnJJPZJ+XcceW4K7V6wtXLgwue7XX3+da9snnJD+ZzKzXM8/XH355ZfJ+mmnndagToaHqmF39zmDLF5ch14A1BGnywJBEHYgCMIOBEHYgSAIOxAEl7gO0RtvvFGx9sQTT9R1252dncl61KG30aNHJ+unnHJKgzoZHtizA0EQdiAIwg4EQdiBIAg7EARhB4Ig7EAQjLNnNmzYkKzPnj27Yi11+asknXvuucn69u3bk/Wurq5kParvv/8+Wd+yZUvFWrV/s+Px3AX27EAQhB0IgrADQRB2IAjCDgRB2IEgCDsQBOPsmXnz5iXr+/btq1i78cYbk+tedNFFyfr8+fOT9UmTJiXrUV166aXJ+vPPP1+xtnPnzuS6HR0dNfXUytizA0EQdiAIwg4EQdiBIAg7EARhB4Ig7EAQYcbZq12/XO169vHjx1esPfzww8l1n3rqqWS9mmrj9FE99NBDyfqLL75Ysfbkk08m112wYEFNPbWyqnt2M+s0s7fNbJOZfWBmv8mWjzezN83so+x2XP3bBVCroRzGH5Y0z93/QdI/S7rbzKZIul/SSnefLGll9juAFlU17O7e6+7rsvsHJG2S1CHpWklLs4ctlXRdvZoEkN8xfUBnZl2SLpa0WtIEd++V+v9DkHRmhXW6zaxsZuW+vr583QKo2ZDDbmajJf1J0j3uvn+o67n7IncvuXupvb29lh4BFGBIYTezE9Uf9D+4+5+zxbvMbGJWnyhpd31aBFCEqkNv1v+duoslbXL33w0orZB0q6QF2e0rdemwIAcPHkzW9+9PH6w89thjFWsXXHBBct3Nmzcn69W+tvj8889P1qOqdhlq6uu/H3/88eS6119/fbI+bdq0ZL0VDWWcfbqkX0l638zWZ8seUH/I/2hmd0j6VFL6om4ATVU17O7+V0mVdj2/KLYdAPXC6bJAEIQdCIKwA0EQdiAIwg4EEeYS1127diXrN910U7Le3d1d87a/+OKLZL3aOHtbW1vN244sdelxtXMfbrjhhmR98eLFyfrMmTOT9WZgzw4EQdiBIAg7EARhB4Ig7EAQhB0IgrADQVi1r1guUqlU8nK53LDtNcqhQ4eS9TFjxiTrY8eOTdZ7e3uPuadG+fTTT5P1CRMmVKyNHDmy6HaG7MiRI8n61q1bk/Vq19KPGjXqmHsqQqlUUrlcHvTEDfbsQBCEHQiCsANBEHYgCMIOBEHYgSAIOxBEmOvZ62nLli3J+nfffZesX3755UW201ALFy5M1u+8886KtSlTphTdzpCNGDEiWa82F8BwxJ4dCIKwA0EQdiAIwg4EQdiBIAg7EARhB4IYyvzsnZKek/T3kn6QtMjdHzez+ZLulNSXPfQBd3+tXo22sg8//DBZr/adAdWujW5lq1evTtbnzp3boE5QzVBOqjksaZ67rzOz0yStNbM3s9pCd/+P+rUHoChDmZ+9V1Jvdv+AmW2SNHx3RUBQx/Se3cy6JF0s6cdjt7lmtsHMlpjZuArrdJtZ2czKfX19gz0EQAMMOexmNlrSnyTd4+77Jf1e0vmSpqp/z//bwdZz90XuXnL3Unt7ewEtA6jFkMJuZieqP+h/cPc/S5K773L3I+7+g6SnJU2rX5sA8qoaduufYnSxpE3u/rsByycOeNgvJW0svj0ARRnKp/HTJf1K0vtmtj5b9oCkOWY2VZJL6pH067p0OAxUu8S12pTM11xzTZHtNNS3336brI8ePbpBnaCaoXwa/1dJg/21hhxTB4YrzqADgiDsQBCEHQiCsANBEHYgCMIOBMFXSRegra0tWb///vuT9RkzZhTYDTA49uxAEIQdCIKwA0EQdiAIwg4EQdiBIAg7EIRV+5rjQjdm1ifpkwGL2iTtaVgDx6ZVe2vVviR6q1WRvZ3j7oN+/1tDw/6zjZuV3b3UtAYSWrW3Vu1LordaNao3DuOBIAg7EESzw76oydtPadXeWrUvid5q1ZDemvqeHUDjNHvPDqBBCDsQRFPCbmZXmdkWM/vYzNIXezeYmfWY2ftmtt7Myk3uZYmZ7TazjQOWjTezN83so+x20Dn2mtTbfDP7PHvt1pvZrCb11mlmb5vZJjP7wMx+ky1v6muX6Kshr1vD37Ob2QhJH0r6F0k7JK2RNMfd/7ehjVRgZj2SSu7e9BMwzOxySX+T9Jy7/2O27N8l7XX3Bdl/lOPc/V9bpLf5kv7W7Gm8s9mKJg6cZlzSdZJuUxNfu0RfN6kBr1sz9uzTJH3s7tvc/XtJyyVd24Q+Wp67r5K096jF10pamt1fqv4/loar0FtLcPded1+X3T8g6cdpxpv62iX6aohmhL1D0mcDft+h1prv3SW9YWZrzay72c0MYoK790r9fzySzmxyP0erOo13Ix01zXjLvHa1TH+eVzPCPthUUq00/jfd3S+RdLWku7PDVQzNkKbxbpRBphlvCbVOf55XM8K+Q1LngN/PlrSzCX0Myt13Zre7Jb2k1puKetePM+hmt7ub3M//a6VpvAebZlwt8No1c/rzZoR9jaTJZnaumZ0kabakFU3o42fMbFT2wYnMbJSkmWq9qahXSLo1u3+rpFea2MtPtMo03pWmGVeTX7umT3/u7g3/kTRL/Z/Ib5X0b83ooUJf50l6L/v5oNm9SXpB/Yd1h9R/RHSHpDMkrZT0UXY7voV6e17S+5I2qD9YE5vU22Xqf2u4QdL67GdWs1+7RF8Ned04XRYIgjPogCAIOxAEYQeCIOxAEIQdCIKwA0EQdiCI/wMeRI73yFe44wAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "LABEL: 8\n", "NN prediction for m=600: 1\n", "NN prediction for m=4000: 8\n" ] } ], "source": [ "NN_prediction = mlp.predict(X_test)\n", "large_NN_prediction = mlp_large.predict(X_test)\n", "i = 0\n", "found = False\n", "while ((not found) and (i" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "Weights with 4000 data points:\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUcAAADuCAYAAACqLcX5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOy9d3xU1do9viYJ6Z0UQiBEAoROkE4QCE26iFIUEQRs94oFC4pyLYjoFQW8ioAFpKqIoii9SY8UIbRACJ0QQnpPJsn8/pjfWhMgl/vJ+9Hreb/vWf+gyWRmzj777L2e9azn2RabzQYTJkyYMHEjnP7qL2DChAkTRoS5OJowYcJEFTAXRxMmTJioAubiaMKECRNVwFwcTZgwYaIKmIujCRMmTFQBc3E0YcKEiSpgLo4mTJgwUQXMxdGECRMmqoBLdV7s7u5u8/LyQnl5Odzd3QEAnp6eAICMjAy9LiAgAACQn58PAPDx8YHFYgEAlJeXAwB8fX0BACUlJaioqOD7AwAqKipQXFwMALh+/ToAIDAwEADg5OSE0tJSAICrqysAoKCgQP/Nf/l3ISEhKC0tRWZmJgoKCizVud7/Jnx8fGzBwcEoKipCSUkJAMBqtQJwjHFpaanGkZVN2dnZumYPDw8AQFFREQCgZs2aej3vBeAY55vHLCsrS+/LexIQEKC/DQkJAQDdGxcXF2RlZSEvLw/FxcWGHVtXV1ebh4cHKioqUKNGDQCO+cfrzcvLg7+/PwD7dQHAlStX9DPOWz8/PwBASkrKLfO2sLAQderU4WcCAM6cOQMA8Pf31/vm5ubqO9x8P3kP09PT9RydP38+3WazBf9hA/IHw83Nzebp6QlfX19kZmYCAJydnQHYn33APiednOxcjGPp4uKiceKcKisrA2Afr4KCAgDQGAHQ/eO/HFObzaaf8e8Ax7x3c3MD4Him3NzcUFZWhtzcXBQVFVU5d6u1ONarVw/z58/HggULEBQUdMOHde7cGQAQHByMhQsX6gsDQJ8+fbB7924AwB133AEA8Pb2BgBs2LBBE+6pp54CAHz77bfo06cPAKBly5YAgEceeQQAEBsbq4Xv2LFjAIDJkyfrpixatAgAMHLkSADA6dOnMXjwYLz++uvVudT/Ojw9PTF48GBcvnwZjz/+OABg586dABw3+OrVq5o8+/fvBwC0adMGOTk5AIDw8HAA9nsA2B8wTs6DBw8CsE/CXr16AQAuXboEADh37hwAYMaMGbpPw4YNA2Cf3Jy4aWlpN7x+9+7d+Oc//4kXXnjhjxyKPxy1a9fGq6++imXLlqF169YA7IsbAHTs2BEAcOHCBZw8eRIA0KxZMwDA+fPnNTcffPBBAMCJEycAAD179tQmNH36dABAUFCQxvvKlSsAgNmzZwMAEhMT9f68n0ePHkW7du0A3LgoAPYHvLCwkN/jwh82GH8CatasiQkTJiA/P19zkAsaF7/du3dj0KBBAIDNmzcDsG8Sbdq0AeCYn127dgUAbNmyRSTh/PnzAOxrx4ABAwAAYWFhABz3Y+vWrdqk+C//BnAsjiRxTk5OaNy4MT744IN/e13VWhyzs7OxevVqODs749lnnwUAXL58GQCwa9cuAPabOnToUACOhWrjxo0atPfeew8A0LRpUwDA3XffjQ0bNgBwLHYNGzbEvn37AADr1q0DADz22GMaDP4td6fy8nI0btxYnw84dqzBgwejqKhIu7xR4eXlhY4dOyIzMxNJSUkAgFatWgFwLHaFhYU4cOAAAGD79u0AgPr16+vG161bFwDw2WefAbCzlvvuu++G9/j111/x+eefAwDuv//+Gz5n4cKFiIuLAwBs27YNgH0y8YEdPXo0AGD16tUA7BvW3r17b2ClRoSTkxM8PDxgs9k0r+69914AwDfffAMAaN26tRZFLy8vAECXLl00J8niyHoAICYmRu8PACNGjMDUqVMB2BdkAPjuu+8AABMnTtRnTZgwAYCdNPD9GQ1xMS4qKlK0ZHS4uLggODgYFosFx48fBwAkJycDABo1agQAePLJJ7Fy5UoAjrnYoEGDG55hwLGeZGZmavPp3r07APsGw8WQ6w1fn5eXJ1LRs2dPAHaSRWbJhZkbe1BQEBISEnTvqoKpOZowYcJEFagWc7TZbCgtLUWrVq20glMPIzNMTU1ViNCvXz8A9t2ZIUX79u0BOFjf9OnT8fTTT+tvAbveNX78eAAQi+Juk5WVhSNHjgCAXnP8+PFb6PEvv/wCwB4eWq1WwzNHwD6+ly9fFnu4eVeLjIwUy3v++ecB2Fn58OHDATh2xUmTJgEADh06pDElg0xOTkaDBg0AOEITSiIeHh6SLKKiogDcqAkzvGYoWFpairp16yp0MipcXFwQGhqKBx98ULop5QOyv6ysLLFKXk+jRo0QGhoKAFi1apV+Btglhi+//BIA9Jr09HS8+uqrABxz+dChQwDsEsbdd98NwH7PADs7orxExkUEBQXhpZdeAgB89NFHf8Qw/GnIzMzE8uXL4erqqme/efPmAKC5OXfuXEWUjEquX78uOSwyMhIA8MMPPwBwaLWAXfIA7FEn15vs7GwAjijm2Wef1dz99NNPAdjvI9+HeiTD/fT0dKxbt06SVFUwmaMJEyZMVIFqMUd/f3/cc889CAwMRHp6OgBINH377bcB2GN7ajfcgaOjo5VAoCA+a9YsAMDjjz+O9evXA7BrjQBQp04dJSO4o44ZMwYA0KNHD2mO1DTPnTuHa9euAYCSDcz67d+/H6NHj9b/GxUWiwWurq7o0aOHdKp69eoBcOhR27dvl+5HHbBfv36YO3cuAOCuu+4CYGdBgF23YsJs69atAOyaFvXKHj16AAB++uknAHZNkzs4d+iEhATtukzCkWmmpqYiKipKrzUqbDYbSkpK8PXXX4s5MiHDedyoUSPNnR07dgCwjyeZ+u+//w7AkTjYuXMnatWqBQD47bffANjHpW3btje8f5MmTQAAS5cuxdWrVwHYExiAXXdn0pJsfMuWLQCAxo0b4+eff/4jh+FPg5+fHwYNGoQjR46IMfO6GCHGxcWJtTEJAwAXL14EAGnpTKZMnjxZY82fMZEDOBg/14ULFy4oicu1JjQ0VEktfg/e20mTJqGoqAi362drMkcTJkyYqALVYo55eXnYtm0bgoODtSKT1UyePBmAXWOhvkjm4+/vL2vIggULAECZpQ0bNuCee+4B4NBu4uPj8e233wJwWEoSExPtX9jFRTs19Utvb29pN/xe1B/q1KmDioqK2+4QRkBWVha+/fZb+Pj4KOPOHZAs0d/fX6yvRYsWAIBTp05JNyHzefLJJwHYmR13TFoYLBaL7BDUcbmjHzlyRLrQF198AcDOYHgfyZ4GDhwIwK5BZmdn35DBNSKys7OxZs0a3HvvvdIAyeg4f7dt2yYGTf06ODhY7grOHzJNuiEAoH///gDszJrRDJ0D/BwnJyfpZNRuIyIixDBnzJgBwGGNGzZsmBgTnxmjwtnZGV5eXsjIyJCLhZEax7t9+/ay5PTu3RuAPRdxM2NkBPjdd99pXr3//vsA7HYn+lPpKPjqq68AAA899JDyIBzn2NhYOTkYOfGzlyxZcoN/sipUa3H08vJCp06dUFJSIi9jREQEAIcHLjo6WhdMBAYGytpA3yLD5eLiYpliGS537txZwiwFXk7i8PBwJRIYjh89elShPC0EHNjMzEw4OTnpM4wKFxcXhISEoLi4WMktTgB6tRITEyU9cME8deqUElqLFy8G4BCpK4OhTFRUlMJjSiIMNTMyMmSrGjt2LAD7hsjfM1H0r3/9C4D9fjG0NDJcXV1Rt25dLFmyRD/jRkwLWE5Ozg3JGcAeanMT54bMh+xvf/ubLExcVM+fPy/7DRMNy5cvB2APpXlfOWYnT56UfESrUHR0NABgzZo1//HhNQqcnJzg6emJKVOmaGPnxlv5ueMcZEK1pKRE/mNamriOhIWFaSwPHz4MwO5JZZKSxIFyhL+/vwjaa6+9BsC+LvD1/F7c4FNSUnDXXXfh6NGj//66qj0SJkyYMPF/ANVijrm5udi0aRPuvfde0ds9e/YAcFg/QkJClDTgrpiQkKBKBIYgXN379u2r1Z3WiGnTpondcBfnzl2rVi0Zwvl6d3d37dSsjGH4Cdh3dDJPo4KhiZubm4RlMkHunI888giGDBkCwGGOj42NFSPZtGkTAMjIHR8fL5mB4nZISIhkD1YjzJkzB4Bd1CZb5Q4+YcIEfPzxxwAg+YP2qiFDhqCwsNDwDCcvLw/bt2/HqFGjZMlhJMMIJioqCnfeeScAe1UVAKxdu1ZhGMeRFpW8vDz8+uuvABwVL7t375Z1haZjMtQWLVoooUDGHhYWJubKeU7mPnr0aFnhyEyNipKSEiQnJ2PFihWKMihD8Ln/8MMPZX7fu3cvADtzpsWsfv36ABzryKFDh8QcGZrfe++9+lveK8pvSUlJkotop5o2bZqeIT7/TIqNGjVKyaB/B5M5mjBhwkQVqBZztFgscHFxwcWLF7Uz0q5D4b93794SpcmAWrVqpYQJTZkUqmvXro1Ro0YBcAipv//+uxICZJDUFLdu3ardgnpnZTsJBVqWxlVUVOD69euGNyqXl5cjNzcXY8eOFbt54oknADhYy+XLl2XzoYZS2XRPbZJjFxkZKZGaOldKSop2dzJA2qsmTJighAPLssrLy5WkoQWF9hY/Pz+4uLjctgTLCPDw8EDLli1x8eJF2ZKoHZL1NmzYEGvXrgXgmNPt2rVT8omRCK0joaGhmoe0NrVt21Y9AGhVo4E+NTVV4zhz5kwA9qiJ34fMnpHP0aNHb2tQNhJq1KiBsLAw1K5dW0w7Ly8PgMPcHRgYqAQqk6t33nmndGAavcnkH3vsMUWGNM+XlJSgQ4cOABxWKUZBQ4cOlY2IzNxiseCBBx4A4LAK8RkpLi5GYWHhbYtDqrU4+vj4oGvXrkhPT5dHjpSXiRDAkXlbsWIFADuVZUjGTOqUKVP0elJlTtg33nhD4QizXlw4ExMTVXPJjGFAQIBCPyZ1mDQICgpCUFCQ/sao8PDwQExMDJycnPRdKTpTpP/222/VLIILVK9evSRmM/xiwopSBOBYADt16qRxY1jIjF56ero+i5/97LPPSgphhpzhztKlS3VfjQyr1YqUlBS4urqqwoUPL8ezRo0aSuIxm2yz2fDjjz8CgEK2F198EYBdFnr44YcB2JMngH2jZ5aa/zJ0CwoKwqlTpwA4iMQPP/ygB5mvZ+gdGxv7vyaszsjIwOLFi/Hqq69i6dKlAByNZVh1NHLkSPzjH/8A4GgW8dhjj+nZ5yLKjfv333+XdMdORwEBAaoWujkzvWnTJv0tN7dz586JNFFCos+xWbNmCAkJUe11VTD2lm/ChAkTfxGq7XP89ddfcfjwYe0ITKUznHB1ddXuwTR+ly5dFE6zgoCJnNTUVFFx/q5Lly6i0mSOZEGPPPKIfGhMRJw7d06v+9vf/gbAEc489NBD8PX1VWhqVGRlZWHlypUoKCjQtZDVsA64X79+8oF16dIFgN3uw4QNGQl3zri4OAnclT+HOyytKwzLt23bpvCbSa8tW7Zo9yXDpKjdpUsXbNy40fA2KX9/fwwcOBAff/yxWDMFeo5nSkqKohWGakePHtU9YJKR7ckuXryoBBjfs1WrVvLysWVZbGwsADsLpZTE2up+/fopPGQihuy1ffv2sgEZHb6+vujTpw/ee+89vPXWWwAcyUFKDi4uLjf0yQTs9+Bmqw076ri5ud3QBhEAVq5cKbmM0h0jqHXr1mnN4BieOHFC3XsoyzFxm5aWhosXL5q11SZMmDBRXVTbBN6hQwewGzjgYDfcDf39/cVgmJ5fuHChLA5kRRT8a9asiXHjxgEA3n33XQB2jY0VHdTfKFTn5eUpgbNs2TIAdmM5rUE0hjPVHxkZiYYNG4p9GRXe3t7o2LEjRo4cKR2Fdh1ey6FDh5QwmTZtGgD7PaFexR2Wu3avXr2QkJAAwGEaHzFihLRG1mJT2+nZs6fuT+XKAyaByDTZDSk1NRV9+vRRYsKoKC8vR0FBAWw2G0aMGAHAEbmQnVy+fPmWWubOnTuL2ZDtUAf08PAQQ+fYLlu2TMybzJQRT3R0tF5Ps/LOnTvVPYrJLrL0tWvXSo80OioqKlBUVAR3d3c9f9Si582bB8CudZO1kUG/+uqrmntMyLDQICQkRBo3k1V8LeCodGElXO3atcU+ycY9PDwUOdFGRH1++/btGDBggJKSVcFkjiZMmDBRBarFHPPz87Fnzx5ER0erBIishf/v7++v1Zk2nI4dOyprxywWNbPo6OhbjjB45513lG0mc6xs4qTFhbv+Z599Ji3o73//OwBHBvzs2bPo27fvLSWNRoPNZkN5eTneffddGY2pJVITLC4uVldkZvZzcnKktTDTSZvVnXfeKe2Vv5s8eTL69u0LwGFQptNg9erV2vHJhtLS0qTX0KhMpvjCCy/g008/1c+Nivz8fOzcuRMtWrRQN27OIXaT4lgDQLdu3QDYr5OaIRk4mUbbtm1lwyGrDA0NVS9NzrfnnnsOgD3jSosVdbWAgAAxWd5PZmHnzJmjvoRGB5nj9OnTNRYsJuDzu2HDBjE7PrcnT56UrYeRH9lh3759FTmR/X3//feyQ9GoT1vU5MmT1c+UuuTixYsVHTEPQi19z549qFev3m0tftU9YAuNGzfG+++/r4uhoMnmBKdPn1YVB8OCHj16qHic9Jn+o2+++UaDxoTMwYMHFaZzIEmVKyoq1AyAnqcOHTposFitwIU5NDQU69evN3zSoKKiAgUFBejUqZM8crTQsNKod+/eso1wEkZHR8sbxmQAa1ibNm2qceE9GTJkiB5A/h0XjFdeeUVhIb9D5fdgNQfrrmfNmoWCggIt3kaFm5sb7rjjDhQVFelhYYKQEgbgeDBpVWrevLnmDUNu2tTCwsI0p9kHYMuWLUpQVj64C7A//AztuFFFRkZq86I1jp+9YMECLaJMfhoVrF0/cOCAKo84T7kRr1y5Us88N9OQkBD5RkmyaKM6duyYki2U8Nzd3dU3gJsV596SJUskCTHU7tSpk7zQnOs806dWrVr/sQG2GVabMGHCRBWoFnNMT0/H559/Dk9PTx3SxFCBu6ePj492Wwr/n3zyCZ555hkAjl2Aq7uvr6/YIc2e48ePF/thmMIdY/z48RJtuRucOHFCOzSTOtx1unfvjuvXr8uyYVR4e3ujS5cu+PLLL9Uen0mkygdnkRlTPkhLS9M4kOGxUmDgwIHarRka9+/fX2NVubMPYG9Dzw4/tJGcOnVKCTZ2NGFVyQMPPIDly5cbvkLG09MTrVu3xpw5cxAfHw/A0QWHc+78+fPqGMVQOjw8XElFhmUMl8+fPy+zPSWcXbt2KYykzYy2oJiYGP2Oc/v7778Xw+R7sOvSmTNnFP0YHaxdnzx5smQzzl0WZQwYMOCGjjiAnVXy+lmBRCnh/fffV3KKjLBBgwaKmDiuZIaxsbGqZuJaU5mZc84PHjwYgKP13O1g7FltwoQJE38RqsUcqYs1a9ZMVhuKoEwU5OXliUkwqTJp0iRpjbSbUICeO3eudhvWtr777rt6DyZ3eKZ1fn6+LAFkoTabTQZTap88xKd3795ISkoyvC2iuLgYiYmJaNKkiWqrqTnRDtG5c2fttGfPngVgN35Tr+JOSxby6aefKllDtjJr1iy9P605PKxr165d2pHJtF966SVZKbjj07517do1uLu7G545pqenY+HChahVq5bmH0vyqHMDjmQLdcimTZvK+M75xZLBXbt26b5w3jZv3lwMiPOQTVyvXLmig83IyisfWUyQ4QQHByv6YQmjUcEjPio3PSaLo0bbsWNHMWeOaVZWlsaHER8jxHr16um/OQ7s/lP5Pfi7mjVrShdmtLR8+XI9G4wK+HmHDx9GSkrKbU3g1Voca9SogVq1auHHH3/UwscQjpNs/vz5WpiY+WzdurUGiQ8rzw0+evSoslEUV2vXri36TOGVrxk2bJgWTD6Uhw8fVlNSfi/6AHmYOJMORkVhYSGOHDmCXr16KQvKqh5OgMqnu/Em9+jRQxsPNyx68ZKSknQP+HedOnXSIsA6Y2Znz58/f0tN+6+//ipfGhdHLgYTJ07Erl27DL/xAPa5UqNGDT1w3BDY3OCJJ55QYpAJEJvNpjFiooSLV0BAgHoHUPaIjo7WPGMtLz2iDz74oDY2JrQKCwvl2uCiynCvYcOGqm83Onx8fNC9e3f8+uuvauxA3yY3FXZiBxwbR1hYmGqlWfHCOZaamqrXUy7y8fHR+9HRwcRaZGSk3rdy0wsmGJkV5/h6eXmhV69eqrWuCsbe8k2YMGHiL0K1mGNBQQEOHDiAN998U2yG4RdX90GDBmkFr8yAGBazGob0eM2aNbKqMOSpfCobj2Pgv3v27FFIyd08KipKAvCbb74JwBG6DBo0CJs2bdLZHEZFeXm5zjEmuyHDYMLl1Vdf1U5JdjN79mydHUPPHu0QcXFxSnyRbTds2FDjx9Dks88+A2APy2mzYJjzzDPPyPpDcZ22oO3bt+P69euGt/L4+fmhX79+2LJli5JJ9GoyhJ43b57m30MPPQTA3plo4sSJABwhN+1Sn332maxk9EweO3ZMlWCUMciMzp8/LxmDSaFu3brhk08+AeBImDHRNnPmTFWIGB2s7lq/fv0tXYnYc2HNmjV6pjmGbm5umuOMFGmF2rVrl5K+vGfvvPOOnnMmd+hTzM3NvUW2qFmzppJmK1euBGBn8IB97i5atOi2iRmTOZowYcJEFagWcwwODsaoUaNw8OBBNa5kjSoTMu3bt1fvQDLC7OxstUvnTkptKzIy8obT2AB7QoHshiIu9bfExETpEqzKefvtt1Vnze/DWuyEhASMGzfutgfpGAE8pOiXX34ReyAzpu1kwYIFstVQQ7ly5YrGitc4depUvYaaGg3cBQUFqkgiI6RYHRAQoKoEaprbt28XG2L3mcp1ycHBwdr1jQqLxQJnZ2fVWAMOjZwa1tixY1V4QJ2qrKxM/Uc5ftS+LRaLGBCtIzabTdaRr7/+GoCDCQ4ePFj3iZFAZmamoit2SmJSoXXr1rr/RkdRURFOnDiBtLS0W3qK0hZ19uxZzV0mbgYMGCAbGec49e0DBw5obKjLJiUlqZKJ9f3UZevXry+bG+9Ly5YtlZxkZQxr2T09PdGwYUPzgC0TJkyYqC7+R+dWz549WzswGSBZxfr161WGRRP4a6+9Jm2FWmNlXYeZUepdKSkp0i3JKslCK7ft526+evVq6Wc0lPOAqH79+uHkyZOGz6j6+/tjyJAhWL58uRgMtURadAIDA1U3yqx869attYuyKzuN8wMHDlQ2jp2Ww8LC1DuQuzS1mrNnz8omRfZksVjEeJi55fEXH3/8MS5dunRDCZ4Rce3aNXz00Udwc3OTq4FaLHVDHx8fZfmZaT5z5oxq+jkPyXDCw8Ol/7KH6JQpU1Tjz9+xVDAqKkrPA/W4yMhIuTGo57Krjaenp3R9o6OsrAzp6elo0qSJOsnzeeO64OnpqXnCksHFixdrHaH2yDUgNTVVY87516lTp1u6tVfu9kW2zvcPDAyUVYcMsrLmmJaWdlu9vFqLI2D3Oh49elQPFkMsthDatGmTKi84aUhlAUfNc+W6RiZzSKOPHTumB5y/42Lp5OR0i13Czc1NXknaK5gASk1NRZ06dQx/hkxRURGOHj2KI0eOaAFk2EvJ4vr162reyQUKcNhu+DoK+U899ZSqbSh0X716VQsEx5EhtKenpzyQ3PQ6deqEl19+GYBj8eXC4urqCqvVKqHdqHBxcYG/vz/OnDmjh5abACWFrVu33nDuEGC/Xm4q3BiYCDh8+LDGnVLEpEmTtCjyXyYgBw0apFCezYL379+ve8Xwksmu9PT0/1j7azT06NFDZ8KwpprS18CBAzUHOeaHDh3SsRNcvPhvly5dlCxh05qzZ89qI+cGzzH94IMPtOmwqunSpUuqgGKCjNV1PXv2xA8//HBbj64ZVpswYcJEFagWc3RxcUFwcDBOnDihGlBaELiLPvzww6qPZGh2xx13aPVnqEDD8po1a8Ty2H6oVq1aMiaTUjO0vHjxolgTqXXdunXx/fffA3DYgWiH6d+/PxISEgwfVjs7O8PPzw8eHh5iaAwPyJ69vb11fdyZN2/eLKsDWR+Z3N13360wkCL18ePHtXNTNOc5v5cuXVK4zqMoTp8+rVCPdiAmClq0aAE3N7cbDlczIkpLS5GSkoJXXnlFNg/O3xdeeAGAvWKGIV7lrjy0kJERMfKJjo4Ws6mcwOF4P/vsswAc9+Krr7664Rx3ADqzGYCsZpSH2rVrd9tGrEZCTk4O1q9fj06dOqnzEOcRW5dVbpDN5OqUKVMwd+5cAA5GTrYcHBysKIkWqP379ytsp2zGZFj37t11b5iwrVGjhqq5tmzZAsAhi/Tt2xf9+/dXpV5VMJmjCRMmTFSBah+T0L59e+Tn56ummis4y6TuuOMOWTveeOMNAPbdgPoZa0kplD744IPaNbmLnzx5UmVv7L/HNH3Dhg3VK456xTvvvKM6ZH6vyv0OV6xYIY3JqHBycoK3tzdCQ0NVl0o2XLmvHQVlar3BwcHaFePi4gA4DOJ33nmnbBBEs2bNJH7f3EPviSeeEAti6VZgYKBK7Xi0JrsuRUdHY/Xq1fp7oyIwMBDDhw9HQUEBhg0bBsAxbxmt/PTTT/od7R1RUVEaK+ro1GLnzp2rpCIZzosvvijGzXlIhIaG6mgQausXL14Uy2Higkzz4MGDiq6MDnY9qqioUEEBn3MyvaSkJI0JrX6rV6/WWNPyw/tx/fp1rQuM+s6dO6e6aYKlytHR0SpZ5L21Wq1aR3h0M5nm5MmT8fjjj99WL6/W4ujm5oaGDRsiODhYniVmoEij+/Xrp4wxM08lJSUKhenPo19s165d8iSy6YHFYtEDx1CRA/T6668r60dvY//+/XUzmCknPT979iyGDx+OOXPmVOdS/+vw8PBA06ZN0b9/fz1szMozbM3KypJkwfDlySeflLTBsJeLWUpKiiqYODFnz56tTYWSBedrppMAACAASURBVOthT5w4IcGa9anr1q3TJsdGpgz9Nm/ejMcff1yLplHBc6ujoqI0/yhVMLl09epVjQfn9O+//65NmZs5hf2//e1v2ki4IGZlZSkkZ0hMUuDi4qI5zSRBYmKinhF6T/nZTk5OkqqMDg8PDzRr1gyrVq2S3MP5Qx9iSEiIpDKecQQ4stSc85R62rVrp3nNTSIhIUGVTfREM2G2aNEi3Y9HH30UgP2Mas57fh8mhx944AHMnTtX978qmGG1CRMmTFSBajFHVhpkZmaK0TE9z0qC0NBQnZZH4bm8vFxWGgq0POrgypUrYpXcBZKTkxW60WbCnSI7O1vpfKbs582bp89k+MMdeOLEiUhKShLTNSpyc3OxceNGlJSUaIw4BmTNGzZs0Jkk3PFee+01MR52diEDb9KkiSoKaC3p1q2bbFdMGjC8tlqtel/u1nFxcdr9mcSghGKz2ZCWlqZd2ahwd3dH06ZNsXnzZllAyH5Zj75mzRolnsj+Ro0apQ5EnD8M2ebMmaMQjVUwn332mcJIsiNWkvXp00edZyiNLFq0SDW/lFBof8vPz1dizuioqKhAfn4+BgwYoKQL5QeOc48ePfRMks2lpKSopRjnHaOYpKQkndzIc3aYlAUczzcrX0pLS5U04z3Lz8/XGkHLDyuRCgoKcOedd4rZVwWTOZowYcJEFagWc6QOYrVatSNw5aU24+LiImGU2taQIUNUAUBWQwPyhx9+KGc7KxQ2bNggQy53W4rY33//vawoZDeNGzfWjsUdnq9577330Lt3b8MbanmCm4eHhxjd4sWLAThY88WLF1WBwYTT4cOHZfkZMGAAAEeXnebNm9/C4k+dOqXemqzj5fu3aNFC48S/c3JyErthtQf1zmvXrqG8vNzwzW5ZwdGsWTPpUtRxqT3GxMRIE6Om/fTTT0tD47wiq/fw8JB1hMmxRo0aicVzLtM6smrVKjF0ss+2bdtKI2eRA5Nr/fv3F8s1OiwWC9zd3fH5559rDt58cmaNGjVUqEE7WuPGjcUG2XuBrO/uu++WHY89Y6OiohTFcM4tWLAAgD36YbKGunxBQYEsQtR5GYUlJyfDycnptgkZY89qEyZMmPiLUC3mWFxcrLNmmWFmhpQ7rJubm/qwUW8JDg5Wq3fqaDywqG/fvuq8w3b8s2fPlmGW2iR/Fx4eLnZDY3lOTo7a18+fP99+Yf9/t5XBgwdjz549Mp4aFWwBf++996qUjLoV7QiAg8HQDtGxY0eZwKnfcKcNCAhQJpna7fjx45WxZTacbCg2NhavvPIKAIct6PTp07qP7DhOBrB69eobWLtR4eXlhQ4dOmDOnDmyRVFXZEb01KlT0r04tsHBwdJXOd9pvXnppZd0GBmZdK9evcQKOd9o3yktLZWGSKdGfn6+bFtknMxyf/DBB9Lxb9et2ggoKCjAb7/9BmdnZ0UhnBNk4Zs2bVIhAnXAkSNHSienVktGOG/ePJUHM8dw7tw5nSRAjZbzNCMj4wZ7IGDX2b/66isADgcHbW65ublo166dItKqUK3F0dvbG127dsXhw4fVHIF0lf/v5eWldDwfKj8/P00qPsiV3f8cEC6Aq1atkm2ECyerEerXry/XOwX0/Px8hX4MkxiWWywWw4fUgD1Ma9myJb766isJ/EyccEOZNm2abCMMcfPy8lQrTWsTa33nzJmj0yHZQGHRokWqd2VdL1vO79u3T6ezcWGOjo5WaMJwiP+OGzcOnp6ehn94AXvyqHv37vLTcQ7RSrN//36NN+fyhQsXNDd5jZQw3njjDc1NbmatW7cWCaCvls1DnJyc1CiFUlRxcbESMLSicaPq1KmT4eUKwtXVFXXq1IG/v782GC5KXB/KysowZcoUAA65rVu3bjdUswAOC96bb76pzZhzuG7dugrXmVhh2H7p0iUtnNyEMjMztdHRT8kFumvXrli/fv1tz5D53zH6JkyYMPFfhqU6HVUsFst1ABf+vK/zp6KezWYL/qu/xL+DObZ/Hv6Xjy1gju+fiX87ttVaHE2YMGHi/wrMsNqECRMmqoC5OJowYcJEFTAXRxMmTJioAubiaMKECRNVwFwcTZgwYaIKmIujCRMmTFQBc3E0YcKEiSpgLo4mTJgwUQWqVVvt6upq8/DwQHl5+Q2nigGORg85OTk6+4ENbt3d3dXSjL9jO/7s7Gy1fmKdZF5enloZ8WdskZafn6/6bL6X1WpVIwG2JOJn5+TkIDg4GBkZGcjLy7NU53r/m/Dx8bEFBQXdUOvJJhNsr+/h4aHrYnMJT09PtcVifSpfn5eXp4YTHLPAwECdsscmCWwQ7OHhcctnWq1W3TvWs/K9srOzYbVaUVxcjNLSUsOOrYeHh83HxwfOzs66Ps4v1q3bbDbNNY5xcXGx5h1fx1rh0tJSzT/WZ7MhB+A4roNNWUtLS9XKj6/nuUGV35/zuKCgQM9IUlJSupErZNzd3W0+Pj7w8vLS9XKOsD7cy8tLY155jrOFHGv/+fzm5ORoPvO+lJWV6f6xXSEbMBcUFKgum00vfH199bdsBMLxzsvLQ2lpKQoLC//t3K3W4ujn54d77rkHhYWFalrAJgbsUOLu7q5u1TwXo1GjRjoY/vjx4wAci93UqVPVo/CTTz4BYF8UZs+erb8FoKL9y5cvq/EBL3Tu3Lk6YIvdfFiEfunSJTRt2lRn3BgVZWVl8Pb2xocffqhOOuzxxy7GZ8+eVbd0Tqr4+HidNcIuymzaYbPZNHnYpCMqKgrDhw8H4Dgwiz30vL29tdBybL/77jv1x+ODy87fS5cuRVBQkHoQGhXh4eF4//33ceXKFW3EPJiJXV3ee+89dYHhXPrll1/UDIUdpznGGRkZWLFiBQDHQnD//feraQXnKxeC48eP6wA59jKNjY1V1xo2bHjiiScAAF9//TU6derEnxm6NK9u3bqYNWsWLl++jL179wJwNOhgQw3A0RCFzTi++eYb3Q92S2KXr65du96ycL799ts6ppWLLrvzhIWF4cCBAwAcC2ZxcTH69+8PwNH1iN/vk08+wfTp09XIoipUa3GsWbMmxo4di82bN2sX4MWxtZOPj4864vAhzcvLU4cetivihNq7d6+6+HChvXbtmnZofvnKg83DsthANzg4WBdPlsPWRH379oW7u7vhO5x4eHigSZMmmDlzphgJbzIPybp06ZLGmy36J06ceEvbJTLCK1eu6IQ8/iwyMlIdkTjeTZo0AWBvsPrTTz8BcCyYtWrV0s/YnYf3t2XLlti1a5fhzwRPT0/HwoULkZWVpUbCPDOaXY6effZZMTt24MnKylLHIj7YlVtkjRo1CoDjeIpTp06pKxSPAqh8+iMZDcfrk08+EcPkKZuMwB588EG8+eabf9AI/LnIzs7G6tWr0atXL7V2Y6NmtnMLCgpSA2UuUAcOHND1s1sSu0JFRERovHiUhZubm7pzjRs3DoAjgnJyclJrPzYV/vHHH2851Iyb25YtW+Dm5qZ1qCoYe8UwYcKEib8I1WKOV69exTvvvIOuXbuqZxr1Ex60VfmsXbY0T01NVRjNRqvshVdWVqYwmY1Bjx49qtCG78F/PTw8FNJzVwgLC1O/Nmpsu3fv1vc6ceKENB2joqSkBOfOncODDz6oIxCofZGZxMXFiZEMGTIEgF12YINf6r+UODp06CDGzTA5ICBAYzFx4kQAEBNftmyZdnIeb7lu3TqFOrwnZPo7duyAr6+v4Vm5r68v+vTpg59++knyAqUKMsjMzExFQ2R7UVFRYibsF8pxWblypdgI/42IiBAbZ0PbrVu3ArA3x2UozwOj5s+fj9GjR+u/AceRC1arVc2I+TOjws/PDwMHDsSyZcv0/A8aNAgAJA1YrVZFl7wHffv21TrCcWO0eerUKR33QU0ccJw/zWMufvjhBwB25s9x4rMREREhmYjHhLC/Y3x8PAYOHKhenFXB2LPahAkTJv4iVIs58vCs9PR0teTnysuuvXfeeadE08pZUbIV/u7VV18FYNcH+DMeztO+fXvtCOyUzB3J398fkyZNAgAlWU6fPq3dnmySTOnq1atwdna+rbZgBLi5uaFBgwY4ffq0vis1MIr7c+bMkXbLTtZHjx5F586dAThEZzLrJUuWqA0/xe3AwEC1kad+SdF60qRJeOmllwAA//znPwHYWQ53bu74zPxFRETAarWKtRsVFRUVyMvLw4gRI5RMorbHa0tPT5cWRlYSERGhiIUHtvEwuMrvRR39k08+UVd2drjmoU9lZWXqQk2m2b59e7zxxhsAoMO9eC937Nih8TY6MjMzsWLFCuTm5ippxyQKGXFKSoqOa6WmPmnSJLFjXiu7sR86dEgJSY6zk5OTNGAycjLDixcvav7zXrVt21YJL/4dO7XXr18f69atu21EaTJHEyZMmKgC1WKOVqv1hkwy4DhKktmmr776SofsMNNstVqVeaN9hAzo9OnT2kl4FGVKSor0Ge7cZJft2rXTzs5M48svv6yjMJnZYhPf2rVrK4NtZFgsFri4uODChQvSTQlmicePHy/7SPfu3QHYNVyOFXVf6ruhoaHScJgN3bx5s5giNTbaKIKDg6Uf8qjMxMREZctpI+L7t2jRAjt27DD8GT0lJSU4f/48Nm3apPGgZ5MMpFWrVmKAY8eOBQD89ttvcgxw/HhcaGRkJObOnQvAcfhWWVmZzo7h4W9kUqmpqTooihntyMhIMUe6CZjJ7t+/P6ZNm/bHDcKfCDc3N0RFRcHT01NRBZk559iZM2d0/bTZzZgxQ/kGskM+q6dPn9b94IF9c+fOVS6B0SMz2lu3bpXOSc3excVFOuTNEcMdd9yBHj16yFpYFaq1OPr7+2PgwIEIDw+XuZI+JvoLO3TooMQKL87JyQm//fYbAOhfHiLF8Lnye9WtW1c/Z2hOL19ERAQ6dOhwwyCMGDFCE5U0na85fPgwGjRoIA+UUeHq6orw8HAkJCQoJKFEQAsEBWrAITBPmDBBPkS+jpsMJ15lnDt3Tifu8fcffvghALvHjg8kTyFMS0vT4ssF+cyZMwDsUkpAQICMuUaFzWZDcXExWrZsqSQXLSFMLtWsWVMbKm1gbm5uSoAxJKbYv2bNGj1wfGCfe+45HUx29epVAJB38tChQ0qYcW7OnTtXmzoPkWLoOXr0aL2Oz4xRYbVakZKSgtLSUi2AlMjodfb391dIS++yxWKRtYqy2Zo1awDYN2duZJXnGzcb3j+G3jk5OUocMhmWmZkp4kWpifdq48aN8PT0NA/YMmHChInqolrM0cnJCV5eXkhISBB9ZrjLFf25556T8XjGjBkA7GyPuwZNtGQtY8aMEdOcN28eALu9gu9L+kzRPz8/X8kfhoAFBQXaEeiYZ8InMjISd911Fz766KPqXOp/HYWFhThy5IjGEYCOsiRjDA8PF5ukTJGcnCzGzZCOO+ewYcMUwsXHxwOwjweZIMvT+J7btm3Trs7wxt/fX2yA9itWOBw5cgQxMTFiqkZFUVEREhMT0axZM7ERFhDwvPWioiKNAxmIl5eXQmbOURri77rrLr2e1S0ZGRli9BxHnrseGhoqFkP5o0uXLopoaMSnxJGcnCzDstFRVlaGjIwM+Pn5aW5MnToVAFQZd/XqVTFtssQGDRooYfLzzz8DcMgce/fuxZdffgnAYf+LiIjAyZMnATiSlQzbi4uLNe+ZaOTcBxzrDefyXXfdhezsbLHSqmAyRxMmTJioAtVijnl5ediyZQvKy8u1KnOHpMD9xBNPiB2SoWRnZ8t0u3TpUgCO3bZly5ZioTTEtm/fXskWJlj4OZ6enmIq/Mz4+HhpNyx7o3506dIlbNy40fAmcHd3dzRu3BhlZWViaJXL0gA7o2ayi1pJQUHBLSL4sGHDANh1YLIblgx6e3tLlyVT5+sTExMxfvx4AA4rj7OzszRNvo52k4YNG2Lv3r2GT3h5e3sjNjYWJ06ckHZIyw3r+e+77z5s3rwZgEMva9SokdgOIxLO7fz8fLERMvdffvlFSYHKxmXAzmb4M5ru/f39lYBhmR0134CAAN0no8NqtSI1NRUxMTFK3nGuMOLbu3ev7Eoc8+eff17WMVrNZs2aBcBu1n744YcBOIzfZ86cUVEDS2b5HDzzzDNK8i5ZsgSA3RbEXAjXDK5b3t7eKi3+d6h244n+/fujffv2eP/99wE4PHJ8mAICAiSScgLu3LlTDzUHYdWqVQDsXkUunBRZV65cqUTMzf7IzZs3y/vIB79Lly560JnZYojp7u6OPXv26LVGRWlpKS5duoScnBw88sgjABz16gxD7rnnHj3ADDUaNWqkUJEbDkMbf39/JbLoMGjQoIFcBAzhfvnlFwD2UIVeMSbMfv75Z9XJ8vX0A/bs2RMjRoxQKGlUuLq6om7dumjbtq3CKGYpOS99fHw0zhzH2rVryx3AjZjzKDw8XGPLjSc5OVnjxkWV/yYmJirzzVB+0aJFqpZZsGDBDd+5d+/eGnejg+tCRUWFZAQuVExINWjQQCE2+wJMnTpVc5XPNxt9NG7cWKSAZKF9+/Z6P0oTzPZ7eXlpXlMy2bhxo0JtvgcrdpKTkzFt2jT5UauCGVabMGHCRBWoFnPMyMjA8uXLYbVaRZEpULOK4/HHH5dHjh6j48ePiwafOHECgMOa4+7urhZOFFevXLkiSwN3c3ba4MoPOHrlNWzYEH369AHgoM0MnyIiInDw4EGFnkZFWVkZrl27hqFDh8omRQbD7i/ffPON7DqUGcrKysSGnnrqKQCORNX58+c1pgyX/fz8ZOUhi+fYbd26VRYUSiIFBQWqf725Tjs6OholJSWG9zkWFBQgPj4eAQEBYnL02lVmiWSHDAWPHj2q37OqiLLN3r17VZdNVjJq1ChVevDvyEwHDx6MXbt2AbAzRsDup+T9YfKFfuGkpKQb/MRGh5OTE4KDgzVOZI6Vex2wWis6OhqA3YLHuco2esuXLwdgf875DFMy279/vxJY9Noymvntt9/0HHDNyMrKkk+a6weTbWFhYRg/frzaJVZ5Tf+TgTBhwoSJ/9dRLebo6emJli1bYufOnVq5mfjgDvDWW29pd6UusGLFCu3KtKCwm09sbKy0COppNHdXfl+adWvXrq3Ppj0lNzdXu2xlwRWwd5rJz8+XjcKoYHXF/v37ZbZm3Sh1lo0bN4p5MzFQWlqqbjncRck8LRaLdmImUU6dOqX7w3H/6quvANhrUbnrUquJjIyU6E2zN7Wg2NhYfPvtt7qXRkVAQADuvfdeWK1WNaMli+M1ffrpp2LoTJL4+vrKYkKtkT0DS0tLdd1kzjt27LilOS4LIaZNm6ZnhUzTzc1Nf0u9jJ+zb98+PSsLFy78w8biz4CzszO8vb3RtWtXFTCwixEjymbNmimnwDyCm5sbfHx8ADiKQZiEWbp0qTR3VmZVVFQoScnngAb5tm3bqt8Ao9pvvvlG34fdphiRWiwW7Nq167bJRJM5mjBhwkQVqBZzdHNzQ6NGjXDt2jVlh7kzsocgu2oAju4bcXFxWtWpR7Kzct++fZUtJSOMjIyU3sDME8u4+vbtq92fn1VRUSFmyp2Cu3OzZs0QGhp626yUERAeHo7p06dj8eLFulaOHzuKpKamSqvlMRLvvPPOLTYlWnr2798vnYeG2kOHDml35jjSMrJv3z6ZcMk0AUcPPFpR+P7Xrl1D3759DZ9VpQncy8tLTJHaIy0n06dPFyPm9W3evFmOC84nMhz2hwQcJa6+vr7ScdlzgGP82GOP6Wdk5xs2bJDWRk2YjLZ79+5ygBgd5eXlyM7ORkJCgqwz7CxFV8W1a9fkcqD1KTw8XM88M/hk0rVq1VIExQgxNjZW+jijHzLBn3/+WfeW97Rjx47SdKkxM7/h6uqK+++//7a9Mqu1ONJuQkEZcFBk3vBDhw7JnkAxdujQoXrgWdvKxfHEiROizaTYrVu3VihBWxDD5dWrV8vlTjtGZGSkhFeGoEws9OjRA88//7weBqOirKwMWVlZcHV1VbjG4wgY5nXq1EmVPrTcnDlzRg8n/Ygci4cfflgLFy0pgCNZwAYeXBAnTZqkCcZJExYWphpfJipY7L99+3a89tpr/yvawUVGRmLnzp2aawzLuOi9/vrrepAYSoeFhanZ7bfffgvAYU/LyMiQ3MAHun79+pIjuLFVHjO2+WO1jZ+fn+raGQpycy8tLdXCafQzepydnREYGIilS5dKdqAkxLaCnTt31lpBS9O6deu0UdMCRRmtTZs2svcwWdWtWzetC5yDnPvz589X7TrHt127dtroWFnDzat58+ZITk6+7REfZlhtwoQJE1WgWsyxsLBQtg6uyKxhpjO+SZMmYjd8Tbdu3dQVgzsjEyZJSUm3dMx5/vnnVdPKpA6ZTOPGjSWusrXUjBkzFGJTCKYlArB3naHFwKiwWq2yMLFZJ8eWnU0++OADhSQM7+rUqSM2w1MfeYLdr7/+ql2XZtmXXnpJ4SMlDhr6y8rK9L60PhQXF99SsUP2P3PmTJSXlyvRYFRkZWXhhx9+QJcuXXTtlAjYnv/IkSOqNWdRQk5OjtghLWTsEdCtWzdZomivcnFxEaPnzzhX09LS9Iww8ZORkSFmyeeK3+vcuXOKkIwOttsbPny4rEvs6sSEx9q1a/XMc2y6d++uvgFkzpTf9uzZo+iPstKoUaPEomnJYau3EydO6LP5OcnJyYpo+Rww7D9z5gxcXV1vG1GazNGECRMmqkC1mSMbr1L0ZOkYy4AiIiJUTsVkQOW+gtwZqC/ec8890meoA3Xs2FG6JjtskDH9+uuvEmGp1zzyyCMyRbMjEHeW2rVrw2q13nJ8qVFRUFAgZsZdjzpZRkaGrp0sJzU1VckTarxkeLt375bmw5LLuLg4ieBkPkxWlZWViU3S4BwSEqKxZKko2f/s2bPh5uYmDdPIKCsrw4YNG6SfsryU8zckJESMnR1fSktLNW+Y8OPf5+bmiqkwaZienn7LofbUL1u3bq05Sp0rJCRELOfJJ58E4EgUpaSk3PbwJyOBJvu+ffvKysT6dNaRd+vWTfYz1j5HRETIEE6dnf0yDx06pN6OnKczZ86UvstxY94hKChIhR58fiZOnKgoijpnmzZtANhZZUxMzG37vFZrcSwvL0dGRoYWQsAhaLMwf/To0RJemRmKjIyUB49hBgdh586dCm04ARMSEpTAYSabZ8Ts2rVLQjUHaNeuXcps0ePHUO+bb77B3r17JdIaFYGBgRgxYgT27dun7BwfREoLFy9elDuAQvSGDRvk8eSDxQn04YcfKsnAh3zIkCHqZn3zgeZr165V1pALXmZmpjKOXFQpeTg5OSE5OdnwYTVPx/viiy8k2nM+si69Zs2auj4SAIvFoo2D4RsfpoEDB6rdFkPvJk2aaJ5xPjJBmJeXp0w076urq6sWESYouWinp6frPYwONrtdunSpFjv6NbmZd+zYUb5FZpyvXr0q/zLHmX/n6+srQsQNLD09He+++y4AR3Pg1157DYBdXmKFDJvbFBcXawwpd/Ce+fv7o7S09LZz1wyrTZgwYaIKVIs51qpVC08++STmzZunVkN0vZNxBAQEyFPHMC8oKEjshrSbloeQkBBRb7aYHzVqlGwsDEsYkgQHB2vnYdjRs2dP/S13AjKm0aNHw83NDTt27KjOpf7XkZGRgWXLlmHkyJGyK7BSiGPbvn17XTtD3cOHD4tBkzGyM0pUVJRCclYfsWa6Mli7OmjQICUjaLFISkqSnYUJAt4vLy8vREdHG/6YhKKiIhw5cgTjx48XQ7n56IcaNWqI9fE1o0aNEovkGLHr0/z58yXhMNGSl5en+UdZiGPn6uoqqaLyPSCjYaca3vvY2FjDt9kj/Pz8MHjwYKxatUqJDzJsPqOzZ8+WZEOGfuHCBSVYKLcRLVu21JgweuzQoYPYJCNJJmtWr14t1k056tixY1pnKEPxs2NiYkzmaMKECRP/E1SLOVZUVKCwsBAPP/ywki7UBhnvT506FePGjQPg0E+OHDmC559/HoBDvKYWk5SUJKM3T2x7/fXXpSuydpKspbi4WLsFxe/ExET1QKTYS/F3zZo1CAwMlK3IqCgtLcXFixdx7tw5GVU5RuyGc+rUKV0nRf0ZM2aoCoO2CbL67777TqyO71kZ1IN4FIWvr68M4bSYLF26VEc38N5VPsHN6AeXAfYook2bNnB2dtY8ZVRDZsgz2QHgxRdfBGCft5xr1Mto6QkODhZDoZ5buWMU9XaO1WuvvabkIhuvxsTEyLpDRkpmFB8fL9O40ZGVlYWvv/4aDRo0kDmb6wKfw7i4OCUHOZ9iY2N1mBmfd64BeXl5qoLhc7Bp0ybZ2gYPHgzAYfiuXbu21hFGUomJiXpOaBWi5hgaGopjx44puVgVTOZowoQJE1WgWnQqMDAQI0eOxOnTp6WHkU1wB1y6dKm0GJZc5ebmitXQdsIdvFevXsqaMs0eHBwsxsPMHrOKXl5eyozzLOG2bdvKyExLAK0UPXr0wNtvv214/aa8vBy5ublITk7Wbkb9r7J1hiVVPGTs73//uzRJshxm8mrVqqVdurKmQ4ZEPYYs55577hFrolH5pZdeUmcf1rvThsVuLEbXHDMzM7F8+XL4+flJe2KtLcfWZrNpjtBalpGRIe2a18wig0cffVQ6N21S+/btU6TDTklk4FevXtV70VaSmpoqtnlz96k6derIsGx0ODk5wdPTE8uWLRPzpX2PzDw8PBz33nsvAMdhYhs2bNDxEDR883m/cOGCypA5h93c3KTvknFyHalbt6604spd2//xj38AcHRr53inpaWhsLDwtr1Iq7U4Xr9+HZ999hnuvvtuTS7efFokjh49qsoVLnYHDhyQPYVNDOhFunbtmqwitAGcOnVKrY54oaytbty4sag7q2by8/M14HxQSckzMjLQoUMHWSWMCi6ONptN1h0u/qyL6LWALwAAIABJREFUfvjhh7WRUEgeM2aMwgmOIx+0iooK+cAY3j399NPyk7KtFkPM48ePa7LQIvHll19qQnEC0761YcMGNGjQwPC11f7+/hg6dCj27dunBBUTWpyXWVlZmkP0LyYnJ6tSiMkEhtIFBQUiBF9//TUAuz2IiSyOKROBPj4+mrf82aOPPqqNh7IH7527u7sWZKOjoqICRUVF6N27t54/zkWuE7/88ouSrNyQBgwYoP+mnY9j6ufnJ0LF+7FixQrVbNPKRkvgzJkz5b3m5t+wYUOF3UzqsFJuxYoV/7FRsxlWmzBhwkQVsFTHwNuoUSPbxx9/jMTERBlfKeozbLPZbLLt8JSxpk2bivKyrpLhR1lZmcKMygZQJmBI08laWrdurR2Y1DouLk6nFbLOmDWwQ4cOxaFDhzB//nxcuXLFsBQnKirKNmPGDKxfv15jRGGZbKKsrEwmdyI9PV0GZTJMhiH9+vW7oUMPYLdScWy5q1ZmU0waVGafZPuUUrjbp6enw93dHV988QWuXr1q2LH18vKyNW3aFHfccYdCNR7wxnArKipKLJFzLTMzU/OP105mExwcrPZxDO0CAgI0r5lE4OsvXbqkowOYrNi+fbvel0kxMv2cnBwVW/z9738/aLPZ2v5hA/IHo27durbnnnsOubm5+v6UECo3l2VhAe1/derUUfTH5BOfbR8fH0lpjHTGjh0rRspeDXzOo6Oj1U+B8s/mzZtl+aGERIP4Tz/9hObNm2PNmjVIT0+vcu6azNGECRMmqkC1NEebzYaSkhJERkbKzE2mQU3r8OHD0qi4az7wwAMydNL6Qc0xLS1N+hmTDSdOnJAWRFbD5ERSUpKEc2ptp06dUv0lTbTUxSZMmAAvLy/tVkZFfn4+9u3bd8OBVtRHyLqvX78uXYV61Lx582R/4PiReb777ruyUvBIhLCwMDESnrHM9x88eLDqsmmBiImJwcSJEwE4kmKsV71w4QLy8vIMn5AJCAjA/fffj127dsnKQYZCthgXFydGXLnDE+fYzeWnPj4+Yj0cn3PnzqlYgaWCZIbe3t6ao9Q7CwsLpbPTDkStslWrVooIjI7s7Gz89NNPqF27thIwzDeQ4VU+apV64apVq3RON8eLTDA3N1dznbrg2bNnpU1SV6Sdb8mSJfpszu/y8nLlRMg4mch85ZVXUFpaqvlQFaq1OFosFri7u8PLy0u1qazGoGjavHlzZaAYAq5Zs0YZVIYbrKJp3bq1fsZMs7u7u2g2BXBm7urUqYNly5YBcFQtVD6cm5Oei2utWrXg4+Nj+Inm4eGBJk2aoG7durqhU6dOBeBoGFpeXq5r50TLz8/XBsLf0REwYsQIhXwU/Hv37q2Hk1IIG40uWLBA3lQ2LV26dKlEb2bFWbWUn5+P+Ph4SSxGRW5uLjZs2ID69etrM2eij/8uXrxYGwlDwh49eqjChQlCPpyZmZnaQNgVf/jw4ZqnXBQp89SsWVP3guPu6+urxYMbYuVkIxMYRgc3nx9//FEnAjC8ZsWMt7e3XABc7AICAuSrpaTGtWDFihV6pjn2HTp0UG8GShOUiAICArTe0FHQqFEj3T8m2/gcODs7Iz09/RaZqjLMsNqECRMmqkC1mKOzszP8/f2xb98+sRsyMtaoxsTEiPIy9B4/frzaQVHop02lWbNmotlc5UePHo3du3cDcDR65S5aq1YtzJw5EwB0JkVGRgbeeecdAI6aS4ZBTk5O6NGjh6xARkVubi62bt2KgIAA7axkOaxlbtWqlY42IGs5fvy4EgiUDhgmt2zZUvIFa0rfeOMNMb3K7wHYQw8mBphsGDlypI5fIKvhd+jevTtCQ0O1ixsVNWvWxNixY3H8+HGFvRzTRx99FIDdusRKIzLwiooKdTpilMKQLTc3V7Yq+vd27dolyYfzlc9CeXm5WDklkS5duohZka3yWIs2bdrcUm9sVJSVlSEtLQ2PPPKIxpDPd+XmwJR7GOk0adJEERAbZPM13bp1k/2G1rZXX31Vc5tRJk9o3L9/v+4V16KEhAR1PWL7M97/+Ph4HDx48LZRj8kcTZgwYaIKVIs5XrlyBZMnT0a9evW0SnPH4w5osViUPGEly3333SfnPy031BgSExMllk6ePBmA/QgAptwJ2lM6duwozYyJiCNHjqjPIXcWiuQVFRVITEwU6zEqrFYrUlNTMWrUKBmDKdLT6lSzZk0xFzLwoKAgaV7EggULANhZHxNltAVt27ZNY0+LDlllt27dxFp5v7744gslXKgnUffZuHEj7rvvPv29UZGWloaPPvpI1V2Aw45EA/fLL7+s8Sab9PLy0lhyjpIljxkzRtENdbPQ0FDZSWhboV48cuRIJXCYmLl8+bIsJjeffFhYWCjDs9Hh4+OD7t27w9vbW7kHMmgmnF588UVFOGyCHRgYqFwEk4P83dmzZzFjxgwAuKE/AKvAaBeknW/RokXqq0ArT/369ZUAZg6CEVFAQADCwsJu2wTbZI4mTJgwUQWqxRz9/f1xzz33ICAgQP0Bb7aD5Ofny+xK3SUkJEQrONkH9bHOnTvL5MksaOVyNGaqWHLl6+ur3YidNt566y1pB7S/VLYF/aeslBHg6uqKunXrYt68edJNecAWu7isXbtWWbfKXbw5HqxL5U6bnZ0tSwVtVRUVFSqFo/WB73nt2jWxSjKeOnXq6G+ZgWWU0Lt3b3z44YeGdwKEhYVhypQpuHTpkvQrzjHOixMnTmgcefyGi4uLtEZGLpyPV69eFetg1NKiRQvMnz8fgMMxQFeH1WpVJpbf4csvv1R0xTFmTbazs7MM5UZHTk4O1q5di9DQUEUVtH1RN6T2BzjG/PTp0zLlsysRNcedO3di+vTpABw9GkaNGiU3ANk3x7tBgwYaL+Yu7rvvPpUNU79nBDlo0CCsWrXqtsyxWotjdnY2fvzxR9x///26QD5oFF7vuOMOnYLHgvy8vDzZQdjokwmTtWvXakC4wDVr1kyThI0CSIc3bdqkScwJNXz4cA0CQxdOzm3btmHChAlagI2KkpISJCcnY/To0Xpg5s6dCwBYv349APsYcEHj4jh27FjVtdOzR6/i77//rnM1eEB83bp1lRjgRGNCYe3atXp/nvP72muvqSErrRjclPbt22d4uQKw225WrFiBli1bqlkEQ2JupitXrlRI+MILLwCwz2mGbZQOOO/HjRuncJpzLiYmRiE55zcrRI4cOSJbWuV6blrWWEfMRaJr166SNowOJycneHl5ISYmRuN1s0zQuHFjJXEZSjdp0kTPNxOB3JQbNmyoeU2/4+XLlzVOlJfefvttAPZmt6wC4z0+ceKEEkNMyFSutOvatasqpaq8rv/RaJgwYcLE/+OoVm21xWK5DuDCn/d1/lTUs9lswf/5ZX8NzLH98/C/fGwBc3z/TPzbsa3W4mjChAkT/1dghtUmTJgwUQXMxdGECRMmqoC5OJowYcJEFTAXRxMmTJioAubiaMKECRNVwFwcTZgwYaIKmIujCRMmTFSBapUPent72wIDA2Gz2dSDkWVVrOctLi5W3WrlQ5pYs8tyNnYccXZ2VpkQX2+xWNTth23Q2b/NxcVFHWbYKTgvL0898lhXydcUFhYiODgY165dQ25urmEPgfLw8LD5+vqiuLhY9akcY5ashYaGqtsJ/anl5eW3lLaxjrSiokKvZ/2zxWJRmRw/h30DXV1ddR/5/q6urnoP3h92tPHy8oKTkxOysrKQn59v2LF1c3OzeXl5wcPDQ2WolecTYL9Odpritbu7u6ummr0D2AMzJCREZZgcj1q1aqlnAN+D/RwLCgpuGT9XV1eVZPJ1HGuLxaI5ff78+XQjm8A5d0tKSvTcsqyUz7azs7POnOY9KCoq0viytJLjxrLLyu9ltVpvWVu4BlgsFt0/3tP8/Hz9njXUnN/sBF5YWIiSkpIq5261FsfAwEC8+OKLKCsrUz0p25SzzvnkyZP6cnz48vPzVavL2l0WotesWVMtznghrq6uKs5n7ePq1av1HdiYgZ+9Y8cO1aiy6S1rKg8ePIjHH39ctZtGha+vL0aNGoXExET06dMHgKNmlw/hs88+q4YfXAhzc3N1zgkfeJ4bXlhYiC1btgBwnATp4uKioy34OWwGEhERoXvG1mW1a9dWrSofVraKa9++Pby9vdWGzqjw8vJCz5490apVKzRt2hQA8N133wFwbCT169fX2PJhbNy4MR544AEA9ibBgOP89Kefflqt5VhH/cILL+j4D77HqFGjANjPTmLjYdb3hoWFaTNnXTCbxbq7u6tp65gxYwxdfcK5m5ycrIYvJDV8jv39/dXQg0dDHD9+XOO7aNEiAI55/eyzz2oh5KJ35coVLaz8l+38XFxc1FOAR1/s2bNHv+fxCCRl3t7eWLhwoXo9VIVqLY5sjlBSUqKVmIPAi/P29lZRODt1BwUFqUsHu8SQSS5btky7Bg9uWrlyJYKD7RslmyLw39zcXJ3j8dNPPwGwn3fCh5/nrrCDTGBgIH788Ud9T6PCy8sL7dq1w86dO3VdvBYyu5dfflkNPNixZODAgVocOTHZNefq1atq5sFO7B07dtTDyfHmYpmXl6f+huz1+Mknn+DJJ5+84f1/+OEHAPZ70ahRI21qRkWNGjVQp04drFmzRps4GxjwrJywsDAt+mTZvr6+6mZPFs95f/bsWS2KXHC3bdumDZtjxQ0uLCxMLPG+++4DYG9+waYKjHTYp9DV1VX3wuhwcnKCq6srPD09tSlwDNnw4cUXX8To0aMBOMa+8mF5ZJxsauLq6qpmKez8ZbFYxNx5r7jhbNmyReyb/V7j4uJEqkiO2CXozJkzsFqtt+3WVa3F0cPDA82aNUNWVpZCOX4hdsp57rnntFtwda9fv74oNUMKngLWpk0bDRBZzjPPPKPwhE1D+XCHhYXpAB129HjrrbcUKrKrBx/uoqIi9O/fX7uKUVFSUoIzZ85gypQpYtlcdCrvlgwL2Go+JSVFk42dUNjerU2bNmJG7F7SoEEDjSVbos2aNQuAfaMbN24cAMcBaF5eXup4xJ2cEzg8PBzHjx+/ocWcEeHs7Axvb2889thjSEhIAOAIy/jwTJgwQQsTN+ZWrVqpXRw3Es7V1NRUzTF2kIqPj1cYyea4HP9u3bqJrXKhvfvuu7Vp83ng4pqdna3vY3QUFhYiISEB48aNU9vBf/zjHwAcR2osWbJEpzqyOfPIkSP1DPPwNm4gO3bsULs3Nrg9ePCgSBUZILsrvfzyy2p2ywgnKSlJbfzef/99ANAhahUVFejXr586MFUFMyFjwoQJE1WgWsyxvLwcOTk5SE9Plzb12WefAXA0AZ06dSpmz54NAGpk2bNnT/UopH5Gvevo0aN6HXfZ9evXSze7uQX/xYsXRc9J2Tt16iTtgCFO5dB01qxZYq5GhYeHB1q0aIHZs2fruzLU4hncw4YNE1uhdltWViaGyfCbYXXlNvBkl4cOHVKfwJubtbZr1049JKnzNG7cWEcDTJkyBQDw1FNPAQBiY2Nx7NgxfT+joqSkRK3zOUYMr6l3X716VXOSY3vmzBnNPzIVjvW6deswfvx4ABAbHTFihHRI9jLl5x44cECfyQPNsrOzxZj4vdauXQvAHonxdUaHr68vevbsiczMTEkGDF/3798PwL4GUP7huHl6ekpS4xiSeXt7eyva41oTExMjBnjzfdm5c6dCc2qOkZGRiqYo9ZGZRkVFYdGiRbc9z95kjiZMmDBRBarFHHNycrBhwwakpaVJcKbWxwy11WqVrsgkTGlpqQ6OpyZI9tK0aVPpP9OmTQNgP4SL1gkeVckMV1BQkDQu7krr169X8oc7N/9/yZIlGDFihLpZGxUZGRlYvHgxmjZtiuHDhwNwMGmy4ueee07JFL5m+vTp2q057itWrABgZ9nvvvsuAMeBYw888IB0FyYLmCSbPXs27r77bgCOzu4ZGRlK/pDNU3ebN28erl27JpZpVJSWluL8+fOIi4sT4+CRBYwwnnrqKSVYyJSLi4sxdOhQAMArr7wCAHj++ecB2BNhTELymInExEQdEcKjRqmZx8TEqLU/E4pDhw69xX7CBENSUpK0Xd4no4J6ua+vr54zZonpLElOTpYOy5+98sormpd0RHAs69atq/nPsbl8+bKiIjJuapDr16+XRsv7eNdddymhRvcKrUPz5s2TI+PfwWSOJkyYMFEFqsUc/fz8MGDAAFitVmkE3GV5fklGRoayq2Q5HTp00E7C1Zo2lfj4eLFJHrjt4uIiSwBN49RuEhISxJS46w4ZMgSHDh0C4GBB3J1ud0aEkRAQEIChQ4fi7Nmz0kx4YBDHhawdcOiQnp6e0rKo8zDjOWDAAI0jM+ADBw7UbksLD48fTUtLk2bLsT179izmzJkDwHGY0VtvvQXAnh0PCQn5jzvwXw0PDw80b94c33//PcaMGQPAYebmPAkICJDWSKYSFxcnNwZ1VkY0NptNehdtY59//rlsT3QEkP1YrVaxGOrG8+fPVzaX78WjWg8ePCjvI48oNSqsViuuX7+OyMhI5R7OnDkDwMEEu3XrJsZM/XvMmDE6eI86JDVBT09PuQZ4XpK7u7uy+wsXLgTgyIpHRETI7ULteMOGDYoQGJ0yk92qVStkZWXpPJ+qUK3FsaioCIcPHxY1Bew+RcDhffvoo48UdjG1Pn78eFlJmAR45plnANj9ka+//joAR/gYEhKiBZYPcmVRlokYhu2Vz7JmWMNBLC0txZEjRwyfNCgrK0N2djaefPJJ2Tp4sBatNO7u7ujfvz8AR1jYoUMHSQk82IwWiHbt2knA5mJQWFgo7yMlEYbZZ8+e1T3geH799deyDdErxrOFKyoq0LVrV4neRoWHhwdiYmLQrVs32cBoK2GicM+eParK4APq4eGhBAnnMq0fy5cv11yjRefdd99VeMgFkAunl5eX7h2TmFFRUTrdkAfAMRnXo0cPPVNGh5ubG6KionDkyBEthiQznCsbN268JcG4bt06yWUMubmInT59WnOQHuf4+HhtbpRHvvnmGwD2zYWJXd7ToUOH6hA0Lsic85cvX0arVq1u69E1w2oTJkyYqALVYo7FxcU4deoU+vTpo1COVJZshYIy4CgR3LZtm8Jd7hAvvfQSAHvoyBCdhlwyGcBhLucRsJcuXdKuTwZUXFwsEZa7LYXe0aNHIzU1VWVZRkVZWRmuX7+OL7/8UvYS2hpoQ1iwYIESAyw7Cw4OFtNhgor34NKlS2J9DAu///57TJ48GYDDvEz861//Ekvl/XF3dxczZTUJWfyDDz6IpKQkw4fVJSUlSEpKwvXr1xUmk0lwjtarV0/SDIX9Jk2ayE7CcJevGTx4sKwjDOOOHTum1zPiYXjdvHlzsXImM4cOHSpJ6aGHHgIAMVtvb2/NaaPDZrPBarVizJgxknGYTOIYdezYUUezMvJr06aNpCDOeVa+1KtXT/ONZbFvvfWWmCCN9Hw2srKy8PHHH9/wu/Lycq0VlPgoWZ09exYnT568bURpMkcTJkyYqALVYo4hISF4+umn4erqqkQM2R5T68OGDZMGw9KdqKgorfjUDLi6JyUlSXeh9gg4GCaTC9SD3n77bZljP/30UwB2vahz584AbjT1AvaysNWrVxu+xK2goADx8fHo2bOnzNbUC8PDwwHYNTCOLdG6dWtdMy081H1WrFih2l2WdHbv3l1jQeZNLWjatGnaie+//34Adg2ZSQvqwLRTbN682fDjCthF+J49e2L37t3SvDl/WY4ZFBQkBs7k4QsvvKDrY+MCCvi9e/dWIouJM3d3dyUmmdwhSz958iTuvPNOAA7mbbPZMHjwYAAOuw41yoyMDMPr5ISvry/i4uKQl5cnJkgtj9eVlZUl3ZvaYG5ursaLc5GNKN5++22xeyYkLRaLmD7rzmkdunDhgsoMOdfDw8PFRJlk4/eLiIhAxv/X3rfHRV2m7d+cZgZBQM6KKJamgeIBUvOASqyYx9TWULTNymOZWWqWWq+67UZrWrxZtmpvqZmHtkzSPCYumKgoHhBRPHEQEZDTIIMwMr8/+FzXjDnr/ng/9dlvn/e5/kFxHJjn+xyu+7qv+35u3aLGaQ9N2hyrqqpkz549Mnv2bGaBUXOKTKbJZGLbIfjAunbtyg+NEA2h3xNPPEFPHTxhTz31FN30yEohjM/NzaWfD1msvLw8DiSSNPABbt26VUJDQ++rtNEaPD09Zfjw4ZKcnMxwGiEKHmxNTQ1DWFS1HD169J6MoIg1pOvduzcXGGSJ9PR0JszwcxD6xcTEUNJAUigqKoqheU5OjohYRfOwsDC5ePEiNxitAj7H8PBwVqIgc2obZuPgxnyMj4/n2COzj/B6wYIFDBmxBr799ltusNgAMC/T09NZ2YV14eTkxLUCiQO/V0hICImB1vsCmM1muXXrluTm5tLLiQMUIfSQIUN4yKNePTAwkJ8RNdZIVt28eZN7ABwUdXV1rKhBEhAHU+fOnfncIH2sXLmS6wVuAIx3+/btpU+fPvz97EGF1QoKCgp20OTa6urqajGZTGQfCDdAdwsKClhXiv54/fv3p/UEQj92+fT0dDJGdNjo1asXO8vAbzd58mQRabQ89O3bV0SstiCdTseabQjb8Ef6+/tLTk4OKb1W4ejoKG5ubqLT6ShEo/8dLCDu7u5ssYTky4gRIyhtoAoAjHDEiBHstQhr06JFi3jqItmVkpIiIo0nLeQLJANWrFjBsB3JLvx/FxcXOXr0KOeCVgGf48qVKynhoFcimMqQIUM4JxGq5eTkUJZAKIxwcebMmWSAGGODwcD5B0YDBhUcHMznBPZ56dIlMldISoiKampq5PHHH//1BuE3RF1dnRQUFMiRI0c4p7CWkRj19fXl9+D9rKioIGMEo8P8fv3119nhB7Yog8HApA78jbASDhkyhK8H4/Tw8KCVCElfWIz69esnq1atom/YHhRzVFBQULCDJlfIxMbGyt///ncmW3DaQqwvLS2l5oW63jVr1rA/IMRuaI4LFy6UBQsWiIiVHVZUVNCsiVMDyRcRua/KYfDgwazBRh0w2M68efMkMTGRJ7JWUVVVJbt27RJPT0+yFLBnjFm3bt1oMsaJm5eXRyEalgdogHV1dTTZwgKUl5fH0x2aLawr27dvpzaM782cOZOVHTiRoeMMHDhQ+vbtSwOzVlFdXS2HDx+W+Ph4JlTAGDA+x48fJ7tGtcbWrVtpKMb8xrzctm0bIxhUXVgsFo4NdFlERfv27aOeCx24qKiIlSR4f2hjbm5ujIa0jrKyMtm8ebNMnTqVpnlUCqFzempqKmvzkeT66aefGHXA5oRn0LJlS1qeMKZr1qzh+ODfoLPX19ezOgcJ27q6uvusP2Do1dXVMnToUDJ8e1DMUUFBQcEOmqw5VlVVidls5qkG8zfYQ3R0NDUbfNXpdEzp215+I9Jo7MRpgff84IMPuPtD48EpfevWLWbGcfdJYmIiTebQGMBMd+/eLV26dOHJrFW4uLhIUFCQbNu2jZ8FZZrI1tXU1PCuHGSOz507x7EEO8Qp+fjjj8uXX34pItYa87Fjx9LGAm0SY3306FEyUjyvcePGMfuH5wm7z+rVqyUuLu6BdggtwMXFRQIDA+XTTz+lRo4aaVjQ9Ho99T/oXxMnTmR2FFlNsHnbrjx4TefOnckU8QwR8YwYMYJzH2wxICCArgowLmiirq6udHZoHW3atJHExES5ePEi5yzWKK46SE9Pp/MEV32cPn2amiSse8h2p6SkcJ7BgXLt2jVmtzFnYdVZsWIFOyyBmd66dYtM85d9TQ0Gg+Tn5z8wovxfbY5TpkyhjxCLCSGXxWLhLw5K3aFDBwqoU6dOFRGruF9WVsZFh0G4c+cOBxWhIsKgurq6+24mdHFxoRcKYQ8+dFhYmOzYsYMWC63CwcFBdDqdDBs2TDZu3Cgi1rG1rXxBCIeFmZ2dzSolCNLw0x07duy+5NWJEyc4AfEMEXoMGTKEzT8Rvl+6dImSxYQJE0TE2mRk4sSJkpiYqPmWZVVVVbJ3715xcHCgpQkbFT5vSEgID2JsXo6OjpQssNlhnmdnZ9Obh3CuW7dutOLge7j/ZPDgwdwAbDdJ/CxIUSAItjdwah0Y327dutH2hc+Pg6a6uppSHJIioaGhbK6MQwFJm44dO3KDRaLx8ccf59xFJReIwL59+/j+2KBramroa4RUgp9dUFAgdXV1/P3sQdtHvoKCgsJ/CE1ijgaDQTp16iTFxcW0ysBUi909OzubBmU0Cu3Xrx+TADhtseN//fXXTLBg57979y6ZEa62hN3E19eXPxOXQe3fv5/sCkwAv0N9fb3ExMSw2kSrcHZ2Fj8/PwkKCqKdAVUWGCudTsfOOLCPVFZWsqoCIQYM8wsWLJBXX31VRKxtyRYvXkwWBNYC5u7v70/DMQTv8PBwPguEOfj/RqNRhg4dKuvWrfv1BuI3AGp/vby8yHqRjALr3bp1q8ybN09ErHOtZcuWNNjjM8LiZDAYOB5o3Hzw4EE+HzS7RdPl8+fPsx4eCbeuXbuSHeIZIEFx7tw5vq/W4e7uLn379pUrV67wM2ItQ27T6/WM3lDfXFhYyIQYGl1j7mZnZ3PuIsI5fvw4mTlYOJI14eHh/Nl4zWeffUaWjtZokFN8fX3/7c2ZijkqKCgo2EGTmKOLi4u0bNlS9Ho9u4dAk0F9bmBgIAV8sMkLFy7QYrN161YRsSZaXnjhBdbuosTPycmJ5lkwVNRLZmVl8TSwPQVg3QFDRZ2xv7+/XLx4kexLy7h7964YjUbaGlDWB0azc+dOnqZgPBMmTKCeiCbAeM3gwYOpwdpe4AT7FXRafC0oKKDOiXHPy8tjGSPuBl+7du0974mEhFbh6+srL774oly4cIE9MjG2YBmDBg2iXoY53a5dO9pPMJdhccrJySHDR5LG3d2dJmUwQCQIARU3AAAgAElEQVRcRKw18mCEYWFhZP1g7OiedOnSJTbC/T3AwcFBNm7cyIgNOQKMzZQpUxiBwJp28OBBllSCcS9ZskREGqNOFJNgXhsMBjJAaJR4fp9++in3GMzdtWvXUnPH8wbz7969u3z99dcPLA5pcsuy7OxsGTVqFP2HCNcQfhQWFnLRoAohPj6eGxk2NtxV8tVXX7FpBL4WFBTwbluE4UBeXh7vo0Fj0N69ezM5g9ASf7dYLLJ//37NV3E0NDRIbW2tXLx4keIxPHmYVM7Ozky2IIQwmUz8rEjWILOakZHBjQ31rGfPnuUY4TlhwsTExNxXUXD+/HlWzSDhA+miW7duYjKZHhiaaAFFRUWSkJAgjz32GBcaEgFYSO3atWMiEaH38ePHWaWCxYVF7O7uTucAkobPPfccNzQcxjhsPv/8c5kyZYqIWEPBW7dukVTgOaHrd319PdfIg+p/tYDy8nL59ttvZebMmTy0UUWFzTIxMZGbou3+gOw+Kl3gWzxy5Mh9vQVSU1PpNsDmi5Zknp6ebI6L11dWVjIpjP0KyMrKknbt2t3TYvGXUGG1goKCgh00iTnqdDoJCgqS8vJy3u+AMAAidocOHbiDw0fn4eHBkAIpfuzos2fPvs/+AAFWxFpNAD/a/v37KXLjjo0bN24wIYM0PvyXlZWVD0zXawU1NTVy4sQJuX37Nk9HjAMYhKurK+8Ex7gXFRUxJIN/DGFbfn4+ww9Yp/75z3+SBUHGQNLrb3/7G209qH/18vJiFQGqZ2C5Ki4uluTkZM1LFuh4VFhYSIsXQmiwXrPZzLkD9taxY0dafxCyIclYXV3NZAP6BtTX17M6CYkF2zAeNh38P09PT1p4IC0hpK+pqeGa0Tpw5/qRI0fIDhEpxsXF8e/4N8ytc+fO0a+LsYfsZnuPFLyJkZGR99w7LWJl7aGhobS0QQJxdnam9QfRDtbN+fPn5fDhw8rKo6CgoNBUNNkEXllZKevXr+euDjYBJrh3715aRGBTaNGiBWP+1NRUEbFeheDj43OP1ihi7SUoYj1lULExaNAgGjlxygQEBPDExvuCfa1evVpcXV0138r/7t27UlFRIUajkVojLnOCbScwMJBsAifoTz/9xDGCuI22/e7u7tR9wbZtdRZ0UsKzfPfdd2mRgH5pMBjI3lFBgoQP9CSto7q6WlJTU2Xx4sWMajBPkFx65JFHaKcBu4RxXMSa6IPWZTAY2LPRtrMPmDr6CoDhtGjRgmzV9vItjO0vG9vu3LmTyR+to6GhQUwmk9TW1nKd4lIsW2aO6AW6tsFg4LqE+RvzNTk5mXos2GFQUBDfF+sdxQplZWWsJEOSZfjw4dTj8dyxRh599FEpKChQmqOCgoJCU9Ek5ujg4CB6vV569uzJGB5sD8bOqqoqMhdcahUVFUXTLVgh9AGz2UyGCXPs6NGjyZCw40NHaNasGU9j/MyGhgZmo6GZwa4yaNAgGTt2LBmrVoG7fy9cuECdC2WEaLXv7+/P72E8o6Ojaf1BeR/Gx83NjTYbMB9PT0+yfGS8Mda5ubk8rW07q8Pqg7GFBurs7CwbNmygbqZV6HQ6CQ4OljVr1lD3gvkYhQQmk4msB/W7p06dIkOHTmubuYdeBj28rq6O9e3owQlWn5SUxPdF5JOVlcXsOfRlWNgiIiLYDUnrMJlMcvbsWXFwcODaRwSCblze3t6MNFBj3apVK1rNwJwxr/v27cvoBUx7xIgRLBvEvgOmmZGRwSw11kjnzp3pKMD1IrhOt127dvd01reHJm2OZrOZd0EgVY+FA1vN7du3GbKkpaWJyL2hH74ihb9w4UL+GRUhEyZM4IRDqh4YM2aMvPjiiyJitUkMHDiQixqX1y9fvlxEGjfo8vJyetK0iqCgIPnzn/8sx48fp2UG4QTE7fT0dFp5EL5duXKFIj5eh01sy5YttOQgfDxx4gSTVfDYwTuWmJhIqwrCoZ9++ombAEIfePdMJpPmr0gQaayRbtasmQQFBdGag8MTtqmff/6ZIRcO9Tlz5nCxYl5h/hYWFnLuw2fq7u7OxY5wHAs1ICCA8gXkiTfffJMNoX+ZwBk5ciSTFBh3raKmpkbS09MlJiaGnwONXpC827BhAzc2JAIHDx7M5hrwKWMPCA8P59yCx3nv3r28c2fatGkiYn0ely5dYgMUHGDfffcdD0FsgkgcHzhwQNzd3flM7EGF1QoKCgp20CTmWFlZKT/88INMmTKFZk+ECEgiVFRUMARGqGtre8BOjyTDTz/9xKQOwrvnn3+e3U/AWmDzOXDgANPzqG0tKipiKAo2iUqFJ598UkpLSx94QmgBt27dki+++EKGDBlCdgOZYcOGDSLSeJETTkV87/Dhw7TwoJ53zpw5ItIY5oHxwFY1depUshkwHmDLli08fcG0X3vtNdmyZYuIWI3htkzJx8eH4YxWodfrpX379tKrVy/W8CKcBlO5evUqEzEI2TZv3swwDCwRId7OnTvZDgvz0cnJiSwPVyKgK8+ePXsYIeEyrcuXLzPiAiNaunSpiDSaoSFdaR1ubm7Sq1cvcXBwYDEIKovAIENDQ2llQsL29OnTHF9EPVj3IlZpB/KPk5MTrw7B/EfPhLFjx9LQD1nJYrGQrWIvQlLM2dlZqqurHzh3FXNUUFBQsIMmX5MwfPhwOXfuHE81nHzYgYODg9mLERg7dizNy2COKGGLiIigLoFd/cCBA9RicBKBAUVHR1NjQwt7b29vnvoQtpEk6NChg2zfvp0pfa3C0dGRtblIWkEDA0uvqalh3TqSWGlpaWwYiqQLRP0uXbpwTFHjmp+fTxYNfQys0tZCBWP+7t27eSKD3cDy0r59e5kzZw5fq1UYjUY5dOiQODo6sjQQff5ggP/uu+8YBYFNvvXWWxwTMEIwkObNm1O/gm3MaDRy3uE9YFMrKSlhNyM0Gx4wYADtOjDSw6qSn5/PpM7HH3/8aw3Fb4K6ujrJz8+XXr16MUJEGSAiuNTUVK59RD87duxgRIhoCabxHj16UPtFUvHw4cOcz0h8IWq6fv06ixWgpffq1YuMHxERuiv16dNHdu3a9cALtpq0OSIr/P777zNhguoAiPq5ubl0nWOxbtmyha/HJodayri4OG6AqPR47LHHKEaj2gN/9/LyYniCzNixY8c4CJiAyEp6eHhIbm4uf65W0bp1a3nvvffkueeeo4sfnxNhRdeuXeW///u/RcSafVu1ahXDQIQyQGBgIMcW9/S0aNGCmx3eH0mJ7OxshtMI6T08PO6pdLJ9/enTp6WyslLzkoWjo6MYDAapq6ujNxE10MiI1tfXc9PCBlpVVcXDCOOIjGtRUREzpbZ17tgcFi1aJCLWrG1wcDATDMjMJiUl0XGBZ4Gvd+/eZXJD63Bzc5OePXuKh4cHxwIVdEj+tWrV6r4klZ+fH++NQoIM9e03btxgGA4i9dxzz/E9EF7jOZ49e5YbMaSMLl26kLShnhv7SVJS0r/tBK7CagUFBQU7aLKVp6KiQnQ6HZmY7R2xIo3eIpzAYJVlZWW8wwRdfCDAjh07lh1ScMq+8cYbDHtwemLn37JlCxMEYEqPPPIIw/yVK1eKiNXHFhoaKleuXNH87YPFxcWyatUq6dWrF8MCMDaEyYcOHZKnn35aRKyCtJeXF09pWIASEhJEpFH4/2Wr+ePHj9MagbAa4WRhYSHfHyF0s2bNKJ1A/gCb8vb2lsrKyt9N9VGPHj1YEYH5AG/js88+S+aByqurV69y3uGzI9QbN24cGTvmHhiU7ethVVm2bBlZKnypRUVFjIiQuEByZ8+ePXzGvwc4ODhIdXU19wGsPzBJEauFCaFweXk55x7CcPzdz8+PHlOsh7q6Ono/EYZj74iKiqLchiRMXV0dxxyJWiSOO3fufE8PB3tQzFFBQUHBDprczzErK0tCQkIopKKCBXachx56iKl61PPapuchruI0mD9/PnVCaD1JSUm0ScBQDrHV19eXzV9x6opYTxAIumChOp1OIiMjKZ5rFbdv35bjx4/L9OnTqZfaVguINOorYNRgLR4eHmTXqJDBKbl+/XpqODi1o6Ki+H5gRbBQeXh4sP4VJ3+PHj3IdNB8FML65s2bJTg4+L66YK3Bz89PXnrpJfn66695/QM0W1StZGZmUrTH3N65cyctYtAGERWtWLGCtiCI+v3796fmCG0LP6+4uJiMGz8zNDSUhn2YoG17ZkJf0zosFovU1dVJZWUl7V5IIqKaKjU1lfPStsEskjNg3NB2W7ZsybHE98rLy2mSBzNHnXZYWBhvekSUNHDgQO5PeG5AXl6edOrUifuQPSjmqKCgoGAHTdYcS0pKJCsri/E6zLSwc8TGxtLaMH36dBFpNCf/8Y9/FBGhRQfsKDExkUZvXMqzevVqls7B4oD3cnFx4WkAvahbt25knTghoElcv35d3N3dNX+3skijNvbpp5+SGaPMCrpJTEwMu0Ij64rMs4hVx4W+O2vWLJ7kyKw6OzvTWTBq1CgRsT67Q4cOkSlBL27dujWfMZgPtKC4uDj54IMPNN/PEVdP/PGPf+R4wCgMi05gYCAdD9C14uLiyMrxb+gJMGHCBLJsaF1///vfqRmivyhYZXZ2NpkgMqjLli3jM0OpLYzPc+fO5bM+dOjQrzYWvxUcHR3l5s2bZHRgZLiupH379uzqjwz17du3+RwQZYJxnz17lhEljPHp6em0SsE1AK22WbNmjBrxmo8++ojRFOY65rDBYJDo6Gj+3R6atDk2b95cYmJipGXLltwcEZKhauLmzZtMJCAF36FDB1YWYBJgElksFnnllVdExEqHAwMDGVLi9VjQXbp04Z/h7zt+/Dg/PKg1mitERUXJkSNHNL+AW7RoIc8884wcOnSIEwxeOYStp0+f5qaFxEJERAQ9nAhNvvzySxFprGjB5glZ4fLly7RIQPBHQqGsrIyTE3d5FBQU8Huot4Z3z8/PTyIjIx84wbQAk8kkZ86cEScnJ1rKbJttiDQmoGAvw+JKT0/nHMP3ICkMGDCA7wVrk8lk4thCuoB95+TJkwwnsRaGDRtGYgDZA9aUGzduaF6uACC3BQQE0L6Hr5CBnnnmGW78IEhxcXHcR2C/gc3n6tWrHDvY0D766CP+GYlGJCNTU1NpQ8MzCg8PZ7IQ+wieY6dOnSQ3N1dZeRQUFBSaCoemXCHg4OBQIiK5//aF2kRbi8Xi95/+Jf4V1Nj+dvidj62IGt/fEv9ybJu0OSooKCj8X4EKqxUUFBTsQG2OCgoKCnagNkcFBQUFO1Cbo4KCgoIdqM1RQUFBwQ7U5qigoKBgB2pzVFBQULCDJpUPurq6Wjw8PO7pqoFOG6gDdnJyYqkeOkQ7ODiwHhclUXh9aWkpa01RQ+zk5MQ6VJTJoS7TbDazYw9+jp+fH//8yzJBNzc3qa2tlZqaGrlz545m25w4OztbdDqdNG/enGVTtnXTIo3jgnIpdDfS6/UsH8TVCagzNxqN7K8HlJSUsG4dzwBfvb29WQaK9ywtLeX74lmgBEuv10ttba0YjUYxmUyaHVuDwWBxc3OThoYGzjXMTXx22zmK8rOqqirWVqPPIP7N3d2d6wAdaFxcXHiNAq4Kxly9devWfb0k79y5w39HDTZ8xz4+PpzzVVVVpVo2gTdr1szi6ekp1dXVLDVF2R7KKQ0GA/cKXLPRunVrlvyh9wFKWZs3b87nga+1tbUcV7weZZeFhYV8f+wxzZo143hiLLGvuLu7i06nk5KSEjEajXbnbpM2R0dHR2nTpo1ER0ezAWtoaKiIWAvHDx8+zBppTJ7OnTuz5hktyNBya/DgwfLFF1+IiHVBHjx4kJeBo3El2kht2bKFNxei/trJyYlttTD5UYN54cIFOXPmDBvkahW+vr4yceJE6dq1K5vVfvTRRyJibQnftm1bNpxAE9Hk5GTWOmPD3LZtm4g0Nr39ZWPg3NxctnpDMxAgIyODDXDRVOGRRx7hM0aTBtyB7ePjI15eXmwuoFXodDrp0aOHREZGcrGgMQIaGNTW1nIDxCEtYm1Vhs+4Zs0aEWlscYYNFgu2uLiYtdq46gItyHJzc9lkwfY+H4wt2nOhXtvT05ObwtKlSzVdfRIUFCQrVqyQxYsXs9UdblhEnX9CQgLvukc9dcuWLVlTjbFHu0Oj0ciNEo1mYmNj2boQfRtsxw8HGZ5HRkYGbyjFpg2kpaVJVlbWr3eHjLe3t4wfP17MZjN7rGFjA6P5wx/+IMOGDRMR6wVYeXl5vCwK/QUxeEVFRVzcuJjL39+f3U8wQdD3bvz48fddQWo2m9mbEP3z5s+fLyKN3UDc3d1F65VAJpNJzp07J1lZWdKvXz8RsW5C2PDLy8t5qID9Xbt2jZ8N3XXQ6dvX15dXhGIT8Pf359ijuQQwefJkTmAs0s8//5xNPMB8cKH6qVOnJDU1lUxSqwgICJA5c+ZIdnY2FwkWGebL+vXreXCjA09DQwOZHfoGYrOMiYnhJU9YA5s2beKzAyPCgj169Kj84Q9/EBFrswudTndf4wOso+zsbImOjv51BuA3hslkkszMTAkPD2cjDcwJ9HnNzMwkWULvyqSkJG6mIEZghEOHDuWeYTsXMT/B+HFgjxs3joRt3LhxItJ4gRe62GNcsY/Ex8fLqVOnVD9HBQUFhaaiScyxvLxctm3bJitWrGCohxANJ0BBQQE1GPS5CwgI4O1ioMhoWdazZ0/u7mhdlpWVJZs3bxYR6xWMuItjyZIlPI1wCri5ud3TO1LEeqdNbW2tjB07lm2StApXV1cJCwuT/Px86qboVYfPe/ToUTIfhL+Ojo5kdGA16BH43XffyZ/+9CcRsd7d88ILL3Asob+A5WRmZvJ9EZr06tWLfQXRhgrhu9FolOnTpzMU0irQZb1t27b83cG2bXVA9BiF5puYmMgWcWBCYInTpk1jz1Dcp3TlyhUy0YULF4qI9YZMPz8/9pJEOH769Gn2PsU1rOh07ebmxpsmtQ5nZ2cJCAiQgQMHci6AQUNCMJvNZI62d0xBDkOUCYaekZHBiNL2LiW0e8Ncf+utt0Sk8SYAyESQKJ544gmONb5iPaxbt44Rwr+CYo4KCgoKdtAk5ujj4yN/+tOfZMmSJbJ8+XIRsQquOPkuXLhAloamrfv372ejSzRMxWu++OILnrLIlJ4/f56nN5I7uNh82rRpZKvQZG7fvs2EzWuvvSYi1hPrzJkz8u2331Kj0yqcnZ3Fx8dHevbsSXaCUxRsMSYmhg1q0dj2ypUrbJCKjByydfv27SN737Vrl4g0Zg8hfoPhI8s3dOhQ3q6HLuteXl4cW3RqR7LGx8dHTp48eV9WXWswGo2yf/9+efzxxzkvoJkj6dWjRw8yNSQJgoODqZGDJYIJNjQ0UItt3bq1iDSOH+Y5sqhILHbs2JH/htfPmjWLXduhsSPR8PLLL3NcoY9qFbdu3ZIvv/xShg4dKqNHjxYRa4IFOvjixYt5rxMSqa1bt2YTWnSbBzPU6/V8RmB9J06coG77y/uxx4wZw/mM5GNmZiZvlIR2jPcKDg7+t7dmKuaooKCgYAdNvn0wOztb/Pz82KYcNwsii+Tt7U0bAzJKSUlJzDhh50ZW6u7du2Q8sAN16tSJ77t48WIRseqX4eHhZJ/IxEZHR1OvhI6AqxT69+8vPj4+ZEJahU6nk+DgYLlw4QLHDTfkTZw4UUQa2TBO1pCQEBFptD2NHTtWRETefvttEbFqgwsWLKCWg3F86aWX+DORnQXrP3jwILN3GP+SkhJ61fbs2SMiVp3Zy8tLdu7cyRNby3B0dJQdO3ZQa4RNBBrh9evXyS4wv8LDw8l24AAA08nMzKTnEWxv2LBhZKbQyzBWOp2OTgO8f2BgIG1DuFYBV2PU1tZS/9U64GJJTU0l2w0PDxcRq6ti+/bt1FeRb9i0aRNzA7Dk4Pnk5ORwrwBLXLNmDccE1kDYgg4cOMA9BvtIbGwsnzOiHkSgAQEBUlpaSr3dHpq0Oer1ennooYekdevWDCkgcOMSoOzsbHn99ddFxHqZzbx582gkxsOHfzE8PJybIwb2yJEjtJvg6sYnn3xSRBoHGx8YiYI2bdrQEoGwE3R727Zt4u/v/7sIq/38/OT69et8yEgCYMPPzMxkogQSxOjRo7k5wfqAhE5ycjLHD5teWloaxxnhB2wUgwYNoliO92zVqhXF9ZUrV4qINUH03nvvyYwZMyiKaxV+fn4yffp0SUlJkR07doiINYmChEtgYCBlIHga6+vruWEieZWamioijdfS4s6Z2bNni0hjMhJJFyw6XFhWUVFBKQQbwOXLl+XFF18UEbnvilZ3d3fZunXrrzkMvxnMZrOUlpZKq1ateHjbJqlEGj3PkIuwbnU6HQ8KSA7YLPfv308LGxKIb775Jl+Pgx1EbPbs2ZQfIFXk5OSQOCDRBQkpPDxczp49S/nOHlRYraCgoGAHTQ6rz58/L08++SSZBZgJ7DgHDhwgRQa7SUpK4qkMFoITpWfPnrThwIKi0+lIn8FMERaOHj2aCQiE0o6OjhRen332WRGxMp9jx47JQw89xFBVq6ipqZGMjAwZOHAgw1gYtxHKiQgTJhD8J02axJMYAGt+6KGHmJxBYmv69Om8+hXCOJJjL7/8MpkpKjvq6+tl5MiRImK9jB7PPDo6WjZt2sTfV6soKSmRTz75REpLS8kkIMkgIjl58qSsWrVKRIRmYpPJxCttkaxBQmbp0qW8wRDJh5CQEM4zSBVgmrGxsUxOzJw5U0REZsyYwWtgkXyAHOTv78/QXOtXs0Juc3Jy4u+PxAlsNQMGDKD9D4UdCQkJ8te//lVErNY02Knu3r17T3Qp0igrgTHi/VEoERYWxugULLG6uppVSbAQIgpbvny5TJ48mXuUPSjmqKCgoGAHTWKONTU1cubMGbFYLGR52K0hnn7yySc8ZcFaXn75ZQrbEFAhRH/++edMIED0Li4upvETpwAM0aWlpbTwIGlgm2yB6Ltv3z4REZk6dark5+c/8ITQAiorK2X37t1y/vx56dq1q4hYWQesH127diXTwaXxTk5ObDRx9OhREbHefSxiLaPE6btkyRKeujDs4vQ1m83UH8ESr127xtMXehKM+aGhoTJjxgzqS1qFp6cnbVHQvRBhQB/39PTk58JcW716NRkmrFDr168XkcZxmTt3rohY2V5ISAhrdTEm0N23bt3KcYc16uDBg0zq4FmAuefn5/N31joaGhqkpqZG4uPjqVlDt42NjRWRxmgGnw17hoi1kQe0Q8zvQ4cOUUPv0aOHiDRq5Khth26OsuGYmBgWmiAJ9uWXXzJJiaQmLIjPPvus6PV6asn20KTNsb6+XgoLC8XR0fGe+kURq49o7969DKsRduTk5Mg777wjIiJ//vOfRcQ6aeLj47n4sJkeOHCACxy+RSQDduzYwTAGm2lFRQXpNhz3SBrk5eXJ119/rfnQLygoSJYtWya7du3iw0XSCx67li1bMtzFRtivXz8eJKhpR3bv5s2bXKTYaC0WCzcGhO1Iko0cOZLvj9B88ODBDOUxsZBZbdWqldTV1Wm+br2yslJ+/PFH6d+/P8M3hLEYu9jYWC5aHDx6vZ7zFW4M1PO//fbbHCPIHqdOnWJCAeEhMtQODg5cvEgePvXUUyQVmLejRo0SkUZ/IA4qrcPd3V2ioqLkxIkTlAKeeuopEbEmQBwdHTm3kKwymUyUGuLj40VE2NWoefPmJAlITD388MOyc+dOEbHKPpAckpOTuV6w4dbU1HDu4jljzu/fv1+8vb2ZCLIHFVYrKCgo2EGTmGNAQIC8/vrrsn37djJFnIKoa0xPT5eSkhIRsdZQlpWV0UoCmo2U/LVr18hqIGKLWFP1EHSBXbt2SXJysohYT+cLFy4wyYAwHKe/m5ubPPnkk/SiaRUWi0Xq6+tl2LBh7JYDZoxOJ5WVlTJgwAARsXoZbTsYISmGtmY3btygPxQMpqSkhMkcSA1IELz//vtk8fA0rlu3jt7IGTNmiIg1WqiqqhIXF5cHnr5aQGBgoMyfP1/Wrl3L6hdICRifW7dukb2B/URFRTF5gGgF/2/w4MGUimA5GTt2LGUMzH08p27dutFmhojqvffeu6f2WsSaHPP29uZc1jrMZrMUFxfLpEmT2C4P8yciIkJEGucf5htC2Z49ezKKwfpE4tZisVAaQz1027Zt6WFEmIzQe+3atfL000+LiLXPQ11dHdcQwnFEB3jG2IfsQTFHBQUFBTtoEnMUaUy8TJs2jfYRaDLQrSIjIymWogvHvn37mIgB64C15OTJkzydIVhfvnyZhk4wTehaRqOReiVOmfj4ePnLX/4iIlZDLrSfy5cvS05ODn+GVnHnzh3Jzc2V4uJiGTNmjIhY7Tpgz2VlZTS4Qle5ePEiT2TYpJCgSk9Pp76D2tX09HQyUTynF154QURE5s6dS60XZuT169ff17wY9bNVVVWi0+lYyaNVoPb32WefJSuH1QuabKdOnTgOSI6kpKSwGAF6FqwgeXl5ZJOzZs0SkUZ2tGzZMhGxPgOMmW3nIkRd0dHRrBbB+oBN6tChQ+z7qHXU19dLUVGRHD9+nAwYcwys+vr161zLSGD17t2bzA1fU1JSRKSR7YNpIzJydXUl04Z+C5Y4depU5hXwDMaPH89mz3hGqBgzmUxy9uzZ+24OsIVijgoKCgp20CTmaDabpby8XHJzc3ni2VpsRBpN1zBnY+fv06cPs85TpkwREav1Jzk5mSVUwOXLl9lnD2V/6N/24YcfktXY1lVOmDBBRKyMAHrF7NmzZf78+ZpnjnV1dXLt2jWJjIykDQJWDoxBamoqtVtksNPS0qijIUOIcitfX18yH2hrnTp14rP6n//5HxGxZp+DgoKYPUTnnREjRlCvRBYc43/37l0xmUz/trvJfxpubm7Ss2dPyczMZAYY7fPRN/Tw4eWlNQgAABQASURBVMNkO9CzHnnkEXbyBosDO3njjTdYKgjm3LdvX1rc8P+Q5b569SqtZ7Bm5ebmysaNG+/5mXj2zz33HNcItE2twsHBQZydncXR0ZFrEhELmOOjjz5KwzuM9R9//PF9DBvRj5+fH+c95mvPnj3575jXuDLl4MGDZJF4tr6+vtQ8YcWC7c3BwUFmzZrF3IQ9NGlzNBgM0r59e/H29mYVwerVq0VEWKFSX1/P9D3S52VlZQyBIYwiZCwtLWU4gwn4zjvvUPxHyydMlN69e7OFFyaNr68vw2iEPxiUjIwMGTFiBENxreLOnTty5coVefjhh+WNN94QEevhAvvJjBkz6LtD+OLq6srmCQjzcBBNmjSJzwDYs2cPrUKYuPA0VlZW8mfBh9qmTRuG2vCi2V5J8f3332v+moS7d++K0WiUrl27ctNCdRASJidOnGBSESHwyJEjGaIh7IOMVF5ezgWKUHvhwoVMmOEwR7IqIiKCVigs9vT0dI4z3h9JxqKiogd68LQGBwcHOXDgAMkSZB9UpqSlpXFeo478/PnzrHxDuItEVu/evelBhaWnrKyMawKbKixZUVFRtBCiKU6vXr24H+A5gwhs375dysvLua/YgwqrFRQUFOzAoSkGXl9fX8vw4cMlNjaWF9tgZ0aYvW3bNrIaGLF37txJRgLrD8Lm6upqngx4j5s3b/J1CHUQFgcHB9PKA8a5du1ashqc2KjJ/sc//iFFRUVy7Ngxqaqq0uxR7OPjYxk2bJjo9XraG5CUgnE1MTGRJybYc/fu3ZlsQWNWCP4FBQU8GVF5kJ+fT1EbEgcqEOrr68neEX76+/vzPcCC8GxeeeUVyczMlOXLl0teXp5mx9bb29syePBgMZlMtIghZHvvvfdEpJElgnmDvYWEhHAcMF9hvp87dy6ZHcanrKyMdjSsDzTTDQkJ4bpAg2hvb292YIKdDT+7Y8eO/H02b958wmKxRP5Kw/Gro3nz5paIiAixWCxMqIDRIaK7evUqWTXmM8beFrDmDB06lBEKnlVRURH7MKCbDpjn0qVLyTqBGzdu8Hn8MlK4evWqbNy4Ufbv3y9lZWV2565ijgoKCgp20CTN0cHBQXQ6nWzbto27P5IBaCYZHh5O1oZyQFdXVwrPAOpSt27dyr6CMCBnZGTwBEJJHDSZf/zjH9RskKb38vJiEgLvBUYbGxsrCQkJmk/IODs7i6+vr3Tp0oVsBZYHW10ELM+2Uw9sDdAeoem888471ChhARo3bhxtVxDBUX8aHBzMJBC0mR9//JGJGOhJYOfl5eXi7u7OxsVaxd27d6Wqqkq8vLyoh6OHKDTZ7Oxssgp8b+3atRIXFyciVt0LXXcyMjL4Z9yRnpaWRraPhMR//dd/iYjIqlWryFphRO7fvz+fD34OoqJ+/frxd9Q60OfVtj8ibGJYt0OHDmUZIKxhHTt25JwCswP7O3r0KNk6GOSSJUvI1tHhB7mJ+vp6Pg/0zTx27BgTi0jEIFk8evRoMZvNDyx9bdLmiKyfwWDggkFiBb/Yli1bmMVDZm/AgAGsGcUCRjKgY8eOTKyAAr/22mtMoCDzhPf/7LPPGH5jcY8fP54DiLAT4Yler5f58+eztlurqKyslKSkJDl27BgbpMKDZ+uFQ/YTgvdjjz0m69atExFrdhsbwMWLF/k6NDs4cOAAM4poCWWbhEHYYtsU9tVXX73n9Vi03t7esnfv3gd6xbSAhoYGMRqNMmDAAM5bzDmEbM888wwPHNzUGBYWxsoNJLEQGsfExHCcEUKnpaVx8aFSBo1Q7ty5w4WIvgRnzpwhIUANMDLZmZmZrAbROoxGoxw8eFC8vLx4KOCzYpOsra1lwgv/Vl1dzSQNDnhIQpmZmTJ16lQRsc7njIwMjh3mHOa1baMKVNTY3t6IRA4OfTc3N/Hz8+M+ZA/aPvIVFBQU/kP4X12TUFpayma1CI+RFOjUqRN3elvBGowEiRM0HY2Li2OIDr/cuXPnGDrj5+BkXblyJU9UsMrTp0/TXwZmCmvR6NGjxWQykaJrFU5OTuLh4SHh4eH0geFUBIsGaxSxhiYIF0SsHk+wj5KSEt5pgpZl1dXVrDhAo1Gcnhs2bGCYh2c3aNAgvh9YERIyp0+fltraWs135WnRooWMGzdOWrRowZZiYDjosnPt2jUybzD2srIyWskwzmAeDg4OfA9cl6DX69n2DCE0wr7OnTtTbkJU4+fnR/kIzxrh+549e2g70joCAwNlwYIF/Kwi1so2yFlnz55lpRe8isXFxWwhh88N609dXR2TWRh7265USEgiUnzxxRfpo0R0sGvXLtrMYM/C75OVlSVPP/00/dD2oJijgoKCgh00iTmaTCbJysqSgoICaoAQUpF8CQkJue8WsB07dpD9QHuEHhkaGkoBHP/v+eefpzEZiRWwqdLSUpqScVIVFBTwPX55g9wPP/wgKSkpZJZahbe3t0ycOFGSk5NZcYETEyehTqcja0MHHtvrH2wTYCKNQjb0Sox3jx49aCXBSQ4W+tJLL5F5I7FQXFzM0xxGftu7nwcNGsS/axW4E/zWrVvsFwhmjOqfd999l0kr6FmDBg3iPIRNCuzv+++/p/4FE3hNTQ2NyGAkqFrq2LEjnw804l69epFF4j3AcIKDg/lvtsk3LaKsrEw2btwoQUFBTB5Cl8Y4DB8+nIUZSDqNGjWK7Bv6IKKTLl26MNqDQby6upr5DCSAkQDavn07mSP6K5SXl1NfR5SEzktGo1EWLlz4wH1BMUcFBQUFO2gSc6yqqpLdu3dLdXU16xhRF41OMrZ1tuhNt2zZMmalYEBG+WFsbCxPEpwyH3/8MWuHYV0BG9q8eTPrUJGtbt68OU9lvC9OEb1eLyNHjrzPIKo1uLi4SEBAgLRt25a6mG2XapHGemfog9Ba9Ho9M9Kw/oBxGgwGPg8YcPV6PZkjTlNYc+7cuXPPySrSqM1Bg/tllNC5c2dZt24d9WGt4ubNm/LRRx9JfHw8mQ16jk6ePFlEGucqLEnIMG/evJmmePRWRPmgt7c37WmIePbu3ctO1WCoeF7ffPMNNWN03ikqKmJUgGw4mKm7uzvdGx9++OGvNha/Berq6iQ/P18KCgqYU0DBALTBAwcOMPuM/qA///wzIx98D+4Xg8HAfQHj5ePjwygFr8P83rBhA68IRk/JiIgIOjHgQMBziYyMlDZt2jxw7japQqZly5aWF154QVasWMH2YmhCAIf/zJkzGUJjAlZUVPCXxIKE8F9ZWUm6jbCmWbNmpNewAsCbl5+fz3Adm8CiRYvYXBOWHiz4c+fOSX5+vnzzzTdSXFys2SqO9u3bW1asWCGbNm1i81lUB6EZgW1CAV65S5cuUcxHSIbQ2NY+Au/Xm2++KYmJiSJiDdcxkePi4ugXxe/w448/8u4PHIgIRdu3by8lJSUya9YsuXjxombHtkOHDpaVK1fK8uXLaffA5e5YnK+88goPVoyHn58fDxxsnAjrpk+fztsKcafR7t27GU5iEaLZRIsWLRji4TW7d++mzIT1gwWekpJCKWTPnj2arpBB5ZzFYuEdOyBNCHt3797NJhGoJ/fx8SFpwvyExSotLY19G5CseeaZZ3hQI4kLX7PRaOSaR5IrODiYhAtzGHP30UcflW7dusnatWulsLBQVcgoKCgo/P+iSWE1bhkLCgpiyIwTEszw+++/pwgN1peZmUmhGgkWsJzz58/zdLG9NeyX9wuDhQYEBPB1EHOTkpJ4GkNARziUkpIiNTU1DAW1ivLycvnmm28kIiKCjYSRXMKJm5SUxGQUkiSBgYFsMwbWjIRCfn4+689xdYKIVaqAGI1w7/Dhw2TxYDlhYWEMSWCDQO33q6++Kq1bt9b8NQkWi0UaGhqkR48elGLAgmFxOnjwIOcyQq3du3eTYWIeIsT95z//yRAdMkVERATlGyR8IAeZzWbaVjBH4+LiaE5GM1aYofV6/e/GymMymSQzM/OeEBVzFyxu4MCB7EoEqcHNzY2hNkJh2Hyio6NZdIDnEhISQuaH9YzxiouL436DZzp58mTKFCiseP/990Wkcd8ZMWLEAy1+ijkqKCgo2EGTrTynTp2SOXPmMPVuG/PjK8qpIFQfO3aMFxOhHhqMpnfv3tQToeu89dZbZClo7ood/uLFi2Sd0NpGjBjBZpZgodDkunXrJllZWfy+VuHs7CxeXl7SvXt32nVw+kI3vHbtGk9nsD8/Pz8+CwjXMMdnZGTwGcD4nZ2dTX0LyTHYSMLCwtgLDyd+nz59qCvDTgVN+fnnn5cbN26QHWkVpaWlsmbNGnF1dWVRARIsYM3t2rVj8YJtY1SMG8YUTGXPnj3UzqANe3l50ToFszLsUvYucrp9+zZZJMYdZZsuLi7sfap16HQ6CQkJoQYtYrXxYR17eHjweg1EP507d2ZUgiIRzM0TJ04w2sF8GzJkCPV0aMXokvTDDz+QkSIfkpGRwUQPok3YgqZNmyZ//etfH5iQadLmqNPppE2bNrJjxw5mP5FRwi+7evVqCv62hfO4BQzNQyHERkREMMTBZJg3bx4nL+g2PvBXX33FdkhLly4VEZG//e1vnNgI8UCxz507J+3atbvHD6hFmM1mKS0tlXfffZdCPzLSqL8tLi6m0I0QOikpiQeBbWWASGNYgaQVDoe//OUvzILj/4WEhIhIYw0rwhwsVh8fHz47JCcgmkdFRUmXLl00vznq9XouXDSXwGKBUJ+SksImBcj+v/rqq0wgYtyxmR47doz106gg+uyzzziWgO396TjQ0IAhPz+fcx9JHWS7W7duze/9smmL1tCiRQt56qmn5IMPPuCtfjgkMF6TJ0/mxo/PnJCQwJZjSLqABHXv3p2VRzicCwsLuUdAukPYvH//fia3EL6PGDFCunfvzn8Xsd5yuG/fPnn44Ycf2GVdhdUKCgoKdtAk5ujh4SGDBw+WK1eu8IQbOXKkiFjvNPn555/ZpBMncGxsLNPsYDWoaR0zZgy7vqBrx/z58xkOwseEU+f111/n+6N+OiYmhjWteA/4oFJTU6VNmzaar602m81SUVEhaWlpbHMF2QC3CyYkJPD1CNvOnj3LahkwE7D6/Px8hjUIuUtLS/m+YFE4PRMSElgFA9ZfXV3NLiewluBrdna2fPjhh5q/E9zT01OGDh0qeXl5lG4gWXz66aci0mgNAetDPe6qVasoFyEhgxB6zJgxHEcw/Hnz5tEnigoORDeTJk3ifSVI6nzzzTeMiNCmDmF2cHAwb9TUOmprayUnJ0eGDBnCSiJEE0jw5eTkMIGF9mzHjx9nVIJIEn/fu3cv/b2w7+zZs4d3RUGmwP5TVFTE2nhIfe+++y4TOPiKROPatWslISGBka89KOaooKCgYAdNYo64BEqn01E7hN0Amt6RI0doJcFJMXfuXF6WA1Ms2OLt27f5PSQiHn30UXZDwckDnUyn01Ggxemcnp5OZgozOCwVb775pqxcuZIns1Zx584duXjxojRv3pynJ6oNPvnkExFpNCDjBETd7aRJk2hPQLIBzGf8+PG0iERFRYlI4x3fhw4dEhGr4RgMUqfTcZxhiC4vL2cyDY2NIYKHhYWJ2Wxm0kGrqK6ulpSUFOncuTNNytCkMY+PHDlCCwg0RKPRyKogtPtHhcymTZs4x8AEly9fzggHOiF0Qz8/Pxr4wUbbtm0rK1euFBGrhQUmc5PJxJ+pdaBCZsyYMUweQtNFDsDR0ZHFDPhcr732Gucq2CGin9raWtqhpk2bJiKNfRjwerwv+nH279+fyRXsDw8//DCTw9CWkXw0Go2SnJzMZ2EPijkqKCgo2MH/Klu9efNm6n44BaBDHTlyhKwGu7yXlxcZzy/vRZ4xYwZ1H+iWN27cYPYKJwNqND/77DOav8EC+vTpQ2MyesBBk0hKSpIOHTrwZNIqkFHt3r07tVSUUCKD3LJlS7IVaDtlZWXUb6EdovflV199xRMcjDoyMpIaJdgTvkZGRrJ2Ffdc79q1i1170E0dGcb3339f+vXrp/lrEkwmk5w9e1YKCgqY5QR7RsTTvHlzMg5oqCdPnmTEAa0cncBTU1NZioguSpcvX5b4+HgRsV5VgeipqqqKV37AWJ6QkEAjPn4m+qJWVFSQoWsdjo6O4ubmJu7u7ry0De4RjOnmzZtptYHdZ82aNYwakSOA7vv222+zRBCvnzRpEucuXCyIUtu0acPxQtQzcOBAZqvhzEAU9MQTT4i/v/8DOx41aXOsqamREydOyKJFi+5zr2MTW7RoET1ySJ937NiRmxvS87A4XLt2jZsBWg116NCBGyYmJ4Ru27AQoXpeXh7DcDjnIXSHhobK7du3+f+1CgcHB3FxcZHLly9TLrANAUQabRGoNIKcYdvyDVVKmECFhYWcKLA/GQwGNluAZQhjk5WVxcmNw8RgMDB5AV8kknHXrl0TLy8vzVcfeXl5yahRo8TLy4sCPCqwkNhq0aIFD1RsgFFRUbSK4Hvw5NbU1HCeYwPw8PCgJIKNEEmVDh06cCPErXtFRUVsigtygYTBzp07ZcmSJSJivZ5Cq9DpdBIcHCwNDQ1sdQcvI8iQXq/nvMYh3qNHD0oZuB0TfseoqCjKEBj7NWvW8P8iTMbG6erqysQlNlU/Pz+SN1jTcJCbzWaJjIzkpmkP2j7yFRQUFP5DaFJXHgcHhxIRyf3tfp3fFG0tFovff/qX+FdQY/vb4Xc+tiJqfH9L/MuxbdLmqKCgoPB/BSqsVlBQULADtTkqKCgo2IHaHBUUFBTsQG2OCgoKCnagNkcFBQUFO1Cbo4KCgoIdqM1RQUFBwQ7U5qigoKBgB2pzVFBQULCD/wdqgltL/uCo7wAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "print(\"Weights with 600 data points:\")\n", "\n", "fig, axes = plt.subplots(4, 4)\n", "vmin, vmax = mlp.coefs_[0].min(), mlp.coefs_[0].max()\n", "for coef, ax in zip(mlp.coefs_[0].T, axes.ravel()):\n", " ax.matshow(coef.reshape(28, 28), cmap=plt.cm.gray, vmin=.5 * vmin,\n", " vmax=.5 * vmax)\n", " ax.set_xticks(())\n", " ax.set_yticks(())\n", "\n", "plt.show()\n", "\n", "print(\"Weights with 4000 data points:\")\n", "\n", "fig, axes = plt.subplots(4, 4)\n", "vmin, vmax = mlp_large.coefs_[0].min(), mlp_large.coefs_[0].max()\n", "for coef, ax in zip(mlp_large.coefs_[0].T, axes.ravel()):\n", " ax.matshow(coef.reshape(28, 28), cmap=plt.cm.gray, vmin=.5 * vmin,\n", " vmax=.5 * vmax)\n", " ax.set_xticks(())\n", " ax.set_yticks(())\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## QUESTION 5\n", "\n", "Describe what do you observe by looking at the weights" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##### [ANSWER TO QUESTION 5]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### TO DO 9\n", "\n", "Take the best SVM model and its parameters, you found in the last notebook. Fit it on a few data points and compute its training and test scores. Then fit also a logistic regression model with C=1. " ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "RESULTS FOR SVM\n", "Training score SVM:\n", "0.0\n", "Test score SVM:\n", "0.0796\n" ] } ], "source": [ "m_training = 5000\n", "\n", "X_train, X_test = X[:m_training], X[m_training:2*m_training]\n", "y_train, y_test = y[:m_training], y[m_training:2*m_training]\n", "\n", "# best parameters found in the SVM notebook\n", "SVM = SVC(kernel='rbf', C=10, gamma=0.01)\n", "SVM.fit(X_train, y_train)\n", "\n", "print ('RESULTS FOR SVM')\n", "\n", "SVM_training_error = 1. - SVM.score(X_train,y_train)\n", "\n", "print(\"Training score SVM:\")\n", "print(SVM_training_error)\n", "\n", "SVM_test_error = 1. - SVM.score(X_test,y_test)\n", "print(\"Test score SVM:\")\n", "print(SVM_test_error)" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "C:\\Users\\Anay Deshpande\\Anaconda3\\lib\\site-packages\\sklearn\\linear_model\\logistic.py:469: FutureWarning: Default multi_class will be changed to 'auto' in 0.22. Specify the multi_class option to silence this warning.\n", " \"this warning.\", FutureWarning)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "RESULTS FOR LOGISTIC REGRESSION WITH REGULARIZATION\n", "Training error (reg): 0.066000\n", "Test error (reg): 0.238400\n" ] } ], "source": [ "from sklearn import linear_model\n", "\n", "#lr = linear_model.LogisticRegression(penalty='l2', C=9999999, solver='newton-cg') # regularization not included\n", "#lr.fit(X_train, y_train)\n", "#\n", "#print ('RESULTS FOR LOGISTIC REGRESSION WITHOUT REGULARIZATION')\n", "#\n", "#training_error = 1. - lr.score(X_train,y_train)\n", "#test_error = 1. - lr.score(X_test,y_test)\n", "\n", "#print (\"Training error (no reg): %f\" % training_error)\n", "#print (\"Test error (no reg): %f\" % test_error)\n", "\n", "\n", "regL2 = linear_model.LogisticRegression(penalty='l2', C=1, solver='newton-cg')\n", "regL2.fit(X_train, y_train)\n", "\n", "print ('\\nRESULTS FOR LOGISTIC REGRESSION WITH REGULARIZATION')\n", "\n", "training_error = 1. - regL2.score(X_train,y_train)\n", "test_error = 1. - regL2.score(X_test,y_test)\n", "\n", "print (\"Training error (reg): %f\" % training_error)\n", "print (\"Test error (reg): %f\" % test_error)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## QUESTION 6\n", "Compare the results of Logistic Regression, SVM and NN. Which one would you prefer? " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "###### [ANSWER TO QUESTION 6]" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.4" } }, "nbformat": 4, "nbformat_minor": 4 }