{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Bevezető példa - Mnist MLP-vel"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1 Könyvtárak importálása, új sesson létrehozása és felkonfigurálása"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Most elrejtjük a GPU-t a Tensorflow elől, így a CPU használatára kényszerítjük."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import os #GPU-t elrejtjuk a Tensorflow meg a Keras elol\n",
    "os.environ['CUDA_DEVICE_ORDER']='PCI_BUS_ID'\n",
    "os.environ['CUDA_VISIBLE_DEVICES']= ''"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Szükséges könyvtárakat beimportáljuk, új Tensorflow session-t inicializálunk, melyet átadunk a Kerasnak"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "from keras import backend as K\n",
    "config = tf.ConfigProto()\n",
    "session = tf.Session(config=config) # Letrehozunk egy TF session-t a megfelelo mem. korlattal\n",
    "K.set_session(session) # atadjuk azt a Kerasnak"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Importáljunk be olyan köynvtárakat, melyekre a továbbiakban szükségünk lesz:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from keras.models import Model\n",
    "from keras.layers import Input, Dense, Flatten\n",
    "from keras.optimizers import SGD\n",
    "from keras.datasets import mnist\n",
    "from keras.callbacks import ModelCheckpoint, EarlyStopping\n",
    "from keras.utils import to_categorical\n",
    "from matplotlib import pyplot as plt\n",
    "from random import sample as RS"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Definiáljunk egy függvényt, mely beolvassa az MNIST mintákat és one-hot coding-ra alakítja át az elvárt kimenetetet (ez elengedhetelten, ha több osztályos osztályozási feladatot kívánunk megoldani):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def mintak_mnist():\n",
    "    (x_tan,y_tan),(x_test,y_test)=mnist.load_data()\n",
    "    x_tan=x_tan.astype('float32')\n",
    "    x_test=x_test.astype('float32')\n",
    "    idx=RS(set(range(y_tan.size)),5)\n",
    "    print('Mintakepek es a cimkeik: '+str([y for y in y_tan[idx]]))\n",
    "    \n",
    "    (f,sf)=plt.subplots(1,5)\n",
    "    f.set_size_inches(16,6)\n",
    "    \n",
    "    for i in range(5):\n",
    "        img = x_tan[idx[i]]\n",
    "        sf[i].imshow(img)\n",
    "    plt.show()\n",
    "    \n",
    "    y_tan=to_categorical(y_tan).astype('float32')\n",
    "    y_test=to_categorical(y_test).astype('float32')   \n",
    "    return (x_tan,y_tan,x_test,y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Hívjuk is meg a függvényt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mintakepek es a cimkeik: [3, 7, 5, 0, 5]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA6IAAAC8CAYAAABizBPxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3XmcVPWZ7/Hv002zg2EXEEFEEGMm\nalCjThK9xiWRkRjHJI5ynUTFmJjBaKJccyWaxZvrEmPGJeIyuDuOuzExLtfEaNQI6KiIAuIaEdwl\nqNDLb/6gcgf9PQWnuqpO1/n15/16+Wr64elzflV+uzi/rq6nLIQgAAAAAADy0tTVCwAAAAAAdC9s\nRAEAAAAAuWIjCgAAAADIFRtRAAAAAECu2IgCAAAAAHLFRhQAAAAAkCs2ogAAAACAXLERBQAAAADk\nqqqNqJnta2bPmNlSM5tVq0UBjYKMI3VkHCkj30gdGUeRWQihc19o1ixpsaS9JL0s6RFJB4cQnir3\nNT2tV+itfp06H5DVKr31eghhWLXHIeNoVF2VcfKNPPAYjtSRcaQua8Z7VHGOnSQtDSEskyQzu1bS\nNEllw99b/bSz7VnFKYGNuztc/0KNDkXG0ZC6KuPkG3ngMRypI+NIXdaMV/OruaMlvbTe5y+XakAq\nyDhSR8aRMvKN1JFxFFo1z4iaU4t+z9fMZkiaIUm91beK0wG5I+NI3UYzTr5RYDyGI3VkHIVWzTOi\nL0sas97nm0l65aNNIYQ5IYQpIYQpLepVxemA3JFxpG6jGSffKDAew5E6Mo5Cq2Yj+oikrcxsCzPr\nKelrkm6tzbKAhkDGkToyjpSRb6SOjKPQOv2ruSGENjM7RtLvJDVLujSEsLBmKwO6GBlH6sg4Uka+\nkToyjqKr5jWiCiH8RtJvarQWoOGQcaSOjCNl5BupI+Mosmp+NRcAAAAAgIqxEQUAAAAA5IqNKAAA\nAAAgV2xEAQAAAAC5YiMKAAAAAMgVG1EAAAAAQK7YiAIAAAAAcsVGFAAAAACQKzaiAAAAAIBcsREF\nAAAAAOSKjSgAAAAAIFdsRAEAAAAAuWIjCgAAAADIFRtRAAAAAECuenT1AlA7a/eZEtVe2L/Z7V12\nwIVRbWX7ard39znfj2pjfvynClcHAAAayetH7eLWH5p9blRrMf96ojW0V9X7iUuOcXvHzn7QrQNI\nB8+IAgAAAAByxUYUAAAAAJArNqIAAAAAgFyxEQUAAAAA5KqqYUVm9rykVZLaJbWFEOJpOcikechg\nt977pvgF/3sOfdrtPWxgPFygl7W4ve3Botqgpt5u7xFfvSOq3fWL0W5vx6pVbr2oyDhSR8Zro6lf\nP78+cEBUCwP7u71LT+0b1RZ9Zm7mNTSb/7Pl9tCR+Rj1MukP34hrx7/qN79S23OT8Q0IftkbKlRO\ntb0MJaoO+a6/5o9tEtXaJo9ze5ceHV+3n7/rVW7vnn3eq2pd33zpc279L0ePjWrh0YVVnateajE1\nd48Qwus1OA7QqMg4UkfGkToyjpSRbxQSv5oLAAAAAMhVtRvRIOlOM5tvZjO8BjObYWbzzGxeq9ZU\neTogd2Qcqdtgxsk3EkDGkTKuU1BY1f5q7m4hhFfMbLiku8zs6RDCfes3hBDmSJojSQNtcJlXIwAN\ni4wjdRvMOPlGAsg4UsZ1CgqrqmdEQwivlD6ulHSTpJ1qsSigUZBxpI6MI3VkHCkj3yiyTj8jamb9\nJDWFEFaV/ry3pB/VbGUJe+V7u0a1Hx91udu7X993KjiyPyG3Wt8ZtCSqXXnIPm7vsF+lM/2uqBl/\ne/ouUe2dCfGUZEmSOT8YdSYql3PoAf/PrZ84JJ7O1mLxJDnJn6J402p/ivTsaw6Japv9/gO3t/ne\nBW4d/62oGc9L88CBbj2M3yyqLT7M7336K+dVtYaV7e+79d+9Nz6qnbXo85mP2z7/Y279hEOvj2p/\nXhWfS5L+eN0Omc/Xx6m9/4n4fpRU06m5ZHzD5p9ygVtvDfHjdZP8fxu8x/ZKeifN869dHpgTD38d\nOieda4xaIN+19do34+snSZp29B+i2qyhd7u9Tc5zfB3yJ5dXO898ZG9/j/ByL/96qxFV86u5IyTd\nZGZ/O87VIYT4fT6A4iLjSB0ZR+rIOFJGvlFond6IhhCWSfpkDdcCNBQyjtSRcaSOjCNl5BtFx9u3\nAAAAAAByxUYUAAAAAJCrat++BRuw+KId3fqCfc+MagOberu91b6QuV5Wj/Hrw/JdRrfhDSD6wezL\n3N7tet0f1UY093J7K3lRfSW8I7SWGRjvnW9av9fd3mlHnBPVnjksHnYkSV//P9+NasOvftJfw6pV\n/uLQbTQPGhTV/vJvm7q983a8IvNxvXyf9OrObu8tv4+HXY76o//92OfmP8e9eirzusq55iejnKo/\nEGyU/lT1+dC1vGFx5eqVDJyrpPeskQ/5vbMfiGo7bDbT7R07myFGkJoGDIhqr31tW7d3zRfjQT8P\n7xhfY0hSs8XDt55c61/UTL/0O1Fti7kvur3V6ni7zEDTVY/X5Xz1wDOiAAAAAIBcsREFAAAAAOSK\njSgAAAAAIFdsRAEAAAAAuWIjCgAAAADIFVNza6R5m4lR7e69z3Z7+zf1qcsaZr0aT+m97Xf+dMZF\nh52X+bg3rR4c1cb/+1tub6NO+S26Nicy+/QtMy1N/oTcVE1q8aczPjT73Kg2cbuj3d6JR8cTSJGm\n5oED3bo3IXfejldmPu6tq+Opu5I06+ZDotr4E/wJn1vKnx4KVKJ50gS3Pv7Kl6Jauem2nibFk0PL\nHaNevXt/cZ7bu/TKLaNa++Jn3V4UX/OI4W697/XxVeit4+NrAancOwT4WVy0Nu49eb/pbu+Yp+Jp\n4m1uJySeEQUAAAAA5IyNKAAAAAAgV2xEAQAAAAC5YiMKAAAAAMgVw4oq1NSvn1tf9P0BUW1cj76Z\nj9ts/s8E7nivd1T7l9v+2e0d4cy5+Nxxj2c/X/BHDZ2xeJ+oNvTZeOgBGpf3QntJerFtk6jWGvyH\nhR9cGw9d6V+DGLy5bYhqZ3zhard3h16vRLVRPbIPZzr78/5xf7X9tKgWHl2Y+bgojqd/OtmtP7Pj\n+VUdd/bcQ936+NPiwRVAPa343DC3fv3I+PGvNfjDilpDe1QrN9jI6/3sqTPd3p1mPBrVzhn1QObj\nnjXSH+g186p4bc/G8xuRiFW7buHWb3IGEy0sc/3z5fvi4YXj5/rn67nir1Gt/anF5ReIzHhGFAAA\nAACQKzaiAAAAAIBcsREFAAAAAOSKjSgAAAAAIFcb3Yia2aVmttLMnlyvNtjM7jKzJaWPg+q7TKB+\nyDhSR8aROjKOlJFvpCrL1Ny5ks6VdPl6tVmS7gkh/MzMZpU+P7H2y2s8HatXu/XJxy2LahNnf8vt\nHTl5ZVT7y3ND3d6tL4wndW3Z9323d+0p70S18ze7z+1tDxbV1oQ2t/etRUOi2uDVSU0Lm6sGz/iw\n+e9Gta1/G098k6ReA9dEtc0uaHF7m+9dkHkNY/Vg5t5KxOmS5swc7/a+dXs8wfmP2/mTcD1f6PuW\nWz/u8P5RbatjMh+2COaqwTOelx5DP6j6GJOv+XZUm3DGPLc3ngmNOpkrMi5Jmn/KBW7dm5BbbhKu\n5873/XcNOPOY6VFtyB3+vxfPXhTXmv4SX4+UW1uT/F5v8u5ut3/N7R203xK33uDminz/f2uOeNOt\nNznPrx275CC3d+uZz0a19rfj62hJiuc3o1Y2+oxoCOE+SR/9Pz5N0mWlP18m6Us1XheQGzKO1JFx\npI6MI2XkG6nq7GtER4QQlktS6ePw2i0JaAhkHKkj40gdGUfKyDcKL8uv5lbFzGZImiFJvdW33qcD\nckfGkTLyjdSRcaSOjKNRdfYZ0RVmNlKSSh/jFz2WhBDmhBCmhBCmtKhXJ08H5I6MI3WZMk6+UWBk\nHCnjOgWF19lnRG+VdJikn5U+3lKzFRVU+1vxEJQJ330o89dPVDzsSJLad/1kVDvxiivd3s/2Xpv5\nfJ6P3+pPZ5n4/foMqWlwDZXxMH9hVJt4RBcspIsN/UY8SODUOz7l9p424vGo1lpmcsy+O/9nVIvH\nGCSnoTJeD83DhkW1trXZh7NsfV08lEiSJpzwSFQLHYyzaEDJZ/yFH+0S1VrDfLe3NWTPqNd7ytP7\nu72D74i/Hyox4ddHufWF+50X1coNV/LWG5yhjIlJPt/ltFzmjTmUln48HtZ418dvcHt/8oe/i2q/\nPv+zbu/QC7vldXAusrx9yzWSHpQ0ycxeNrPDtS70e5nZEkl7lT4HComMI3VkHKkj40gZ+UaqNvqM\naAjh4DJ/tWeN1wJ0CTKO1JFxpI6MI2XkG6nq7GtEAQAAAADoFDaiAAAAAIBcsREFAAAAAOSq7u8j\nmpoeo0e59fZR8QSvNYP9Edntx70R1c6e+O9u77Dm+6PayOY+G1piJpPv+3pU2+bHL7q9bVWfDaiN\n9hXxdPprHtnZ7T11v0ejWoc6/OOmP12xW7rtsd9FtQ75o5P/2hFPWxxzd5kpo0zIRc5enxFPx5Wk\nJw4/N6o1yX888ybOluv93vJ4eujgqYs3tMROGzzfvxRtmZp9vd5tMyszJh2F1/86/10pvnffAVFt\nwA2tbu+5m98W1U48Ob5ukKQzvrVdVHvw8B3c3jDvSbcOH8+IAgAAAAByxUYUAAAAAJArNqIAAAAA\ngFyxEQUAAAAA5IphRRvQY9zmUe218/wBRA9sN7eqczWV+V/RofgF+LUw/sx4BFHb8lfrci4AaHRL\n2lqi2pqB/uNvn379olrH6tU1XxPwN7scucCtt4Z4cJY3uKdcrzeUSJKeP2S0U322/AKrMOL38RA6\nSZp55G5R7ZxRD7i93m0LDKHrdtpeXRHV3opjJEna43vfj2o/OPIat3fW0P+Mi7c4NUl7HP+dqDbg\nWn+4EnhGFAAAAACQMzaiAAAAAIBcsREFAAAAAOSKjSgAAAAAIFdsRAEAAAAAuWJqboVO3/oGt96k\n6qazNVuZnwmEjqqOW87J/3FFVDttzwPc3rbnXqjLGgCgnu56v09U27PPe27v9j3jx+D7zjzP7T38\nX/aIag889Gm3d7N7sj+G93vunajWOii+DZLUdP9jmY+LYlnzhR2j2jmjLnR7vcn65a5HvGm681aO\ncXsHLV6yoSXWVPtifxrvvJVbRbWmUdlv2y6bPuf2Lp24ZeY1IF2jzvxTVLv89s+7vatv/GNUO3Tg\nS27vV06+I6rdOX8Ht7d9ybINLbFb4BlRAAAAAECu2IgCAAAAAHLFRhQAAAAAkCs2ogAAAACAXG10\nWJGZXSppqqSVIYRtS7VTJB0p6bVS20khhN/Ua5Fdpe35F6PawjWj3d7dej+f+bivt78f1b66aLrb\nu+q2kVFtxEPvur2nX39xVJvc0uL2frpX/IL/xd8c5faOPzHtYUXdOeNF0zxieFQ7eMeHu2AlxdJd\nM/7LaV+Kav/3Fx+4vXduc2Pm416y+b1x0atJ0lcyH1Z3vt8vqm3V8obbe9aKeKjGS1/f3O1tX/hM\n9kUUUGr5Xv3teGhVh4Lb2xrao5o3uKdcbwjVDVqsJ29tldwPZ418yO3d5bNTotqQBh9WlFrGG1X7\nIn9I1w377xrVxt1xi9t77KDno9q5R+3r9m75PYYVZXlGdK4k7x48O4SwXek/go8imysyjrTNFRlH\nuuaKfCNtc0XGkaCNbkRDCPdJejOHtQBdgowjdWQcKSPfSB0ZR6qqeY3oMWb2uJldamaDyjWZ2Qwz\nm2dm81q1porTAbkj40jdRjNOvlFgPIYjdWQchdbZjegFkraUtJ2k5ZLOKtcYQpgTQpgSQpjSol6d\nPB2QOzKO1GXKOPlGQfEYjtSRcRRepzaiIYQVIYT2EEKHpIsk7VTbZQFdi4wjdWQcKSPfSB0ZRwo2\nOjXXY2YjQwjLS58eIOnJ2i2psV1z8hfd+plT26Jaj9d6ur3jbo+nNvb5w6Nubx89F9X8mXHSB8Gf\nlJfV8HkdVX19SrpzxhvZK1+ZENVuGf5bt9ebHNla5ptn8exto1pPPVLZ4gqmO2Tcmxbb96hxbu82\nM46Jaj3f8SeKduywKvMaejwyIF6XP8xcJ0y/Pqrt3We123vu6Puj2sSZ33R7J87YwAITVeR8m8UP\nVE3ys+g9zlXSO3jq4gpXV3vNk+LHdUnaeUQ8sb8W98NOR8XXW8/GbzrQ8Iqc8aJpXxJPtz3y9iPc\n3mcOPD+qDdqal/eWk+XtW66RtLukoWb2sqQfStrdzLbTuj3R85KOquMagboi40gdGUfKyDdSR8aR\nqo1uREMIBzvlS+qwFqBLkHGkjowjZeQbqSPjSFU1U3MBAAAAAKgYG1EAAAAAQK46NayoO+t3w8Nu\nfeIN9Tlf++47RLU3j/OHV2zb8ufMx93lsa9GtWH3x4MBJCkewwTUV4/Ro9z6rv9zQVTrkD9kyxtM\ntPdTX3Z7e//+Cee4SFHbsufd+vhZfj1Pp9s/RrVDjv7XzF/fe1A8CA/FE0I8ZKejzJjC1tAe1bzB\nPeV6G8GKzw1z69ePvDqqdSj7bSt3P/z5wu2j2hA9uKElApFxt5a5Oj4w33UUHc+IAgAAAAByxUYU\nAAAAAJArNqIAAAAAgFyxEQUAAAAA5IqNKAAAAAAgV0zNbRBvHLGLW//7ox+Jamds6k/uVZlpcp4+\nFwyKam3Ll2T+eqCelu8/1q3fOOqWqo77/NIRbn3iB/7EaCBPTVWOKB/zc362nAKzeEJuk+JJupI/\nGbaS3kbw5o5+8Ku9bbs99jW3d8jFTMhF9T4Y2tLVS0gC/2oBAAAAAHLFRhQAAAAAkCs2ogAAAACA\nXLERBQAAAADkimFFddT88Ulu/elvfSyqLTvgAre3PXQ4Vf/F+p4H1vg/a+izfHVUi8cjALXTPGSw\nW3/ji/H3ya9O+GXV5/vtewOi2janr3B7q5wRA9RE2PGdqr7e2r1/L1A0IcT/xneU+Re6NbRHtXJD\nibzeF37kD0ocO7u6gT5rvrCjW1/97Tjji7f/ldtb7W3r96/xtRZQKyumrunqJSSBZ0QBAAAAALli\nIwoAAAAAyBUbUQAAAABArtiIAgAAAABytdGNqJmNMbN7zWyRmS00s5ml+mAzu8vMlpQ+Dqr/coHa\nI+NIHRlHysg3UkfGkaosU3PbJB0fQlhgZgMkzTezuyT9s6R7Qgg/M7NZkmZJOrF+S61M89Ah/l84\nUwWtX9/Mx112+Fi3fsxXb4tqn+5zhdv7dz3jqW/tzpQ8qfykPM+ja+Pb9r9OPNrt7T//4czH7QYK\nmfGiGfprfzbtTZtXPyHXc95WE53qC3U5VwGQ8ZKmvv7jfZi8RVybv7Dq8/XYIv43Y/U2w93ek7e9\nPvNxpy3+h6jWvOwVtzeeJ5qcpPK9akF8/dK0vX+N4E2RbSozWd/rXXj4eW5vx+HxtUe543rXKU1a\nUEFv9vXe+X4/t/eUp/ePaoPveMTtLaikMl6t5o9t4tbf2XtyVOt/3UNVn2/tPlOi2g27+d87LdYr\nqq1+cKjbO1iLq1tYAjb6jGgIYXkIYUHpz6skLZI0WtI0SZeV2i6T9KV6LRKoJzKO1JFxpIx8I3Vk\nHKmq6DWiZjZO0vaSHpY0IoSwXFr3DSLJ/xEvUCBkHKkj40gZ+UbqyDhSknkjamb9Jd0g6dgQwrsV\nfN0MM5tnZvNaxZu/onGRcaSuMxkn3ygKHsOROjKO1GTaiJpZi9YF/6oQwo2l8gozG1n6+5GSVnpf\nG0KYE0KYEkKY0qL496aBRkDGkbrOZpx8owh4DEfqyDhStNFhRWZmki6RtCiE8PP1/upWSYdJ+lnp\n4y11WWEGtv3Ho9rEi59xez8zIH5h8P793nJ7vRfQVzI8SIpfaF+pf3t3TFS78sWd3d7+//ROXHuD\noUQbU4SMN6q3p+/i1id9Kx7ycvnY+9ze1pD9FQLPtX0Q1Y6Y+V23t4/+nPm4qSPj/23pDz/p1vf/\nfPxY+eSnsh/3pf+9q1s/4dB4ANEhA5ZnPu4v3vKGbkl2cDyCqO31NzIfNyWp5Xvs7Aej2oQRR7m9\nC/eLB6Z4Q34kqTXEmSla75nHTHd7ExtMFEkt49V66Yj4ul+Stj/wyai24rrsx23b03/Q/8kFc6La\n5J7+tcs/Pvv5qDbu4qVubzcYJLdRWabm7iZpuqQnzOyxUu0krQv9dWZ2uKQXJR1UnyUCdUfGkToy\njpSRb6SOjCNJG92IhhDul8rM1pb2rO1ygPyRcaSOjCNl5BupI+NIVUVTcwEAAAAAqBYbUQAAAABA\nrtiIAgAAAABylWVYUcNbdtDAqHbbpo07LXZRa2tUO/vVvdzelf80JKr1Wfac28v0LdRC86QJbn30\n5fGkz4tGnuH2jmiOx8OXm47boY6o9ugav/e4k46PagNufsjtBZqHDYtqVxx0rts7ovn9qHb+gs9k\nPtdVw/zvhU2aemc+xgFLpka1tu8PdXvDq09kPi6Kb+zNfv3u//GxqLZf37+6vd50Wu/dAfLu/cQl\nx7i93vTgnkp7Oi6y6bPHa259WM84+8sO9N9posd78bXH6eef7/Z+smdce3Kt/y4ab546Nqq1rJjv\n9oJnRAEAAAAAOWMjCgAAAADIFRtRAAAAAECu2IgCAAAAAHKVxLCiRjXp7iP9+tkfRLWOx54qc5RV\nNVwR8GHNE7aIapOvXub2nuYOAIuHElXKG0x07Mnfdns3uZbBRMjOesSDUT5VNrJ9osppI+ZVcLbs\nQ4kOXraPW+84cE1UC28wlAhSr9/6Q3rO/s4hUe1HY8pc2jmzVR481R/e1Rri8YfeoKFyvZ89dWbm\nNYy9OB5KBGzI+/fGg+gk6bTvXh3Vmn7pf+94gxLLWbQ27j15v+lub8tTDCaqBM+IAgAAAAByxUYU\nAAAAAJArNqIAAAAAgFyxEQUAAAAA5IqNKAAAAAAgV0lMzd3ipHji2tSTPtUFK/mwrbTArWef0wXU\n10tfHhnVrhkRT51bJ/vDhTdhbvpjh7m9Qy/sF9U2uYPpuKhe24rXotpOP/2O27t2z3ei2mOfvjzz\nuSbd409Jn3xyvIaO1990eztWr858PkCSet4RTwQdUsHXT724PtdKQ8QkXNTPmIsXuvVtNosf358+\n6LzMxz38hb3c+ptHDI9q7U8tznxclMczogAAAACAXLERBQAAAADkio0oAAAAACBXbEQBAAAAALna\n6PQRMxsj6XJJm2rdnJ05IYRzzOwUSUdK+tskhpNCCL+p10KBeunOGR91+p+i2h+O9Edd7NM3Huby\nyQe+4fZudkFLfK57/eFdqK/unG91tEel4efHmZcknR+Xpir7IJdyw+naMh8BndWtM45ugYx/WPvb\n8fWIJE04Nh50OPXYSgZyvV1hHdXKMgazTdLxIYQFZjZA0nwzu6v0d2eHEM6s3/KAXJBxpIx8I3Vk\nHKkj40jSRjeiIYTlkpaX/rzKzBZJGl3vhQF5IeNIGflG6sg4UkfGkaqKXiNqZuMkbS/p4VLpGDN7\n3MwuNbNBZb5mhpnNM7N5rVpT1WKBeiPjSBn5RurIOFJHxpGSzBtRM+sv6QZJx4YQ3pV0gaQtJW2n\ndT+lOcv7uhDCnBDClBDClBb1qsGSgfog40gZ+UbqyDhSR8aRmkwbUTNr0brgXxVCuFGSQggrQgjt\nIYQOSRdJ2ql+ywTqi4wjZeQbqSPjSB0ZR4qyTM01SZdIWhRC+Pl69ZGl31mXpAMkPVmfJQL1RcY/\n7LytJvp1pzZWT9R3Maga+UbqyDhSR8aRqixTc3eTNF3SE2b2WKl2kqSDzWw7SUHS85KOqssKgfoj\n40gZ+UbqyDhSR8aRpCxTc++XZM5fJf8+RegeyDhSRr6ROjKO1JFxpKqiqbkAAAAAAFSLjSgAAAAA\nIFdsRAEAAAAAuWIjCgAAAADIFRtRAAAAAECu2IgCAAAAAHLFRhQAAAAAkCs2ogAAAACAXFkIIb+T\nmb0m6YXSp0MlvZ7byfPFbetaY0MIw7rixOtlvAj3U2dx27pel2Scx/AkFOG2NcJjuFSM+6qzuG1d\ni4zXH7eta2XKeK4b0Q+d2GxeCGFKl5y8zrhtSPl+4rZBSvu+4rZBSvu+4rZBSvu+4rYVA7+aCwAA\nAADIFRtRAAAAAECuunIjOqcLz11v3DakfD9x2yClfV9x2yClfV9x2yClfV9x2wqgy14jCgAAAADo\nnvjVXAAAAABArtiIAgAAAABylftG1Mz2NbNnzGypmc3K+/y1ZGaXmtlKM3tyvdpgM7vLzJaUPg7q\nyjV2lpmNMbN7zWyRmS00s5mlehK3r57IeDGQ8c4j48VAxjsnpXxL6WacfHdeShlPNd9S98h4rhtR\nM2uWdJ6kL0jaRtLBZrZNnmuosbmS9v1IbZake0IIW0m6p/R5EbVJOj6EMFnSpyV9u/T/KpXbVxdk\nvFDIeCeQ8UIh4xVKMN9Suhkn352QYMbnKs18S90g43k/I7qTpKUhhGUhhLWSrpU0Lec11EwI4T5J\nb36kPE3SZaU/XybpS7kuqkZCCMtDCAtKf14laZGk0Urk9tURGS8IMt5pZLwgyHinJJVvKd2Mk+9O\nSyrjqeZb6h4Zz3sjOlrSS+t9/nKplpIRIYTl0roASRrexeupmpmNk7S9pIeV4O2rMTJeQGS8ImS8\ngMh4Zt0h31JiGSDfFekOGU8uA6lmPO+NqDk13j+mgZlZf0k3SDo2hPBuV6+nAMh4wZDxipHxgiHj\nFSHfBUO+K0bGCybljOe9EX1Z0pj1Pt9M0is5r6HeVpjZSEkqfVzZxevpNDNr0brgXxVCuLFUTub2\n1QkZLxAy3ilkvEDIeMW6Q76lRDJAvjulO2Q8mQyknvG8N6KPSNrKzLYws56Svibp1pzXUG+3Sjqs\n9OfDJN3ShWvpNDMzSZdIWhRC+Pl6f5XE7asjMl4QZLzTyHhBkPFO6Q75lhLIAPnutO6Q8SQy0B0y\nbiHk+2y8mX1R0i8kNUu6NITw01wXUENmdo2k3SUNlbRC0g8l3SzpOkmbS3pR0kEhhI++iLrhmdnf\nS/qjpCckdZTKJ2nd76YX/vbVExkvBjLeeWS8GMh456SUbyndjJPvzksp46nmW+oeGc99IwoAAAAA\n6N7y/tVcAAAAAEA3x0YUAADGdJmIAAAANElEQVQAAJArNqIAAAAAgFyxEQUAAAAA5IqNKAAAAAAg\nV2xEAQAAAAC5YiMKAAAAAMjVfwHcI9Qe4P83ogAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1152x432 with 5 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "(x_tan,y_tan,x_tst,y_tst)=mintak_mnist() # tanito / teszt mintak lekerese\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2 rejtett rétegű MLP konstruálása"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Definiáljuk először is az MLP architekturáját: első rejtett rétegében 256, másodikban 128 tanh nemlinearitásos neuron szerepel.\n",
    "Továbbá lefordítjuk:\n",
    "- definiáljuk a tanításánál későbbiekben alkalmazott optimalizációt\n",
    "- továbbá meghatározzuk a veszteségfüggvényt, illetve a metrikákat is"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def get_MLP(x_p, y_p):\n",
    "    bem = Input(shape=x_p[0].shape, dtype='float32')\n",
    "    x = Flatten()(bem)\n",
    "    x = Dense(256, activation='tanh')(x)\n",
    "    x = Dense(128, activation='tanh')(x)\n",
    "    y = Dense(y_p[0].size, activation='softmax')(x)\n",
    "    \n",
    "    model=Model(inputs=bem, outputs=y)\n",
    "    model.compile(optimizer=SGD(lr=0.005, decay=0, momentum=0.9, nesterov=True), loss='categorical_crossentropy', metrics=['accuracy'])\n",
    "    return(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Definiáljuk a hálót tanító függvényt is:\n",
    "A Kereas beépített fit függvényét fogjuk meghívni, továbbá tanulás görbét is rajzolunk."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def tanitas(model, x_tan, y_tan, epn):\n",
    "    hst=model.fit(x=x_tan, y=y_tan, verbose=1, batch_size=32, epochs=epn, validation_split=0.1)\n",
    "    \n",
    "    plt.title('Tanito gorbek')\n",
    "    plt.xlabel('Epoch')\n",
    "    plt.ylabel('Loss')\n",
    "    epi=np.linspace(1,epn,epn)\n",
    "    plt.plot(epi, hst.history['loss'], 'b-', epi, hst.history['val_loss'], 'r-')\n",
    "    plt.legend(['tanito', 'validacios'], loc='upper left')\n",
    "    plt.show()\n",
    "    return model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Hívjuk meg az előző két függvényt, hogy létrehozzuk és betanítsuk az MLP-t."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "input_8 (InputLayer)         (None, 28, 28)            0         \n",
      "_________________________________________________________________\n",
      "flatten_8 (Flatten)          (None, 784)               0         \n",
      "_________________________________________________________________\n",
      "dense_22 (Dense)             (None, 256)               200960    \n",
      "_________________________________________________________________\n",
      "dense_23 (Dense)             (None, 128)               32896     \n",
      "_________________________________________________________________\n",
      "dense_24 (Dense)             (None, 10)                1290      \n",
      "=================================================================\n",
      "Total params: 235,146\n",
      "Trainable params: 235,146\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "None\n",
      "Train on 54000 samples, validate on 6000 samples\n",
      "Epoch 1/15\n",
      "54000/54000 [==============================] - 17s 311us/step - loss: 0.9600 - acc: 0.6812 - val_loss: 0.8286 - val_acc: 0.7263\n",
      "Epoch 2/15\n",
      "54000/54000 [==============================] - 14s 267us/step - loss: 0.8401 - acc: 0.7160 - val_loss: 0.6342 - val_acc: 0.7898\n",
      "Epoch 3/15\n",
      "54000/54000 [==============================] - 14s 268us/step - loss: 0.7300 - acc: 0.7557 - val_loss: 0.7128 - val_acc: 0.7708\n",
      "Epoch 4/15\n",
      "54000/54000 [==============================] - 15s 279us/step - loss: 0.7461 - acc: 0.7479 - val_loss: 0.6506 - val_acc: 0.7840\n",
      "Epoch 5/15\n",
      "54000/54000 [==============================] - 15s 269us/step - loss: 0.7790 - acc: 0.7308 - val_loss: 0.6627 - val_acc: 0.7772\n",
      "Epoch 6/15\n",
      "54000/54000 [==============================] - 14s 259us/step - loss: 0.7065 - acc: 0.7616 - val_loss: 0.6307 - val_acc: 0.7782\n",
      "Epoch 7/15\n",
      "54000/54000 [==============================] - 15s 284us/step - loss: 0.6835 - acc: 0.7618 - val_loss: 0.5988 - val_acc: 0.7917\n",
      "Epoch 8/15\n",
      "54000/54000 [==============================] - 14s 256us/step - loss: 0.6583 - acc: 0.7758 - val_loss: 0.5852 - val_acc: 0.7975\n",
      "Epoch 9/15\n",
      "54000/54000 [==============================] - 14s 257us/step - loss: 0.6604 - acc: 0.7755 - val_loss: 0.5807 - val_acc: 0.8013\n",
      "Epoch 10/15\n",
      "54000/54000 [==============================] - 15s 280us/step - loss: 0.7057 - acc: 0.7587 - val_loss: 0.6503 - val_acc: 0.7828\n",
      "Epoch 11/15\n",
      "54000/54000 [==============================] - 16s 296us/step - loss: 0.7156 - acc: 0.7567 - val_loss: 0.6728 - val_acc: 0.7723\n",
      "Epoch 12/15\n",
      "54000/54000 [==============================] - 16s 288us/step - loss: 0.7235 - acc: 0.7516 - val_loss: 0.5613 - val_acc: 0.8002\n",
      "Epoch 13/15\n",
      "54000/54000 [==============================] - 16s 291us/step - loss: 0.7329 - acc: 0.7492 - val_loss: 0.6439 - val_acc: 0.7785\n",
      "Epoch 14/15\n",
      "54000/54000 [==============================] - 16s 288us/step - loss: 0.6508 - acc: 0.7803 - val_loss: 0.5270 - val_acc: 0.8167\n",
      "Epoch 15/15\n",
      "54000/54000 [==============================] - 16s 303us/step - loss: 0.6748 - acc: 0.7692 - val_loss: 0.6381 - val_acc: 0.7677\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xd4VGX2wPHvoTcpAkoTgbXQBQw2\nlCLq2sCGCosFC4hYEHdVXHVV1NV1FRFFEfsqi7DgDysWFHsDlCIoioIYeu8IJOf3x5mEIaRMkrlz\nZ5LzeZ55EmYm954Zkjn3becVVcU555wDKBN2AM4555KHJwXnnHPZPCk455zL5knBOedcNk8Kzjnn\nsnlScM45l82Tgiv1ROQKEXkj7DhyIyKnisjCRP+sK708KbiUICJbom6ZIrI96t99i3NsVX1WVXtE\nzlNJRFREGsUncudSS7mwA3AuFqpaLet7EVkMXKmqU8OLKHgi4n+fLuG8peBKBBHpJCJfi8hGEVkm\nIo9kfahGXf33F5FfRGS9iDwS9bMDRSQrwXwS+bog0go5O/KcayI/u1ZEXhWRA/OJ5UoRWSIiq0Xk\nZhFZISLHRx6rLCKjRGS5iKSLyL9FpHzksVNFZKGI3CEiK4Eno455t4isE5FfReT8qPsri8gIEfk9\ncp7HRKRiHnHdJCJzRKRekd5kVyp4UnAlxS7gWmB/4ASgB3BljuecBrQHOgCXiUjXXI7TOfL1cFWt\npqqTReR04A7gHKAhsAZ4ObcgRKQdMBy4AGgUudWJesrdQFugDXAk0BW4OerxJkB54CDg+qj7KgD1\ngAHAiyLSNPLYI5FztAEOBw4DhuYS131AL6Crqq7ILXbnwJOCKyFU9RtVna6qGar6C/AM0CXH0/6p\nqptUdRHWImgX4+H7AmNUdY6q7sA+xLvnccV9ATBJVb9S1T+A29n776wvcKeqrlHVlcC9wMVRj/8B\n3KOqO1V1e+S+3cDdkfumAlOBXpGW0OXAYFXdoKobgQeA3lHHExEZBRwHnKSq62J8za6U8j5LVyKI\nSEvgYawVUBn73f48x9Oir5C3AdWITQPgw6x/qOoGEdmEtRpyXnU3AJZEPXeTiGyMxCjY1f5vUc//\nLXKc7BhVdVeOY66OJKPon2kQuZUH5tmhARAsiWQ5ALgM6KGqmwt+qa6085aCKymeBr4F/qSq1YFh\n2AdkYeVWNngZcHDWP0SkBlAdWJrLc5dj3TlZz60O1ABQK0m8IvpYQOMcx8nt/HVEpFKOn1kWOddu\n7DXXjNxqqGrtqOeuBM4G/isiR+VybOf24knBlRT7ARtVdYuItAL6F+UgkS6fjUCzqLvHAf1FpHXk\nw/lfwId59M1PAM4TkY4iUgFLTpk5jnWniNQWkQOA28hjfCJKeeAOEakgIicCJ2NdVLuA54BHRaSO\nmINE5OQcr+k9rJvpDRFpH9s74UorTwqupBgCXCkiW4BRwPhiHOsfwP9EZIOI9FTVN4H7gdexK/R6\n7D0OkE1VvwNuAv4PawEsx5LMH1HHng/MA2ZhXVwPFhDPYqxFsAJLApep6q+Rx26IxDQjcp53gENy\niestYCAwRUTaFnA+V4qJb7LjXHBEpBawDmigqsvDjse5gnhLwbk4E5GekfUD1bDpqV97QnCpwpOC\nc/F3PtbVk47NLCpWGQ7nEsm7j5xzzmXzloJzzrlsKbd4rU6dOtqkSZOww3DOuZQyc+bMNapat6Dn\npVxSaNKkCTNmzAg7DOecSyki8lvBz/LuI+ecc1E8KTjnnMvmScE551y2lBtTyM2uXbtIT09nx44d\nBT/ZxaRSpUo0atSI8uXLhx2Kcy6BSkRSSE9PZ7/99qNJkyZElRB2RaSqrF27lvT0dJo2bVrwDzjn\nSowS0X20Y8cOateu7QkhTkSE2rVre8vLuVKoRCQFwBNCnPn76VzpVGKSQkG2boX0dPCqHs45l7dS\nlRRWrIBt2+J/7A0bNvDEE08U+ef/8Y9/MHXqVABGjBjBtiCCdM65GJSapLD//lCmDKxZE/9jFzcp\nDBs2jJNOOgnwpOCcC1epSQrlykGtWrB2LWRkxPfYQ4cO5ZdffqFdu3YMGTKE7t2706FDB9q0acNr\nr70GwOLFi2nRogX9+/enVatWnHLKKWzfvh2Afv36MXHiREaOHMmyZcvo1q0b3bp1A2DcuHG0adOG\n1q1bc8stt8Q3cOecy6FETEmNdsMNMGtW7o9lZFj3UaVKUJjp9+3awYgReT/+wAMP8P333zNr1ix2\n797Ntm3bqF69OmvWrOGYY46hZ8+eAPz888+MGzeOp59+mgsuuIBJkyZx0UUXZR/n+uuvZ/jw4Uyb\nNo06deqwbNkybrnlFmbOnEmtWrU45ZRTmDx5MmeffXbswTvnXCGUmpYCQNmy1oW0a1dw51BV/v73\nv9O2bVtOOukkli5dysqVKwFo2rQp7dq1A+DII49k8eLF+R5r+vTpdO3albp161KuXDn69u3LJ598\nElzwzrlSr8S1FPK7ogdYuRJ+/x1atoQqVeJ//rFjx7J69WpmzpxJ+fLladKkSfZ8/4oVK2Y/r2zZ\nstndR3nxDZCcc4lWqloKALVrgwisXh2/Y+63335s3rwZgI0bN3LAAQdQvnx5pk2bxm+/xVStNtdj\nHX300Xz88cesWbOGjIwMxo0bR5cuXeIXuHPO5VDiWgoFKVfOZiKtXQuNGlmXUnHVrl2bTp060bp1\nazp27MiPP/5IWloa7dq1o3nz5oU61oABAzjttNOoX78+06ZN4/7776dbt26oKqeffjpnnXVW8QN2\nzrk8pNwezWlpaZpzk50ffviBFi1axHyMzZthwQJo0gTq1IlzgCVIYd9X51zyEpGZqppW0PNKXfcR\nQLVqNgMpnl1IzjlXEpTKpCACdevaKmdfJ+acc3uUyqQAewacg1jh7JxzqarUJoUgVzg751yqKrVJ\nAawLKSMD1q8POxLnnEsOpTopVKsGFSt6F5JzzmUp1Ukha8B5yxYoYHFxXFWrVg2AZcuW0atXr1yf\n07VrV3JOvY3FjBkzuP7664sVn3Ou9Cp1i9dyql0bli611sJBByX23A0aNGDixIlxPWZaWhppaQVO\nRXbOuVyV6pYCWLXUmjVtwDkzs2jHuOWWW/baT+Guu+7i7rvvzrWEdrTFixfTunVrALZv307v3r1p\n27YtF1544V51ka6++mrS0tJo1aoVd955Z/b906dP57jjjuOII47gqKOOYvPmzXz00UeceeaZAKxb\nt46zzz6btm3bcswxxzBnzhwAPv74Y9q1a0e7du1o3759dlkN55wreS2F/Gpn5+HgDNi+DTIqQZnc\nSmoXUDu7d+/e3HDDDQwaNAiACRMm8M477zBkyJB9Smjntffxk08+SZUqVZgzZw5z5syhQ4cO2Y/d\nd9997L///mRkZNC9e3fmzJlD8+bNufDCCxk/fjwdO3Zk06ZNVK5cea9j3nnnnbRv357Jkyfz4Ycf\ncskllzBr1iweeughRo0aRadOndiyZQuVKlUq1PvlnCu5Sl5SKIKyZUEiJbULs89Clvbt27Nq1SqW\nLVvG6tWrqVWrFvXr12fIkCF88sknlClTJruEdr169XI9xieffJI9FtC2bVvatm2b/diECRMYM2YM\nu3fvZvny5cyfPx8RoX79+nTs2BGA6tWr73PMzz77jEmTJgFw4oknsnbtWjZu3EinTp248cYb6du3\nL+eeey6NGjUq/It2zpVIJS8pFFQ7OxcCbF5uYwutW1sJjMLq1asXEydOZMWKFfTu3TvfEtp5xpFL\nK2LRokU89NBDTJ8+nVq1atGvXz927NiBqubZ6siSW10rEWHo0KGcccYZvP322xxzzDFMnTq10IX7\nnHMlU6kfU8hSp07xSmr37t2bV155hYkTJ9KrV69Cl9Du3LkzY8eOBeD777/P7v/ftGkTVatWpUaN\nGqxcuZIpU6YA0Lx5c5YtW8b06dMB2Lx5M7t3787zmB999BF16tShevXq/PLLL7Rp04ZbbrmFtLQ0\nfvzxx6K9aOdciVPyWgpFFD3g3LCh7dBWGK1atWLz5s00bNiQ+vXr07dvX3r06BFzCe2rr76ayy67\njLZt29KuXTuOOuooAI444gjat29Pq1ataNasGZ06dQKgQoUKjB8/nuuuu47t27dTuXJlpk6dutcx\n77rrruxjVqlShRdffBGAESNGMG3aNMqWLUvLli057bTTCvdinXMlVqksnZ2XjRvh55+hWTPbc6G0\n89LZzpUcSVE6W0ROFZEFIrJQRIbm8vjBIvKBiMwRkY9EJNQRz+rVoUIFL6ntnCu9AksKIlIWGAWc\nBrQE+ohIyxxPewj4j6q2BYYB9wcVTyyyVjhv3gwFjAk751yJFGRL4Shgoar+qqo7gVeAnHtJtgQ+\niHw/LZfHYxavbrDate1raa+HlGrdis65+AgyKTQEfo/6d3rkvmizgfMi358D7CcitQt7okqVKrF2\n7dq4fJBVqGADzmvWFH2Fc6pTVdauXeuL2pwrhYKcfZTbJPqcn9p/Ax4XkX7AJ8BSYHfOHxKRAcAA\ngMaNG+9z0EaNGpGens7qOA0GbN8Oq1ZZWe0qVeJyyJRTqVIlX9TmXCkUZFJIB6JLzDUClkU/QVWX\nAecCiEg14DxV3ZjzQKo6BhgDNvso5+Ply5enadOmcQs8IwOaNoUWLeDdd+N2WOecS3pBdh9NBw4V\nkaYiUgHoDbwe/QQRqSMiWTHcCjwXYDwxK1sWrrgC3nsPFi0KOxrnnEucwJKCqu4GrgXeBX4AJqjq\nPBEZJiI9I0/rCiwQkZ+AA4H7goqnsC6/3BawPfNM2JE451zilIjFa0Hp0QNmzoTffitaoTznnEsW\nSbF4LdX17w/Ll8Nbb4UdiXPOJYYnhXycfjo0aABjxoQdiXPOJYYnhXyUK2cDzu+8A0uWhB2Nc84F\nz5NCAa64wr4++2y4cTjnXCJ4UijAwQfDn/9sSWH3PsvqnHOuZPGkEIMBA2xXtsj+Ns45V2J5UojB\nmWdCvXrw9NNhR+Kcc8HypBCD8uXhsstsamp6etjROOdccDwpxOjKK61q6nNJUYjDOeeC4UkhRs2a\nwckn24BzRkbY0TjnXDA8KRRC//62XuG998KOxDnnguFJoRDOOsu26/QVzs65ksqTQiFUqGADzm+8\nYTWRnHOupPGkUEhXXmljCs8/H3YkzjkXf54UCunQQ6FbN1uzUFr3cHbOlVyeFIpgwABYvBimTg07\nEueciy9PCkVwzjlQu7avcHbOlTyeFIqgYkW49FKYPBlWrgw7Guecix9PCkXUv79VTX3hhbAjcc65\n+PGkUETNm0PnzvDMMz7g7JwrOTwpFEP//rBwIXz0UdiROOdcfHhSKIbzzoNateCpp8KOxDnn4sOT\nQjFUrgz9+sGrr9omPM45l+o8KRTTtdfaCucnnww7EuecKz5PCsXUrBn06GFdSDt2hB2Nc84VjyeF\nOBg8GNasgXHjwo7EOeeKp/QkhU8/hb/+FVTjfuhu3aB1a3j00UAO75xzCVN6ksLcuTB8OCxaFPdD\ni8D118Ps2fDJJ3E/vHPOJUzpSQpdutjXjz8O5PB9+8L++8PIkYEc3jnnEqL0JIWWLaFOncCSQpUq\nVj118mSroOqcc6mo9CQFETjhhED7dwYNstOMGhXYKZxzLlClJymAdSEtWgS//x7I4Q86CM491+oh\nbd0ayCmccy5QpS8pQGBdSGDTUzdsgJdeCuwUzjkXmNKVFNq0gRo1Ak0Kxx0HRx5pA84+PdU5l2pK\nV1IoWzbwcQURay388AO8/35gp3HOuUCUrqQA1oX000+wfHlgp7jgAjjwQJ+e6pxLPaUzKUCgrYWK\nFWHgQHjrLfj558BO45xzcRdoUhCRU0VkgYgsFJGhuTzeWESmich3IjJHRE4PMh4A2reHatUCX3o8\ncCCULw+PPRboaZxzLq4CSwoiUhYYBZwGtAT6iEjLHE+7HZigqu2B3sATQcWTrVw56NQp0MFmgHr1\n4MIL4fnnYePGQE/lnHNxE2RL4Shgoar+qqo7gVeAs3I8R4Hqke9rAMsCjGePLl1g3jwrbRqgwYNh\nyxZ44YVAT+Occ3ETZFJoCESvEkuP3BftLuAiEUkH3gauy+1AIjJARGaIyIzVq1cXP7IEjCsApKXZ\nFNXHHrONeJxzLtkFmRQkl/tyztzvA7ygqo2A04GXRGSfmFR1jKqmqWpa3bp1ix9ZWprtpZmAkqaD\nB8Mvv8Dbbwd+KuecK7Ygk0I6cFDUvxuxb/fQFcAEAFX9EqgE1AkwJlOhAhx7bODjCgDnnAMNG/r0\nVOdcaggyKUwHDhWRpiJSARtIfj3Hc5YA3QFEpAWWFOLQPxSDLl1sA4T16wM9TfnycM01MHWqDWM4\n51wyCywpqOpu4FrgXeAHbJbRPBEZJiI9I0/7K9BfRGYD44B+qgkqDtG5s9Wh+OyzwE81YABUquSt\nBedc8pNEfQbHS1pams6YMaP4B9q+HWrWtC3T/v3v4h+vAP37w9ixkJ5um/E451wiichMVU0r6Hml\nb0VzlsqV4eijEzKuAJZ7tm+3strOOZesSm9SABtX+PZb2Lw58FO1aQPdusHjj8Pu3YGfzjnniqR0\nJ4XOnW0BweefJ+R0gwfb/j6TJyfkdM45V2ilOykcd5yVvUjAegWAM8+Epk3h0UcTcrqE2L4dhgyB\n44+HTZvCjsY5V1ylOylUrWoL2RI0rlC2LFx7rU14+vbbhJwyUDNmQIcOMGIEfPEF3HJL2BE554qr\ndCcFsHGF6dNh27aEnO7yyy0XpfL01F27YNgwW/+3ZYttJjRkCIweDR9+GHZ0zrni8KTQubN9yn31\nVUJOV7Mm9OsH48bBypUJOWVcLVhgRWbvvNOqwM6ZAyedBPfcA3/6E1x5JWzdGnaUzrmi8qRw/PFQ\npkzCupAArrsOdu6EMWMSdspiy8y0mVPt21stpwkT4OWXoVYte7xKFXj2WVi0CG67LdxYnXNF50mh\nenX7pEtgUjj8cDj1VHjiCUsOyS493eK97jro2hXmzoXzz9/3eV26WEmPkSMTNqHLORdnnhTAupC+\n+gp27EjYKQcPhhUr4H//S9gpi2TcOFtj8fnnNmbw1lvQoEHez7//fmjc2MZOtm9PXJzOufjwpAB2\nifvHHzbgnCCnnAKHHWbTU5Ox0si6ddC7N/zlL9CihdUOvOoqkNwKokfZbz94+mn46Se4++7ExOqc\nix9PCgAnnGCfdgnsQipTxkpfTJ8OX3+dsNPG5J13oHVrmDQJ7rvPlnEcckjsP3/yyXDFFVZSKoF5\n1jkXBzElBRH5k4hUjHzfVUSuF5GawYaWQPvvb30kCUwKAJdeCjVqJM9itq1bYdAgOO00G0D+5hv4\n+99tfV9hPfSQ7VN9+eWpMW7inDOxthQmARkicgjwLNAU+G9gUYWhc2dbgbVrV8JOWa2aXVFPnAhL\nlybstLn66ito187GDW68EWbOtPH3oqpZE556Cr7/3lobzrnUEGtSyIzsj3AOMEJVhwD1gwsrBF26\n2AK2mTMTetprrrHyS08+mdDTZtu5E26/3dYe7Nxpi88eftj2fyiuM8+Evn3hn/+0MQnnXPKLNSns\nEpE+wKXAm5H7ygcTUkg6d7avCe5CatYMeva0q+oETn4CYP58W5V8331wySW2EK1r1/ie49FHrXfu\nsssS2ghzzhVRrEnhMuBY4D5VXSQiTYGXgwsrBAccYNNsEpwUwKanrlkD/01Qh1xmJjzyiNUtWrIE\nXn0Vnn/exjfirXZtW4/x3Xc2zuCcS26F3nlNRGoBB6nqnGBCyl/cdl7LzcCB9sm8bl3RRleLSBWO\nOMImQM2aVfC0z+JYssTKbEybBj162PTRAw8M7nxZzj8fXn/dXl+LFsGfzzm3t7juvCYiH4lIdRHZ\nH5gNPC8iw4sbZNLp0sU23ElwB7iITU+dMyeYKt67d8PUqbYlaOvWNk30mWfgtdcSkxDASmRUq2az\nkTIyEnNO51zhxdp9VENVNwHnAs+r6pHAScGFFZKQxhXABmRr147f9NSMDHsZgwZBw4a2duCVV2z8\nYvZsm/UUZIskpwMPtPIXX32V2hVinSvpYk0K5USkPnABewaaS56GDa3UZwhJoXJlGDDArt4XLy7a\nMTIzbVbt4MFw0EE2aPzCC/Z14kRYtcqK2DVrFr+4C+Mvf7EZSbfdBgsXhhODcy5/sSaFYcC7wC+q\nOl1EmgE/BxdWiLp0gU8/tU/YBLv6art6HzUq9p9RtUVmf/0rNGliU0ufegqOOcZaBqtXw/jxcN55\nlnjCJGLrIMqXtxLbIbzFzrkCxJQUVPV/qtpWVa+O/PtXVT0v2NBC0qULrF9vq64S7KCD7MP7mWfy\n35NA1WbzDB1qDZujj4bHHrPB6pdeshbBq6/afgdVqyYu/lg0bAjDh1tjbPTosKNxzuUU60BzIxH5\nPxFZJSIrRWSSiDQKOrhQdOliX0PoQgLr+tmwAf7zn30f+/57uOMOK73doYNN8TzsMHjuOduw5403\n4KKLrBp4Mrv8ctuY5+abi95V5pwLRkxTUkXkfaysxUuRuy4C+qrqyQHGlqtAp6RmOfhg6NjROuIT\nTBWOOsq2uZw3D37+2bp/xo+3xWZlykC3btYKOOccqFMn4SHGxeLFNhPquOPg3XcTO+jtSqfMTKve\n++23Vrjg229tYeWECbZ/ekkX65TUWCfj11XV56P+/YKI3FC00FJAly5WKlQ14Z9WWdNTL7nEWgQL\nF9p9J5xgYw3nnZe4aaRBatIEHnzQynw8/7y1HpyLl9274ccf904As2bZxRZAxYpW+fejj6yl3b9/\nqOEmlVhbClOBF4Bxkbv6AJepavfgQstdQloKzz5rI6Hz54ey0uqPP6z8RKVK1iLo1cv64kuazExr\n9cyeba2ikvgaXfB27rQ/1egEMHv2nk2eqlSxYo8dOtjtyCPtz7pcObv+++EHa5HXLDl1n3MV75bC\n5cDjwCOAAl9gpS9KpuhxhRCSQsWK9otd0pUpY/m3bVtbTP76696N5PK3Y4eNrUUngDlz9pRn328/\nq+571VX24d+hg7W48+oeevRRe96wYTYBwhWhzEX2D4rcoKoj4hxPgRLSUlC1y9YuXWw/Sheo4cNt\nSu3YsbaWwTlVWLbMWpDz5u1JBN9/b11DYFf20Vf/HTpYl1CZQm4dNmCAdWHOmVOyS7DE2lIoTlJY\noqqNi/TDxZCQpADQp4/VnEhP98vXgGVk2PqKn3+2boCSMGbiYqMKy5fv+fCfN89+B+bNg40b9zyv\nTp29P/w7dICmTePzp7l6NRx6qK3tmTKl5P65x7v7KNdzFONnk1+XLrb665dfCrcXpSu0smVtsK99\ne7juOpsN4koWVVixYu8P/6wEsGHDnufVrg2tWlmLsVUraNnSvh5wQHCx1a0Ld95pm0u9/TaccUZw\n50oF3lLIy/z59tv4zDNWKMgF7p//tBIYEyfaLCuXelRtzUxuH/7r1+953v77259X1i36wz+MK/Vd\nu2xsKyPDuqgqVEh8DEGLS/eRiGzGBpb3eQiorKqJqy8dkbCkoGr9GKeemvtKMhd3u3bZ6uylS+1D\npHbtsCNyBVmxwqruRt/Wrt3zeK1auX/4H3hg8nXTvPOO7U/+73/D3/4WdjTxF/iYQlgSlhTA5oLO\nmJH4Zbeq8K9/WSW7Y45J7LlDNns2pKVB795WssMlj40bbcbPN9/sSQC//26PlSljH/YdO1q5lawP\n/3r1ku/DPz9nnmlDiT/9ZLGXJIkYUyj5unSBSZPgt99slXOijBsHt95ql8rffWdFkUqJI46Av//d\npgheeKH9kbrE27HDFntlffh/8w0sWLDn8T/9ySYHdOxotw4dkq/OVlEMH24r7W+7zaZLl0beUsjP\n7Nm26uXFF22JcSJs3AjNm1tCWLLEfkM/+qhkdnLmYedOm2Wybp31R5f0RUVhy8iw7rroFsCcOXum\nftarZx/8Rx1lX9PSSnbX3k03wcMP2/uRVuB1derw7qN4yMy0uXDnnmsDzokweLCVPP3mG1i0CC64\nAG64wTZVLkWmT7ees27d7OqtbduwIypZvvzSBvSnT7cuoW3b7P4aNeyDMDoJNGyYWl1AxbVpk01R\nPeQQ+OyzkvPaY00KqGpgN+BUYAGwEBiay+OPALMit5+ADQUd88gjj9SE6tlT9ZBDEnOu775TLVNG\n9eqr99w3eLAqqP7vf4mJIYmMGqVarZq9/J49Vb/+OuyISobx41XLlVOtWFH1mGNUr79e9aWXVH/8\nUTUjI+zoksOzz9rv3dixYUcSP8AMjeVzO5YnFeUGlAV+AZoBFbC9nVvm8/zrgOcKOm7Ck8LDD9vb\ntHRpsOfJyFA99ljVunVV163bc/8ff9hf7n77qS5YEGwMSWjtWtW77lKtVcv+G04+WfXjj8OOKnU9\n+6xdd5xwguqGDWFHk7wyMlSPPFK1YUPVLVvCjiY+Yk0KhVwQXihHAQvVNuTZCbwCnJXP8/uwp+Be\n8kjUvs0vvGBt+gcftHl8WSpUsNVcFSrYbKisdn4psf/+trDot99sQtbs2Tb+37kzvPeeTdRysRk5\n0pbcnHSSTb+sUSPsiJJXmTL2fi1dCg88EHY0CRZL5ijKDegFPBP174uBx/N47sHAcqBsHo8PAGYA\nMxo3bhxIFs3Trl12lT5wYHDnWLNGtXZt1eOPz7v9/s47qiKql1yimpkZXCxJbutW1ZEjVRs1spZD\nWprq5Mne7VGQ++6z9+ucc1R37Ag7mtTxl79YN9uiRWFHUnwkQUsht+GZvK7regMTVTUjtwdVdYyq\npqlqWt26deMWYEzKlYPjjw+2pfD3v9ta/yeeyLua15//DP/4hy2kK61z5bAyyNddZ/tMjBljM5TO\nPtsmib3yis2kcXuo2uzm226zXfkmTLAqvC42//qXlWEpiYvZ8hJkUkgHoifYNwKW5fHc3iRj11GW\nrKLrq1bF/9jffANPP20767Rpk/9z77gDTjkFrr3W1i+UYhUr2sYoCxbYIrfdu62GYcuW1hO3a1fY\nEYYvM9MS6AMPWCnpF1+0axwXu0aNLKlOmgTTpoUdTYLE0pwoyg1bGPcr0JQ9A82tcnne4cBiItNj\nC7olfKBZVfWLLzSQGUC7d6s1soj/AAAdLElEQVR26KBav77qxo2x/cyqVdZ30qyZ6vr18Y0nhWVk\nqE6cqNqunf1XHXyw6hNPqG7fHnZk4di1S7VfP3sv/vrXUt3jWGzbtqk2aaLatq29r6mKsLuPVHU3\ncC3wLvADMEFV54nIMBHpGfXUPsArkaCTU1qa9Vt88kl8jzt6tBWJf+QRqF49tp+pW9f6AJYsgX79\nfKQ1okwZK6L37bfw5ptQvz4MGgTNmtnbu3Vr2BEmzs6d1mp64QW4+26r5VNS5tqHoXJleOghW9D3\n9NNhR5MAsWSOZLqF0lJQVT3pJLtUiJcVK1Rr1LDjFuUybsQIuwz897/jF1MJkpmp+sEHqt262dtU\np44Ntpb0aZjbtqmedpq95ocfDjuakiMzU7VrV5sPsnZt2NEUDWGvUwjqFlpSGDbMZv/E6zfikktU\ny5e3FUNFkZmp2quXatmyPnG/AJ9/rnr66fbbXqOG6u23p+4fdn42bbIPLhHVp54KO5qSZ/ZsW+Nx\n3XVhR1I0nhTi7eOP7e2aPDl+x7r11uIdZ+NG1UMPVa1XT3X58uLHVcLNnKl63nl7ksN995WchUnr\n1qkefbRdI5SkVbjJ5uqr7T3+/vuwIym8WJNCkLOPSpajjrIpL8UdV9i1yzq7Dz4Ybr+9eMeqXt2m\nRWzcaFtVZVUwc7nq0MHq/cyZYxPKbrvNqn2OGrVn4/dUtHKlVVn/7jv7dfB9roNzzz32ZzdkSMkd\nzvOkEKtKlaxCW3HXK4wcaaU/H33UBq+Lq00bePJJmy93553FP14p0KYNvPYafPEFHH64zfBt3hxe\nfjn11jn8/rut7l640AbYz8qvZoArttq1bfD+/ffh9dfDjiYgsTQnkukWWveRquodd1inYlFHK9PT\nrcLbGWfEf45g//7WL/Lmm/E9bgmXmak6ZYpq+/b29rVpo/r666kxhXPhQpt6W7266qefhh1N6bFz\np2rLljYrPFGrw3ftUn3xRdWVK4t+DLz7KABdutiKoM8/L9rP33ijdfGMHBn/OYIjR9rO9xdfnPid\n4lKYiO24OmOGrYjesQN69rRF7PGegRxP8+fDCSfAli3w4YcWb+j++AMmT7ba0yVY+fLW0P/11+Ar\n2mdkwNixtovdpZfaAsTAxZI5kukWakth61abMXTLLYX/2XfftUvRYcPiH1eWhQttBDUtzQvcFNHO\nnTZzp0ED++867TSraJ5MZs60qZH16iXJgOemTaoPPmiLMEF10KCwI0qIs85SrVo1mALKGRmq48ap\nNm++pwX76qvFq/GFzz4KyHHHWSnrwtixw2YJHXJI8EtsJ0+2/9boPRlcoW3bZp9zWSW7e/dW/fnn\nsKNS/ewz6y46+OAkiGfVKpvfW7OmvUndu9utatWSvyBE7RqsQgXVSy+N3zEzMlQnTFBt1cre0lat\nrJBCPAo+elIIytChtkNJYeYy3nuvvdXvvBNcXNFuusnO9/LLiTlfCbZ+veptt6lWqWL/7VddFfzW\nGnl5/32L47DDVJcsCScGVVX97TebrF+5si2KOPdc1W++scdmzLDfvREjQgwwcYYOtZf71VfFO05G\nhuqkSdYiANUWLVRfeSW+1X89KQRlyhR7295/P7bn//qraqVKNkE+UXbutF1UqlRRnTcvcectwZYv\nV732Wus9rFRJ9eabE7sAbvJkuypt08YWw4di3jy7LC5Xzm79+qnOn7/v84491lrFpaCe+aZN1mt2\n9NFFe7mZmfZ/m1Wz67DDbJ3J7t3xj9WTQlA2bbIZSLffHtvze/Sw5nSiL+2WLlU94ADrlNy8Of7H\nz8y01dhPPql6/vk2o2ru3PifJ8n8+qvqxRfbBXKiFsD997+2YOqoo0Jaif3116pnn20fF1Wq2Bax\n+f0+jxtnz33rrcTFGKIXX7SX++KLsf9MZqbqG2/Y7m5gOfQ//wm24F6sSUHsuakjLS1NZ8yYEW4Q\nHTtalayCpqe88YZNZXnwQbjppsTEFm3aNNtm64IL4L//Ld6MJ1VYtMiOOW2aTXlZvtwea9TIpu1s\n2mSTuP/2txJfo3nuXFv89sYbcOCBVtW8f3/bIC+Lqk3I2brVZgnF+jXn959+amsR3ngD9tsvQS9Q\nFT74AO6/3/6va9a0OtzXXw916uT/s7t22eLMI46AKVMSE2+IMjPh2GNtzciCBfn/H6narnd33gnT\np1vBxjvusL0ugv6TEZGZqppW4BNjyRzJdAu9paBqtYgrVLDRyLxs3Wr1dlu2tO6csGRtuTVqVOF/\ndskSu/zp189GNu13WvXAA23kdcwYG+3MzLRBx1697PGjj1b94Ye4v5Rk9Pnnqp0728uuV8/mEzRo\nYIPBZcvuectiuZUrZ2O2DRuqHn64VVXv3NnmDGzdmqAXlFWDPC3NgqpfX/Whh6yFXBh3320/X9Ta\nXinmq68038o1mZk2AfHoo+15TZrYftmJ/GjAWwoBymoBfPSRrV3Ize23w3332VV1166JjG5vmZkW\n63vvwWefWbmOvKxYYa/pww8t7oUL7f7ate01dOsGJ55oy3/zanWMH29lPLZutdd/ww22dVUJpgrv\nvgvPPWcvtWpVqFat8F+jWxkJt3OnTYj/17/scveQQ+Dmm+GSS4q2VdvKlXDQQTBwoK2hKQUuvdTW\nusyfb+VTwH43PvzQNk384gto3Ng+Gi69NPH/395SCNK6ddapfPfduT++YIG1JC66KLFx5WXtWrvS\nb9zY9oPOsmaNXRVec41Nd8i6ZK1RQ7VnT9VHHlGdNavwI2jLl9skblDt1En1p5/i+nJcHG3ZYv/P\nWZtet2unOn58fEY6L7rI9jePdQOpFLd0qQ0fnn22/XvatD2tyEaNbPgtzOVD+EBzwI44QvXEE/e9\nPzNT9eSTrf8gmSqXTp9uierEE1VvuMHiz0oCVauqnnqqTcyfPj0+HwiZmaovvWT9IZUrq44cWSpm\no6SMdevsoqZ2bfsd6NzZZtbFs77H11/bsUeOjN8xk9z999tLzup9a9BA9fHHk2MtqSeFoF1/vX3Y\n/fHH3vdPmJC8fwhPPGGxVapki4zuvde2Gg2yYzM9fc+uL1272vQdF64tW2zQAmx23OefB3euo4+2\neZal5IJg+3Z7a+vVU3300eTaDtaTQtAmTbK3L/oPatMmuzRo1y55N3P98cfEX7ZkZtqo2n77WUHA\n0aNTo+JcSXXttfa7O2VK8Od6+WVN6MLNJLB1677Xiskg1qTgBfGK6oQT7Gt0Ke2774Zly6yUdbJO\nyTz88KINHBaHCFx+OXz/vZUfHzgQ/vxnm8PnEuuDD+Dxx2HwYKsEGLTzz7c5u8k42Lxjx55p1XFU\npUrIkwaKyZNCUdWtCy1b7kkK338PI0bAlVfaB5/bV+PGNgtq9GibitG6tU3Z0dSaAZeyNm6Eyy6z\nC4P770/MOStUsIuAKVP2zGZLFpdcYptrbNsWdiRJxZNCcXTpYmW0s3ZTq1EjcX9sqUoErrrKVn91\n6ABXXAFnnmktLBeswYPtff7Pf2zxZaJcdZXN1R01KnHnLMi0afC//8HatTaN2mXzpFAcXbrYktMb\nb7Rlpw88UPBqT2eaNrWujJEj7Q+0VSvb+sxbDcF47TUrxn/rrfmvVQlC/fq2qv655+zvJWwZGbZ+\npkkTazU9+WTYERUsI8MWN3zxRfDnimXgIZluSTPQrKq6bJlmT+ssakUsZ6uiO3Wy9/Gss0Ks+FZC\nrVpldbDatw9vBPTLL7XIK+vjbfRoi+V//7MpQmDVXZPZa6/tibmI8NlHCXLooVYg79tvw44kte3e\nrfrww6oVK9rc+VdeCTuikiEz00pbV6gQbsHCzEybvN+8ebgzz9ats9+vLl0sjvXrbWp5//7hxRSL\n7t1tBVwxZjXGmhS8+6i47rnH+krbtw87ktRWtqx1w82aZTUCeve2LofVq8OOLLWNHQuvvmq/p61b\nhxeHiBXT+/FHmDo1vDiGDYN162xSiIgV+uvTxwpGbtwYXlz5mTfPulqvuSYxsxpjyRzJdEu6loKL\nv127bGlohQqqdetaKQ5XeL//biVLOnUKpkB/Ye3YYd1YPXqEc/4ffrCqgwMG7H3/N99Yp8njj4cT\nV0EGDrQW9OrVxToM3lJwKatcORg6FGbOtGmsvXrZ1dzatWFHljpUbWbXrl3wwgvJUZSwYkUYMADe\nfNN2vU+0G2+06oP33rv3/WlpNhNu9Ojkm+iwfr3NFuvbN2GTWDwpuOTVujV8+aV1fUyaZOtC/u//\nwo4qNYwebWtCHnrIKp4mi4EDw5me+vbbtlbiH/+wNUbRRODqq22tUSJm9xTGc8/ZOorrrkvYKb10\ntksNc+ZAv37w3Xfwl7/YVNbatcOOKjktXGgb3Bx/vO3oUpzNlYLQu7fFlZ5uNcODtnMntG1rrYC5\nc3NfbrxlCzRsaGXmX3op+JhikZEBhx5qm1gVtKFXDGItne0tBZca2raFr7+2UiITJlgr4vXXw44q\n+WRkWPIsXx6efTb5EgLYVe/GjbYuJRFGjbI9IoYPz7v+RLVqcPHFtqBtzZrExFWQt96y3Q6vvz6h\np/Wk4FJH+fLW/J8+3erpnHWW/SGvWxd2ZMnj4Ydtlf3jj9sVZjI67jjrw3/88eD78FevtguJU0+F\n00/P/7kDB9r+qS+8EGxMsRo50v4Pzz47oaf1pOBST7t28M03ttHtK69Yq+HNN8OOKnxz59qGv+ee\nawOTyUrEWgvz5tlq9iDdcYftAjh8eMGtptatrcvtqadsx8IwzZ9v01AHDUp4cU1PCi41VagAd91l\nyaFOHejRw7pNNmwIO7Jw7NxpBd5q1rRB5mTsNorWu7f9vwVZPXX2bHj6aZvf36JFbD8zcKCNyXz4\nYXBxxeKxx2y2Vv/+CT+1JwWX2tq3hxkz7Irw5ZethtLbb4cdVeLdc48t/BszZt/ZNcmoUiWbnvrG\nG7B4cfyPr2r1jWrVshZlrM47zyYwjB4d/5hiFcI01GieFFzqq1DBVqp+/bV9CJxxhu3fUFpaDd98\nY9V5L73UxllSxdVXW4vmiSfif+xXX4WPPrI1CbVqxf5zlSpZefHJk8Or3Pv88wmfhhrNp6S6kuWP\nPyxBPPCAVed85pnEbCYTlu3brbW0bZuNKdSoEXZEhXPBBVb2Ij3ddqeJhx07rLtov/3g228L3ye/\ncKFNBR02zFqgiRTnaajRkmJKqoicKiILRGShiAzN4zkXiMh8EZknIv8NMh5XClSsCPfdB199ZR+Q\np51mGx8la12b4rr1Vptu+fzzqZcQwK6G16+3Gk3xMny4dUmNGFG0QdpDDoGTT7auuN274xdXLEKa\nhrqXWGphFOUGlAV+AZoBFYDZQMsczzkU+A6oFfn3AQUd12sfuZht3646dKhVsW3USPXdd8OOKL4+\n/NBq9lx7bdiRFF1mpuoRR6i2aROf6qnp6apVq6qec07xjpO1B/vrrxc/psI46aRiV0PNC0lQ++go\nYKGq/qqqO4FXgJwdnv2BUaq6PpKgVgUYjyttKlWyvvYvvrDFSX/+sw1ubtoUdmTFt2mT9X0feij8\n619hR1N0WdVT587de7/zorr1Vqv39NBDxTtOjx7QoEFiN+CZP9+60kKYhhotyKTQEIjemT09cl+0\nw4DDRORzEflKREpw568LzdFHW3mMm26yVb5t2ljXwoIFyVcALVZDhsDvv9sslXj1xYelTx+b8fPY\nY8U7ztdfW4mKv/4VmjUr3rHKl7dux3fese6cRAhxGmq0IJNCbhOlc/4FlsO6kLoCfYBnRKTmPgcS\nGSAiM0Rkxmqvr++KolIlePBB+Owz2H9/+1Bt3tz2brjmGlv8tnVr2FHG5o03rFDaLbfAMceEHU3x\nVa5sH8CTJ8OSJUU7Rmam7UFdr561FuLhyiutJfP00/E5Xn42bLAE/5e/hL6lb5BJIR04KOrfjYCc\nc7zSgddUdZeqLgIWYEliL6o6RlXTVDWtbirMwXbJ69hjrdXw669WE6d1aytr0KOHXa2ecgo88oht\nBpOMrYg1a+xKsm3bws2/T3aDBtnXok5PHTvWWgoPPGCzjuLhoIPgzDOtdblzZ3yOmZcQqqHmKZaB\nh6LcsFbAr0BT9gw0t8rxnFOBFyPf18G6m2rnd1wfaHZxt2OH6vvvq954o2qLFpq973aTJrbByWuv\nqW7eHHaUNhB7/vmq5curzp4ddjTxd+65qvvvr7ptW+F+bvNm1QYNVDt2jP8+6VOm2O9CkNvD7t6t\n2rSp6gknBHcOjX2gOdBd0oDTgZ+wWUi3Re4bBvSMfC/AcGA+MBfoXdAxPSm4wC1erPrkk6o9e9pM\nFrBd4Lp3V33oIdV588LZZ/i//7VY/vnPxJ87ET76yF7fM88U7uduu81+7osv4h9TRoZdHHTtGv9j\nZ3n9dYt/woTgzqGxJwVfvOZcfnbutHGIKVPsNm+e3d+4sS2KO+006N49fl0WeVm2zLq6Dj8cPv00\n1NkpgVG1fSDKlLEuvljqNy1aZAvVevUKrhT3Aw/YOMUPP9g4VLydfLJ1V/76qw1wByQpFq85l/Iq\nVIATT4R//9t25vrtN6ui2aGDbfZ+zjk2FnHiiTb9dfx4m1r50082bTQeF10a2Vpzxw4bjCyJCQH2\nVE+dPdsScSxuvtl2cnvggeDiuvxy+7B+6qn4Hzt6GmqACaEwvKXgXFHt3GlrILJaEXPn7vucKlVs\nRkz9+vl/PeCAvPdRHjMGrrrKpixee22wryls27ZZiYfu3W3Dm/x8/DF07ZqYchR9+tj01KVL4zsF\neNAgG2ROTw981lGsLQVPCs7Fy/r11s2zYgUsX57319wK9ZUpY9VNcyaLOnVsY6FjjrE9l8uUgsb9\nzTdbqYpFi2wGUG4yMuDII+09//FHm9YapKwE9PzzVqI9HjZssC1AL7zQEkPAYk0KJbQd6lwIatWy\nW6tW+T9vxw5LEPklj7lzYeVKq71Tp459aJSGhAB29fzww1a++r77cn/Os89aN9P48cEnBIDOnW08\nYfTo+CWFkKuh5sVbCs4lq8xMWLvWuiuqVg07msQ65xwbV/j9d1t4GG3DBivv0aKFXcEnakOhRx+1\nPRq+/dYq0xZHRgYcdpiV0vj00/jEVwAfaHYu1WV1KZW2hAB29bxmjW23mtM991iyfPTRxO4wd8kl\n1iqJxwY8b79ts43CrIaaB08Kzrnk062bdcONHLn3DK4FC+y+K64o/tV6YdWqZduIjh1b/KKKjz1m\nA+pnnx2f2OLIk4JzLvlkTU/97jub4ZXlxhutO+3ee8OJa+BAq5FVnP0f5s+H99+3neeSZBpqNE8K\nzrnkdNFFULPmnuqpU6ZYt8sdd8CBB4YTU8eO1kJ58smir0F5/PGkqIaaF08KzrnkVLWqdRNNmmQ7\nqQ0ZYgPMYfbDi9gV/ty58OWXhf/56GqoSVrc05OCcy55DRpkM3X+/GcbTxg+3FaZh6lPHytrUpQB\n5+eft+6nJJuGGs2TgnMueTVrZuWrf/rJypqfcUbYEdkufhdfDBMm2CyoWGVkWNfR8ccnfpC8EDwp\nOOeS29ChtrJ5xIjETkHNz8CB8Mcf8OKLsf/MlCk2DTWJWwngi9ecc65ojj8eVq2ybq1YktUpp9jM\no0WLQpl15IvXnHMuSAMHws8/w4cfFvzcH36waahJVA01L54UnHOuKHr1srLpsQw4J/k01GieFJxz\nrigqVYLLLoPJk62QYV42bLCxhz59knYaajRPCs45V1QDBlgl2/xKX6fANNRonhScc66oDj0UTjrJ\nNkLKyNj38axpqJ062W59KcCTgnPOFcfAgbBkiU05zSlrGmoSVkPNiycF55wrjp49bae83AacR460\n3dXOOSfxcRWRJwXnnCuO8uXhyiutWN9vv+25P2saapJWQ82LJwXnnCuuK6+0BWxjxuy5L2sa6oAB\n4cVVBJ4UnHOuuBo3trpMzz4LO3fCxo0pNQ01WrmwA3DOuRLh6qvhjTfgtddg6dKUmoYazZOCc87F\nwymnQJMmMGoUpKen1DTUaN595Jxz8VC2rI0ffPwx/PJLSk1DjeZJwTnn4uXyy22mUYpNQ43m3UfO\nORcvBx5os47q10+paajRPCk451w8pdgU1Jy8+8g551w2TwrOOeeyeVJwzjmXzZOCc865bJ4UnHPO\nZfOk4JxzLpsnBeecc9k8KTjnnMsmqhp2DIUiIquB3wp8YmLVAdaEHUQhpFK8HmtwUineVIoVkjPe\ng1W1wDreKZcUkpGIzFDVtLDjiFUqxeuxBieV4k2lWCH14o3m3UfOOeeyeVJwzjmXzZNCfIwp+ClJ\nJZXi9ViDk0rxplKskHrxZvMxBeecc9m8peCccy6bJwXnnHPZPCkUg4gcJCLTROQHEZknIoPDjqkg\nIlJWRL4TkTfDjqUgIlJTRCaKyI+R9/jYsGPKi4gMifwOfC8i40SkUtgxRROR50RklYh8H3Xf/iLy\nvoj8HPlaK8wYs+QR678jvwdzROT/RKRmmDFGyy3eqMf+JiIqInXCiK0oPCkUz27gr6raAjgGuEZE\nWoYcU0EGAz+EHUSMHgXeUdXmwBEkadwi0hC4HkhT1dZAWaB3uFHt4wXg1Bz3DQU+UNVDgQ8i/04G\nL7BvrO8DrVW1LfATcGuig8rHC+wbLyJyEHAysCTRARWHJ4ViUNXlqvpt5PvN2IdWw3CjypuINALO\nAJ4JO5aCiEh1oDPwLICq7lTVDeFGla9yQGURKQdUAZaFHM9eVPUTYF2Ou88CXox8/yJwdkKDykNu\nsarqe6q6O/LPr4BGCQ8sD3m8twCPADcDKTWbx5NCnIhIE6A98HW4keRrBPZLmhl2IDFoBqwGno90\ndz0jIlXDDio3qroUeAi7IlwObFTV98KNKiYHqupysAsc4ICQ44nV5cCUsIPIj4j0BJaq6uywYyks\nTwpxICLVgEnADaq6Kex4ciMiZwKrVHVm2LHEqBzQAXhSVdsDW0me7o29RPrizwKaAg2AqiJyUbhR\nlUwichvWbTs27FjyIiJVgNuAf4QdS1F4UigmESmPJYSxqvpq2PHkoxPQU0QWA68AJ4rIy+GGlK90\nIF1Vs1peE7EkkYxOAhap6mpV3QW8ChwXckyxWCki9QEiX1eFHE++RORS4Eygryb3Aqs/YRcIsyN/\nb42Ab0WkXqhRxciTQjGIiGB93j+o6vCw48mPqt6qqo1UtQk2CPqhqibt1ayqrgB+F5HDI3d1B+aH\nGFJ+lgDHiEiVyO9Ed5J0UDyH14FLI99fCrwWYiz5EpFTgVuAnqq6Lex48qOqc1X1AFVtEvl7Swc6\nRH6nk54nheLpBFyMXXXPitxODzuoEuQ6YKyIzAHaAf8MOZ5cRVozE4FvgbnY31VSlTkQkXHAl8Dh\nIpIuIlcADwAni8jP2CyZB8KMMUsesT4O7Ae8H/k7Gx1qkFHyiDdleZkL55xz2byl4JxzLpsnBeec\nc9k8KTjnnMvmScE551w2TwrOOeeyeVJwLgcRyYiaYjxLROK2klpEmuRWTdO5ZFEu7ACcS0LbVbVd\n2EE4FwZvKTgXIxFZLCL/EpFvIrdDIvcfLCIfRGr9fyAijSP3Hxip/T87cssqfVFWRJ6O7L/wnohU\nDu1FOZeDJwXn9lU5R/fRhVGPbVLVo7AVtiMi9z0O/CdS638sMDJy/0jgY1U9AqvbNC9y/6HAKFVt\nBWwAzgv49TgXM1/R7FwOIrJFVavlcv9i4ERV/TVSCHGFqtYWkTVAfVXdFbl/uarWEZHVQCNV/SPq\nGE2A9yMb2yAitwDlVfXe4F+ZcwXzloJzhaN5fJ/Xc3LzR9T3GfjYnksinhScK5wLo75+Gfn+C/Zs\nv9kX+Czy/QfA1ZC9N3b1RAXpXFH5FYpz+6osIrOi/v2OqmZNS60oIl9jF1R9IvddDzwnIjdhu8Vd\nFrl/MDAmUjUzA0sQywOP3rli8DEF52IUGVNIU9U1YcfiXFC8+8g551w2byk455zL5i0F55xz2Twp\nOOecy+ZJwTnnXDZPCs4557J5UnDOOZft/wF6DQDHyPX/fwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "mlp=get_MLP(x_tan,y_tan)\n",
    "print(mlp.summary())\n",
    "mlp=tanitas(mlp,x_tan,y_tan,15)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Háló elmentése, betöltése"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Írjuk meg a háló kimentéséhez, valamint visszatöltéséhez szükséges függvényeket, illetve importáljuk be a json file-ból model objektumot konvertáló Keras függvényt."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from keras.models import model_from_json\n",
    "\n",
    "def halo_mentes(model,nev):\n",
    "    model_json=model.to_json()\n",
    "    with open(nev+'.json','w') as json_file:\n",
    "        json_file.write(model_json)\n",
    "    model.save_weights(nev+'.h5')\n",
    "    \n",
    "\n",
    "def halo_betoltes(nev):\n",
    "    json_file=open(nev+'.json','r')\n",
    "    model_json=json_file.read()\n",
    "    json_file.close()\n",
    "    model=model_from_json(model_json)\n",
    "    model.load_weights(nev+'.h5')\n",
    "    return(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Hívjuk meg a két függvényt:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "halo_mentes(mlp, 'mlp_mnist')\n",
    "mlp=halo_betoltes('mlp_mnist')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Háló tesztelése"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Top3 Accuracy-val való bővítése a háló metrikáinak"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Definiáljunk a kiértékelést végző függvényt:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.metrics import confusion_matrix\n",
    "\n",
    "def teszt(halo, x_test, y_test):\n",
    "    \n",
    "    tmp=halo.evaluate(x_test,y_test)\n",
    "        \n",
    "    y_pred=np.argmax(halo.predict(x_test),axis=1)\n",
    "    print('Confusion Matrix:')\n",
    "    print(confusion_matrix(np.argmax(y_test, axis=1), y_pred))\n",
    "    print()\n",
    "    print('Loss:'+str(tmp[0])+'; Acc:'+str(tmp[1])+'; Top3Acc:'+str(tmp[2]))\n",
    "    print()\n",
    "    return({'loss':tmp[0], 'acc':tmp[1], 'top3acc':tmp[2]})\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Hívjuk is meg a tesztpontokra."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10000/10000 [==============================] - 1s 85us/step\n",
      "Confusion Matrix:\n",
      "[[ 834    0    1   60    1   23   41    3   17    0]\n",
      " [   0 1033   39    4    1    1    7   29   21    0]\n",
      " [  24    5  796   47   20    1   38   32   65    4]\n",
      " [  33   11    9  667    3   43   50   12  155   27]\n",
      " [   2    2    1    3  784    2   32    4   19  133]\n",
      " [  39    1    2  149   16  487   51   17  101   29]\n",
      " [  35    2   13   11   34    6  824    2   30    1]\n",
      " [   4   18   20   23    6    5    1  851   28   72]\n",
      " [  14    9    6  135   12   27   30   11  691   39]\n",
      " [   3    8    1   78  130    9   12   54   62  652]]\n",
      "\n",
      "Loss:0.680050016212; Acc:0.7619; Top3Acc:0.9504\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'acc': 0.76190000000000002,\n",
       " 'loss': 0.68005001621246342,\n",
       " 'top3acc': 0.95040000000000002}"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "teszt(mlp,x_tst,y_tst)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Vegyünk fel egy új metrikát, mely azt méri, hogy egy adott mintához tartozó elvárt kimenet, a háló által adott bemenethez 3 legnagyobb konfidenciával taksált kimeneti osztály egyikének a tagja. Ennek használatához újra kell fordítanunk a hálót."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from keras.metrics import top_k_categorical_accuracy\n",
    "\n",
    "def inTop3(x,y):\n",
    "    return(top_k_categorical_accuracy(x,y,k=3))\n",
    "\n",
    "mlp.compile(optimizer=SGD(lr=0.01, decay=0, momentum=0.9, nesterov=True), loss='categorical_crossentropy', metrics=['accuracy', inTop3])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10000/10000 [==============================] - 1s 118us/step\n",
      "Confusion Matrix:\n",
      "[[ 834    0    1   60    1   23   41    3   17    0]\n",
      " [   0 1033   39    4    1    1    7   29   21    0]\n",
      " [  24    5  796   47   20    1   38   32   65    4]\n",
      " [  33   11    9  667    3   43   50   12  155   27]\n",
      " [   2    2    1    3  784    2   32    4   19  133]\n",
      " [  39    1    2  149   16  487   51   17  101   29]\n",
      " [  35    2   13   11   34    6  824    2   30    1]\n",
      " [   4   18   20   23    6    5    1  851   28   72]\n",
      " [  14    9    6  135   12   27   30   11  691   39]\n",
      " [   3    8    1   78  130    9   12   54   62  652]]\n",
      "\n",
      "Loss:0.680050016212; Acc:0.7619; Top3Acc:0.9504\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'acc': 0.76190000000000002,\n",
       " 'loss': 0.68005001621246342,\n",
       " 'top3acc': 0.95040000000000002}"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "teszt(mlp,x_tst,y_tst)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Végezetül töröljük a létrehozott session-t, ezáltal felszabadítva a GPU memóriát"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "K.clear_session()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}