ipynb • Lines: 2199{
"cells": [
{
"cell_type": "markdown",
"id": "09818f05",
"metadata": {},
"source": [
"# Probability Distributions\n",
"\n",
"Data plays a central role in various fields, including Data Science and Machine Learning Engineering. As time progresses, data has become increasingly crucial for making informed decisions and developing innovative products. In this assignment, you will work with data that follows different probability distributions.\n",
"\n",
"## Sections\n",
"\n",
"1. **Generating Data**: Learn how to generate data that follows specific probability distributions.\n",
"\n",
"2. **Naive Bayes Classifier (Continuous)**: Implement a Naive Bayes classifier for continuous data generated in Section 1.\n",
"\n",
"3. **Real-Life Problem (Spam Detection)**: Enhance the Naive Bayes implementation to address a real-life problem, specifically spam detection.\n"
]
},
{
"cell_type": "code",
"execution_count": 38,
"id": "9d0c4973",
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": [
"import numpy as np\n",
"import pandas as pd\n",
"import matplotlib.pyplot as plt\n",
"from math import factorial\n",
"from scipy.special import erfinv, comb\n",
"from scipy.stats import uniform, binom, norm\n",
"from dataclasses import dataclass\n",
"from sklearn.metrics import accuracy_score\n",
"\n",
"import utils"
]
},
{
"cell_type": "markdown",
"id": "29c8a7f2",
"metadata": {},
"source": [
"# Section 1: Generating Data from a Specific Distribution\n",
"\n",
"Let's recap some concepts and formalize them to facilitate coding. Don't worry, you will be guided throughout the entire assignment!\n",
"\n",
"A random variable $X$ is a function that represents a random phenomenon, meaning its exact value cannot be determined. However, probabilities can be assigned to a set of possible values it can take. For example, if $X$ has a uniform distribution on $[2, 4]$, we cannot determine the exact value of $X$, but we can say with a probability of $1$ that the value lies between $[2, 4]$. We can also say that:\n",
"\n",
"$$\\mathbf P(X \\leq 3) = \\frac{1}{2}$$\n",
"\n",
"where $3$ is the midpoint of the interval $[2, 4]$. Therefore, you have learned that a continuous random variable is associated with a function called the probability density function (PDF), which encodes the probability of the random variable falling within a given range. In other words, if $X$ is a continuous random variable and $f$ is its PDF, then:\n",
"\n",
"$$\\mathbf P(a \\leq X \\leq b) = \\text{Area under } f \\text{ between } a \\text{ and } b$$\n",
"\n",
"In the discrete case, $\\mathbf P(X = a) = f(a)$. In any case, $\\mathbf P(-\\infty < X < +\\infty) = 1$ because a random variable takes values on real numbers, and you're covering all possible outcomes.\n",
"\n",
"Another function associated with a random variable is the cumulative distribution function (CDF), denoted as $F$. It represents the probability that a random variable $X$ will be less than or equal to $x$, for any $x \\in \\mathbb{R}$:\n",
"\n",
"$$F(x) := \\mathbf P(X \\leq x), \\quad \\forall x \\in \\mathbb{R}$$\n",
"\n",
"The CDF is a non-decreasing function and approaches $1$ as $x$ approaches infinity because it represents a probability and must yield a value between $0$ and $1$.\n",
"\n",
"In the lectures, you learned that if $X$ is a random variable with CDF $F$, then $F(X)$ follows a uniform distribution between $0$ and $1$. In other words, the new random variable $F(X)$ will be uniformly distributed between $0$ and $1$. This opens up the possibility of generating artificial data with any desired distribution, given that we know $F$. The process is as follows:\n",
"\n",
"1. Generate a random value $y$ uniformly from the interval $[0, 1]$.\n",
"2. Compute $F^{-1}(y)$, which is the inverse function of $F$ evaluated at $y$.\n",
"\n",
"It can be shown that if $Y$ follows a uniform distribution between $0$ and $1$, then the random variable $F^{-1}(Y)$ has the same distribution as $X$.\n",
"\n",
"Therefore, by computing the inverse of $F$, you can generate artificial data from any known distribution! This is an incredibly powerful technique, isn't it?\n",
"\n",
"So far in the course, you have encountered three common probability distributions:\n",
"- Uniform\n",
"- Gaussian\n",
"- Binomial\n",
"\n",
"In the first part of this assignment, you will code a random generator for each of the above distributions!\n"
]
},
{
"cell_type": "markdown",
"id": "001b9b4a",
"metadata": {},
"source": [
"## Exercise 1: Uniform Generator\n",
"\n",
"The natural first step is to create a function capable of generating random data that comes from the uniform distribution. You will not be coding a [pseudo-random number generator](https://en.wikipedia.org/wiki/Pseudorandom_number_generator) (this is outside the scope of this assignment) but instead you will use a predefined function that handles this for you. If you are unsure where you can find such a function take a look at the [numpy.random.uniform](https://numpy.org/doc/stable/reference/random/generated/numpy.random.uniform.html) function."
]
},
{
"cell_type": "code",
"execution_count": 39,
"id": "33b2dcd2",
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": [
"def uniform_generator(a, b, num_samples=100):\n",
" \"\"\"\n",
" Generates an array of uniformly distributed random numbers within the specified range.\n",
"\n",
" Parameters:\n",
" - a (float): The lower bound of the range.\n",
" - b (float): The upper bound of the range.\n",
" - num_samples (int): The number of samples to generate (default: 100).\n",
"\n",
" Returns:\n",
" - array (ndarray): An array of random numbers sampled uniformly from the range [a, b).\n",
" \"\"\"\n",
"\n",
" np.random.seed(42)\n",
"\n",
" ### START CODE HERE ###\n",
" array = np.random.uniform(a,b,num_samples)\n",
" ### END CODE HERE ###\n",
"\n",
" return array"
]
},
{
"cell_type": "code",
"execution_count": 40,
"id": "02cb0af5",
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"6 randomly generated values between 0 and 1:\n",
"[0.375 0.951 0.732 0.599 0.156 0.156]\n",
"\n",
"3 randomly generated values between 20 and 55:\n",
"[33.109 53.275 45.62 ]\n",
"\n",
"1 randomly generated value between 0 and 100:\n",
"[37.454]\n"
]
}
],
"source": [
"# Test your function\n",
"\n",
"print(f\"6 randomly generated values between 0 and 1:\\n{np.array2string(uniform_generator(0, 1, num_samples=6), precision=3)}\\n\")\n",
"print(f\"3 randomly generated values between 20 and 55:\\n{np.array2string(uniform_generator(20, 55, num_samples=3), precision=3)}\\n\")\n",
"print(f\"1 randomly generated value between 0 and 100:\\n{np.array2string(uniform_generator(0, 100, num_samples=1), precision=3)}\")"
]
},
{
"cell_type": "markdown",
"id": "871446b7",
"metadata": {},
"source": [
"##### __Expected Output__\n",
"\n",
"```\n",
"6 randomly generated values between 0 and 1:\n",
"[0.375 0.951 0.732 0.599 0.156 0.156]\n",
"\n",
"3 randomly generated values between 20 and 55:\n",
"[33.109 53.275 45.62 ]\n",
"\n",
"1 randomly generated value between 0 and 100:\n",
"[37.454]\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "c86f5e76",
"metadata": {},
"source": [
"## Exercise 2: Gaussian Generator\n",
"\n",
"With your uniform data generator ready you can go ahead and create generators for the other distributions. In order to do this you will need the inverse `CDF` for the distribution you wish to create data for. "
]
},
{
"cell_type": "markdown",
"id": "19d63b3b",
"metadata": {},
"source": [
"### 2.1 Inverse CDF - Gaussian\n",
"\n",
"Let's start with a Normal Distribution generator. In the case of the Gaussian distribution finding the inverse is a bit challenging because the CDF doesn't have a closed analytical expression.\n",
"\n",
"The closed formula uses a function called the Gaussian error function, denoted as $\\text{erf}(x)$. However, you don't need to implement it or its inverse for this assignment. These functions are important in statistics, and there are many libraries available that provide their implementations.\n",
"\n",
"For instance, you can use [scipy.special.erf](https://docs.scipy.org/doc/scipy/reference/generated/scipy.special.erf.html) and [scipy.special.erfinv](https://docs.scipy.org/doc/scipy/reference/generated/scipy.special.erfinv.html#scipy.special.erfinv) to compute the erf function and its inverse. Alternatively, you can find an implementation of the erf function in [math.erf](https://docs.python.org/3/library/math.html#math.erf) from the Python math library.\n",
"\n",
"If $X \\sim N(\\mu, \\sigma)$, then the CDF $F(x)$ can be expressed as:\n",
"\n",
"$$y = F(x) = \\frac{1}{2} \\left[ 1 + \\text{erf}\\left( \\frac{x - \\mu}{\\sigma \\sqrt{2}} \\right) \\right].$$\n",
"\n",
"With some simple calculations and denoting ${\\text{erf}}^{-1}$ as the inverse of the $\\text{erf}$ function, it can be shown that:\n",
"\n",
"$$x = F^{-1}(y) = \\sigma \\sqrt{2} \\cdot \\text{erf}^{-1}(2y - 1) + \\mu.$$\n"
]
},
{
"cell_type": "code",
"execution_count": 41,
"id": "b20aaede",
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": [
"def inverse_cdf_gaussian(y, mu, sigma):\n",
" \"\"\"\n",
" Calculates the inverse cumulative distribution function (CDF) of a Gaussian distribution.\n",
"\n",
" Parameters:\n",
" - y (float or ndarray): The probability or array of probabilities.\n",
" - mu (float): The mean of the Gaussian distribution.\n",
" - sigma (float): The standard deviation of the Gaussian distribution.\n",
"\n",
" Returns:\n",
" - x (float or ndarray): The corresponding value(s) from the Gaussian distribution that correspond to the given probability/ies.\n",
" \"\"\"\n",
" ### START CODE HERE ###\n",
" x = mu + erfinv(2 * y - 1) * sigma*(2**0.5)\n",
" ### END CODE HERE ###\n",
"\n",
" return x"
]
},
{
"cell_type": "code",
"execution_count": 42,
"id": "dc12aaba",
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Inverse of Gaussian CDF with mu 15 and sigma 5 for value 1e-10: -16.807\n",
"Inverse of Gaussian CDF with mu 15 and sigma 5 for value 0: -inf\n",
"Inverse of Gaussian CDF with mu 20 and sigma 0.5 for value 0.4: 19.873\n",
"Inverse of Gaussian CDF with mu 15 and sigma 5 for value 1: inf\n"
]
}
],
"source": [
"# Test your function\n",
"\n",
"print(f\"Inverse of Gaussian CDF with mu {15} and sigma {5} for value {1e-10}: {inverse_cdf_gaussian(1e-10, 15, 5):.3f}\")\n",
"print(f\"Inverse of Gaussian CDF with mu {15} and sigma {5} for value {0}: {inverse_cdf_gaussian(0, 15, 5)}\")\n",
"print(f\"Inverse of Gaussian CDF with mu {20} and sigma {0.5} for value {0.4}: {inverse_cdf_gaussian(0.4, 20, 0.5):.3f}\")\n",
"print(f\"Inverse of Gaussian CDF with mu {15} and sigma {5} for value {1}: {inverse_cdf_gaussian(1, 15, 5)}\")"
]
},
{
"cell_type": "markdown",
"id": "4a401b18",
"metadata": {},
"source": [
"##### __Expected Output__\n",
"\n",
"```\n",
"Inverse of Gaussian CDF with mu 15 and sigma 5 for value 1e-10: -16.807\n",
"Inverse of Gaussian CDF with mu 15 and sigma 5 for value 0: -inf\n",
"Inverse of Gaussian CDF with mu 20 and sigma 0.5 for value 0.4: 19.873\n",
"Inverse of Gaussian CDF with mu 15 and sigma 5 for value 1: inf\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "55fbc248",
"metadata": {},
"source": [
"### 2.2 Gaussian Generator\n",
"\n",
"Now that you have all the necessary information, you can create a generator for data that follows a Gaussian distribution with a specified $\\mu$ and $\\sigma$. Similar to the generator for uniformly distributed data, the `gaussian_generator` function should allow you to specify the number of samples to generate. **Make sure to utilize the functions you have defined earlier in the assignment.**"
]
},
{
"cell_type": "code",
"execution_count": 43,
"id": "5df743c1",
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": [
"def gaussian_generator(mu, sigma, num_samples):\n",
" ### START CODE HERE ###\n",
"\n",
" # Generate an array with num_samples elements that distribute uniformally between 0 and 1\n",
" u = uniform_generator(0,1,num_samples)\n",
"\n",
" # Use the uniform-distributed sample to generate Gaussian-distributed data\n",
" # Hint: You need to sample from the inverse of the CDF of the distribution you are generating\n",
" array = inverse_cdf_gaussian(u, mu, sigma)\n",
" ### END CODE HERE ###\n",
"\n",
" return array"
]
},
{
"cell_type": "code",
"execution_count": 44,
"id": "07885432",
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA0oAAAGHCAYAAACOM6KuAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy88F64QAAAACXBIWXMAAA9hAAAPYQGoP6dpAABWAklEQVR4nO3deVxWZf7/8fetssrizqKoqJArppKkjYkpmKlpNGVqLqOOOtiCpuZSI5aCu2Zm/WxKaExzmtRxmjLJvdQGF9yysiTESaJMAVFB5fz+8MH97b5BBQQO4uv5eJzHcK5znet87nPu09wfr+tcx2IYhiEAAAAAgFUVswMAAAAAgIqGRAkAAAAA7JAoAQAAAIAdEiUAAAAAsEOiBAAAAAB2SJQAAAAAwA6JEgAAAADYIVECAAAAADskSgAAAABgh0QJQIURFxcni8Wiffv2Fbq9T58+aty4sU1Z48aNNXz48GIdZ/fu3YqOjtb58+dLFigK9dJLL6lhw4aqVq2aatSoccv6X3zxhQYOHKiGDRvKyclJ1atXV6tWrfTCCy/om2++KfuAS0H+d/bHH380OxSr0NBQhYaGWtd//PFHWSwWxcXFFaud1atXa8mSJcXap7BjRUdHy2Kx6Ndffy1WWzfz9ddfKzo6utDzPnz48AL/nQCAkiBRAnBHW79+vV5++eVi7bN7927NnDmTRKkU/etf/9Ls2bM1dOhQ7dixQ59//vlN67/00kvq0qWLUlJS9NJLL2nTpk3asGGDRowYoYSEBLVo0ULXrl0rp+hLrnfv3tqzZ498fHzMDuWGfHx8tGfPHvXu3btY+5UkUSrpsYrr66+/1syZMwtNlF5++WWtX7++TI8P4O5QzewAAOB2tGvXzuwQiu3KlSuyWCyqVq3y/Cf46NGjkqTnnntO9erVu2ndNWvWaPbs2Ro7dqyWL18ui8Vi3RYWFqYJEyZo+fLlZRpvaalbt67q1q1rdhg35eTkpPvvv79Mj3Ht2jVdvXq1XI51K02bNjX1+AAqD3qUANzR7Ife5eXladasWbrnnnvk4uKiGjVqKCgoSK+99pqk68OAJk2aJEny9/eXxWKRxWLR9u3brfvPmzdPzZs3l5OTk+rVq6ehQ4fq9OnTNsc1DEMxMTFq1KiRnJ2dFRwcrISEhALDnrZv3y6LxaK///3veuGFF1S/fn05OTnp+++/1y+//KLIyEi1bNlSbm5uqlevnh566CHt2rXL5lj5w5nmz5+vuXPnqnHjxnJxcVFoaKi+++47XblyRVOmTJGvr688PT312GOPKT093aaNrVu3KjQ0VLVr15aLi4saNmyoxx9/XBcvXrzp+S3K+WjcuLFeeuklSZKXl5csFouio6Nv2OasWbNUp04dLV682CZJymexWDRu3DhVrVrVWpaQkKB+/fqpQYMGcnZ2VrNmzTRmzJgCw7luNOwqf/jX73344YcKCQmRp6enXF1d1aRJE40YMcLms9/suyQVPvSuqLHmx3Ts2DENHDhQnp6e8vLy0ogRI5SRkXHD85fPMAzNmzfP+h1s3769Pv300wL1ChsO98svv2j06NHy8/OTk5OT6tatqwceeMDaExgaGqr//Oc/SklJsd4j+ecvv7158+Zp1qxZ8vf3l5OTk7Zt23bTYX6pqamKiIiQh4eHPD099fTTT+uXX36xqXOj787v7/O4uDg98cQTkqRu3bpZY8s/ZmHfgcuXL2vq1Kny9/eXo6Oj6tevr3HjxhXoVW7cuLH69OmjTZs2qX379nJxcVHz5s317rvv2tS7ePGiJk6cKH9/fzk7O6tWrVoKDg7WmjVrCsQO4M5Vef45E0Clkf+v0/YMw7jlvvPmzVN0dLReeuklPfjgg7py5Yq++eYb6w+iUaNG6bffftPrr7+udevWWYdMtWzZUpL0l7/8RStWrNAzzzyjPn366Mcff9TLL7+s7du368CBA6pTp44kafr06YqNjdXo0aMVERGh1NRUjRo1SleuXFFgYGCBuKZOnapOnTrprbfeUpUqVVSvXj3rj8QZM2bI29tbFy5c0Pr16xUaGqotW7bYJFyS9MYbbygoKEhvvPGGzp8/rxdeeEF9+/ZVSEiIHBwc9O677yolJUUTJ07UqFGjtHHjRknXf9j27t1bXbp00bvvvqsaNWrof//7nzZt2qTc3Fy5urre8HwW5XysX79eb7zxht555x1t2rRJnp6eatCgQaHt/fTTT/r66681cOBAOTs73/J65vvhhx/UqVMnjRo1Sp6envrxxx+1aNEi/eEPf9CRI0fk4OBQ5LYkac+ePRowYIAGDBig6OhoOTs7KyUlRVu3brXWudV3qbRiffzxxzVgwACNHDlSR44c0dSpUyWpwI9zezNnztTMmTM1cuRI/fGPf1Rqaqr+/Oc/69q1a7rnnntuuu+QIUN04MABzZ49W4GBgTp//rwOHDigs2fPSpKWL1+u0aNH64cffrjhMLalS5cqMDBQCxYskIeHhwICAm56zMcee0xPPvmkxo4dq2PHjunll1/W119/ra+++qpY1693796KiYnRtGnT9MYbb6h9+/aSbtyTZBiG+vfvry1btmjq1Knq0qWLDh8+rBkzZmjPnj3as2ePnJycrPUPHTqkF154QVOmTJGXl5f+9re/aeTIkWrWrJkefPBBSdKECRP097//XbNmzVK7du2UnZ2to0ePWs8fgErCAIAKYuXKlYakmy6NGjWy2adRo0bGsGHDrOt9+vQx7r333pseZ/78+YYkIzk52ab8+PHjhiQjMjLSpvyrr74yJBnTpk0zDMMwfvvtN8PJyckYMGCATb09e/YYkoyuXbtay7Zt22ZIMh588MFbfv6rV68aV65cMbp372489thj1vLk5GRDktG2bVvj2rVr1vIlS5YYkoxHH33Upp2oqChDkpGRkWEYhmH885//NCQZSUlJt4zh94p6PgzDMGbMmGFIMn755Zebtrl3715DkjFlypQC2/I/f/6Sl5dXaBt5eXnGlStXjJSUFEOS8a9//cu6bdiwYQW+I7+PL9+CBQsMScb58+dvGGtRvkv531n771JRYs2Pad68eTb7REZGGs7Ozjf8/IZhGOfOnTOcnZ1tvieGYRhffvllge9g/vdn5cqV1jI3NzcjKirqpp+td+/ehZ7L/PaaNm1q5ObmFrrt98fK/5zjx4+3qfv+++8bkoxVq1ZZyyQZM2bMKHBM+/v8ww8/NCQZ27ZtK1DX/juwadOmQs/z2rVrDUnGihUrbI7j7OxspKSkWMsuXbpk1KpVyxgzZoy1rHXr1kb//v0LHBtA5cLQOwAVznvvvafExMQCyx/+8Idb7tuxY0cdOnRIkZGR+uyzz5SZmVnk427btk2SCsyi17FjR7Vo0UJbtmyRJO3du1c5OTl68sknberdf//9N5xt6/HHHy+0/K233lL79u3l7OysatWqycHBQVu2bNHx48cL1H3kkUdUpcr//We7RYsWklTgwfn88lOnTkmS7r33Xjk6Omr06NGKj4/XyZMnC43FXlHPR2mpXbu2HBwcrMtHH31k3Zaenq6xY8fKz8/Pep4aNWokSYWeq1u57777JElPPvmk/vGPf+h///tfgTol/S4VN9ZHH33UZj0oKEiXL18uMHzy9/bs2aPLly9r8ODBNuWdO3e2HutmOnbsqLi4OM2aNUt79+7VlStXivLRCsRdnJ4g+1iffPJJVatWzfo9Kyv5vYT23+MnnnhC1atXL/A9vvfee9WwYUPrurOzswIDA5WSkmIt69ixoz799FNNmTJF27dv16VLl8ruAwAwDYkSgAqnRYsWCg4OLrB4enrect+pU6dqwYIF2rt3r3r16qXatWure/fuN5xy/Pfyh80UNoOZr6+vdXv+/3p5eRWoV1jZjdpctGiR/vKXvygkJEQfffSR9u7dq8TERD388MOF/vCqVauWzbqjo+NNyy9fvizp+pCkzz//XPXq1dO4cePUtGlTNW3a1OZZm8IU9XwUh5+fnyTZ/OjMt337diUmJuqtt96yKc/Ly1N4eLjWrVunyZMna8uWLfrvf/+rvXv3SlKJfqQ++OCD2rBhg65evaqhQ4eqQYMGat26tc0zJiX5LpUk1tq1a9us5w8Du9nnyj/33t7eBbYVVmZv7dq1GjZsmP72t7+pU6dOqlWrloYOHaq0tLRb7puvuDP92cdVrVo11a5du8yHq509e1bVqlUrMOmGxWKRt7d3gePbXw/p+jX5/fVYunSpXnzxRW3YsEHdunVTrVq11L9/f504caJsPgQAU5AoAahUqlWrpgkTJujAgQP67bfftGbNGqWmpqpnz563nLgg/wfSmTNnCmz76aefrM8n5df7+eefC9S70Q/NwiYtWLVqlUJDQ/Xmm2+qd+/eCgkJUXBwsLKysm7+IUugS5cu+ve//62MjAzt3btXnTp1UlRUlD744IMb7lPU81Ecvr6+atWqlRISEqyJXL57771XwcHBBZ6vOXr0qA4dOqT58+fr2WefVWhoqO67775Cf9A6OzsrJyenQHlh7/Dp16+ftmzZooyMDG3fvl0NGjTQoEGDtGfPHkkl+y4VJ9bbkd9eYd+3oiQ7derU0ZIlS/Tjjz8qJSVFsbGxWrduXbHeSVbYd/pm7OO6evWqzp49a3NunJycCr1+t5NM1a5dW1evXi0wcYRhGEpLSyvR97h69eqaOXOmvvnmG6WlpenNN9/U3r171bdv3xLHCaDiIVECUGnVqFFDf/zjHzVu3Dj99ttv1pnJbvQv9g899JCk6wnM7yUmJur48ePq3r27JCkkJEROTk5au3atTb29e/cW2lNyIxaLxeYhckk6fPiw9Yd6WahatapCQkL0xhtvSJIOHDhww7pFPR/FNX36dP3666+aMGFCkSboyP9Bbn+u/t//+38F6jZu3Fjp6ek2SWxubq4+++yzG7bv5OSkrl27au7cuZKkgwcPFqhzo+/S7cR6O+6//345Ozvr/ffftynfvXt3sb6DktSwYUM988wzCgsLs/k+2Pei3C77WP/xj3/o6tWrNpOWNG7cWIcPH7apt3XrVl24cMGmrCi9bvnyv6f23+OPPvpI2dnZJf4e5/Py8tLw4cM1cOBAffvtt7f8BxkAdw5mvQNQqfTt21etW7dWcHCw6tatq5SUFC1ZskSNGjWyzsrVpk0bSdJrr72mYcOGycHBQffcc4/uuecejR49Wq+//rqqVKmiXr16WWd58/Pz0/jx4yVdH+o2YcIExcbGqmbNmnrsscd0+vRpzZw5Uz4+PjbPEd1Mnz599Oqrr2rGjBnq2rWrvv32W73yyivy9/cvdNa/knrrrbe0detW9e7dWw0bNtTly5etM6r16NHjhvsV9XwU18CBA3Xs2DHNnj1bhw4d0vDhwxUQEKC8vDylpqbq73//uyTJ3d1dktS8eXM1bdpUU6ZMkWEYqlWrlv79738rISGhQNsDBgzQX//6Vz311FOaNGmSLl++rKVLlxZ4ee1f//pXnT59Wt27d1eDBg10/vx5vfbaa3JwcFDXrl0lFe27ZK84sd6OmjVrauLEiZo1a5ZGjRqlJ554QqmpqYqOjr7l0LuMjAx169ZNgwYNUvPmzeXu7q7ExERt2rRJERER1npt2rTRunXr9Oabb6pDhw6qUqWKgoODSxzzunXrVK1aNYWFhVlnvWvbtq3Ns35DhgzRyy+/rL/+9a/q2rWrvv76ay1btqzAsNvWrVtLklasWCF3d3c5OzvL39+/0J67sLAw9ezZUy+++KIyMzP1wAMPWGe9a9eunYYMGVLszxISEqI+ffooKChINWvW1PHjx/X3v/9dnTp1uukskgDuMObOJQEA/yd/BrHExMRCtxc2C5f9bFgLFy40OnfubNSpU8dwdHQ0GjZsaIwcOdL48ccfbfabOnWq4evra1SpUsVm9qxr164Zc+fONQIDAw0HBwejTp06xtNPP22kpqba7J+Xl2fMmjXLaNCggeHo6GgEBQUZH3/8sdG2bVubmcjyZ7378MMPC3yenJwcY+LEiUb9+vUNZ2dno3379saGDRsKzNqVP5PY/Pnzbfa/Udv253HPnj3GY489ZjRq1MhwcnIyateubXTt2tXYuHFjoef594p6Poo6693v7dy50xgwYIDRoEEDw8HBwXB1dTVatmxp/OUvfzH27dtnU/frr782wsLCDHd3d6NmzZrGE088YZw6darQWdI++eQT49577zVcXFyMJk2aGMuWLSsw693HH39s9OrVy6hfv77h6Oho1KtXz3jkkUeMXbt2WesU5btU2Kx3RY31RufsVjPp5cvLyzNiY2MNPz8/63fw3//+t9G1a9ebznp3+fJlY+zYsUZQUJDh4eFhuLi4GPfcc48xY8YMIzs727rfb7/9Zvzxj380atSoYVgsFuv5u9H3sbBj/f5z7t+/3+jbt6/h5uZmuLu7GwMHDjR+/vlnm/1zcnKMyZMnG35+foaLi4vRtWtXIykpqcB9bhjXZ3309/c3qlatanPMwmY+vHTpkvHiiy8ajRo1MhwcHAwfHx/jL3/5i3Hu3Dmbeo0aNTJ69+5d4HPZn9MpU6YYwcHBRs2aNQ0nJyejSZMmxvjx441ff/21wL4A7lwWwyjCuAcAwC0lJyerefPmmjFjhqZNm2Z2OAAA4DaQKAFACRw6dEhr1qxR586d5eHhoW+//Vbz5s1TZmamjh49esPZ7wAAwJ2BZ5QAoASqV6+uffv26Z133tH58+fl6emp0NBQzZ49myQJAIBKgB4lAAAAALDD9OAAAAAAYIdECQAAAADskCgBAAAAgJ1KP5lDXl6efvrpJ7m7u1vfmA4AAADg7mMYhrKysuTr63vLF8RX+kTpp59+kp+fn9lhAAAAAKggUlNT1aBBg5vWqfSJkru7u6TrJ8PDw8PkaAAAAACYJTMzU35+ftYc4WYqfaKUP9zOw8ODRAkAAABAkR7JYTIHAAAAALBDogQAAAAAdkiUAAAAAMBOpX9GCQAAAHcnwzB09epVXbt2zexQUE6qVq2qatWqlcprgUiUAAAAUOnk5ubqzJkzunjxotmhoJy5urrKx8dHjo6Ot9UOiRIAAAAqlby8PCUnJ6tq1ary9fWVo6NjqfQwoGIzDEO5ubn65ZdflJycrICAgFu+VPZmSJQAAABQqeTm5iovL09+fn5ydXU1OxyUIxcXFzk4OCglJUW5ublydnYucVtM5gAAAIBK6XZ6E3DnKq3rzrcHAAAAAOyQKAEAAACAHRIlAAAAAEWyfft2WSwWnT9/3uxQyhyTOaDCWJzwXYGy8WGBJkSCUrcttvj7dJta+nGgVC1PWl5mbUfeG1lmbQO4uxX2e6MsVbbfMp07d9aZM2fk6elZbsfMycnRxIkTtWbNGl26dEndu3fX8uXL1aBBgzI9Lj1KAAAAAIrE0dFR3t7e5TrdelRUlNavX68PPvhAX3zxhS5cuKA+ffqU+YuESZQAAACACiQrK0uDBw9W9erV5ePjo8WLFys0NFRRUVGSpFWrVik4OFju7u7y9vbWoEGDlJ6ebt0/Li5ONWrUsGlzw4YNNsnNoUOH1K1bN7m7u8vDw0MdOnTQvn37JEkpKSnq27evatasqerVq6tVq1b65JNPJBUcenf27FkNHDhQDRo0kKurq9q0aaM1a9bYHDs0NFTPPfecJk+erFq1asnb21vR0dFFOhcZGRl65513tHDhQvXo0UPt2rXTqlWrdOTIEX3++efFOKvFR6IEAAAAVCATJkzQl19+qY0bNyohIUG7du3SgQMHrNtzc3P16quv6tChQ9qwYYOSk5M1fPjwYh1j8ODBatCggRITE7V//35NmTJFDg4OkqRx48YpJydHO3fu1JEjRzR37ly5ubkV2s7ly5fVoUMHffzxxzp69KhGjx6tIUOG6KuvvrKpFx8fr+rVq+urr77SvHnz9MorryghIeGWce7fv19XrlxReHi4tczX11etW7fW7t27i/WZi4tnlAAAAIAKIisrS/Hx8Vq9erW6d+8uSVq5cqV8fX2tdUaMGGH9u0mTJlq6dKk6duyoCxcu3DChsXfq1ClNmjRJzZs3lyQFBATYbHv88cfVpk0b6zFupH79+po4caJ1/dlnn9WmTZv04YcfKiQkxFoeFBSkGTNmWI+1bNkybdmyRWFhYTeNMy0tTY6OjqpZs6ZNuZeXl9LS0or0WUuKHiUAAACggjh58qSuXLmijh07Wss8PT11zz33WNcPHjyofv36qVGjRnJ3d1doaKik6wlOUU2YMEGjRo1Sjx49NGfOHP3www/Wbc8995xmzZqlBx54QDNmzNDhw4dv2M61a9c0e/ZsBQUFqXbt2nJzc9PmzZsLxBIUFGSz7uPjYzNcsLgMwyjz56RIlAAAAIAKwjAMSSqQBOSXZ2dnKzw8XG5ublq1apUSExO1fv16SdeH5ElSlSpVrPXzXblyxWY9Ojpax44dU+/evbV161a1bNnS2s6oUaN08uRJDRkyREeOHFFwcLBef/31QuNduHChFi9erMmTJ2vr1q1KSkpSz549rbHkyx/Wl89isSgvL++W58Pb21u5ubk6d+6cTXl6erq8vLxuuf/tMDVRunr1ql566SX5+/vLxcVFTZo00SuvvGJz0gzDUHR0tHx9feXi4qLQ0FAdO3bMxKgBAACAstG0aVM5ODjov//9r7UsMzNTJ06ckCR98803+vXXXzVnzhx16dJFzZs3L9AzU7duXWVlZSk7O9talpSUVOBYgYGBGj9+vDZv3qyIiAitXLnSus3Pz09jx47VunXr9MILL+jtt98uNN5du3apX79+evrpp9W2bVs1adLEGmtp6NChgxwcHGyeZzpz5oyOHj2qzp07l9pxCmNqojR37ly99dZbWrZsmY4fP6558+Zp/vz5NhnrvHnztGjRIi1btkyJiYny9vZWWFiYsrKyTIwcAAAAKH3u7u4aNmyYJk2apG3btunYsWMaMWKEqlSpIovFooYNG8rR0VGvv/66Tp48qY0bN+rVV1+1aSMkJESurq6aNm2avv/+e61evVpxcXHW7ZcuXdIzzzyj7du3KyUlRV9++aUSExPVokULSden4/7ss8+UnJysAwcOaOvWrdZt9po1a6aEhATt3r1bx48f15gxY0r12SFPT0+NHDlSL7zwgrZs2aKDBw/q6aefVps2bdSjR49SO05hTJ3MYc+ePerXr5969+4tSWrcuLHWrFljnZrQMAwtWbJE06dPV0REhKTrM2Z4eXlp9erVGjNmjGmxAwAA4M5zJ7wAdtGiRRo7dqz69OkjDw8PTZ48WampqXJ2dlbdunUVFxenadOmaenSpWrfvr0WLFigRx991Lp/rVq1tGrVKk2aNEkrVqxQjx49FB0drdGjR0uSqlatqrNnz2ro0KH6+eefVadOHUVERGjmzJmSrj93NG7cOJ0+fVoeHh56+OGHtXjx4kJjffnll5WcnKyePXvK1dVVo0ePVv/+/ZWRkVFq52Px4sWqVq2annzySesLZ+Pi4lS1atVSO0ZhLIb9AMZyNGfOHL311lvavHmzAgMDdejQIYWHh2vJkiUaOHCgTp48qaZNm+rAgQNq166ddb9+/fqpRo0aio+PL9BmTk6OcnJyrOuZmZny8/NTRkaGPDw8yuVzoWQKe1P2nfAfMxTBttji79NtaunHgVK1PGl5mbUdeW9kmbUNoPK7fPmykpOT5e/vL2dnZ7PDuW3Z2dmqX7++Fi5cqJEjR5odToV3s+ufmZkpT0/PIuUGpvYovfjii8rIyFDz5s1VtWpV66wZAwcOlCRrt539g1peXl5KSUkptM3Y2FhrNgwAAADcaQ4ePKhvvvlGHTt2VEZGhl555RVJ1zsLUH5MfUZp7dq1WrVqlVavXq0DBw4oPj5eCxYsKNBTVNisHzeaDnDq1KnKyMiwLqmpqWUWPwAAAFAWFixYoLZt26pHjx7Kzs7Wrl27VKdOHbPDKnXvv/++3NzcCl1atWplamym9ihNmjRJU6ZM0VNPPSVJatOmjVJSUhQbG6thw4bJ29tb0vWeJR8fH+t+N5sO0MnJSU5OTmUfPAAAAFAG2rVrp/3795sdRrl49NFHbV5M+3v2U4qXN1MTpYsXL6pKFdtOrapVq1qnB/f395e3t7cSEhKszyjl5uZqx44dmjt3brnHCwAAAKD0uLu7y93d3ewwCmVqotS3b1/Nnj1bDRs2VKtWrXTw4EEtWrRII0aMkHR9yF1UVJRiYmIUEBCggIAAxcTEyNXVVYMGDTIzdJQT+wkemNzhLsIEEHe1spoogkkiAABFZWqi9Prrr+vll19WZGSk0tPT5evrqzFjxuivf/2rtc7kyZN16dIlRUZG6ty5cwoJCdHmzZsrbOYJAAAA4M5naqLk7u6uJUuWaMmSJTesY7FYFB0drejo6HKLCwAAAMDdzdRZ7wAAAACgIiJRAgAAAAA7JEoAAAAAimT79u2yWCw6f/682aGUOVOfUQIAAADKVUlmVb0dlWxG1s6dO+vMmTPy9PQst2OuWLFCq1ev1oEDB5SVlaVz586pRo0aZX5cepQAAAAAFImjo6O8vb1lsVjK7ZgXL17Uww8/rGnTppXbMSUSJQAAAKBCycrK0uDBg1W9enX5+Pho8eLFCg0NVVRUlCRp1apVCg4Olru7u7y9vTVo0CClp6db94+LiyvQ47Jhwwab5ObQoUPq1q2b3N3d5eHhoQ4dOmjfvn2SpJSUFPXt21c1a9ZU9erV1apVK33yySeSCg69O3v2rAYOHKgGDRrI1dVVbdq00Zo1a2yOHRoaqueee06TJ09WrVq15O3tXawZraOiojRlyhTdf//9Rd6nNJAoAQAAABXIhAkT9OWXX2rjxo1KSEjQrl27dODAAev23Nxcvfrqqzp06JA2bNig5ORkDR8+vFjHGDx4sBo0aKDExETt379fU6ZMkYODgyRp3LhxysnJ0c6dO3XkyBHNnTtXbm5uhbZz+fJldejQQR9//LGOHj2q0aNHa8iQIfrqq69s6sXHx6t69er66quvNG/ePL3yyitKSEgo3okpZzyjBAAAAFQQWVlZio+P1+rVq9W9e3dJ0sqVK+Xr62utM2LECOvfTZo00dKlS9WxY0dduHDhhgmNvVOnTmnSpElq3ry5JCkgIMBm2+OPP642bdpYj3Ej9evX18SJE63rzz77rDZt2qQPP/xQISEh1vKgoCDNmDHDeqxly5Zpy5YtCgsLK1K8ZqBHCQAAAKggTp48qStXrqhjx47WMk9PT91zzz3W9YMHD6pfv35q1KiR3N3dFRoaKul6glNUEyZM0KhRo9SjRw/NmTNHP/zwg3Xbc889p1mzZumBBx7QjBkzdPjw4Ru2c+3aNc2ePVtBQUGqXbu23NzctHnz5gKxBAUF2az7+PjYDBesiEiUAAAAgArCMAxJKjBZQn55dna2wsPD5ebmplWrVikxMVHr16+XdH1IniRVqVLFWj/flStXbNajo6N17Ngx9e7dW1u3blXLli2t7YwaNUonT57UkCFDdOTIEQUHB+v1118vNN6FCxdq8eLFmjx5srZu3aqkpCT17NnTGku+/GF9+SwWi/Ly8op8XsxAogQAAABUEE2bNpWDg4P++9//WssyMzN14sQJSdI333yjX3/9VXPmzFGXLl3UvHnzAj0zdevWVVZWlrKzs61lSUlJBY4VGBio8ePHa/PmzYqIiNDKlSut2/z8/DR27FitW7dOL7zwgt5+++1C4921a5f69eunp59+Wm3btlWTJk2ssd7pSJQAAACACsLd3V3Dhg3TpEmTtG3bNh07dkwjRoxQlSpVZLFY1LBhQzk6Our111/XyZMntXHjRr366qs2bYSEhMjV1VXTpk3T999/r9WrVysuLs66/dKlS3rmmWe0fft2paSk6Msvv1RiYqJatGgh6fosc5999pmSk5N14MABbd261brNXrNmzZSQkKDdu3fr+PHjGjNmjNLS0kr1nKSlpSkpKUnff/+9JOnIkSNKSkrSb7/9VqrHscdkDgAAALh73AEvgF20aJHGjh2rPn36yMPDQ5MnT1ZqaqqcnZ1Vt25dxcXFadq0aVq6dKnat2+vBQsW6NFHH7XuX6tWLa1atUqTJk3SihUr1KNHD0VHR2v06NGSpKpVq+rs2bMaOnSofv75Z9WpU0cRERGaOXOmpOvPHY0bN06nT5+Wh4eHHn74YS1evLjQWF9++WUlJyerZ8+ecnV11ejRo9W/f39lZGSU2vl46623rLFJ0oMPPijp+iQXxZ3trzgshv0AxkomMzNTnp6eysjIkIeHh9nh4CYWJ3x3yzrjwwLLIRKUuvJ6C/od8H9+lcnypOVmh1BskfdGmh0CgHJw+fJlJScny9/fX87OzmaHc9uys7NVv359LVy4UCNHjjQ7nArvZte/OLkBPUoAAABABXLw4EF988036tixozIyMvTKK69Ikvr162dyZHcXnlECAAAAKpgFCxaobdu26tGjh7Kzs7Vr1y7VqVPH7LBK3fvvvy83N7dCl1atWpkaGz1KAAAAQAXSrl077d+/3+wwysWjjz5q82La37OfUry8kSgBAAAAMIW7u7vc3d3NDqNQDL0DAAAAADskSgAAAABgh0QJAAAAAOzwjBLKjf17kngnUgVQkvcb8a6iMlNW7yXi3UEAABQfPUoAAAAAYIdECQAAAECRbN++XRaLRefPnzc7lDJn6tC7xo0bKyUlpUB5ZGSk3njjDRmGoZkzZ2rFihU6d+6cQkJC9MYbb5j+8ikAAADcmcpqmPONVLbhz507d9aZM2fk6elZLsf77bffNGPGDG3evFmpqamqU6eO+vfvr1dffbXMYzC1RykxMVFnzpyxLgkJCZKkJ554QpI0b948LVq0SMuWLVNiYqK8vb0VFhamrKwsM8MGAAAA7kqOjo7y9vaWxWIpl+P99NNP+umnn7RgwQIdOXJEcXFx2rRpk0aOHFnmxzY1Uapbt668vb2ty8cff6ymTZuqa9euMgxDS5Ys0fTp0xUREaHWrVsrPj5eFy9e1OrVq80MGwAAACgzWVlZGjx4sKpXry4fHx8tXrxYoaGhioqKkiStWrVKwcHBcnd3l7e3twYNGqT09HTr/nFxcapRo4ZNmxs2bLBJbg4dOqRu3brJ3d1dHh4e6tChg/bt2ydJSklJUd++fVWzZk1Vr15drVq10ieffCKp4NC7s2fPauDAgWrQoIFcXV3Vpk0brVmzxubYoaGheu655zR58mTVqlVL3t7eio6OLtK5aN26tT766CP17dtXTZs21UMPPaTZs2fr3//+t65evVqMs1p8FeYZpdzcXK1atUojRoyQxWJRcnKy0tLSFB4ebq3j5OSkrl27avfu3TdsJycnR5mZmTYLAAAAcKeYMGGCvvzyS23cuFEJCQnatWuXDhw4YN2em5urV199VYcOHdKGDRuUnJys4cOHF+sYgwcPVoMGDZSYmKj9+/drypQpcnBwkCSNGzdOOTk52rlzp44cOaK5c+fKzc2t0HYuX76sDh066OOPP9bRo0c1evRoDRkyRF999ZVNvfj4eFWvXl1fffWV5s2bp1deecU6mqy4MjIy5OHhoWrVyvYpogozPfiGDRt0/vx560VOS0uTJHl5ednU8/LyKvS5pnyxsbGaOXNmmcUJAAAAlJWsrCzFx8dr9erV6t69uyRp5cqV8vX1tdYZMWKE9e8mTZpo6dKl6tixoy5cuHDDhMbeqVOnNGnSJDVv3lySFBAQYLPt8ccfV5s2bazHuJH69etr4sSJ1vVnn31WmzZt0ocffqiQkBBreVBQkGbMmGE91rJly7RlyxaFhYUVKd58Z8+e1auvvqoxY8YUa7+SqDA9Su+884569epl8yWQVGD8o2EYNx0TOXXqVGVkZFiX1NTUMokXAAAAKG0nT57UlStX1LFjR2uZp6en7rnnHuv6wYMH1a9fPzVq1Eju7u4KDQ2VdD3BKaoJEyZo1KhR6tGjh+bMmaMffvjBuu25557TrFmz9MADD2jGjBk6fPjwDdu5du2aZs+eraCgINWuXVtubm7avHlzgViCgoJs1n18fGyGCxZFZmamevfurZYtW1qTrrJUIRKllJQUff755xo1apS1zNvbW9L/9SzlS09PL9DL9HtOTk7y8PCwWQAAAIA7gWEYkgrvLJCk7OxshYeHy83NTatWrVJiYqLWr18v6fqQPEmqUqWKtX6+K1eu2KxHR0fr2LFj6t27t7Zu3aqWLVta2xk1apROnjypIUOG6MiRIwoODtbrr79eaLwLFy7U4sWLNXnyZG3dulVJSUnq2bOnNZZ8+cP68lksFuXl5RX5vGRlZenhhx+Wm5ub1q9fX6C9slAhEqWVK1eqXr166t27t7XM399f3t7eNmMXc3NztWPHDnXu3NmMMAEAAIAy1bRpUzk4OOi///2vtSwzM1MnTpyQJH3zzTf69ddfNWfOHHXp0kXNmzcv0DNTt25dZWVlKTs721qWlJRU4FiBgYEaP368Nm/erIiICK1cudK6zc/PT2PHjtW6dev0wgsv6O233y403l27dqlfv356+umn1bZtWzVp0sQaa2nJzMxUeHi4HB0dtXHjRjk7O5dq+zdieqKUl5enlStXatiwYTYPZFksFkVFRSkmJkbr16/X0aNHNXz4cLm6umrQoEEmRgwAAACUDXd3dw0bNkyTJk3Stm3bdOzYMY0YMUJVqlSRxWJRw4YN5ejoqNdff10nT57Uxo0b9eqrr9q0ERISIldXV02bNk3ff/+9Vq9erbi4OOv2S5cu6ZlnntH27duVkpKiL7/8UomJiWrRooUkKSoqSp999pmSk5N14MABbd261brNXrNmzZSQkKDdu3fr+PHjGjNmTIERYbcjKytL4eHhys7O1jvvvKPMzEylpaUpLS1N165dK7XjFMb0yRw+//xznTp1yuahtHyTJ0/WpUuXFBkZaX3h7ObNm+Xu7m5CpAAAALjT3QkvgF20aJHGjh2rPn36yMPDQ5MnT1ZqaqqcnZ1Vt25dxcXFadq0aVq6dKnat2+vBQsW6NFHH7XuX6tWLa1atUqTJk3SihUr1KNHD0VHR2v06NGSpKpVq+rs2bMaOnSofv75Z9WpU0cRERHWCdGuXbumcePG6fTp0/Lw8NDDDz+sxYsXFxrryy+/rOTkZPXs2VOurq4aPXq0+vfvr4yMjFI5F/v377fOoNesWTObbcnJyWrcuHGpHKcwFsN+AGMlk5mZKU9PT+s0gjDP4oTvbNbHhwXedHth7PfBbdoWW/x9uk0tn+OUREliq0DK6m3xZfWjoLzfbl8a7oQfSABu3+XLl5WcnCx/f/9yG6ZVlrKzs1W/fn0tXLiwXF60eqe72fUvTm5geo8SAAAAgP9z8OBBffPNN+rYsaMyMjL0yiuvSJL69etncmR3F9OfUQIAAABga8GCBWrbtq169Oih7Oxs7dq1S3Xq1DE7rFL3/vvvy83NrdClVatWpsZGjxIAAABQgbRr10779+83O4xy8eijj9q8mPb3ymMK8JshUQIAAABgCnd39wo7URuJEspEUSZmQCkqr8kSIOnOnMQAAO5GlXzOMtxAaV13nlECAABApZI/ZOvixYsmRwIz5F/32x26R48SAAAAKpWqVauqRo0aSk9PlyS5urrKYrGYHBXKmmEYunjxotLT01WjRg1VrVr1ttojUQIAAECl4+3tLUnWZAl3jxo1aliv/+0gUQIAAEClY7FY5OPjo3r16unKlStmh4Ny4uDgcNs9SflIlAAAAFBpVa1atdR+OOPuwmQOAAAAAGCHHiWUCqYDL0VM9V1qmMYbAACUFD1KAAAAAGCHRAkAAAAA7JAoAQAAAIAdEiUAAAAAsEOiBAAAAAB2SJQAAAAAwA7Tg8M0TCl+h2L68jsO06QDAFB89CgBAAAAgB0SJQAAAACwQ6IEAAAAAHZIlAAAAADADokSAAAAANghUQIAAAAAO6YnSv/73//09NNPq3bt2nJ1ddW9996r/fv3W7cbhqHo6Gj5+vrKxcVFoaGhOnbsmIkRAwAAAKjsTE2Uzp07pwceeEAODg769NNP9fXXX2vhwoWqUaOGtc68efO0aNEiLVu2TImJifL29lZYWJiysrLMCxwAAABApWbqC2fnzp0rPz8/rVy50lrWuHFj69+GYWjJkiWaPn26IiIiJEnx8fHy8vLS6tWrNWbMmPIOGQAAAMBdwNQepY0bNyo4OFhPPPGE6tWrp3bt2untt9+2bk9OTlZaWprCw8OtZU5OTuratat2795daJs5OTnKzMy0WQAAAACgOExNlE6ePKk333xTAQEB+uyzzzR27Fg999xzeu+99yRJaWlpkiQvLy+b/by8vKzb7MXGxsrT09O6+Pn5le2HAAAAAFDpmJoo5eXlqX379oqJiVG7du00ZswY/fnPf9abb75pU89isdisG4ZRoCzf1KlTlZGRYV1SU1PLLH4AAAAAlZOpiZKPj49atmxpU9aiRQudOnVKkuTt7S1JBXqP0tPTC/Qy5XNycpKHh4fNAgAAAADFYepkDg888IC+/fZbm7LvvvtOjRo1kiT5+/vL29tbCQkJateunSQpNzdXO3bs0Ny5c8s93rvV4oTvbNbHhwWaFInJtsUWf59uU0s/DhMtP3+4TNqNrBFUJu0CAACUlKmJ0vjx49W5c2fFxMToySef1H//+1+tWLFCK1askHR9yF1UVJRiYmIUEBCggIAAxcTEyNXVVYMGDTIzdAAAAACVmKmJ0n333af169dr6tSpeuWVV+Tv768lS5Zo8ODB1jqTJ0/WpUuXFBkZqXPnzikkJESbN2+Wu7u7iZEDAAAAqMxMTZQkqU+fPurTp88Nt1ssFkVHRys6Orr8ggIAAABwVzN1MgcAAAAAqIhIlAAAAADADokSAAAAANghUQIAAAAAOyRKAAAAAGCHRAkAAAAA7JAoAQAAAIAd09+jBBTH4oTvCpSNDws0IRJUSNtibdfPHy7afv5dSj8WVEjLk5aXSbuR90aWSbsAAPPQowQAAAAAdkiUAAAAAMAOQ+9Q6dgPz2NoHspE8q7i78MQPxQTQwUBwDz0KAEAAACAHRIlAAAAALBTKolSZmamNmzYoOPHj5dGcwAAAABgqhIlSk8++aSWLVsmSbp06ZKCg4P15JNPKigoSB999FGpBggAAAAA5a1EidLOnTvVpcv1h5LXr18vwzB0/vx5LV26VLNmzSrVAAEAAACgvJUoUcrIyFCtWrUkSZs2bdLjjz8uV1dX9e7dWydOnCjVAAEAAACgvJUoUfLz89OePXuUnZ2tTZs2KTw8XJJ07tw5OTs7l2qAAAAAAFDeSvQepaioKA0ePFhubm5q2LChQkNDJV0fktemTZvSjA8AAAAAyl2JEqXIyEh17NhRqampCgsLU5Uq1zummjRpwjNKAAAAAO54JUqUJCk4OFhBQUFKTk5W06ZNVa1aNfXu3bs0YwMAAAAAU5ToGaWLFy9q5MiRcnV1VatWrXTq1ClJ0nPPPac5c+aUaoAAAAAAUN5KlChNnTpVhw4d0vbt220mb+jRo4fWrl1basEBAAAAgBlKNPRuw4YNWrt2re6//35ZLBZrecuWLfXDDz+UWnAAAAAAYIYS9Sj98ssvqlevXoHy7Oxsm8TpVqKjo2WxWGwWb29v63bDMBQdHS1fX1+5uLgoNDRUx44dK0nIAAAAAFBkJUqU7rvvPv3nP/+xrucnR2+//bY6depUrLZatWqlM2fOWJcjR45Yt82bN0+LFi3SsmXLlJiYKG9vb4WFhSkrK6skYQMAAABAkZRo6F1sbKwefvhhff3117p69apee+01HTt2THv27NGOHTuKF0C1aja9SPkMw9CSJUs0ffp0RURESJLi4+Pl5eWl1atXa8yYMSUJHQAAAABuqUQ9Sp07d9aXX36pixcvqmnTptq8ebO8vLy0Z88edejQoVhtnThxQr6+vvL399dTTz2lkydPSpKSk5OVlpam8PBwa10nJyd17dpVu3fvvmF7OTk5yszMtFkAAAAAoDhK/B6lNm3aKD4+/rYOHhISovfee0+BgYH6+eefNWvWLHXu3FnHjh1TWlqaJMnLy8tmHy8vL6WkpNywzdjYWM2cOfO24gIAAABwdytyopSZmSkPDw/r3zeTX+9WevXqZf27TZs26tSpk5o2bar4+Hjdf//9klRgcgjDMG46YcTUqVM1YcIEm7j9/PyKFA8AAAAASMVIlGrWrKkzZ86oXr16qlGjRqHJSn4Sc+3atRIFU716dbVp00YnTpxQ//79JUlpaWny8fGx1klPTy/Qy/R7Tk5OcnJyKtHxAQAAAEAqRqK0detW1apVS5K0bdu2MgkmJydHx48fV5cuXeTv7y9vb28lJCSoXbt2kqTc3Fzt2LFDc+fOLZPjAwAAAIBUjESpa9euhf59OyZOnKi+ffuqYcOGSk9P16xZs5SZmalhw4bJYrEoKipKMTExCggIUEBAgGJiYuTq6qpBgwaVyvEBAAAAoDAlmsxh5cqVcnNz0xNPPGFT/uGHH+rixYsaNmxYkdo5ffq0Bg4cqF9//VV169bV/fffr71796pRo0aSpMmTJ+vSpUuKjIzUuXPnFBISos2bN8vd3b0kYQMAAABAkZQoUZozZ47eeuutAuX16tXT6NGji5woffDBBzfdbrFYFB0drejo6JKECQAAAAAlUqL3KKWkpMjf379AeaNGjXTq1KnbDgoAAAAAzFSiRKlevXo6fPhwgfJDhw6pdu3atx0UAAAAAJipRInSU089peeee07btm3TtWvXdO3aNW3dulXPP/+8nnrqqdKOEQAAAADKVYmeUZo1a5ZSUlLUvXt3Vat2vYm8vDwNHTpUMTExpRogAAAAAJS3EiVKjo6OWrt2rV599VUdOnRILi4uatOmjXW2OqA8LU74zuwQcKdL3mV2BLjDLU9abnYIAIBSVqJEKV9gYKACAwNLKxYAAAAAqBBKlChdu3ZNcXFx2rJli9LT05WXl2ezfevWraUSHAAAAACYoUSJ0vPPP6+4uDj17t1brVu3lsViKe24AAAAAMA0JUqUPvjgA/3jH//QI488UtrxAAAAAIDpSjQ9uKOjo5o1a1basQAAAABAhVCiROmFF17Qa6+9JsMwSjseAAAAADBdiYbeffHFF9q2bZs+/fRTtWrVSg4ODjbb161bVyrBAQAAAIAZSpQo1ahRQ4899lhpxwIAAAAAFUKJEqWVK1eWdhwAAAAAUGGU+IWzV69e1fbt2/XDDz9o0KBBcnd3108//SQPDw+5ubmVZowoZ4sTvjM7hLK1LdbsCMrc8vOHzQ4BAADgjlaiRCklJUUPP/ywTp06pZycHIWFhcnd3V3z5s3T5cuX9dZbb5V2nAAAAABQbko0693zzz+v4OBgnTt3Ti4uLtbyxx57TFu2bCm14AAAAADADCWe9e7LL7+Uo6OjTXmjRo30v//9r1QCAwAAAACzlChRysvL07Vr1wqUnz59Wu7u7rcdFCq2Sv8MU2m6C56HKg08UwUAACqaEg29CwsL05IlS6zrFotFFy5c0IwZM/TII4+UVmwAAAAAYIoS9SgtXrxY3bp1U8uWLXX58mUNGjRIJ06cUJ06dbRmzZrSjhEAAAAAylWJEiVfX18lJSVpzZo1OnDggPLy8jRy5EgNHjzYZnIHAAAAALgTlfg9Si4uLhoxYoRGjBhRmvEAAAAAgOlKlCi99957N90+dOjQEgUDAAAAABVBiRKl559/3mb9ypUrunjxohwdHeXq6lqiRCk2NlbTpk3T888/b50owjAMzZw5UytWrNC5c+cUEhKiN954Q61atSpJ2AAAAABQJCWa9e7cuXM2y4ULF/Ttt9/qD3/4Q4kmc0hMTNSKFSsUFBRkUz5v3jwtWrRIy5YtU2Jiory9vRUWFqasrKyShA0AAAAARVKiRKkwAQEBmjNnToHeplu5cOGCBg8erLfffls1a9a0lhuGoSVLlmj69OmKiIhQ69atFR8fr4sXL2r16tWlFTYAAAAAFFBqiZIkVa1aVT/99FOx9hk3bpx69+6tHj162JQnJycrLS1N4eHh1jInJyd17dpVu3fvvmF7OTk5yszMtFkAAAAAoDhK9IzSxo0bbdYNw9CZM2e0bNkyPfDAA0Vu54MPPtCBAweUmJhYYFtaWpokycvLy6bcy8tLKSkpN2wzNjZWM2fOLHIMuDvtOXnWZr1Tk9omRQLcQvKu4u/j36X04wCKYHnS8jJpN/LeyDJpFwBupkSJUv/+/W3WLRaL6tatq4ceekgLFy4sUhupqal6/vnntXnzZjk7O9+wnsVisVk3DKNA2e9NnTpVEyZMsK5nZmbKz8+vSDEBAAAAgFTCRCkvL++2D7x//36lp6erQ4cO1rJr165p586dWrZsmb799ltJ13uWfHx8rHXS09ML9DL9npOTk5ycnG47PgAAAAB3r1J9Rqk4unfvriNHjigpKcm6BAcHa/DgwUpKSlKTJk3k7e2thIQE6z65ubnasWOHOnfubFbYAAAAAO4CJepR+v3QtltZtGhRoeXu7u5q3bq1TVn16tVVu3Zta3lUVJRiYmIUEBCggIAAxcTEyNXVVYMGDSpJ2AAAAABQJCVKlA4ePKgDBw7o6tWruueeeyRJ3333napWrar27dtb693sWaKimDx5si5duqTIyEjrC2c3b94sd3f322oXAAAAAG6mRIlS37595e7urvj4eOu7j86dO6c//elP6tKli1544YUSBbN9+3abdYvFoujoaEVHR5eoPQAAAAAoiRI9o7Rw4ULFxsbavCC2Zs2amjVrVpFnvQMAAACAiqpEiVJmZqZ+/vnnAuXp6enKysq67aAAAAAAwEwlSpQee+wx/elPf9I///lPnT59WqdPn9Y///lPjRw5UhEREaUdIwAAAACUqxI9o/TWW29p4sSJevrpp3XlypXrDVWrppEjR2r+/PmlGiAAAAAAlLcSJUqurq5avny55s+frx9++EGGYahZs2aqXr16accHAAAAAOXutl44e+bMGZ05c0aBgYGqXr26DMMorbgAAAAAwDQlSpTOnj2r7t27KzAwUI888ojOnDkjSRo1alSJpwYHAAAAgIqiREPvxo8fLwcHB506dUotWrSwlg8YMEDjx49nivA7yOKE78wO4bbcf2rFrSttq132gZTQ8vOHzQ4BAAAAhShRorR582Z99tlnatCggU15QECAUlJSSiUwAAAAADBLiYbeZWdny9XVtUD5r7/+Kicnp9sOCgAAAADMVKJE6cEHH9R7771nXbdYLMrLy9P8+fPVrVu3UgsOAAAAAMxQoqF38+fPV2hoqPbt26fc3FxNnjxZx44d02+//aYvv/yytGMEbsuek2fNDgG4LnmX2REAkqTlScvNDgEAKrwS9Si1bNlShw8fVseOHRUWFqbs7GxFRETo4MGDatq0aWnHCAAAAADlqtg9SleuXFF4eLj+3//7f5o5c2ZZxAQAAAAApip2j5KDg4OOHj0qi8VSFvEAAAAAgOlKNPRu6NCheuedd0o7FgAAAACoEEo0mUNubq7+9re/KSEhQcHBwapevbrN9kWLFpVKcAAAAABghmIlSidPnlTjxo119OhRtW/fXpL03Xff2dRhSB4AAACAO12xEqWAgACdOXNG27ZtkyQNGDBAS5culZeXV5kEBwAAAABmKNYzSoZh2Kx/+umnys7OLtWAAAAAAMBsJZrMIZ994gQAAAAAlUGxEiWLxVLgGSSeSQIAAABQ2RTrGSXDMDR8+HA5OTlJki5fvqyxY8cWmPVu3bp1pRchAAAAAJSzYiVKw4YNs1l/+umnSzUYAAAAAKgIipUorVy5sqziAAAAAIAK47Ymc7hdb775poKCguTh4SEPDw916tRJn376qXW7YRiKjo6Wr6+vXFxcFBoaqmPHjpkYMQAAAIC7gamJUoMGDTRnzhzt27dP+/bt00MPPaR+/fpZk6F58+Zp0aJFWrZsmRITE+Xt7a2wsDBlZWWZGTYAAACASs7URKlv37565JFHFBgYqMDAQM2ePVtubm7au3evDMPQkiVLNH36dEVERKh169aKj4/XxYsXtXr1ajPDBgAAAFDJmZoo/d61a9f0wQcfKDs7W506dVJycrLS0tIUHh5urePk5KSuXbtq9+7dN2wnJydHmZmZNgsAAAAAFIfpidKRI0fk5uYmJycnjR07VuvXr1fLli2VlpYmSfLy8rKp7+XlZd1WmNjYWHl6eloXPz+/Mo0fAAAAQOVjeqJ0zz33KCkpSXv37tVf/vIXDRs2TF9//bV1u/0LbQ3DuOlLbqdOnaqMjAzrkpqaWmaxAwAAAKicijU9eFlwdHRUs2bNJEnBwcFKTEzUa6+9phdffFGSlJaWJh8fH2v99PT0Ar1Mv+fk5GR9IS4AAAAAlITpPUr2DMNQTk6O/P395e3trYSEBOu23Nxc7dixQ507dzYxQgAAAACVnak9StOmTVOvXr3k5+enrKwsffDBB9q+fbs2bdoki8WiqKgoxcTEKCAgQAEBAYqJiZGrq6sGDRpkZtgAAAAAKjlTE6Wff/5ZQ4YM0ZkzZ+Tp6amgoCBt2rRJYWFhkqTJkyfr0qVLioyM1Llz5xQSEqLNmzfL3d3dzLABAAAAVHKmJkrvvPPOTbdbLBZFR0crOjq6fAICAAAAAFXAZ5QAAAAAwGwkSgAAAABgh0QJAAAAAOyY/h4loCLYc/JsgbJOTWqbEAlQCpJ3lWw//y6lGwdQSpYnLS+TdiPvjSyTdgFUDvQoAQAAAIAdEiUAAAAAsEOiBAAAAAB2SJQAAAAAwA6JEgAAAADYIVECAAAAADskSgAAAABgh0QJAAAAAOyQKAEAAACAHRIlAAAAALBDogQAAAAAdkiUAAAAAMAOiRIAAAAA2CFRAgAAAAA7JEoAAAAAYIdECQAAAADskCgBAAAAgB0SJQAAAACwQ6IEAAAAAHZIlAAAAADAjqmJUmxsrO677z65u7urXr166t+/v7799lubOoZhKDo6Wr6+vnJxcVFoaKiOHTtmUsQAAAAA7gamJko7duzQuHHjtHfvXiUkJOjq1asKDw9Xdna2tc68efO0aNEiLVu2TImJifL29lZYWJiysrJMjBwAAABAZVbNzINv2rTJZn3lypWqV6+e9u/frwcffFCGYWjJkiWaPn26IiIiJEnx8fHy8vLS6tWrNWbMGDPCBgAAAFDJVahnlDIyMiRJtWrVkiQlJycrLS1N4eHh1jpOTk7q2rWrdu/eXWgbOTk5yszMtFkAAAAAoDgqTKJkGIYmTJigP/zhD2rdurUkKS0tTZLk5eVlU9fLy8u6zV5sbKw8PT2ti5+fX9kGDgAAAKDSqTCJ0jPPPKPDhw9rzZo1BbZZLBabdcMwCpTlmzp1qjIyMqxLampqmcQLAAAAoPIy9RmlfM8++6w2btyonTt3qkGDBtZyb29vSdd7lnx8fKzl6enpBXqZ8jk5OcnJyalsAwYAAABQqZnao2QYhp555hmtW7dOW7dulb+/v812f39/eXt7KyEhwVqWm5urHTt2qHPnzuUdLgAAAIC7hKk9SuPGjdPq1av1r3/9S+7u7tbnjjw9PeXi4iKLxaKoqCjFxMQoICBAAQEBiomJkaurqwYNGmRm6HeExQnfFSgbHxZYLse+/9SKYu+zt+Ho2z7uxirf33Yb1rZ+/L+2Hs1rpk5Napda20CFlLyr+Pv4dyn9OIBysjxpeZm0G3lvZJm0C6B8mZoovfnmm5Kk0NBQm/KVK1dq+PDhkqTJkyfr0qVLioyM1Llz5xQSEqLNmzfL3d29nKMFAAAAcLcwNVEyDOOWdSwWi6KjoxUdHV32AQEAAACAJItRlGzlDpaZmSlPT09lZGTIw8PD7HDKVWFD78pLSYbelYbSHHp3K341XMrtWEClwnA9oEQY0gfcvuLkBhVmenAAAAAAqChIlAAAAADADokSAAAAANghUQIAAAAAOyRKAAAAAGCHRAkAAAAA7JAoAQAAAIAdEiUAAAAAsEOiBAAAAAB2SJQAAAAAwA6JEgAAAADYIVECAAAAADskSgAAAABgp5rZAaD0LE74rkzavf/UilJvc2OV70u9TQAAAKC00KMEAAAAAHZIlAAAAADADokSAAAAANghUQIAAAAAOyRKAAAAAGCHRAkAAAAA7JAoAQAAAIAdEiUAAAAAsEOiBAAAAAB2TE2Udu7cqb59+8rX11cWi0UbNmyw2W4YhqKjo+Xr6ysXFxeFhobq2LFj5gQLAAAA4K5RzcyDZ2dnq23btvrTn/6kxx9/vMD2efPmadGiRYqLi1NgYKBmzZqlsLAwffvtt3J3dzch4oplccJ3xap//6kVxT7GxirfayP9jgAAALjLmJoo9erVS7169Sp0m2EYWrJkiaZPn66IiAhJUnx8vLy8vLR69WqNGTOmPEMFAAAAcBepsH0FycnJSktLU3h4uLXMyclJXbt21e7du2+4X05OjjIzM20WAAAAACiOCpsopaWlSZK8vLxsyr28vKzbChMbGytPT0/r4ufnV6ZxAgAAAKh8KmyilM9isdisG4ZRoOz3pk6dqoyMDOuSmppa1iECAAAAqGRMfUbpZry9vSVd71ny8fGxlqenpxfoZfo9JycnOTk5lXl8AAAAACqvCtuj5O/vL29vbyUkJFjLcnNztWPHDnXu3NnEyAAAAABUdqb2KF24cEHff/+9dT05OVlJSUmqVauWGjZsqKioKMXExCggIEABAQGKiYmRq6urBg0aZGLUwI2lnr9ks+5Xw8WkSIBKJnlX8ffx71L6cQCV0PKk5WXSbuS9kWXSLlBeTE2U9u3bp27dulnXJ0yYIEkaNmyY4uLiNHnyZF26dEmRkZE6d+6cQkJCtHnzZt6hBAAAAKBMmZoohYaGyjCMG263WCyKjo5WdHR0+QUFAAAA4K5XYZ9RAgAAAACzkCgBAAAAgB0SJQAAAACwQ6IEAAAAAHZIlAAAAADADokSAAAAANghUQIAAAAAOyRKAAAAAGDH1BfOougWJ3xns37/qRW6/3frG6t8f8s2NpIWAwAAAEXCT2cAAAAAsEOiBAAAAAB2SJQAAAAAwA6JEgAAAADYIVECAAAAADskSgAAAABgx2IYhmF2EGUpMzNTnp6eysjIkIeHh9nhaHnS8oKFybsKFKWev1QO0cAMfjVczA4BwM34dzE7AgA3EXlvpNkh4A5WnNyAHiUAAAAAsEOiBAAAAAB2SJQAAAAAwA6JEgAAAADYIVECAAAAADskSgAAAABgp5rZAQB3O/up4Jk+HLgDFfKah1tiGnKgQin0FS6lgOnM71z0KAEAAACAHRIlAAAAALBzRwy9W758uebPn68zZ86oVatWWrJkibp0YcgC7l72w/UkhuwBuAmGBgIoIoYg/p8K36O0du1aRUVFafr06Tp48KC6dOmiXr166dSpU2aHBgAAAKCSqvCJ0qJFizRy5EiNGjVKLVq00JIlS+Tn56c333zT7NAAAAAAVFIVeuhdbm6u9u/frylTptiUh4eHa/fu3YXuk5OTo5ycHOt6RkaGJCkzM7PsAi2GSxcKDpnSxSsFinIuFSxD5XDJ0fa2s7/W9tsLU9j3oyj7ASiCwv47fSuF/He8TI5TUhU9PqAYyuo3XaG/0UpBRfkNWlSV/Tzkx2EYxi3rVuhfVr/++quuXbsmLy8vm3IvLy+lpaUVuk9sbKxmzpxZoNzPz69MYgQAVDYfVbLjlFRFjw93q4maaHYIxXKnxVtWKtp5yMrKkqen503rVOhEKZ/FYrFZNwyjQFm+qVOnasKECdb1vLw8/fbbb6pdu/YN94G5MjMz5efnp9TUVHl4eJgdDkoB17Ry4XpWPlzTyoXrWflwTcuOYRjKysqSr6/vLetW6ESpTp06qlq1aoHeo/T09AK9TPmcnJzk5ORkU1ajRo2yChGlyMPDg/8YVDJc08qF61n5cE0rF65n5cM1LRu36knKV6Enc3B0dFSHDh2UkJBgU56QkKDOnTubFBUAAACAyq5C9yhJ0oQJEzRkyBAFBwerU6dOWrFihU6dOqWxY8eaHRoAAACASqrCJ0oDBgzQ2bNn9corr+jMmTNq3bq1PvnkEzVq1Mjs0FBKnJycNGPGjAJDJnHn4ppWLlzPyodrWrlwPSsfrmnFYDGKMjceAAAAANxFKvQzSgAAAABgBhIlAAAAALBDogQAAAAAdkiUAAAAAMAOiRJMt3z5cvn7+8vZ2VkdOnTQrl27zA4JJRAdHS2LxWKzeHt7mx0WimHnzp3q27evfH19ZbFYtGHDBpvthmEoOjpavr6+cnFxUWhoqI4dO2ZOsLilW13P4cOHF7hn77//fnOCxS3Fxsbqvvvuk7u7u+rVq6f+/fvr22+/tanDPXpnKco15T41F4kSTLV27VpFRUVp+vTpOnjwoLp06aJevXrp1KlTZoeGEmjVqpXOnDljXY4cOWJ2SCiG7OxstW3bVsuWLSt0+7x587Ro0SItW7ZMiYmJ8vb2VlhYmLKysso5UhTFra6nJD388MM29+wnn3xSjhGiOHbs2KFx48Zp7969SkhI0NWrVxUeHq7s7GxrHe7RO0tRrqnEfWompgeHqUJCQtS+fXu9+eab1rIWLVqof//+io2NNTEyFFd0dLQ2bNigpKQks0NBKbBYLFq/fr369+8v6fq/VPv6+ioqKkovvviiJCknJ0deXl6aO3euxowZY2K0uBX76yld/5fq8+fPF+hpwp3hl19+Ub169bRjxw49+OCD3KOVgP01lbhPzUaPEkyTm5ur/fv3Kzw83KY8PDxcu3fvNikq3I4TJ07I19dX/v7+euqpp3Ty5EmzQ0IpSU5OVlpams396uTkpK5du3K/3sG2b9+uevXqKTAwUH/+85+Vnp5udkgoooyMDElSrVq1JHGPVgb21zQf96l5SJRgml9//VXXrl2Tl5eXTbmXl5fS0tJMigolFRISovfee0+fffaZ3n77baWlpalz5846e/as2aGhFOTfk9yvlUevXr30/vvva+vWrVq4cKESExP10EMPKScnx+zQcAuGYWjChAn6wx/+oNatW0viHr3TFXZNJe5Ts1UzOwDAYrHYrBuGUaAMFV+vXr2sf7dp00adOnVS06ZNFR8frwkTJpgYGUoT92vlMWDAAOvfrVu3VnBwsBo1aqT//Oc/ioiIMDEy3Mozzzyjw4cP64svviiwjXv0znSja8p9ai56lGCaOnXqqGrVqgX+pSs9Pb3Av4jhzlO9enW1adNGJ06cMDsUlIL8GQy5XysvHx8fNWrUiHu2gnv22We1ceNGbdu2TQ0aNLCWc4/euW50TQvDfVq+SJRgGkdHR3Xo0EEJCQk25QkJCercubNJUaG05OTk6Pjx4/Lx8TE7FJQCf39/eXt729yvubm52rFjB/drJXH27FmlpqZyz1ZQhmHomWee0bp167R161b5+/vbbOcevfPc6poWhvu0fDH0DqaaMGGChgwZouDgYHXq1EkrVqzQqVOnNHbsWLNDQzFNnDhRffv2VcOGDZWenq5Zs2YpMzNTw4YNMzs0FNGFCxf0/fffW9eTk5OVlJSkWrVqqWHDhoqKilJMTIwCAgIUEBCgmJgYubq6atCgQSZGjRu52fWsVauWoqOj9fjjj8vHx0c//vijpk2bpjp16uixxx4zMWrcyLhx47R69Wr961//kru7u7XnyNPTUy4uLrJYLNyjd5hbXdMLFy5wn5rNAEz2xhtvGI0aNTIcHR2N9u3bGzt27DA7JJTAgAEDDB8fH8PBwcHw9fU1IiIijGPHjpkdFoph27ZthqQCy7BhwwzDMIy8vDxjxowZhre3t+Hk5GQ8+OCDxpEjR8wNGjd0s+t58eJFIzw83Khbt67h4OBgNGzY0Bg2bJhx6tQps8PGDRR2LSUZK1eutNbhHr2z3Oqacp+aj/coAQAAAIAdnlECAAAAADskSgAAAABgh0QJAAAAAOyQKAEAAACAHRIlAAAAALBDogQAAAAAdkiUAAAAAMAOiRIAAAAA2CFRAgBUOqGhoYqKijI7DADAHYxECQBQofTt21c9evQodNuePXtksVh04MCBco4KAHC3IVECAFQoI0eO1NatW5WSklJg27vvvqt7771X7du3NyEyAMDdhEQJAFCh9OnTR/Xq1VNcXJxN+cWLF7V27Vr1799fAwcOVIMGDeTq6qo2bdpozZo1N23TYrFow4YNNmU1atSwOcb//vc/DRgwQDVr1lTt2rXVr18//fjjj9bt27dvV8eOHVW9enXVqFFDDzzwQKHJHACgciBRAgBUKNWqVdPQoUMVFxcnwzCs5R9++KFyc3M1atQodejQQR9//LGOHj2q0aNHa8iQIfrqq69KfMyLFy+qW7ducnNz086dO/XFF1/Izc1NDz/8sHJzc3X16lX1799fXbt21eHDh7Vnzx6NHj1aFoulND4yAKACqmZ2AAAA2BsxYoTmz5+v7du3q1u3bpKuD7uLiIhQ/fr1NXHiRGvdZ599Vps2bdKHH36okJCQEh3vgw8+UJUqVfS3v/3NmvysXLlSNWrU0Pbt2xUcHKyMjAz16dNHTZs2lSS1aNHiNj8lAKAio0cJAFDhNG/eXJ07d9a7774rSfrhhx+0a9cujRgxQteuXdPs2bMVFBSk2rVry83NTZs3b9apU6dKfLz9+/fr+++/l7u7u9zc3OTm5qZatWrp8uXL+uGHH1SrVi0NHz5cPXv2VN++ffXaa6/pzJkzpfVxAQAVEIkSAKBCGjlypD766CNlZmZq5cqVatSokbp3766FCxdq8eLFmjx5srZu3aqkpCT17NlTubm5N2zLYrHYDOOTpCtXrlj/zsvLU4cOHZSUlGSzfPfddxo0aJCk6z1Me/bsUefOnbV27VoFBgZq7969ZfPhAQCmI1ECAFRITz75pKpWrarVq1crPj5ef/rTn2SxWLRr1y7169dPTz/9tNq2basmTZroxIkTN22rbt26Nj1AJ06c0MWLF63r7du314kTJ1SvXj01a9bMZvH09LTWa9eunaZOnardu3erdevWWr16del/cABAhUCiBACokNzc3DRgwABNmzZNP/30k4YPHy5JatasmRISErR7924dP35cY8aMUVpa2k3beuihh7Rs2TIdOHBA+/bt09ixY+Xg4GDdPnjwYNWpU0f9+vXTrl27lJycrB07duj555/X6dOnlZycrKlTp2rPnj1KSUnR5s2b9d133/GcEgBUYiRKAIAKa+TIkTp37px69Oihhg0bSpJefvlltW/fXj179lRoaKi8vb3Vv3//m7azcOFC+fn56cEHH9SgQYM0ceJEubq6Wre7urpq586datiwoSIiItSiRQuNGDFCly5dkoeHh1xdXfXNN9/o8ccfV2BgoEaPHq1nnnlGY8aMKcuPDwAwkcWwH7QNAAAAAHc5epQAAAAAwA6JEgAAAADYIVECAAAAADskSgAAAABgh0QJAAAAAOyQKAEAAACAHRIlAAAAALBDogQAAAAAdkiUAAAAAMAOiRIAAAAA2CFRAgAAAAA7/x/X5hQ4nhP7wwAAAABJRU5ErkJggg==\n",
"text/plain": [
"<Figure size 1000x400 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# Test your function\n",
"\n",
"gaussian_0 = gaussian_generator(0, 1, 1000)\n",
"gaussian_1 = gaussian_generator(5, 3, 1000)\n",
"gaussian_2 = gaussian_generator(10, 5, 1000)\n",
"\n",
"utils.plot_gaussian_distributions(gaussian_0, gaussian_1, gaussian_2)"
]
},
{
"cell_type": "markdown",
"id": "c3296d4a",
"metadata": {},
"source": [
"##### __Expected Output__\n",
"\n",
"<img src=\"./assets/gaussian.png\" style=\"height: 300px;\"/>\n"
]
},
{
"cell_type": "markdown",
"id": "d78cc376",
"metadata": {},
"source": [
"## Exercise 3: Binomial Generator\n",
"\n",
"### 3.1 Inverse CDF - Binomial\n",
"\n",
"If $X \\sim \\text{Binomial}(n,p)$, then its PMF is given by:\n",
"\n",
"$$P(X = k) = {n \\choose k}p^{k}(1-p)^{n-k}.$$\n",
"\n",
"Therefore, if $0 \\leq x \\leq n$, its CDF is given by:\n",
"\n",
"$$F(x) = P(X \\leq x) = P(X = 0) + P(X = 1) + \\ldots + P(X = \\lfloor x \\rfloor) = \\sum_{k=0}^{\\lfloor x \\rfloor} {n \\choose k}p^{k}(1-p)^{n-k}.$$\n",
"\n",
"Here, $\\lfloor x \\rfloor$ denotes the [floor function](https://en.wikipedia.org/wiki/Floor_and_ceiling_functions), which returns the greatest integer less than or equal to $x$. For example, $\\lfloor 2.9 \\rfloor = 2$ and $\\lfloor 1.2 \\rfloor = 1$. This function is necessary because the domain of $F$ is the entire set of real numbers, but $P(X = k)$ is non-zero only for positive integer values between 0 and $n$\n",
"\n",
"If $x > n$, then $F(x) = 1$. It is important to note that the expression for $F(x)$ can become complex and messy, and there is no closed-form expression for the inverse function $F^{-1}$ in this case. However, statistical libraries provide implementations of the inverse CDF using [generalized quantile functions](https://en.wikipedia.org/wiki/Cumulative_distribution_function#Inverse_distribution_function_.28quantile_function.29). You can refer to [scipy.stats.binom](https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.binom.html) for an example of a library that implements these functions. In particular, the `scipy.stats.binom.ppf` function is what you need. Since the `binom` class is already imported, you can use `help(binom)` to explore its parameters and functions. The function you will need is located in the \"Methods\" section: `ppf`.\n"
]
},
{
"cell_type": "code",
"execution_count": 45,
"id": "1819bfed",
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": [
"def inverse_cdf_binomial(y, n, p):\n",
" \"\"\"\n",
" Calculates the inverse cumulative distribution function (CDF) of a binomial distribution.\n",
"\n",
" Parameters:\n",
" - y (float or ndarray): The probability or array of probabilities.\n",
" - n (int): The number of trials in the binomial distribution.\n",
" - p (float): The probability of success in each trial.\n",
"\n",
" Returns:\n",
" - x (float or ndarray): The corresponding value(s) from the binomial distribution that correspond to the given probability/ies.\n",
" \"\"\"\n",
"\n",
" ### START CODE HERE ###\n",
" x = binom.ppf(y, n, p)\n",
" ### END CODE HERE ###\n",
"\n",
" return x"
]
},
{
"cell_type": "code",
"execution_count": 46,
"id": "c6c769c3",
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Inverse of Binomial CDF with n 15 and p 0.9 for value 1e-10: 3.000\n",
"Inverse of Binomial CDF with n 15 and p 0.5 for value 0: -1.0\n",
"Inverse of Binomial CDF with n 20 and p 0.2 for value 0.4: 3.000\n",
"Inverse of Binomial CDF with n 15 and p 0.5 for value 1: 15.0\n"
]
}
],
"source": [
"# Test your function\n",
"\n",
"print(f\"Inverse of Binomial CDF with n {15} and p {0.9} for value {1e-10}: {inverse_cdf_binomial(1e-10, 15, 0.9):.3f}\")\n",
"print(f\"Inverse of Binomial CDF with n {15} and p {0.5} for value {0}: {inverse_cdf_binomial(0, 15, 0.5)}\")\n",
"print(f\"Inverse of Binomial CDF with n {20} and p {0.2} for value {0.4}: {inverse_cdf_binomial(0.4, 20, 0.2):.3f}\")\n",
"print(f\"Inverse of Binomial CDF with n {15} and p {0.5} for value {1}: {inverse_cdf_binomial(1, 15, 0.5)}\")"
]
},
{
"cell_type": "markdown",
"id": "9f4fa672",
"metadata": {},
"source": [
"##### __Expected Output__\n",
"\n",
"```\n",
"Inverse of Binomial CDF with n 15 and p 0.9 for value 1e-10: 3.000\n",
"Inverse of Binomial CDF with n 15 and p 0.5 for value 0: -1.0\n",
"Inverse of Binomial CDF with n 20 and p 0.2 for value 0.4: 3.000\n",
"Inverse of Binomial CDF with n 15 and p 0.5 for value 1: 15.0\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "6cc766a4",
"metadata": {},
"source": [
"### 3.2 Binomial Generator\n"
]
},
{
"cell_type": "code",
"execution_count": 47,
"id": "ea27b236",
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": [
"def binomial_generator(n, p, num_samples):\n",
" \"\"\"\n",
" Generates an array of binomially distributed random numbers.\n",
"\n",
" Args:\n",
" n (int): The number of trials in the binomial distribution.\n",
" p (float): The probability of success in each trial.\n",
" num_samples (int): The number of samples to generate.\n",
"\n",
" Returns:\n",
" array: An array of binomially distributed random numbers.\n",
" \"\"\"\n",
" ### START CODE HERE ###\n",
"\n",
" # Generate an array with num_samples elements that distribute uniformally between 0 and 1\n",
" u = uniform_generator(0,1,num_samples)\n",
"\n",
" # Use the uniform-distributed sample to generate binomial-distributed data\n",
" # Hint: You need to sample from the inverse of the CDF of the distribution you are generating\n",
" array = inverse_cdf_binomial(u, n ,p)\n",
" ### END CODE HERE ###\n",
"\n",
" return array"
]
},
{
"cell_type": "code",
"execution_count": 48,
"id": "e06e9f1e",
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA1IAAAGHCAYAAAC6SmOyAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy88F64QAAAACXBIWXMAAA9hAAAPYQGoP6dpAABIVElEQVR4nO3deVRV9f7/8deRSUAEAWVQnNE0UVNyoGtiqTiWw01z1jQrlRsXvV1JS7zXtOyrWJqW17FB81ep+e1aSTlkoWYamn1N7UYOCZdyAAUFhf37w8VZHgFlH48e0OdjrbPy7P3Zn/3eh83Ol5+9P8diGIYhAAAAAECZVXJ2AQAAAABQ0RCkAAAAAMAkghQAAAAAmESQAgAAAACTCFIAAAAAYBJBCgAAAABMIkgBAAAAgEkEKQAAAAAwiSAFAAAAACYRpABUOCtWrJDFYtF3331X4vpevXqpbt26Nsvq1q2rkSNHmtpPSkqKEhMTdfbsWfsKRYmmTp2q2rVry9XVVX5+fqW2S0xMlMVisb4qVaqkkJAQ9ejRQ998841N219//VUWi0UrVqy4tcXfpKJjut3bStLIkSOL/V5YLBYlJiaa6mfjxo2mtylpXzf6PbbHyZMnlZiYqNTU1GLrbvbzA4BruTq7AAC4HdatW6eqVaua2iYlJUXTp0/XyJEjr/sXfpTdxx9/rJdeeklTpkxR9+7d5eHhccNtPvvsM/n6+qqwsFDHjh3T7NmzFR0drV27dqlVq1aSpJCQEO3YsUMNGjS41YdwU8aMGaNu3bo5uwyrHTt2qFatWqa22bhxo9544w3TYcqefZl18uRJTZ8+XXXr1lXLli1t1pW3zx5AxUeQAnBXuO+++5xdgmmXLl2SxWKRq+udc6k+cOCAJOkvf/mLatSoUaZtWrdurcDAQElSVFSU2rRpowYNGujDDz+0BikPDw+1a9fu1hTtQLVq1brlYcKMW/2ZGYahixcvytPT0+k/n/L22QOo+Li1D8Bd4dpb+woLCzVjxgw1btxYnp6e8vPzU/PmzfXaa69JunIb0N/+9jdJUr169ay3l23dutW6/ezZs3XPPffIw8NDNWrU0PDhw3XixAmb/RqGoZkzZ6pOnTqqXLmyIiMjlZycrOjoaEVHR1vbbd26VRaLRe+8844mTpyomjVrysPDQz///LN+//13jRs3Tk2bNlWVKlVUo0YNPfTQQ9q+fbvNvopub3v11Vf1yiuvqG7duvL09FR0dLQOHz6sS5cuafLkyQoNDZWvr6/69u2rzMxMmz42b96s6OhoBQQEyNPTU7Vr11b//v2Vm5t73c+3LJ9H3bp1NXXqVElSUFCQXbeVSZKvr68kyc3NrdixX31rX9GtXD/++KMGDRokX19fBQUF6YknnlBWVpZNnxcvXlRCQoLq1asnd3d31axZU+PHjy92W2fdunXVq1cvffLJJ7rvvvvk6empJk2a6JNPPpF05Xa1Jk2ayNvbW23atCl221pJt5etWbNGXbt2VUhIiLW/yZMnKycnx/RnU2TFihVq3LixPDw81KRJE7399tsltrv2Z5Cbm6tJkyapXr16qly5svz9/RUZGanVq1dLunJ74BtvvGHdtuj166+/WpdNmDBBb775ppo0aSIPDw+tXLmyxH0VOXPmjEaNGiV/f395e3urd+/e+uWXX2zalHZr7tW/R1u3btX9998vSRo1apS1tqJ9lvTZl/X3ODo6Ws2aNdPu3bvVoUMHeXl5qX79+nr55ZdVWFho09/1risA7ix3zj9zArjrFBQU6PLly8WWG4Zxw21nz56txMRETZ06VQ8++KAuXbqkn376yfoX5zFjxuj06dOaP3++1q5dq5CQEElS06ZNJUnPPPOMFi9erAkTJqhXr1769ddf9cILL2jr1q3au3evdQRlypQpmjVrlsaOHat+/frp+PHjGjNmjC5duqRGjRoVqyshIUHt27fXm2++qUqVKqlGjRr6/fffJUnTpk1TcHCwzp8/r3Xr1ik6OlpffvmlTSCTpDfeeEPNmzfXG2+8obNnz2rixInq3bu32rZtKzc3Ny1btkxHjx7VpEmTNGbMGG3YsEHSlTDSs2dPdejQQcuWLZOfn59+++03ffbZZ8rPz5eXl1epn2dZPo9169bpjTfe0NKlS62365VlhKDo51x0a9/UqVPl4eGhP//5zzfcVpL69++vgQMHavTo0frhhx+UkJAgSVq2bJmkK+dLnz599OWXXyohIUEdOnTQ/v37NW3aNO3YsUM7duywuQVx3759SkhI0JQpU+Tr66vp06erX79+SkhI0JdffqmZM2fKYrHo73//u3r16qW0tDR5enqWWt+RI0fUo0cPxcXFydvbWz/99JNeeeUVffvtt9q8eXOZjvFqK1as0KhRo/Too49qzpw5ysrKUmJiovLy8lSp0vX//TQ+Pl7vvPOOZsyYofvuu085OTk6cOCATp06JUl64YUXlJOTow8//FA7duywblf0+yFJ69ev1/bt2/Xiiy8qODj4hiOPo0ePVpcuXbRq1SodP35cU6dOVXR0tPbv32/qltpWrVpp+fLlGjVqlKZOnaqePXtK0nXPsbL+HktSRkaGhgwZookTJ2ratGlat26dEhISFBoaquHDh0u68XUFwB3GAIAKZvny5Yak677q1Kljs02dOnWMESNGWN/36tXLaNmy5XX38+qrrxqSjLS0NJvlBw8eNCQZ48aNs1m+a9cuQ5Lx/PPPG4ZhGKdPnzY8PDyMgQMH2rTbsWOHIcno2LGjddmWLVsMScaDDz54w+O/fPmycenSJePhhx82+vbta12elpZmSDJatGhhFBQUWJfPmzfPkGQ88sgjNv3ExcUZkoysrCzDMAzjww8/NCQZqampN6zhamX9PAzDMKZNm2ZIMn7//fcb9lvU9tpX1apVjbVr19q0LTr25cuXF9t+9uzZNm3HjRtnVK5c2SgsLDQMwzA+++yzEtutWbPGkGQsXrzYuqxOnTqGp6enceLECeuy1NRUQ5IREhJi5OTkWJevX7/ekGRs2LChWE2lKSwsNC5dumRs27bNkGTs27evzNsahmEUFBQYoaGhRqtWrazHZxiG8euvvxpubm7Ffi8kGdOmTbO+b9asmdGnT5/r7mP8+PGl1iHJ8PX1NU6fPl3iuqv3VfR7fPU5bBiG8c033xiSjBkzZliXXfv7W6Rjx442v0e7d+8udh4UufbzM3PeduzY0ZBk7Nq1y6Zt06ZNjZiYGOv7slxXANw5uLUPQIX19ttva/fu3cVef/rTn264bZs2bbRv3z6NGzdOn3/+ubKzs8u83y1btkhSsVuN2rRpoyZNmujLL7+UJO3cuVN5eXkaMGCATbt27doVmz2tSP/+/Utc/uabb6pVq1aqXLmyXF1d5ebmpi+//FIHDx4s1rZHjx42Iw9NmjSRJOu/0F+7/NixY5Kkli1byt3dXWPHjtXKlSuL3V5VmrJ+Hvb64osvtHv3bn377bf65JNP1LlzZz3++ONat25dmbZ/5JFHbN43b95cFy9etN7WWDTqc239jz32mLy9vYvV37JlS9WsWdP6vuhzjI6Othm1K1p+9OjR69b3yy+/aPDgwQoODpaLi4vc3NzUsWNHSSrx53s9hw4d0smTJzV48GCb29jq1KmjqKioG27fpk0bffrpp5o8ebK2bt2qCxcumNq/JD300EOqVq1amdsPGTLE5n1UVJTq1KljPa9uFbPnbXBwsNq0aWOzrHnz5jY/35u5rgCoeAhSACqsJk2aKDIystir6Bma60lISND//M//aOfOnerevbsCAgL08MMPl2kq5qLbnK6+nalIaGiodX3Rf4OCgoq1K2lZaX3OnTtXzzzzjNq2bauPPvpIO3fu1O7du9WtW7cS/6Lr7+9v897d3f26yy9evChJatCggb744gvVqFFD48ePV4MGDdSgQYMbPt9R1s/DXi1atFBkZKTuv/9+9ezZUx988IEaNmyo8ePHl2n7gIAAm/dFt+kVfXanTp2Sq6urqlevbtPOYrEoODi4WP32fr4lOX/+vDp06KBdu3ZpxowZ2rp1q3bv3q21a9fa1FhWRbUGBwcXW1fSsmu9/vrr+vvf/67169erU6dO8vf3V58+fXTkyJEy11DSeXA9pdV6s+fNjZg9b689j6Qr59LVP6Obua4AqHgIUgDuSq6uroqPj9fevXt1+vRprV69WsePH1dMTMwNJ1Yo+gtVenp6sXUnT560PldR1O6///1vsXYZGRkl9l3S99y8++67io6O1qJFi9SzZ0+1bdtWkZGROnfu3PUP0g4dOnTQ//7v/yorK0s7d+5U+/btFRcXp/fff7/Ubcr6eThKpUqVdO+99yo9Pb3YZBn2CAgI0OXLl63PohUxDEMZGRkOr/9qmzdv1smTJ7Vs2TKNGTNGDz74oCIjI+Xj42NXf0U/i5LOr9LOuat5e3tr+vTp+umnn5SRkaFFixZp586d6t27d5lrMPtdTaXVenVwqVy5svLy8oq1++OPP0zt62q34ry9mesKgIqHIAXgrufn56c///nPGj9+vE6fPm2dgezakYsiDz30kKQrAedqu3fv1sGDB/Xwww9Lktq2bSsPDw+tWbPGpt3OnTtveLvX1SwWS7HvW9q/f7/Nw/6O5uLiorZt21pnaNu7d2+pbcv6eThKQUGBfvjhB3l4eJj+brCSFNV3bf0fffSRcnJyHF7/1YpCx7U/37feesuu/ho3bqyQkBCtXr3aZtKVo0ePKiUlxVRfQUFBGjlypAYNGqRDhw5Zg0Bpvxf2eu+992zep6Sk6OjRozaTqNStW1f79++3aXf48GEdOnTIZpmZ2m71eVvadQXAnYNZ+wDclXr37q1mzZopMjJS1atX19GjRzVv3jzVqVNH4eHhkqSIiAhJ0muvvaYRI0bIzc1NjRs3VuPGjTV27FjNnz9flSpVUvfu3a2zfYWFhemvf/2rpCu3esXHx2vWrFmqVq2a+vbtqxMnTmj69OkKCQm54QxqRXr16qV//vOfmjZtmjp27KhDhw7pH//4h+rVq1firIX2evPNN7V582b17NlTtWvX1sWLF60z23Xu3LnU7cr6edhrz5491ts1//vf/2rZsmX66aef9Ne//lWVK1e+qb4lqUuXLoqJidHf//53ZWdn64EHHrDO2nffffdp2LBhN72P0kRFRalatWp6+umnNW3aNLm5uem9997Tvn377OqvUqVK+uc//6kxY8aob9++evLJJ3X27FklJiaW6da+tm3bqlevXmrevLmqVaumgwcP6p133lH79u2tz38V/V688sor6t69u1xcXNS8eXPrrYxmfffddxozZowee+wxHT9+XFOmTFHNmjU1btw4a5thw4Zp6NChGjdunPr376+jR49q9uzZxW7HbNCggTw9PfXee++pSZMmqlKlikJDQxUaGlpsv7fivC3LdQXAnYMgBeCu1KlTJ3300UdasmSJsrOzFRwcrC5duuiFF16wfj9RdHS0EhIStHLlSv3rX/9SYWGhtmzZYr3NrkGDBlq6dKneeOMN+fr6qlu3bpo1a5bNLUkvvfSSvL299eabb2r58uW65557tGjRIk2ZMqXMUztPmTJFubm5Wrp0qWbPnq2mTZvqzTff1Lp166zfa+UILVu21KZNmzRt2jRlZGSoSpUqatasmTZs2KCuXbted9uyfh726Natm/XP/v7+Cg8P17JlyzRixIib6reIxWLR+vXrlZiYqOXLl+ull15SYGCghg0bppkzZxYbLXKkgIAA/fvf/9bEiRM1dOhQeXt769FHH9WaNWusXzZs1ujRoyVdCTr9+vVT3bp19fzzz2vbtm03PF8eeughbdiwQUlJScrNzVXNmjU1fPhwTZkyxdpm8ODB+uabb7Rw4UL94x//kGEYSktLK3UClRtZunSp3nnnHT3++OPKy8tTp06d9Nprr9k8czZ48GCdPHnS+nvUrFkzLVq0SNOnT7fpy8vLS8uWLdP06dPVtWtXXbp0SdOmTSv1+8ocfd6W5boC4M5hMYwyfOEKAMBh0tLSdM8992jatGl6/vnnnV0OAACwA0EKAG6hffv2afXq1YqKilLVqlV16NAhzZ49W9nZ2Tpw4ECps/cBAIDyjVv7AOAW8vb21nfffaelS5fq7Nmz8vX1VXR0tF566SVCFAAAFRgjUgAAAABgEtOfAwAAAIBJBCkAAAAAMIkgBQAAAAAmMdmEpMLCQp08eVI+Pj7Wb5kHAAAAcPcxDEPnzp1TaGioKlUqfdyJICXp5MmTCgsLc3YZAAAAAMqJ48ePq1atWqWuJ0hJ8vHxkXTlw6pataqTqwEAAADgLNnZ2QoLC7NmhNIQpCTr7XxVq1YlSAEAAAC44SM/TDYBAAAAACYRpAAAAADAJIIUAAAAAJjEM1IAAAC46xmGocuXL6ugoMDZpeAWc3Fxkaur601/7RFBCgAAAHe1/Px8paenKzc319ml4Dbx8vJSSEiI3N3d7e6DIAUAAIC7VmFhodLS0uTi4qLQ0FC5u7vf9EgFyi/DMJSfn6/ff/9daWlpCg8Pv+6X7l4PQQoAAAB3rfz8fBUWFiosLExeXl7OLge3gaenp9zc3HT06FHl5+ercuXKdvXDZBMAAAC469k7KoGKyRE/b84YAAAAADCJIAUAAAAAJhGkAAAAgAooOjpacXFxpa6vW7eu5s2bd9vqKc3WrVtlsVh09uzZMm9zo2MrD5hsArhNkpIP3/Z9/rVLo9u+TwAA7hS3+//djv7/9u7du+Xt7e3QPu0RFRWl9PR0+fr63pL+DcPQ9OnTtXjxYp05c0Zt27bVG2+8oXvvvfeW7K8II1IAAADAHah69erlYiZCd3d3BQcH37Jp5WfPnq25c+dqwYIF2r17t4KDg9WlSxedO3fuluyvCEEKAAAAqKAuX76sCRMmyM/PTwEBAZo6daoMw5BU/NY+i8WiJUuWqG/fvvLy8lJ4eLg2bNhg09+2bdvUpk0beXh4KCQkRJMnT9bly5et66OjoxUbG6u4uDhVq1ZNQUFBWrx4sXJycjRq1Cj5+PioQYMG+vTTT63bXHtr36lTpzRo0CDVqlVLXl5eioiI0OrVq+06fsMwNG/ePE2ZMkX9+vVTs2bNtHLlSuXm5mrVqlV29VlWBCkAAACgglq5cqVcXV21a9cuvf7660pKStKSJUtKbT99+nQNGDBA+/fvV48ePTRkyBCdPn1akvTbb7+pR48euv/++7Vv3z4tWrRIS5cu1YwZM4rtMzAwUN9++61iY2P1zDPP6LHHHlNUVJT27t2rmJgYDRs2TLm5uSXWcPHiRbVu3VqffPKJDhw4oLFjx2rYsGHatWuX6eNPS0tTRkaGunbtal3m4eGhjh07KiUlxXR/ZhCkAAAAgAoqLCxMSUlJaty4sYYMGaLY2FglJSWV2n7kyJEaNGiQGjZsqJkzZyonJ0fffvutJGnhwoUKCwvTggULdM8996hPnz6aPn265syZo8LCQmsfLVq00NSpUxUeHq6EhAR5enoqMDBQTz75pMLDw/Xiiy/q1KlT2r9/f4k11KxZU5MmTVLLli1Vv359xcbGKiYmRh988IHp48/IyJAkBQUF2SwPCgqyrrtVCFIAAABABdWuXTubZ4/at2+vI0eOqKCgoMT2zZs3t/7Z29tbPj4+yszMlCQdPHhQ7du3t+nvgQce0Pnz53XixIkS+3BxcVFAQIAiIiKsy4pCTVG/1yooKNBLL72k5s2bKyAgQFWqVNGmTZt07NgxM4du49rnrwzDuGXPZBUhSAEAAAB3CTc3N5v3FovFOtpUUvgoet7q6uUl9XH1sqK2V49iXW3OnDlKSkrSc889p82bNys1NVUxMTHKz883fTzBwcGSVGz0KTMzs9golaMRpAAAAIAKaufOncXeh4eHy8XFxXRfTZs2VUpKijU8SVJKSop8fHxUs2bNm661yPbt2/Xoo49q6NChatGiherXr68jR47Y1Ve9evUUHBys5ORk67L8/Hxt27ZNUVFRjiq5RAQpAAAAoII6fvy44uPjdejQIa1evVrz58/Xs88+a1df48aN0/HjxxUbG6uffvpJH3/8saZNm6b4+HhVquS42NCwYUMlJycrJSVFBw8e1FNPPWX380wWi0VxcXGaOXOm1q1bpwMHDmjkyJHy8vLS4MGDHVZzSfhCXgAAAKAEFeGL7YcPH64LFy6oTZs2cnFxUWxsrMaOHWtXXzVr1tTGjRv1t7/9TS1atJC/v79Gjx6tqVOnOrTmF154QWlpaYqJiZGXl5fGjh2rPn36KCsry67+nnvuOV24cEHjxo2zfiHvpk2b5OPj49C6r2Uxrh67u0tlZ2fL19dXWVlZqlq1qrPLwR3qdn87ulQx/gcAAIAzXbx4UWlpaapXr54qV67s7HJwm1zv517WbMCtfQAAAABgEkEKAAAAQLm0fft2ValSpdSXM/GMFAAAAIByKTIyUqmpqc4uo0QEKQAAAADlkqenpxo2bOjsMkrErX0AAAAAYBJBCgAAAABMIkgBAAAAgEkEKQAAAAAwiSAFAAAAACYRpAAAAIAKKDo6WnFxcaWur1u3rubNm3fb6inN1q1bZbFYdPbs2TJvc6NjKw+Y/hwAAAAoyZZZt3d/nRIc2t3u3bvl7e3t0D7tERUVpfT0dPn6+t6S/teuXau33npLe/bs0alTp/T999+rZcuWt2RfV2NECgAAALgDVa9eXV5eXs4uQ+7u7goODpbFYrkl/efk5OiBBx7Qyy+/fEv6Lw1BCgAAAKigLl++rAkTJsjPz08BAQGaOnWqDMOQVPzWPovFoiVLlqhv377y8vJSeHi4NmzYYNPftm3b1KZNG3l4eCgkJESTJ0/W5cuXreujo6MVGxuruLg4VatWTUFBQVq8eLFycnI0atQo+fj4qEGDBvr000+t21x7a9+pU6c0aNAg1apVS15eXoqIiNDq1avt/gyGDRumF198UZ07d7a7D3sQpAAAAIAKauXKlXJ1ddWuXbv0+uuvKykpSUuWLCm1/fTp0zVgwADt379fPXr00JAhQ3T69GlJ0m+//aYePXro/vvv1759+7Ro0SItXbpUM2bMKLbPwMBAffvtt4qNjdUzzzyjxx57TFFRUdq7d69iYmI0bNgw5ebmlljDxYsX1bp1a33yySc6cOCAxo4dq2HDhmnXrl2O+2BuA4IUAAAAUEGFhYUpKSlJjRs31pAhQxQbG6ukpKRS248cOVKDBg1Sw4YNNXPmTOXk5Ojbb7+VJC1cuFBhYWFasGCB7rnnHvXp00fTp0/XnDlzVFhYaO2jRYsWmjp1qsLDw5WQkCBPT08FBgbqySefVHh4uF588UWdOnVK+/fvL7GGmjVratKkSWrZsqXq16+v2NhYxcTE6IMPPnDsh3OLEaQAAACACqpdu3Y2zx61b99eR44cUUFBQYntmzdvbv2zt7e3fHx8lJmZKUk6ePCg2rdvb9PfAw88oPPnz+vEiRMl9uHi4qKAgABFRERYlwUFBUmStd9rFRQU6KWXXlLz5s0VEBCgKlWqaNOmTTp27JiZQ3c6Zu0DAAAA7hJubm427y0Wi3W0yTCMYhNCFD1vdfXykvq4ellR26tHsa42Z84cJSUlad68eYqIiJC3t7fi4uKUn59v51E5B0EKd52k5MPOLgEAAMAhdu7cWex9eHi4XFxcTPfVtGlTffTRRzaBKiUlRT4+PqpZs6ZD6pWk7du369FHH9XQoUMlXQlcR44cUZMmTRy2j9uBW/sAAACACur48eOKj4/XoUOHtHr1as2fP1/PPvusXX2NGzdOx48fV2xsrH766Sd9/PHHmjZtmuLj41WpkuNiQ8OGDZWcnKyUlBQdPHhQTz31lDIyMuzu7/Tp00pNTdX//d//SZIOHTqk1NTUm+qzLBiRAgAAAEri4C/IvRWGDx+uCxcuqE2bNnJxcVFsbKzGjh1rV181a9bUxo0b9be//U0tWrSQv7+/Ro8eralTpzq05hdeeEFpaWmKiYmRl5eXxo4dqz59+igrK8uu/jZs2KBRo0ZZ3z/++OOSpGnTpikxMdERJZfIYhTd+HgXy87Olq+vr7KyslS1alVnl4Nb7G66te+vXRo5uwQAAMq1ixcvKi0tTfXq1VPlypWdXQ5uk+v93MuaDbi1DwAAAABMIkgBAAAAKJe2b9+uKlWqlPpyJp6RAgAAAFAuRUZGKjU11dlllIggBcC5tsxy7v4rwIPEAADcrTw9PdWwYUNnl1Eibu0DAAAAAJOcGqRmzZql+++/Xz4+PqpRo4b69OmjQ4cO2bQxDEOJiYkKDQ2Vp6enoqOj9eOPP9q0ycvLU2xsrAIDA+Xt7a1HHnlEJ06cuJ2HAgAAAOAu4tQgtW3bNo0fP147d+5UcnKyLl++rK5duyonJ8faZvbs2Zo7d64WLFig3bt3Kzg4WF26dNG5c+esbeLi4rRu3Tq9//77+vrrr3X+/Hn16tVLBQUFzjgsAAAAAHc4pz4j9dlnn9m8X758uWrUqKE9e/bowQcflGEYmjdvnqZMmaJ+/fpJklauXKmgoCCtWrVKTz31lLKysrR06VK988476ty5syTp3XffVVhYmL744gvFxMTc9uMCAAAAcGcrV89IFX2bsb+/vyQpLS1NGRkZ6tq1q7WNh4eHOnbsqJSUFEnSnj17dOnSJZs2oaGhatasmbXNtfLy8pSdnW3zAgAAAICyKjdByjAMxcfH609/+pOaNWsmScrIyJAkBQUF2bQNCgqyrsvIyJC7u7uqVatWaptrzZo1S76+vtZXWFiYow8HAAAAuKWio6MVFxdX6vq6detq3rx5t62e0mzdulUWi0Vnz54t8zY3OrbyoNxMfz5hwgTt379fX3/9dbF1FovF5r1hGMWWXet6bRISEhQfH299n52dTZgCAACAjYWpC2/r/sa1HOfQ/nbv3i1vb2+H9mmPqKgopaeny9fX1+F9X7p0SVOnTtXGjRv1yy+/yNfXV507d9bLL7+s0NBQh+/vauViRCo2NlYbNmzQli1bVKtWLevy4OBgSSo2spSZmWkdpQoODlZ+fr7OnDlTaptreXh4qGrVqjYvAAAA4E5SvXp1eXl5ObsMubu7Kzg4+IYDIfbIzc3V3r179cILL2jv3r1au3atDh8+rEceecTh+7qWU4OUYRiaMGGC1q5dq82bN6tevXo26+vVq6fg4GAlJydbl+Xn52vbtm2KioqSJLVu3Vpubm42bdLT03XgwAFrGwAAAOBOdPnyZU2YMEF+fn4KCAjQ1KlTZRiGpOK39lksFi1ZskR9+/aVl5eXwsPDtWHDBpv+tm3bpjZt2sjDw0MhISGaPHmyLl++bF0fHR2t2NhYxcXFqVq1agoKCtLixYuVk5OjUaNGycfHRw0aNNCnn35q3ebaW/tOnTqlQYMGqVatWvLy8lJERIRWr15t1/H7+voqOTlZAwYMUOPGjdWuXTvNnz9fe/bs0bFjx+zqs6ycGqTGjx+vd999V6tWrZKPj48yMjKUkZGhCxcuSLryw46Li9PMmTO1bt06HThwQCNHjpSXl5cGDx4s6cqHN3r0aE2cOFFffvmlvv/+ew0dOlQRERHWWfwAAACAO9HKlSvl6uqqXbt26fXXX1dSUpKWLFlSavvp06drwIAB2r9/v3r06KEhQ4bo9OnTkqTffvtNPXr00P333699+/Zp0aJFWrp0qWbMmFFsn4GBgfr2228VGxurZ555Ro899piioqK0d+9excTEaNiwYcrNzS2xhosXL6p169b65JNPdODAAY0dO1bDhg3Trl27HPKZZGVlyWKxyM/PzyH9lcapQWrRokXKyspSdHS0QkJCrK81a9ZY2zz33HOKi4vTuHHjFBkZqd9++02bNm2Sj4+PtU1SUpL69OmjAQMG6IEHHpCXl5f+93//Vy4uLs44LAAAAOC2CAsLU1JSkho3bqwhQ4YoNjZWSUlJpbYfOXKkBg0apIYNG2rmzJnKycnRt99+K0lauHChwsLCtGDBAt1zzz3q06ePpk+frjlz5qiwsNDaR4sWLTR16lSFh4crISFBnp6eCgwM1JNPPqnw8HC9+OKLOnXqlPbv319iDTVr1tSkSZPUsmVL1a9fX7GxsYqJidEHH3xw05/HxYsXNXnyZA0ePPiWP77j1MkmioYdr8disSgxMVGJiYmltqlcubLmz5+v+fPnO7A6AAAAoHxr166dzbNH7du315w5c1RQUFBi++bNm1v/7O3tLR8fH2VmZkqSDh48qPbt29v098ADD+j8+fM6ceKEateuXawPFxcXBQQEKCIiwrqsaJ6Con6vVVBQoJdffllr1qzRb7/9pry8POXl5d30xBiXLl3S448/rsLCQi1ceOsnCik3s/YBAAAAuLXc3Nxs3lssFutoU0mzXhcNfFy9vKQ+rl5W1PbqUayrzZkzR0lJSZo3b54iIiLk7e2tuLg45efn23lUV0LUgAEDlJaWps2bN9+WyeQIUgAAAEAFtXPnzmLvw8PD7XrEpWnTpvroo49sAlVKSop8fHxUs2ZNh9QrSdu3b9ejjz6qoUOHSroSuI4cOaImTZrY1V9RiDpy5Ii2bNmigIAAh9V6PeVi+nMAAAAA5h0/flzx8fE6dOiQVq9erfnz5+vZZ5+1q69x48bp+PHjio2N1U8//aSPP/5Y06ZNU3x8vCpVclxsaNiwoZKTk5WSkqKDBw/qqaeeKvZ1R2V1+fJl/fnPf9Z3332n9957TwUFBdYJ7G5mhKssGJECAAAASuDoL8i9FYYPH64LFy6oTZs2cnFxUWxsrMaOHWtXXzVr1tTGjRv1t7/9TS1atJC/v79Gjx6tqVOnOrTmF154QWlpaYqJiZGXl5fGjh2rPn36KCsry3RfJ06csE7h3rJlS5t1W7ZsUXR0tAMqLpnFKMuMD3e47Oxs+fr6Kisriy/nvQskJR92dgm3zV+7NHJ2CTe2ZZZz998pwbn7BwA41cWLF5WWlqZ69eqpcuXKzi4Ht8n1fu5lzQbc2gcAAAAAJhGkAAAAAJRL27dvV5UqVUp9ORPPSAEAAAAolyIjI5WamursMkpEkAIAAABQLnl6eqphw4bOLqNE3NoHAACAux7zr91dHPHzJkgBAADgruXm5iZJys3NdXIluJ2Kft5FP397cGsfAAAA7louLi7y8/NTZmamJMnLy0sWi8XJVeFWMQxDubm5yszMlJ+fn1xcXOzuiyAFAACAu1pwcLAkWcMU7nx+fn7Wn7u9CFIAAAB3uYWpC51dgtONazlONWrU0KVLl5xdCm4xNze3mxqJKkKQAgAAAHTlNj9H/AUbdwcmmwAAAAAAkxiRAu52W2Y5uwIAAIAKhxEpAAAAADCJIAUAAAAAJhGkAAAAAMAkghQAAAAAmESQAgAAAACTCFIAAAAAYBJBCgAAAABMIkgBAAAAgEkEKQAAAAAwiSAFAAAAACYRpAAAAADAJIIUAAAAAJhEkAIAAAAAkwhSAAAAAGASQQoAAAAATCJIAQAAAIBJBCkAAAAAMIkgBQAAAAAmEaQAAAAAwCSCFAAAAACY5OrsAgAAAO52C1MXOrsEACYxIgUAAAAAJhGkAAAAAMAkghQAAAAAmESQAgAAAACTCFIAAAAAYBJBCgAAAABMIkgBAAAAgEkEKQAAAAAwiSAFAAAAACa5OrsA3N2Skg87uwQAAADANIIUcAcrS1Btd+yUQ/fZvn6AQ/sDAAAoj7i1DwAAAABMIkgBAAAAgEnc2gc4Ubtji51dAgAAAOzg1BGpr776Sr1791ZoaKgsFovWr19vs37kyJGyWCw2r3bt2tm0ycvLU2xsrAIDA+Xt7a1HHnlEJ06cuI1HAQAAAOBu49QglZOToxYtWmjBggWltunWrZvS09Otr40bN9qsj4uL07p16/T+++/r66+/1vnz59WrVy8VFBTc6vIBAAAA3KUccmtfdna2Nm/erMaNG6tJkyZl3q579+7q3r37ddt4eHgoODi4xHVZWVlaunSp3nnnHXXu3FmS9O677yosLExffPGFYmJiyn4QAAAAAFBGdo1IDRgwwDqKdOHCBUVGRmrAgAFq3ry5PvroI4cWuHXrVtWoUUONGjXSk08+qczMTOu6PXv26NKlS+ratat1WWhoqJo1a6aUlJRS+8zLy1N2drbNCwAAAADKyq4g9dVXX6lDhw6SpHXr1skwDJ09e1avv/66ZsyY4bDiunfvrvfee0+bN2/WnDlztHv3bj300EPKy8uTJGVkZMjd3V3VqlWz2S4oKEgZGRml9jtr1iz5+vpaX2FhYQ6rGQAAAMCdz64glZWVJX9/f0nSZ599pv79+8vLy0s9e/bUkSNHHFbcwIED1bNnTzVr1ky9e/fWp59+qsOHD+vf//73dbczDEMWi6XU9QkJCcrKyrK+jh8/7rCaAQAAANz57ApSYWFh2rFjh3JycvTZZ59Zb607c+aMKleu7NACrxYSEqI6depYw1pwcLDy8/N15swZm3aZmZkKCgoqtR8PDw9VrVrV5gUAAAAAZWVXkIqLi9OQIUNUq1YthYSEKDo6WtKVW/4iIiIcWZ+NU6dO6fjx4woJCZEktW7dWm5ubkpOTra2SU9P14EDBxQVFXXL6gAAAABwd7Nr1r5x48apTZs2On78uLp06aJKla7ksfr165t6Rur8+fP6+eefre/T0tKUmpoqf39/+fv7KzExUf3791dISIh+/fVXPf/88woMDFTfvn0lSb6+vho9erQmTpyogIAA+fv7a9KkSYqIiLDO4gcAAAAAjmb39OeRkZFq3ry50tLS1KBBA7m6uqpnz56m+vjuu+/UqVMn6/v4+HhJ0ogRI7Ro0SL98MMPevvtt3X27FmFhISoU6dOWrNmjXx8fKzbJCUlydXVVQMGDNCFCxf08MMPa8WKFXJxcbH30AAAAADguuwKUrm5uYqNjdXKlSslSYcPH1b9+vX1l7/8RaGhoZo8eXKZ+omOjpZhGKWu//zzz2/YR+XKlTV//nzNnz+/bMUDAAAAwE2y6xmphIQE7du3T1u3brWZXKJz585as2aNw4oDAAAAgPLIrhGp9evXa82aNWrXrp3NNONNmzbVf/7zH4cVBwAAAADlkV0jUr///rtq1KhRbHlOTs51v78JAAAAAO4EdgWp+++/3+ZLcYvC07/+9S+1b9/eMZUBAAAAQDll1619s2bNUrdu3fR///d/unz5sl577TX9+OOP2rFjh7Zt2+boGgEAAACgXLFrRCoqKkrffPONcnNz1aBBA23atElBQUHasWOHWrdu7egaAQAAAKBcsft7pCIiIqzTnwMAAADA3aTMQSo7O1tVq1a1/vl6itoBAAAAwJ2ozEGqWrVqSk9PV40aNeTn51fi7HyGYchisaigoMChRQIAAABAeVLmILV582b5+/tLkrZs2XLLCgIAAACA8q7MQapjx44l/hkAKrQts5xdgdQpwdkVAAAAk+yatW/58uX64IMPii3/4IMPmIACAAAAwB3PriD18ssvKzAwsNjyGjVqaObMmTddFAAAAACUZ3YFqaNHj6pevXrFltepU0fHjh276aIAAAAAoDyzK0jVqFFD+/fvL7Z83759CggIuOmiAAAAAKA8sytIPf744/rLX/6iLVu2qKCgQAUFBdq8ebOeffZZPf74446uEQAAAADKlTLP2ne1GTNm6OjRo3r44Yfl6nqli8LCQg0fPpxnpAAAAFDhLExd6NT9j2s5zqn7h3l2BSl3d3etWbNG//znP7Vv3z55enoqIiJCderUcXR9AAAAAFDu2BWkijRq1EiNGjVyVC0AAAAAUCHYFaQKCgq0YsUKffnll8rMzFRhYaHN+s2bNzukOAAAAAAoj+wKUs8++6xWrFihnj17qlmzZrJYLI6uCwAAAADKLbuC1Pvvv6//9//+n3r06OHoegAAAACg3LNr+nN3d3c1bNjQ0bUAAAAAQIVgV5CaOHGiXnvtNRmG4eh6AAAAAKDcs+vWvq+//lpbtmzRp59+qnvvvVdubm4269euXeuQ4gAAAACgPLIrSPn5+alv376OrgUAAAAAKgS7gtTy5csdXQcAAAAAVBh2PSMlSZcvX9YXX3yht956S+fOnZMknTx5UufPn3dYcQAAAABQHtk1InX06FF169ZNx44dU15enrp06SIfHx/Nnj1bFy9e1JtvvunoOgEAAACg3LBrROrZZ59VZGSkzpw5I09PT+vyvn376ssvv3RYcQAAAABQHtk9a98333wjd3d3m+V16tTRb7/95pDCAAAAbpeFqQudXQKACsauEanCwkIVFBQUW37ixAn5+PjcdFEAAAAAUJ7ZFaS6dOmiefPmWd9bLBadP39e06ZNU48ePRxVGwAAAACUS3bd2peUlKROnTqpadOmunjxogYPHqwjR44oMDBQq1evdnSNAAAAAFCu2BWkQkNDlZqaqtWrV2vv3r0qLCzU6NGjNWTIEJvJJwAAAADgTmRXkJIkT09PPfHEE3riiSccWQ8AAAAAlHt2Bam33377uuuHDx9uVzEAAAAAUBHYFaSeffZZm/eXLl1Sbm6u3N3d5eXlRZACAAAAcEeza9a+M2fO2LzOnz+vQ4cO6U9/+hOTTQAAAAC449kVpEoSHh6ul19+udhoFQAAAADcaRwWpCTJxcVFJ0+edGSXAAAAAFDu2PWM1IYNG2zeG4ah9PR0LViwQA888IBDCgMAAACA8squINWnTx+b9xaLRdWrV9dDDz2kOXPmOKIuAAAAACi37ApShYWFjq4DAAAAACoMhz4jBQAAAAB3A7tGpOLj48vcdu7cufbsAgAAAADKLbuC1Pfff6+9e/fq8uXLaty4sSTp8OHDcnFxUatWraztLBaLY6oEAAAAgHLEriDVu3dv+fj4aOXKlapWrZqkK1/SO2rUKHXo0EETJ050aJEAAAAAUJ7Y9YzUnDlzNGvWLGuIkqRq1appxowZzNoHAAAA4I5nV5DKzs7Wf//732LLMzMzde7cuZsuCgAAAADKM7uCVN++fTVq1Ch9+OGHOnHihE6cOKEPP/xQo0ePVr9+/RxdIwAAAACUK3Y9I/Xmm29q0qRJGjp0qC5dunSlI1dXjR49Wq+++qpDCwRupXbHFju7hDvOjl9OOWW/7esHOGW/AADg7mTXiJSXl5cWLlyoU6dOWWfwO336tBYuXChvb+8y9/PVV1+pd+/eCg0NlcVi0fr1623WG4ahxMREhYaGytPTU9HR0frxxx9t2uTl5Sk2NlaBgYHy9vbWI488ohMnTthzWAAAAABQJjf1hbzp6elKT09Xo0aN5O3tLcMwTG2fk5OjFi1aaMGCBSWunz17tubOnasFCxZo9+7dCg4OVpcuXWyew4qLi9O6dev0/vvv6+uvv9b58+fVq1cvFRQU3MyhAQAAAECp7Lq179SpUxowYIC2bNkii8WiI0eOqH79+hozZoz8/PzKPHNf9+7d1b179xLXGYahefPmacqUKdbnrlauXKmgoCCtWrVKTz31lLKysrR06VK988476ty5syTp3XffVVhYmL744gvFxMTYc3gAAAAAcF12jUj99a9/lZubm44dOyYvLy/r8oEDB+qzzz5zSGFpaWnKyMhQ165drcs8PDzUsWNHpaSkSJL27NmjS5cu2bQJDQ1Vs2bNrG1KkpeXp+zsbJsXAAAAAJSVXUFq06ZNeuWVV1SrVi2b5eHh4Tp69KhDCsvIyJAkBQUF2SwPCgqyrsvIyJC7u7vN91ld26Yks2bNkq+vr/UVFhbmkJoBAAAA3B3sClI5OTk2I1FF/vjjD3l4eNx0UVezWCw27w3DKLbsWjdqk5CQoKysLOvr+PHjDqkVAAAAwN3BrmekHnzwQb399tv65z//KelK2CksLNSrr76qTp06OaSw4OBgSVdGnUJCQqzLMzMzraNUwcHBys/P15kzZ2xGpTIzMxUVFVVq3x4eHg4PfABgty2znLv/TgnO3T8AABWQXSNSr776qt566y11795d+fn5eu6559SsWTN99dVXeuWVVxxSWL169RQcHKzk5GTrsvz8fG3bts0aklq3bi03NzebNunp6Tpw4MB1gxQAAAAA3Ay7RqSaNm2q/fv3a9GiRXJxcVFOTo769eun8ePH24we3cj58+f1888/W9+npaUpNTVV/v7+ql27tuLi4jRz5kyFh4crPDxcM2fOlJeXlwYPHixJ8vX11ejRozVx4kQFBATI399fkyZNUkREhHUWPwAAAABwNNNBqmiWvLfeekvTp0+/qZ1/9913NrcCxsfHS5JGjBihFStW6LnnntOFCxc0btw4nTlzRm3bttWmTZvk4+Nj3SYpKUmurq4aMGCALly4oIcfflgrVqyQi4vLTdUGAAAAAKUxHaTc3Nx04MCBG074UBbR0dHX/RJfi8WixMREJSYmltqmcuXKmj9/vubPn3/T9QAAAABAWdj1jNTw4cO1dOlSR9cCAAAAABWCXc9I5efna8mSJUpOTlZkZKS8vb1t1s+dO9chxQEAAABAeWQqSP3yyy+qW7euDhw4oFatWkmSDh8+bNPGEbf8AQAAAEB5ZipIhYeHKz09XVu2bJEkDRw4UK+//rr1e50AAAAA4G5g6hmpayeG+PTTT5WTk+PQggAAAACgvLNrsoki15txDwAAAADuVKaClMViKfYMFM9EAQAAALjbmHpGyjAMjRw5Uh4eHpKkixcv6umnny42a9/atWsdVyEAAAAAlDOmgtSIESNs3g8dOtShxQAAAABARWAqSC1fvvxW1QEAAAAAFcZNTTYBAAAAAHcjghQAAAAAmESQAgAAAACTCFIAAAAAYBJBCgAAAABMIkgBAAAAgEkEKQAAAAAwiSAFAAAAACYRpAAAAADAJIIUAAAAAJhEkAIAAAAAkwhSAAAAAGASQQoAAAAATCJIAQAAAIBJrs4uAAAAALjbLUxd6NT9j2s5zqn7r4gYkQIAAAAAkwhSAAAAAGASQQoAAAAATCJIAQAAAIBJBCkAAAAAMIkgBQAAAAAmEaQAAAAAwCSCFAAAAACYRJACAAAAAJNcnV0Ayoek5MPOLgEAAACoMBiRAgAAAACTCFIAAAAAYBJBCgAAAABMIkgBAAAAgElMNgEAAJxqYepCZ5cAAKYRpOBU7Y4tdnYJAAAAgGnc2gcAAAAAJhGkAAAAAMAkghQAAAAAmESQAgAAAACTCFIAAAAAYBJBCgAAAABMIkgBAAAAgEkEKQAAAAAwiSAFAAAAACYRpAAAAADAJIIUAAAAAJhEkAIAAAAAk8p1kEpMTJTFYrF5BQcHW9cbhqHExESFhobK09NT0dHR+vHHH51YMQAAAIC7QbkOUpJ07733Kj093fr64YcfrOtmz56tuXPnasGCBdq9e7eCg4PVpUsXnTt3zokVAwAAALjTlfsg5erqquDgYOurevXqkq6MRs2bN09TpkxRv3791KxZM61cuVK5ublatWqVk6sGAAAAcCcr90HqyJEjCg0NVb169fT444/rl19+kSSlpaUpIyNDXbt2tbb18PBQx44dlZKSct0+8/LylJ2dbfMCAAAAgLIq10Gqbdu2evvtt/X555/rX//6lzIyMhQVFaVTp04pIyNDkhQUFGSzTVBQkHVdaWbNmiVfX1/rKyws7JYdAwAAAIA7T7kOUt27d1f//v0VERGhzp0769///rckaeXKldY2FovFZhvDMIotu1ZCQoKysrKsr+PHjzu+eAAAAAB3rHIdpK7l7e2tiIgIHTlyxDp737WjT5mZmcVGqa7l4eGhqlWr2rwAAAAAoKwqVJDKy8vTwYMHFRISonr16ik4OFjJycnW9fn5+dq2bZuioqKcWCUAAACAO52rswu4nkmTJql3796qXbu2MjMzNWPGDGVnZ2vEiBGyWCyKi4vTzJkzFR4ervDwcM2cOVNeXl4aPHiws0sHAAAAcAcr10HqxIkTGjRokP744w9Vr15d7dq1086dO1WnTh1J0nPPPacLFy5o3LhxOnPmjNq2batNmzbJx8fHyZUDAAAAuJOV6yD1/vvvX3e9xWJRYmKiEhMTb09BAAAAAKAK9owUAAAAAJQHBCkAAAAAMIkgBQAAAAAmEaQAAAAAwCSCFAAAAACYVK5n7QOAstrxy6nbvs/29QNu+z5viS2znLv/TgnO3T8AAHZgRAoAAAAATCJIAQAAAIBJ3NoHAMBdbmHqQmeXAAAVDiNSAAAAAGASQQoAAAAATCJIAQAAAIBJBCkAAAAAMIkgBQAAAAAmEaQAAAAAwCSCFAAAAACYRJACAAAAAJMIUgAAAABgEkEKAAAAAEwiSAEAAACASQQpAAAAADCJIAUAAAAAJhGkAAAAAMAkghQAAAAAmESQAgAAAACTCFIAAAAAYBJBCgAAAABMIkgBAAAAgEkEKQAAAAAwydXZBQAAAABwroWpC51dgsa1HOfsEkxhRAoAAAAATCJIAQAAAIBJBCkAAAAAMIlnpMqhpOTDzi4BAAAAwHUwIgUAAAAAJhGkAAAAAMAkghQAAAAAmMQzUgAAOFl5+P4WAIA5BCkAsNOOX07d9n22rx9w2/cJAACKI0gBAJxryyzn7r9TgnP3DwCokHhGCgAAAABMIkgBAAAAgEkEKQAAAAAwiSAFAAAAACYRpAAAAADAJIIUAAAAAJhEkAIAAAAAkwhSAAAAAGASQQoAAAAATCJIAQAAAIBJBCkAAAAAMIkgBQAAAAAmuTq7AEdZuHChXn31VaWnp+vee+/VvHnz1KFDB2eXBQAo77bMcnYFUjVfZ1cAADDpjhiRWrNmjeLi4jRlyhR9//336tChg7p3765jx445uzQAAAAAd6A7YkRq7ty5Gj16tMaMGSNJmjdvnj7//HMtWrRIs2aVg39pBAAH2fHLKafst339AKfsFwCA8qrCB6n8/Hzt2bNHkydPtlnetWtXpaSklLhNXl6e8vLyrO+zsrIkSdnZ2beuUBMu5px3dgm3Tc6FvBs3AuB02TkXnV3CHe2Cm7uzSwAApysvfxcvqsMwjOu2q/BB6o8//lBBQYGCgoJslgcFBSkjI6PEbWbNmqXp06cXWx4WFnZLagQAAABwfZM0ydkl2Dh37px8fUt/hrXCB6kiFovF5r1hGMWWFUlISFB8fLz1fWFhoU6fPq2AgACbbbKzsxUWFqbjx4+ratWqt6Zw3BU4l+AInEdwFM4lOArnEhylPJ1LhmHo3LlzCg0NvW67Ch+kAgMD5eLiUmz0KTMzs9goVREPDw95eHjYLPPz8yt1H1WrVnX6DxR3Bs4lOALnERyFcwmOwrkERykv59L1RqKKVPhZ+9zd3dW6dWslJyfbLE9OTlZUVJSTqgIAAABwJ6vwI1KSFB8fr2HDhikyMlLt27fX4sWLdezYMT399NPOLg0AAADAHeiOCFIDBw7UqVOn9I9//EPp6elq1qyZNm7cqDp16txUvx4eHpo2bVqx2wABsziX4AicR3AUziU4CucSHKUinksW40bz+gEAAAAAbFT4Z6QAAAAA4HYjSAEAAACASQQpAAAAADCJIAUAAAAAJhGkSrFw4ULVq1dPlStXVuvWrbV9+3Znl4QKJjExURaLxeYVHBzs7LJQAXz11Vfq3bu3QkNDZbFYtH79epv1hmEoMTFRoaGh8vT0VHR0tH788UfnFIty7Ubn0siRI4tdp9q1a+ecYlFuzZo1S/fff798fHxUo0YN9enTR4cOHbJpw3UJZVGWc6kiXZcIUiVYs2aN4uLiNGXKFH3//ffq0KGDunfvrmPHjjm7NFQw9957r9LT062vH374wdkloQLIyclRixYttGDBghLXz549W3PnztWCBQu0e/duBQcHq0uXLjp37txtrhTl3Y3OJUnq1q2bzXVq48aNt7FCVATbtm3T+PHjtXPnTiUnJ+vy5cvq2rWrcnJyrG24LqEsynIuSRXnusT05yVo27atWrVqpUWLFlmXNWnSRH369NGsWbOcWBkqksTERK1fv16pqanOLgUVmMVi0bp169SnTx9JV/7VNzQ0VHFxcfr73/8uScrLy1NQUJBeeeUVPfXUU06sFuXZteeSdOVffs+ePVtspAq4nt9//101atTQtm3b9OCDD3Jdgt2uPZekinVdYkTqGvn5+dqzZ4+6du1qs7xr165KSUlxUlWoqI4cOaLQ0FDVq1dPjz/+uH755Rdnl4QKLi0tTRkZGTbXKA8PD3Xs2JFrFOyydetW1ahRQ40aNdKTTz6pzMxMZ5eEci4rK0uS5O/vL4nrEux37blUpKJclwhS1/jjjz9UUFCgoKAgm+VBQUHKyMhwUlWoiNq2bau3335bn3/+uf71r38pIyNDUVFROnXqlLNLQwVWdB3iGgVH6N69u9577z1t3rxZc+bM0e7du/XQQw8pLy/P2aWhnDIMQ/Hx8frTn/6kZs2aSeK6BPuUdC5JFeu65OrsAsori8Vi894wjGLLgOvp3r279c8RERFq3769GjRooJUrVyo+Pt6JleFOwDUKjjBw4EDrn5s1a6bIyEjVqVNH//73v9WvXz8nVobyasKECdq/f7++/vrrYuu4LsGM0s6linRdYkTqGoGBgXJxcSn2LyiZmZnF/qUFMMPb21sRERE6cuSIs0tBBVY08yPXKNwKISEhqlOnDtcplCg2NlYbNmzQli1bVKtWLetyrkswq7RzqSTl+bpEkLqGu7u7WrdureTkZJvlycnJioqKclJVuBPk5eXp4MGDCgkJcXYpqMDq1aun4OBgm2tUfn6+tm3bxjUKN+3UqVM6fvw41ynYMAxDEyZM0Nq1a7V582bVq1fPZj3XJZTVjc6lkpTn6xK39pUgPj5ew4YNU2RkpNq3b6/Fixfr2LFjevrpp51dGiqQSZMmqXfv3qpdu7YyMzM1Y8YMZWdna8SIEc4uDeXc+fPn9fPPP1vfp6WlKTU1Vf7+/qpdu7bi4uI0c+ZMhYeHKzw8XDNnzpSXl5cGDx7sxKpRHl3vXPL391diYqL69++vkJAQ/frrr3r++ecVGBiovn37OrFqlDfjx4/XqlWr9PHHH8vHx8c68uTr6ytPT09ZLBauSyiTG51L58+fr1jXJQMleuONN4w6deoY7u7uRqtWrYxt27Y5uyRUMAMHDjRCQkIMNzc3IzQ01OjXr5/x448/OrssVABbtmwxJBV7jRgxwjAMwygsLDSmTZtmBAcHGx4eHsaDDz5o/PDDD84tGuXS9c6l3Nxco2vXrkb16tUNNzc3o3bt2saIESOMY8eOObtslDMlnUOSjOXLl1vbcF1CWdzoXKpo1yW+RwoAAAAATOIZKQAAAAAwiSAFAAAAACYRpAAAAADAJIIUAAAAAJhEkAIAAAAAkwhSAAAAAGASQQoAAAAATCJIAQAAAIBJBCkAwF0pOjpacXFxzi4DAFBBEaQAABVO79691blz5xLX7dixQxaLRXv37r3NVQEA7iYEKQBAhTN69Ght3rxZR48eLbZu2bJlatmypVq1auWEygAAdwuCFACgwunVq5dq1KihFStW2CzPzc3VmjVr1KdPHw0aNEi1atWSl5eXIiIitHr16uv2abFYtH79eptlfn5+Nvv47bffNHDgQFWrVk0BAQF69NFH9euvv1rXb926VW3atJG3t7f8/Pz0wAMPlBj2AAAVH0EKAFDhuLq6avjw4VqxYoUMw7Au/+CDD5Sfn68xY8aodevW+uSTT3TgwAGNHTtWw4YN065du+zeZ25urjp16qQqVaroq6++0tdff60qVaqoW7duys/P1+XLl9WnTx917NhR+/fv144dOzR27FhZLBZHHDIAoJxxdXYBAADY44knntCrr76qrVu3qlOnTpKu3NbXr18/1axZU5MmTbK2jY2N1WeffaYPPvhAbdu2tWt/77//vipVqqQlS5ZYw9Hy5cvl5+enrVu3KjIyUllZWerVq5caNGggSWrSpMlNHiUAoLxiRAoAUCHdc889ioqK0rJlyyRJ//nPf7R9+3Y98cQTKigo0EsvvaTmzZsrICBAVapU0aZNm3Ts2DG797dnzx79/PPP8vHxUZUqVVSlShX5+/vr4sWL+s9//iN/f3+NHDlSMTEx6t27t1577TWlp6c76nABAOUMQQoAUGGNHj1aH330kbKzs7V8+XLVqVNHDz/8sObMmaOkpCQ999xz2rx5s1JTUxUTE6P8/PxS+7JYLDa3CUrSpUuXrH8uLCxU69atlZqaavM6fPiwBg8eLOnKCNWOHTsUFRWlNWvWqFGjRtq5c+etOXgAgFMRpAAAFdaAAQPk4uKiVatWaeXKlRo1apQsFou2b9+uRx99VEOHDlWLFi1Uv359HTly5Lp9Va9e3WYE6ciRI8rNzbW+b9WqlY4cOaIaNWqoYcOGNi9fX19ru/vuu08JCQlKSUlRs2bNtGrVKscfOADA6QhSAIAKq0qVKho4cKCef/55nTx5UiNHjpQkNWzYUMnJyUpJSdHBgwf11FNPKSMj47p9PfTQQ1qwYIH27t2r7777Tk8//bTc3Nys64cMGaLAwEA9+uij2r59u9LS0rRt2zY9++yzOnHihNLS0pSQkKAdO3bo6NGj2rRpkw4fPsxzUgBwhyJIAQAqtNGjR+vMmTPq3LmzateuLUl64YUX1KpVK8XExCg6OlrBwcHq06fPdfuZM2eOwsLC9OCDD2rw4MGaNGmSvLy8rOu9vLz01VdfqXbt2urXr5+aNGmiJ554QhcuXFDVqlXl5eWln376Sf3791ejRo00duxYTZgwQU899dStPHwAgJNYjGtvCAcAAAAAXBcjUgAAAABgEkEKAAAAAEwiSAEAAACASQQpAAAAADCJIAUAAAAAJhGkAAAAAMAkghQAAAAAmESQAgAAAACTCFIAAAAAYBJBCgAAAABMIkgBAAAAgEn/HwMvB6YxAg08AAAAAElFTkSuQmCC\n",
"text/plain": [
"<Figure size 1000x400 with 1 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# Test your function\n",
"\n",
"binomial_0 = binomial_generator(12, 0.4, 1000)\n",
"binomial_1 = binomial_generator(15, 0.5, 1000)\n",
"binomial_2 = binomial_generator(25, 0.8, 1000)\n",
"\n",
"utils.plot_binomial_distributions(binomial_0, binomial_1, binomial_2)"
]
},
{
"cell_type": "markdown",
"id": "c40d2b52",
"metadata": {},
"source": [
"##### __Expected Output__\n",
"\n",
"<img src=\"./assets/binomial2.png\" style=\"height: 300px;\"/>\n"
]
},
{
"cell_type": "markdown",
"id": "664454c7",
"metadata": {},
"source": [
"# Section 2: Building a Dog Breed Classifier using Naive Bayes\n",
"\n",
"Data generated from simulations plays an important role in Machine Learning to train and evaluate models. In this section, you will simulate a synthetic dataset containing information about three different dog breeds. Each of them will have four features following various distributions, and you will use the corresponding generator functions for each of them. Then you will build and test a simple Naive Bayes classified to predict the breed from the features."
]
},
{
"cell_type": "markdown",
"id": "9e560546",
"metadata": {},
"source": [
"## Section 2.1: Generating the Dataset\n",
"\n",
"In this section, we will generate a dataset that consists of four features for each dog:\n",
"\n",
"- `height` in centimeters, which follows a Gaussian distribution.\n",
"- `weight` in kilograms, which follows a Gaussian distribution.\n",
"- `bark_days`, representing the number of days (out of 30) that the dog barks. It follows a Binomial distribution with `n = 30`.\n",
"- `ear_head_ratio`, which is the ratio between the length of the ears and the length of the head. It follows a Uniform distribution.\n"
]
},
{
"cell_type": "code",
"execution_count": 49,
"id": "45961a80",
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": [
"FEATURES = [\"height\", \"weight\", \"bark_days\", \"ear_head_ratio\"]"
]
},
{
"cell_type": "markdown",
"id": "4064e165",
"metadata": {},
"source": [
"You will generate synthetic data using the generator functions defined earlier to create a diverse and representative dataset for our dog breed classification problem. \n",
"\n",
"In order to avoid a dependency between section 1 and this one, a pre-loaded version of the dataset is available. If you implemented all of the graded functions in section 1 correctly then the resulting dataset should be equal to the pre-loaded one. But in case something went off you can savely work on section 2:"
]
},
{
"cell_type": "code",
"execution_count": 50,
"id": "c34979c7",
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"The dataset generated from the generator functions is not identical to the expect one.\n",
"\n",
"Falling back to the pre-loaded one.\n"
]
},
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>height</th>\n",
" <th>weight</th>\n",
" <th>bark_days</th>\n",
" <th>ear_head_ratio</th>\n",
" <th>breed</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>2836</th>\n",
" <td>39.697810</td>\n",
" <td>31.740980</td>\n",
" <td>9.0</td>\n",
" <td>0.193120</td>\n",
" <td>2</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1002</th>\n",
" <td>36.710641</td>\n",
" <td>21.140427</td>\n",
" <td>26.0</td>\n",
" <td>0.163527</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1075</th>\n",
" <td>34.726930</td>\n",
" <td>19.817954</td>\n",
" <td>24.0</td>\n",
" <td>0.386113</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1583</th>\n",
" <td>32.324884</td>\n",
" <td>30.812210</td>\n",
" <td>18.0</td>\n",
" <td>0.463242</td>\n",
" <td>1</td>\n",
" </tr>\n",
" <tr>\n",
" <th>248</th>\n",
" <td>37.691499</td>\n",
" <td>21.794333</td>\n",
" <td>28.0</td>\n",
" <td>0.118190</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>814</th>\n",
" <td>36.688852</td>\n",
" <td>21.125901</td>\n",
" <td>26.0</td>\n",
" <td>0.165052</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1407</th>\n",
" <td>30.844078</td>\n",
" <td>27.110196</td>\n",
" <td>16.0</td>\n",
" <td>0.399051</td>\n",
" <td>1</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3376</th>\n",
" <td>38.616784</td>\n",
" <td>30.814387</td>\n",
" <td>8.0</td>\n",
" <td>0.169269</td>\n",
" <td>2</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2700</th>\n",
" <td>44.655532</td>\n",
" <td>35.990456</td>\n",
" <td>12.0</td>\n",
" <td>0.281653</td>\n",
" <td>2</td>\n",
" </tr>\n",
" <tr>\n",
" <th>533</th>\n",
" <td>35.209095</td>\n",
" <td>20.139397</td>\n",
" <td>24.0</td>\n",
" <td>0.322284</td>\n",
" <td>0</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" height weight bark_days ear_head_ratio breed\n",
"2836 39.697810 31.740980 9.0 0.193120 2\n",
"1002 36.710641 21.140427 26.0 0.163527 0\n",
"1075 34.726930 19.817954 24.0 0.386113 0\n",
"1583 32.324884 30.812210 18.0 0.463242 1\n",
"248 37.691499 21.794333 28.0 0.118190 0\n",
"814 36.688852 21.125901 26.0 0.165052 0\n",
"1407 30.844078 27.110196 16.0 0.399051 1\n",
"3376 38.616784 30.814387 8.0 0.169269 2\n",
"2700 44.655532 35.990456 12.0 0.281653 2\n",
"533 35.209095 20.139397 24.0 0.322284 0"
]
},
"execution_count": 50,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Read the pre-loaded dataset\n",
"pre_loaded_df = pd.read_pickle(\"df_all_breeds.pkl\")\n",
"\n",
"try:\n",
" # Generate the dataset using the graded functions from section 1\n",
" df_all_breeds = utils.generate_data(gaussian_generator, binomial_generator, uniform_generator)\n",
"except:\n",
" # In case of an error\n",
" print(\"There was an error when generating the dataset using the generator functions.\\n\\nFalling back to the pre-loaded one.\")\n",
" df_all_breeds = pre_loaded_df\n",
"else:\n",
" # In case that the generated dataset does not match the pre-loaded one\n",
" if not df_all_breeds.equals(pre_loaded_df):\n",
" print(\"The dataset generated from the generator functions is not identical to the expect one.\\n\\nFalling back to the pre-loaded one.\")\n",
" df_all_breeds = pre_loaded_df\n",
"\n",
" \n",
"# Print the first 10 rows of the dataframe\n",
"df_all_breeds.head(10)"
]
},
{
"cell_type": "markdown",
"id": "2773730b",
"metadata": {},
"source": [
"All that is left is to divide the generated dataset into training and testing splits. You will use the 70% of the data for training and the remaining 30% for testing:"
]
},
{
"cell_type": "code",
"execution_count": 51,
"id": "722e8ec7",
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": [
"# Define a 70/30 training/testing split\n",
"split = int(len(df_all_breeds)*0.7)\n",
"\n",
"# Do the split\n",
"df_train = df_all_breeds[:split].reset_index(drop=True)\n",
"df_test = df_all_breeds[split:].reset_index(drop=True)"
]
},
{
"cell_type": "markdown",
"id": "7f31e13a",
"metadata": {},
"source": [
"## Section 2.2 Brief Recap on Naive Bayes Algorithm\n",
"\n",
"Let's recap how the Naive Bayes algorithm works and formalize the notation. \n",
"\n",
"Let $X$ be a set of training data. An element $x \\in X$ is a vector in the form $x = (x_1, x_2, \\ldots, x_n)$, where $n$ is the number of attributes of each sample. For instance, $X$ can be a set of 100 dog breeds, and each dog breed might have 4 attributes, such as ear head ratio, number of days it barks in the month, weight, and height. So, $X = \\{ \\text{dog}_1, \\text{dog}_2, \\ldots, \\text{dog}_{100} \\}$, and each dog breed, for instance, dog 5, will be represented as a 4-dimensional vector: $\\text{dog}_5 = (\\text{ear head ratio}_{\\text{dog}_5}, \\text{bark days}_{\\text{dog}_5}, \\text{weight}_{\\text{dog}_5}, \\text{height}_{\\text{dog}_5})$.\n",
"\n",
"Suppose that there are $m$ classes $C_1, C_2, \\ldots, C_m$. Using the same example above, suppose there are $m = 3$ different types of dog breeds in the training data. The idea is to predict the class of a sample $x \\in X$ by looking at its attributes. Naive Bayes does so by computing the posterior probabilities of a sample belonging to class $C_i$, i.e., it computes\n",
"\n",
"$$\\mathbf P(C_i \\mid x), \\quad i = 1, \\ldots, m.$$\n",
"\n",
"The predicted class is the $C_i$ with the highest probability. More formally, considering the set of every posterior probability of a given sample, what Naive Bayes computes is:\n",
"\n",
"$$\\text{predicted class for } x = \\arg \\max \\left\\{ \\mathbf P(C_1 \\mid x), \\mathbf P(C_2 \\mid x), \\ldots, \\mathbf P(C_m \\mid x) \\right\\}$$\n",
"\n",
"So, if the highest value is $\\mathbf P(C_2 \\mid x)$, then $\\arg \\max \\left\\{\\mathbf P(C_1 \\mid x), \\mathbf P(C_2 \\mid x), \\ldots, \\mathbf P(C_m \\mid x) \\right\\} = 2$.\n",
"\n",
"To compute the posterior probability $\\mathbf P(C_i \\mid x)$, use the Bayes' Theorem:\n",
"\n",
"$$\\mathbf P(C_i \\mid x) = \\frac{\\mathbf P(x \\mid C_i)\\mathbf P(C_i)}{\\mathbf P(x)}.$$\n",
"\n",
"In this equation we use $\\mathbf P(x \\mid C_i)$ generically to indicate the distriution of $X|C_i$. If $X$ is a contiuous random variable, $\\mathbf P(x|C_i)$ should be interpreted as $f_{X|C_i}(x)$. Note that $\\mathbf P(x)$ is positive and constant for every class $C_i$, therefore, to maximize $\\mathbf P(C_i \\mid x)$, it is sufficient to maximize $\\mathbf P(x \\mid C_i)\\mathbf P(C_i)$. The probabilities $\\mathbf P(C_i)$ are called the class prior probabilities, and they denote how likely a random sample from $X$ (without knowing any of its attributes) is to belong to each class. This value is usually not known and can be estimated from the training set by computing the proportion of each class in the training set. If the training set is too small, it is common to assume that each class is equally likely, i.e., $\\mathbf P(C_1) = \\mathbf P(C_2) = \\ldots = \\mathbf P(C_m)$, thus only maximizing $\\mathbf P(x \\mid C_i)$ remains. You will work here with the case when the prior probabilities will be estimated from the training set.\n",
"\n",
"In general, it would be computationally expensive to compute $\\mathbf P(x \\mid C_i)$ for each $x$ and each class, this is why a **naive** assumption of **class-conditional independence** is made. This assumption states that each attribute is independent of each other attribute within each class. It is a strong assumption. For example, in our dog breed example, it would mean that for a specific type of dog breed, there is no correlation between its weight, height, and ear head ratio.\n",
"\n",
"Assuming class-conditional independence, for an $x = (x_1, \\ldots x_n)$ in $X$:\n",
"\n",
"$$\\mathbf P(x \\mid C_i) = \\mathbf P(x_1 \\mid C_i) \\cdot \\mathbf P(x_2 \\mid C_i) \\cdot \\ldots \\cdot \\mathbf P(x_n \\mid C_i) = \\prod_{k = 1}^{n} \\mathbf P(x_k \\mid C_i).$$\n",
"\n",
"The probabilities $\\mathbf P(x_k \\mid C_i)$ can be estimated from the training data. The computation of $\\mathbf P(x_k \\mid C_i)$ depends on whether $x_k$ is categorical or not.\n",
"\n",
"- If $x_k$ is categorical, then $\\mathbf P(x_k \\mid C_i)$ is the number of samples in $X$ that have attribute $x_k$ divided by the number of samples in class $C_i$.\n",
"\n",
"- If $x_k$ is continuous-valued or discrete-valued, we need to make an assumption about its distribution and estimate its parameters using the training data. For instance, if $x_k$ is continuous-valued, we can assume that $\\mathbf P(x_k \\mid C_i)$ follows a Gaussian distribution with parameters $\\mu_{C_i}$ and $\\sigma_{C_i}$. Therefore, we need to estimate $\\mu$ and $\\sigma$ from the training data, and then $\\mathbf P(x_k \\mid C_i) = \\text{PDF}_{\\text{gaussian}}(x_k, \\mu_{C_i}, \\sigma_{C_i})$."
]
},
{
"cell_type": "markdown",
"id": "b86787fc",
"metadata": {},
"source": [
"## Exercise 4: PDF for Distributions\n",
"\n",
"To calculate the probabilities of predicting each class using Naive Bayes, you need to compute the prior probabilities. Although you already know the prior for each feature, you still need a way to compute the probability. In the next exercise, you are required to write a function that takes a value `x` and the relevant parameters and returns the value of the Probability Density Function (`PDF`) for each distribution.\n",
"\n",
"You can choose to implement this function on your own or utilize the implementation from the [scipy.stats](https://docs.scipy.org/doc/scipy/reference/stats.html) module."
]
},
{
"cell_type": "markdown",
"id": "30569fed",
"metadata": {},
"source": [
"### 4.1 Uniform PDF\n",
"\n",
"If $X \\sim \\text{Uniform}(a,b)$, then the PDF for $X$ is given by:\n",
"\n",
"\n",
"$$f_X(x) = \n",
"\\begin{cases}\n",
"\\frac{1}{b-a}, \\quad \\text{if } x \\in [a,b]. \\\\\n",
"0, \\quad \\text{otherwise.}\n",
"\\end{cases}\n",
"$$"
]
},
{
"cell_type": "code",
"execution_count": 52,
"id": "4930247d",
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": [
"def pdf_uniform(x, a, b):\n",
" \"\"\"\n",
" Calculates the probability density function (PDF) for a uniform distribution between 'a' and 'b' at a given point 'x'.\n",
"\n",
" Args:\n",
" x (float): The value at which the PDF is evaluated.\n",
" a (float): The lower bound of the uniform distribution.\n",
" b (float): The upper bound of the uniform distribution.\n",
"\n",
" Returns:\n",
" float: The PDF value at the given point 'x'. Returns 0 if 'x' is outside the range [a, b].\n",
" \"\"\"\n",
" ### START CODE HERE ###\n",
" pdf = (0) if (x < a or x > b) else 1/(b-a)\n",
" ### END CODE HERE ###\n",
"\n",
" return pdf"
]
},
{
"cell_type": "code",
"execution_count": 53,
"id": "f894d948",
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Uniform PDF with a=0 and b=5 for value 1e-10: 0.200\n",
"Uniform PDF with a=20 and b=25 for value 5: 0.000\n",
"Uniform PDF with a=2 and b=10 for value 5.4: 0.125\n",
"Uniform PDF with a=35 and b=60 for value 42: 0.040\n"
]
}
],
"source": [
"# Test your function\n",
"\n",
"print(f\"Uniform PDF with a={0} and b={5} for value {1e-10}: {pdf_uniform(1e-10, 0, 5):.3f}\")\n",
"print(f\"Uniform PDF with a={20} and b={25} for value {5}: {pdf_uniform(5, 20, 25):.3f}\")\n",
"print(f\"Uniform PDF with a={2} and b={10} for value {5.4}: {pdf_uniform(5.4, 2, 10):.3f}\")\n",
"print(f\"Uniform PDF with a={35} and b={60} for value {42}: {pdf_uniform(42, 35, 60):.3f}\")"
]
},
{
"cell_type": "markdown",
"id": "276c9f5e",
"metadata": {},
"source": [
"##### __Expected Output__\n",
"\n",
"```\n",
"Uniform PDF with a=0 and b=5 for value 1e-10: 0.200\n",
"Uniform PDF with a=20 and b=25 for value 5: 0.000\n",
"Uniform PDF with a=2 and b=10 for value 5.4: 0.125\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "bd0bc5b7",
"metadata": {},
"source": [
"### 4.2 Gaussian PDF\n",
"\n",
"You will need to implement the PDF for the Gaussian Distribution. The PDF for $X$ if $X \\sim \\text{Normal}(\\mu,\\sigma^2)$ is given by:\n",
"\n",
"$$f(x) = \\frac{1}{\\sigma \\sqrt{2 \\pi}} e^{-\\frac{1}{2}\\left(\\frac{x - \\mu}{\\sigma}\\right)^2}$$ "
]
},
{
"cell_type": "code",
"execution_count": 54,
"id": "2be18fe6",
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": [
"def pdf_gaussian(x, mu, sigma):\n",
" \"\"\"\n",
" Calculate the probability density function (PDF) of a Gaussian distribution at a given value.\n",
"\n",
" Args:\n",
" x (float or array-like): The value(s) at which to evaluate the PDF.\n",
" mu (float): The mean of the Gaussian distribution.\n",
" sigma (float): The standard deviation of the Gaussian distribution.\n",
"\n",
" Returns:\n",
" float or ndarray: The PDF value(s) at the given point(s) x.\n",
" \"\"\"\n",
"\n",
" ### START CODE HERE ###\n",
" coefficient = 1.0 / (sigma * np.sqrt(2 * np.pi))\n",
" exponent = -0.5 * ((x - mu) / sigma) ** 2\n",
" pdf = coefficient * np.exp(exponent)\n",
" ### END CODE HERE ###\n",
"\n",
" return pdf"
]
},
{
"cell_type": "code",
"execution_count": 55,
"id": "32bb2b1e",
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Gaussian PDF with mu=15 and sigma=5 for value 10: 0.048\n",
"Gaussian PDF with mu=15 and sigma=5 for value 0: 0.001\n",
"Gaussian PDF with mu=20 and sigma=0.5 for value 20: 0.798\n",
"Gaussian PDF with mu=15 and sigma=5 for value 1: 0.002\n"
]
}
],
"source": [
"# Test your function\n",
"\n",
"print(f\"Gaussian PDF with mu={15} and sigma={5} for value {10}: {pdf_gaussian(10, 15, 5):.3f}\")\n",
"print(f\"Gaussian PDF with mu={15} and sigma={5} for value {0}: {pdf_gaussian(0, 15, 5):.3f}\")\n",
"print(f\"Gaussian PDF with mu={20} and sigma={0.5} for value {20}: {pdf_gaussian(20, 20, 0.5):.3f}\")\n",
"print(f\"Gaussian PDF with mu={15} and sigma={5} for value {1}: {pdf_gaussian(1, 15, 5):.3f}\")"
]
},
{
"cell_type": "markdown",
"id": "a77e3a5f",
"metadata": {},
"source": [
"##### __Expected Output__\n",
"\n",
"```\n",
"Gaussian PDF with mu=15 and sigma=5 for value 10: 0.048\n",
"Gaussian PDF with mu=15 and sigma=5 for value 0: 0.001\n",
"Gaussian PDF with mu=20 and sigma=0.5 for value 20: 0.798\n",
"Gaussian PDF with mu=15 and sigma=5 for value 1: 0.002\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "b2f27cce",
"metadata": {},
"source": [
"### 4.3 Binomial Probability Mass Function (PMF)\n",
"\n",
"For the binomial distribution, since it is a discrete distribution, we will be using the Probability Mass Function (PMF) instead of the Probability Density Function (PDF).\n",
"\n",
"Remember that if we have a random variable X following a binomial distribution with parameters n and p, its PMF is given by:\n",
"$$p_X(x) = {n \\choose x} p^x (1-p)^{n-x},\\quad x=0,1,2,\\ldots, n$$\n",
"\n",
"Here, you can calculate the combination ${n \\choose x}$ using either the definition: ${n \\choose x} = \\frac{n!}{x!(n-x)!}$, utilizing the `math.factorial` function, or you can use the `scipy.special.comb` function to obtain the combination. You can also refer to the `binom` documentation to find any other relevant functions that may assist you."
]
},
{
"cell_type": "code",
"execution_count": 56,
"id": "64167eef",
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": [
"def pmf_binomial(x, n, p):\n",
" \"\"\"\n",
" Calculate the probability mass function (PMF) of a binomial distribution at a specific value.\n",
"\n",
" Args:\n",
" x (int): The value at which to evaluate the PMF.\n",
" n (int): The number of trials in the binomial distribution.\n",
" p (float): The probability of success for each trial.\n",
"\n",
" Returns:\n",
" float: The probability mass function (PMF) of the binomial distribution at the specified value.\n",
" \"\"\"\n",
"\n",
" ### START CODE HERE ###\n",
" pmf = binom.pmf(x, n, p)\n",
" ### END CODE HERE ###\n",
"\n",
" return pmf"
]
},
{
"cell_type": "code",
"execution_count": 57,
"id": "e15b91ec",
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Binomial PMF with n=15 and p=0.9 for value 15: 0.206\n",
"Binomial PMF with n=30 and p=0.5 for value 15: 0.144\n",
"Binomial PMF with n=20 and p=0.9 for value 15: 0.032\n",
"Binomial PMF with n=15 and p=0.5 for value 20: 0.000\n"
]
}
],
"source": [
"# Test your function\n",
"\n",
"print(f\"Binomial PMF with n={15} and p={0.9} for value {15}: {pmf_binomial(15, 15, 0.9):.3f}\")\n",
"print(f\"Binomial PMF with n={30} and p={0.5} for value {15}: {pmf_binomial(15, 30, 0.5):.3f}\")\n",
"print(f\"Binomial PMF with n={20} and p={0.9} for value {15}: {pmf_binomial(15, 20, 0.9):.3f}\")\n",
"print(f\"Binomial PMF with n={15} and p={0.5} for value {20}: {pmf_binomial(20, 15, 0.5):.3f}\")"
]
},
{
"cell_type": "markdown",
"id": "f50dbf59",
"metadata": {},
"source": [
"##### __Expected Output__\n",
"\n",
"```\n",
"Binomial PMF with n=15 and p=0.9 for value 15: 0.206\n",
"Binomial PMF with n=30 and p=0.5 for value 15: 0.144\n",
"Binomial PMF with n=20 and p=0.9 for value 15: 0.032\n",
"Binomial PMF with n=15 and p=0.5 for value 20: 0.000\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "f8bb2dc5",
"metadata": {},
"source": [
"## 2.3 Estimated Parameters\n",
"\n",
"Now that you have the `PDF`s ready you need a way of estimating the parameters of the distributions for the features in the training split, this translates to estimating:\n",
"\n",
"- `mu` and `sigma` for the `height` feature\n",
"- `mu` and `sigma` for the `weight` feature \n",
"- `n` and `p` for the `bark_days` feature \n",
"- `a` and `b` for the `ear_head_ratio` feature\n",
"\n",
"\n",
"Since the interpretation and way of computing these parameters has not been covered in the lectures, the assignment provides functions that can accomplish this for you (**you don't need to use them in any of your code but it is good that you are aware what is going on under the hood**). These have already been imported into this environment and are called:\n",
"\n",
"- `estimate_gaussian_params`\n",
"- `estimate_binomial_params`\n",
"- `estimate_uniform_params`\n",
"\n",
"All of these functions work in the same way. They expect an array of numbers (a numpy array, a pandas series or a regular python list) and will return the relevant parameters depending on the distribution selected. An example of how to use these functions can be seen by running the following cell:"
]
},
{
"cell_type": "code",
"execution_count": 58,
"id": "febc3031",
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Gaussian:\n",
"mu = 24.583 and sigma = 7.232 for sample: [26.31 32.45 14.99]\n",
"\n",
"Binomial:\n",
"n = 30 and p = 0.480 for sample: [ 9 26 18 14 5]\n",
"\n",
"Uniform:\n",
"a = 0.070 and b = 0.900 for sample: [0.9 0.26 0.18 0.07 0.5 ]\n"
]
}
],
"source": [
"m, s = utils.estimate_gaussian_params(np.array([26.31, 32.45, 14.99]))\n",
"print(f\"Gaussian:\\nmu = {m:.3f} and sigma = {s:.3f} for sample: {np.array([26.31, 32.45, 14.99])}\\n\")\n",
"\n",
"n, p = utils.estimate_binomial_params(np.array([9, 26, 18, 14, 5]))\n",
"print(f\"Binomial:\\nn = {n} and p = {p:.3f} for sample: {np.array([9, 26, 18, 14, 5])}\\n\")\n",
"\n",
"a, b = utils.estimate_uniform_params(np.array([0.9, 0.26, 0.18, 0.07, 0.5]))\n",
"print(f\"Uniform:\\na = {a:.3f} and b = {b:.3f} for sample: {np.array([0.9, 0.26, 0.18, 0.07, 0.5])}\")"
]
},
{
"cell_type": "markdown",
"id": "dba92e67",
"metadata": {},
"source": [
"By using these functions on every corresponding feature of the training set, the training parameters can be estimated. The next cell will perform this operation and save these parameters in the `train_params` variable, which is a dictionary with the following structure:\n",
"\n",
"```\n",
"{breed: {feature_1: {param_1: value, param_2: value},\n",
" feature_2: {param_1: value, param_2: value},\n",
" ... \n",
"}\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 59,
"id": "2a343e08",
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"data": {
"text/plain": [
"{0: {'height': {'mu': 35.03, 'sigma': 1.518},\n",
" 'weight': {'mu': 20.02, 'sigma': 1.012},\n",
" 'bark_days': {'n': 30, 'p': 0.801},\n",
" 'ear_head_ratio': {'a': 0.1, 'b': 0.597}},\n",
" 1: {'height': {'mu': 29.971, 'sigma': 2.01},\n",
" 'weight': {'mu': 24.927, 'sigma': 5.025},\n",
" 'bark_days': {'n': 30, 'p': 0.498},\n",
" 'ear_head_ratio': {'a': 0.201, 'b': 0.5}},\n",
" 2: {'height': {'mu': 39.814, 'sigma': 3.572},\n",
" 'weight': {'mu': 31.841, 'sigma': 3.061},\n",
" 'bark_days': {'n': 30, 'p': 0.296},\n",
" 'ear_head_ratio': {'a': 0.101, 'b': 0.3}}}"
]
},
"execution_count": 59,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Compute the estimated parameters from the training dataset\n",
"train_params = utils.compute_training_params(df_train, FEATURES)\n",
"\n",
"# Print the parameters dictionary with rounded values - this is just for prettier visualizations\n",
"utils.round_dict(train_params)"
]
},
{
"cell_type": "markdown",
"id": "738a0478",
"metadata": {},
"source": [
"To get any estimated parameter out of this dictionary you use the syntax `train_params[breed][feature][parameter]` \n",
"\n",
"Check two examples of this in the following cell:"
]
},
{
"cell_type": "code",
"execution_count": 60,
"id": "46a70a37",
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"estimated value of n for the feature bark_days for the breed 0: 30\n",
"\n",
"estimated value of sigma for the feature height for the breed 2: 3.572\n"
]
}
],
"source": [
"# Get the estimated value of n for the feature bark_days for the breed 0\n",
"estimated_param = train_params[0]['bark_days']['n']\n",
"print(f\"estimated value of n for the feature bark_days for the breed 0: {estimated_param}\\n\")\n",
"\n",
"# Get the estimated value of sigma for the feature height for the breed 2\n",
"estimated_param = train_params[2]['height']['sigma']\n",
"print(f\"estimated value of sigma for the feature height for the breed 2: {estimated_param:.3f}\")"
]
},
{
"cell_type": "markdown",
"id": "0618af71",
"metadata": {},
"source": [
"## Exercise 5: Computing the probability of each class\n",
"\n",
"Now you have the estimated parameters for every feature you need to compute the probability belonging to each class which is analogous to the proportion in which every class appears in the training set. For this complete the `compute_breed_proportions` function below. This function should return a dictionary that includes the proportion of data belonging to each breed. Notice that all values should sum up to 1. You can use Python's built-in `round` function to avoid very long floats but this is up to you and your grade will not be affected by this. This dict should look like this:\n",
"\n",
"```\n",
"{0: 0.25, 1: 0.5, 2: 0.25} \n",
"```\n",
"\n",
"Notice that the keys of this dictionary are integers while the values are floating point numbers."
]
},
{
"cell_type": "code",
"execution_count": 61,
"id": "4023c2c1",
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": [
"def compute_breed_proportions(df):\n",
" \"\"\"\n",
" Computes the estimated probabilities of each breed.\n",
"\n",
" Args:\n",
" df (pandas.DataFrame): The dataframe containing the training data.\n",
"\n",
" Returns:\n",
" - probs_dict (dict): A dictionary that contains the proportion of data belonging to each breed. \n",
" \"\"\"\n",
" # Dict that should contain the estimated parameters\n",
" params_dict = {}\n",
" \n",
" probs_dict = {}\n",
" \n",
" ### START CODE HERE ###\n",
" \n",
" # Loop over the breeds\n",
" for breed in df[\"breed\"]:\n",
" \n",
" # Slice the original df to only include data for the current breed\n",
" # You can use the syntax df[df['breed'] == group] replacing group with the corresponding variable\n",
" df_breed = df[df[\"breed\"] == breed]\n",
" \n",
" probs_dict[breed] = round(len(df_breed) / len(df), 3)\n",
" \n",
" ### END CODE HERE ###\n",
"\n",
" return probs_dict"
]
},
{
"cell_type": "code",
"execution_count": 63,
"id": "3be3e2df",
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Probability of each class for training split:\n",
"\n",
"{2: 0.26, 0: 0.346, 1: 0.393}\n"
]
}
],
"source": [
"# Test your function\n",
"\n",
"train_class_probs = compute_breed_proportions(df_train)\n",
"\n",
"print(f\"Probability of each class for training split:\\n\\n{train_class_probs}\")\n"
]
},
{
"cell_type": "markdown",
"id": "078d7688",
"metadata": {},
"source": [
"##### __Expected Output__\n",
"\n",
"```\n",
"Probability of each class for training split:\n",
"\n",
"{0: 0.346, 1: 0.393, 2: 0.26}\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "3c0e2308",
"metadata": {},
"source": [
"## Exercise 6: Compute the Probability of X given the Breed -> $P(x \\mid C_{i})$\n",
"\n",
"To code a Naive Bayes classifier, you will assume **class-conditional independence** for a given $\\boldsymbol x = (x_1, \\ldots, x_n)$ in $\\boldsymbol X$. With this assumption, you can compute the probability of $x$ given the class using the following expression:\n",
"\n",
"$$\\mathbf P(\\boldsymbol x \\mid C_{i}) = \\mathbf P(x_1 \\mid C_i) \\cdot \\mathbf P(x_2 \\mid C_i) \\cdot \\ldots \\cdot \\mathbf P(x_n \\mid C_i) = \\prod_{k = 1}^{n} \\mathbf P(x_k \\mid C_i).$$\n",
"\n",
"The probabilities $\\mathbf P(x_k \\mid C_i)$ can be estimated from the training tuples.\n",
"\n",
"If $x_k$ is continuous-valued or discrete-valued, you need to make an assumption about its distribution and estimate its parameters using the training set. For example, if $x_k$ is continuous-valued, it is often assumed that $\\mathbf P(x_k \\mid C_i)$ follows a Gaussian distribution with parameters $\\mu_{C_i}$ and $\\sigma_{C_i}$. Therefore, you need to estimate $\\mu$ and $\\sigma$ from the training set, and then $\\mathbf P(x_k \\mid C_i) = \\text{PDF}_{\\text{gaussian}}(x_k,\\mu_{C_i},\\sigma_{C_i})$.\n",
"\n",
"In this case, you already know the true distributions for every feature, so you just need to compute the appropriate `PDF` for each feature by passing the estimated parameters of that feature to the corresponding `PDF` computation function.\n",
"\n",
"Complete the `prob_of_X_given_C` function below. This function takes the following parameters:\n",
"- `X`: a list containing the values for the features in the `features` parameter (the order matters)\n",
"- `features`: the names of the features being passed\n",
"- `breed`: the breed that will be assumed for the `X` observation\n",
"- `params_dict`: the dictionary containing the estimated parameters from the training split\n",
"\n",
"The function should return the probability of the values of `X` given the selected `breed`.\n"
]
},
{
"cell_type": "code",
"execution_count": 78,
"id": "28f50bb9",
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": [
"def prob_of_X_given_C(X, features, breed, params_dict):\n",
" \"\"\"\n",
" Calculate the conditional probability of X given a specific breed, using the given features and parameters.\n",
"\n",
" Args:\n",
" X (list): List of feature values for which the probability needs to be calculated.\n",
" features (list): List of feature names corresponding to the feature values in X.\n",
" breed (int): The breed for which the probability is calculated.\n",
" params_dict (dict): Dictionary containing the parameters for different breeds and features.\n",
"\n",
" Returns:\n",
" float: The conditional probability of X given the specified breed.\n",
" \"\"\"\n",
" \n",
" if len(X) != len(features):\n",
" print(\"X and list of features should have the same length\")\n",
" return 0\n",
" \n",
" \n",
" probability = 1.0\n",
" \n",
" ### START CODE HERE ###\n",
" \n",
" for feature_val, feature_name in zip(X, features):\n",
" \n",
" # Match the current feature based on its name\n",
" match feature_name:\n",
" case \"height\" | \"weight\": \n",
" # Get the relevant parameters out of the params_dict dictionary\n",
" mu = params_dict[breed][feature_name][\"mu\"]\n",
" sigma = params_dict[0][feature_name][\"sigma\"]\n",
" \n",
" # Compute the relevant pdf given the distribution and the estimated parameters\n",
" probability_f = pdf_gaussian(feature_val, mu, sigma)\n",
" \n",
" case \"bark_days\": \n",
" # Get the relevant parameters out of the params_dict dictionary\n",
" n = params_dict[breed][feature_name][\"n\"]\n",
" p = params_dict[breed][feature_name][\"p\"]\n",
" \n",
" # Compute the relevant pmf given the distribution and the estimated parameters\n",
" probability_f = pmf_binomial(feature_val, n, p)\n",
"\n",
" case \"ear_head_ratio\": \n",
" # Get the relevant parameters out of the params_dict dictionary\n",
" a = params_dict[breed][feature_name][\"a\"]\n",
" b = params_dict[breed][feature_name][\"b\"]\n",
" \n",
" # Compute the relevant pdf given the distribution and the estimated parameters\n",
" probability_f = pdf_uniform(feature_val, a, b)\n",
" \n",
" # Multiply by probability of current feature\n",
" probability *= probability_f\n",
" \n",
" ### END CODE HERE ###\n",
" \n",
" return probability"
]
},
{
"cell_type": "code",
"execution_count": 79,
"id": "2c00030d",
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Example dog has breed 1 and features: height = 28.63, weight = 21.56, bark_days = 13.00, ear_head_ratio = 0.27\n",
"\n",
"Probability of these features if dog is classified as breed 0: 6.989632718589114e-11\n",
"Probability of these features if dog is classified as breed 1: 0.00010647848339178657\n",
"Probability of these features if dog is classified as breed 2: 1.4464091416605823e-36\n"
]
}
],
"source": [
"# Test your function\n",
"\n",
"example_dog = df_test[FEATURES].loc[0]\n",
"example_breed = df_test[[\"breed\"]].loc[0][\"breed\"]\n",
"print(f\"Example dog has breed {example_breed} and features: height = {example_dog['height']:.2f}, weight = {example_dog['weight']:.2f}, bark_days = {example_dog['bark_days']:.2f}, ear_head_ratio = {example_dog['ear_head_ratio']:.2f}\\n\")\n",
"\n",
"print(f\"Probability of these features if dog is classified as breed 0: {prob_of_X_given_C([*example_dog], FEATURES, 0, train_params)}\")\n",
"print(f\"Probability of these features if dog is classified as breed 1: {prob_of_X_given_C([*example_dog], FEATURES, 1, train_params)}\")\n",
"print(f\"Probability of these features if dog is classified as breed 2: {prob_of_X_given_C([*example_dog], FEATURES, 2, train_params)}\")\n"
]
},
{
"cell_type": "markdown",
"id": "29e6a339",
"metadata": {},
"source": [
"##### __Expected Output__\n",
"\n",
"```\n",
"Example dog has breed 1 and features: height = 28.63, weight = 21.56, bark_days = 13.00, ear_head_ratio = 0.27\n",
"\n",
"Probability of these features if dog is classified as breed 0: 6.989632718589114e-11\n",
"Probability of these features if dog is classified as breed 1: 0.0038267778327024894\n",
"Probability of these features if dog is classified as breed 2: 7.959172138800559e-08\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "f9771c79",
"metadata": {},
"source": [
"## Exercise 7: Predict the Breed\n",
"\n",
"If all classes were perfectly balanced, the previous function could be used to compute the maximum posterior. However, this is NOT the case, and you still need to multiply every probability $\\mathbf P(x \\mid C_{i})$ by the probability of belonging to each class $\\mathbf P(C_{i})$. After all, the expression that you need to maximize in order to get a prediction is $\\mathbf P(x \\mid C_{i})\\mathbf P(C_{i})$. You can accomplish this by multiplying the result of `prob_of_X_given_C` by the corresponding proportion found in the `probs_dict` dictionary.\n",
"\n",
"Complete the `predict_breed` function below. This function receives the following parameters:\n",
"- `X`: a list containing the values for the features of each feature in the `features` parameter (the order matters).\n",
"- `features`: the name of the features being passed.\n",
"- `params_dict`: the dictionary containing the estimated parameters from the training split.\n",
"- `probs_dict`: the dictionary containing the proportion of each class from the training split.\n",
"\n",
"The function should return the breed with the highest maximum posterior:\n"
]
},
{
"cell_type": "code",
"execution_count": 80,
"id": "704ec4ac",
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": [
"def predict_breed(X, features, params_dict, probs_dict):\n",
" \"\"\"\n",
" Predicts the breed based on the input and features.\n",
"\n",
" Args:\n",
" X (array-like): The input data for prediction.\n",
" features (array-like): The features used for prediction.\n",
" params_dict (dict): A dictionary containing parameters for different breeds.\n",
" probs_dict (dict): A dictionary containing probabilities for different breeds.\n",
"\n",
" Returns:\n",
" int: The predicted breed index.\n",
" \"\"\"\n",
" \n",
" ### START CODE HERE ###\n",
" \n",
" posterior_breed_0 = prob_of_X_given_C(X, features, 0, params_dict)*probs_dict[0]\n",
" posterior_breed_1 = prob_of_X_given_C(X, features, 1, params_dict)*probs_dict[1]\n",
" posterior_breed_2 = prob_of_X_given_C(X, features, 2, params_dict)*probs_dict[2]\n",
" \n",
" # Save the breed with the maximum posterior\n",
" # Hint: You can create a numpy array with the posteriors and then use np.argmax\n",
" prediction = np.argmax(np.array([posterior_breed_0, posterior_breed_1, posterior_breed_2]))\n",
" ### END CODE HERE ###\n",
" \n",
" return prediction"
]
},
{
"cell_type": "code",
"execution_count": 81,
"id": "7315ee67",
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Example dog has breed 1 and Naive Bayes classified it as 1\n"
]
}
],
"source": [
"# Test your function\n",
"\n",
"example_pred = predict_breed([*example_dog], FEATURES, train_params, train_class_probs)\n",
"print(f\"Example dog has breed {example_breed} and Naive Bayes classified it as {example_pred}\")\n"
]
},
{
"cell_type": "markdown",
"id": "59d4af42",
"metadata": {},
"source": [
"##### __Expected Output__\n",
"\n",
"```\n",
"Example dog has breed 1 and Naive Bayes classified it as 1\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "6563437b",
"metadata": {},
"source": [
"The classifier worked for this particular example, but how will it perform when considering the whole testing split?\n",
"\n",
"Run the following cell to find out:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9ca023fd",
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": [
"preds = df_test.apply(lambda x: predict_breed([*x[FEATURES]], FEATURES, train_params, train_class_probs), axis=1)\n",
"test_acc = accuracy_score(df_test[\"breed\"], preds)\n",
"print(f\"Accuracy score for the test split: {test_acc:.2f}\")"
]
},
{
"cell_type": "markdown",
"id": "4ad8fde0",
"metadata": {},
"source": [
"##### __Expected Output__\n",
"\n",
"```\n",
"Accuracy score for the test split: 1.00\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "b07594b8",
"metadata": {},
"source": [
"The Naive Bayes classifier achieved an accuracy of 100% in the testing data. Nice job! \n",
"\n",
"You might think that something is wrong when reaching such a high accuracy but in this case it makes sense because the data is generated and you know the true distributions for each feature, real-life data won't have this nice behavior, but you can see how generated data can help to build and preliminarily test a machine learning model."
]
},
{
"cell_type": "markdown",
"id": "3d9cb739",
"metadata": {},
"source": [
"# Section 3 - Spam Detector"
]
},
{
"cell_type": "markdown",
"id": "d0b3c87f",
"metadata": {},
"source": [
"In this final section you will once again implement and train a Naive Bayes classifier. The idea is to build a classifier that is able to detect spam from ham (aka not spam) emails. This time your implementation should take into account two major differences:\n",
"\n",
"- The data is a real life dataset that includes over 5500 emails with their corresponding labels\n",
"- The features are categorical\n",
"\n",
"Begin by loading the dataset and doing some pre-processing:"
]
},
{
"cell_type": "code",
"execution_count": 82,
"id": "6aa97724",
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>text</th>\n",
" <th>spam</th>\n",
" <th>words</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>Subject: naturally irresistible your corporate...</td>\n",
" <td>1</td>\n",
" <td>[easy, creativeness, isoverwhelminq, but, conv...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>Subject: the stock trading gunslinger fanny i...</td>\n",
" <td>1</td>\n",
" <td>[perspicuous, waterway, mcdougall, but, diffus...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>Subject: unbelievable new homes made easy im ...</td>\n",
" <td>1</td>\n",
" <td>[easy, unbelievable, advantage, limited, post,...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>Subject: 4 color printing special request add...</td>\n",
" <td>1</td>\n",
" <td>[additional, /, golden, @, mail, version, -, c...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>Subject: do not have money , get software cds ...</td>\n",
" <td>1</td>\n",
" <td>[?, finish, great, get, software, ,, all, ain,...</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" text spam \\\n",
"0 Subject: naturally irresistible your corporate... 1 \n",
"1 Subject: the stock trading gunslinger fanny i... 1 \n",
"2 Subject: unbelievable new homes made easy im ... 1 \n",
"3 Subject: 4 color printing special request add... 1 \n",
"4 Subject: do not have money , get software cds ... 1 \n",
"\n",
" words \n",
"0 [easy, creativeness, isoverwhelminq, but, conv... \n",
"1 [perspicuous, waterway, mcdougall, but, diffus... \n",
"2 [easy, unbelievable, advantage, limited, post,... \n",
"3 [additional, /, golden, @, mail, version, -, c... \n",
"4 [?, finish, great, get, software, ,, all, ain,... "
]
},
"execution_count": 82,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Load the dataset\n",
"emails = pd.read_csv('emails.csv')\n",
"\n",
"# Helper function that converts text to lowercase and splits words into a list\n",
"def process_email(text):\n",
" \"\"\"\n",
" Processes the given email text by converting it to lowercase, splitting it into words,\n",
" and returning a list of unique words.\n",
"\n",
" Parameters:\n",
" - text (str): The email text to be processed.\n",
"\n",
" Returns:\n",
" - list: A list of unique words extracted from the email text.\n",
" \"\"\"\n",
"\n",
" text = text.lower()\n",
" return list(set(text.split()))\n",
"\n",
"# Create an extra column with the text converted to a lower-cased list of words\n",
"emails['words'] = emails['text'].apply(process_email)\n",
"\n",
"# Show the first 5 rows\n",
"emails.head(5)"
]
},
{
"cell_type": "markdown",
"id": "f4bf5d9e",
"metadata": {},
"source": [
"## One last Naive Bayes reminder\n",
"\n",
"Remember that supposing **class-conditional independence**, for a $\\boldsymbol x = (x_1, \\ldots x_n)$ in $\\boldsymbol X$:\n",
"\n",
"$$\\mathbf P(\\boldsymbol x \\mid C_{i}) = \\mathbf P(x_1 \\mid C_i) \\cdot \\mathbf P(x_2 \\mid C_i) \\cdot \\ldots \\cdot \\mathbf P(x_n \\mid C_i) = \\prod_{k = 1}^{n} \\mathbf P(x_k \\mid C_i).$$\n",
"\n",
"The probabilities $\\mathbf P(x_k\\mid C_i)$ can be estimated from the training tuples. The computation of $\\mathbf P(x_k \\mid C_i)$ depends on whether $x_k$ is categorical or not.\n",
"\n",
"If $x_k$ is categorical, then $\\mathbf P(x_k \\mid C_i)$ is the number of samples in $X$ that has attribute $x_k$ divided by the number of samples in class $C_i$. \n",
"\n",
"With this in mind, you need to know the number of times that each word appears in both spam and ham emails, as well as the number of samples for class. Your next two exercises will be about computing these values.\n",
"\n",
"If you need some extra help with this section, check out this [tutorial](https://www.youtube.com/watch?v=Q8l0Vip5YUw) by your instructor Luis."
]
},
{
"cell_type": "markdown",
"id": "53fb834c",
"metadata": {},
"source": [
"## Exercise 8: Frequency of a word in each class\n",
"\n",
"To compute the frequency of each word in the dataset you need to define the `word_freq_per_class` below. This function receives the email dataframe as input and should return a dictionary that has the words in the emails as keys and another dictionary that keeps track of how many times that word appeared in `spam` and `hams` emails as values. This dictionary should look like this:\n",
"\n",
"```python\n",
"{'website': {'spam': 204, 'ham': 135},\n",
" 'collaboration': {'spam': 28, 'ham': 34},\n",
" 'logo': {'spam': 97, 'ham': 13},\n",
" ...\n",
"}\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 85,
"id": "698c0c15",
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": [
"def word_freq_per_class(df):\n",
" \"\"\"\n",
" Calculates the frequency of words in each class (spam and ham) based on a given dataframe.\n",
"\n",
" Args:\n",
" df (pandas.DataFrame): The input dataframe containing email data, \n",
" with a column named 'words' representing the words in each email.\n",
"\n",
" Returns:\n",
" dict: A dictionary containing the frequency of words in each class. \n",
" The keys of the dictionary are words, and the values are nested dictionaries with keys \n",
" 'spam' and 'ham' representing the frequency of the word in spam and ham emails, respectively.\n",
" \"\"\"\n",
" \n",
" word_freq_dict = {}\n",
" \n",
" ### START CODE HERE ###\n",
" \n",
" # Hint: You can use the iterrows() method to iterate over the rows of a dataframe.\n",
" # This method yields an index and the data in the row so you can ignore the first returned value. \n",
" for _, row in df.iterrows():\n",
" # Iterate over the words in each email\n",
" for word in row['words']:\n",
" # Check if word doesn't exist within the dictionary\n",
" if word not in word_freq_dict:\n",
" # If word doesn't exist, initialize the count at 0\n",
" word_freq_dict[word] = {'spam': 0, 'ham': 0}\n",
" \n",
" # Check if the email was spam\n",
" match row['spam']:\n",
" case 0: \n",
" # If ham then add 1 to the count of ham\n",
" word_freq_dict[word]['ham'] += 1\n",
" case 1: \n",
" # If spam then add 1 to the count of spam\n",
" word_freq_dict[word]['spam'] += 1\n",
" \n",
" ### END CODE HERE ###\n",
"\n",
" return word_freq_dict"
]
},
{
"cell_type": "code",
"execution_count": 86,
"id": "ce2d6cbb",
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Frequency in both classes for word 'lottery': {'spam': 8, 'ham': 0}\n",
"\n",
"Frequency in both classes for word 'sale': {'spam': 38, 'ham': 41}\n",
"\n",
"Word 'asdfg' not in corpus\n"
]
}
],
"source": [
"# Test your function\n",
"\n",
"word_freq = word_freq_per_class(emails)\n",
"print(f\"Frequency in both classes for word 'lottery': {word_freq['lottery']}\\n\")\n",
"print(f\"Frequency in both classes for word 'sale': {word_freq['sale']}\\n\")\n",
"\n",
"try:\n",
" word_freq['asdfg']\n",
"except KeyError:\n",
" print(\"Word 'asdfg' not in corpus\")\n",
"\n",
"# grade-up-to-here"
]
},
{
"cell_type": "markdown",
"id": "0aab1fcd",
"metadata": {},
"source": [
"##### __Expected Output__\n",
"\n",
"```\n",
"Frequency in both classes for word 'lottery': {'spam': 8, 'ham': 0}\n",
"\n",
"Frequency in both classes for word 'sale': {'spam': 38, 'ham': 41}\n",
"\n",
"Word 'asdfg' not in corpus\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "f2fe8acc",
"metadata": {},
"source": [
"## Exercise 9: Frequency of classes\n",
"\n",
"To compute the frequency of each class in the dataset you need to define the `class_frequencies` below. This function receives the email dataframe as input and should return a dictionary that has returns the number of spam and ham emails. This dictionary should look like this:\n",
"\n",
"```python\n",
"{'spam': 1000, 'ham': 1000}\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e5f03a74",
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": [
"def class_frequencies(df):\n",
" \"\"\"\n",
" Calculate the frequencies of classes in a DataFrame.\n",
"\n",
" Args:\n",
" df (DataFrame): The input DataFrame containing a column 'spam' indicating class labels.\n",
"\n",
" Returns:\n",
" dict: A dictionary containing the frequencies of the classes.\n",
" The keys are 'spam' and 'ham', representing the class labels.\n",
" The values are the corresponding frequencies in the DataFrame.\n",
" \"\"\"\n",
" \n",
" ### START CODE HERE ###\n",
" \n",
" class_freq_dict = { \n",
" # You can create a df that only includes data for a certain category like this:\n",
" # df[df['category']==category_value] and then use len(df) to check the number of rows.\n",
" \"spam\": None,\n",
" \"ham\": None\n",
" } \n",
" \n",
" ### END CODE HERE ###\n",
" \n",
" return class_freq_dict"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "20eebe98",
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": [
"# Test your function\n",
"\n",
"class_freq = class_frequencies(emails[:10])\n",
"print(f\"Small dataset:\\n\\nThe frequencies for each class are {class_freq}\\n\")\n",
"print(f\"The proportion of spam in the dataset is: {100*class_freq['spam']/len(emails[:10]):.2f}%\\n\")\n",
"print(f\"The proportion of ham in the dataset is: {100*class_freq['ham']/len(emails[:10]):.2f}%\\n\")\n",
"\n",
"class_freq = class_frequencies(emails)\n",
"print(f\"\\nFull dataset:\\n\\nThe frequencies for each class are {class_freq}\\n\")\n",
"print(f\"The proportion of spam in the dataset is: {100*class_freq['spam']/len(emails):.2f}%\\n\")\n",
"print(f\"The proportion of ham in the dataset is: {100*class_freq['ham']/len(emails):.2f}%\")"
]
},
{
"cell_type": "markdown",
"id": "ba1d3d34",
"metadata": {},
"source": [
"##### __Expected Output__\n",
"\n",
"```\n",
"Small dataset:\n",
"\n",
"The frequencies for each class are {'spam': 10, 'ham': 0}\n",
"\n",
"The proportion of spam in the dataset is: 100.00%\n",
"\n",
"The proportion of ham in the dataset is: 0.00%\n",
"\n",
"\n",
"Full dataset:\n",
"\n",
"The frequencies for each class are {'spam': 1368, 'ham': 4360}\n",
"\n",
"The proportion of spam in the dataset is: 23.88%\n",
"\n",
"The proportion of ham in the dataset is: 76.12%\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "53f1a94c",
"metadata": {},
"source": [
"## Exercise 10: Naive Bayes for categorical features\n",
"\n",
"Now you have everything you need to build your Naive Bayes classifier. Complete the `naive_bayes_classifier` below. This function receives any text as a parameter and should return the probability of that text belonging to the `spam` class. Notice that the function also receives the two dictionaries that were created during the previous exercises, which means that this probability will depend on the dataset you used for training. With this in mind, if you submit a text containing words that are not in the training dataset the probability should be equal to the proportion of `spam` in the emails:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5e05f5ff",
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": [
"def naive_bayes_classifier(text, word_freq=word_freq, class_freq=class_freq):\n",
" \"\"\"\n",
" Implements a naive Bayes classifier to determine the probability of an email being spam.\n",
"\n",
" Args:\n",
" text (str): The input email text to classify.\n",
" \n",
" word_freq (dict): A dictionary containing word frequencies in the training corpus. \n",
" The keys are words, and the values are dictionaries containing frequencies for 'spam' and 'ham' classes.\n",
"\n",
" class_freq (dict): A dictionary containing class frequencies in the training corpus. \n",
" The keys are class labels ('spam' and 'ham'), and the values are the respective frequencies.\n",
"\n",
" Returns:\n",
" float: The probability of the email being spam.\n",
"\n",
" \"\"\"\n",
"\n",
" # Convert the text to lowercase-only\n",
" text = text.lower()\n",
"\n",
" # This is a list containing the words of the email \n",
" words = set(text.split())\n",
" \n",
" cumulative_product_spam = 1.0\n",
" cumulative_product_ham = 1.0\n",
" \n",
" ### START CODE HERE ###\n",
" \n",
" # Iterate over the words in the email\n",
" for None in None:\n",
" # You should only include words that exist in the corpus in your calculations\n",
" if None in None:\n",
" # Calculate the probability of the word appearing in a spam email\n",
" cumulative_product_spam *= None/None\n",
" \n",
" # Calculate the probability of the word appearing in a ham email\n",
" cumulative_product_ham *= None/None\n",
" \n",
" # Calculate the likelihood of the words appearing in the email given that it is spam\n",
" likelihood_word_given_spam = None * None\n",
" \n",
" # Calculate the likelihood of the words appearing in the email given that it is ham\n",
" likelihood_word_given_ham = None * None\n",
" \n",
" # Calculate the posterior probability of the email being spam given that the words appear in the email (the probability of being a spam given the email content)\n",
" prob_spam = None / (None + None)\n",
" \n",
" ### END CODE HERE ###\n",
" \n",
" return prob_spam"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b53b92c4",
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": [
"# Test your function\n",
"\n",
"msg = \"enter the lottery to win three million dollars\"\n",
"print(f\"Probability of spam for email '{msg}': {100*naive_bayes_classifier(msg):.2f}%\\n\")\n",
"\n",
"msg = \"meet me at the lobby of the hotel at nine am\"\n",
"print(f\"Probability of spam for email '{msg}': {100*naive_bayes_classifier(msg):.2f}%\\n\")\n",
"\n",
"msg = \"9898 asjfkjfdj\"\n",
"print(f\"Probability of spam for email '{msg}': {100*naive_bayes_classifier(msg):.2f}%\\n\")"
]
},
{
"cell_type": "markdown",
"id": "05a3df24",
"metadata": {},
"source": [
"##### __Expected Output__\n",
"\n",
"```\n",
"Probability of spam for email 'enter the lottery to win three million dollars': 100.00%\n",
"\n",
"Probability of spam for email 'meet me at the lobby of the hotel at nine am': 0.00%\n",
"\n",
"Probability of spam for email '9898 asjfkjfdj': 23.88%\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "27bef59b",
"metadata": {},
"source": [
"**Congratulations on finishing this assignment!**\n",
"\n",
"During this assignment you tested your theoretical and practical skills by coding functions capable of generating random numbers for the probability distributions you saw in the lectures, as well as creating two implementations of the Naive Bayes algorithm.\n",
"\n",
"**Keep up the good work!**\n"
]
}
],
"metadata": {
"grader_version": "2",
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.9"
}
},
"nbformat": 4,
"nbformat_minor": 5
}