ipynb • Lines: 1285{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "EAt-K2qgcIou"
},
"source": [
"# Optimization Using Gradient Descent: Linear Regression"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "FZYK-0rin5x7"
},
"source": [
"In this assignment, you will build a simple linear regression model to predict sales based on TV marketing expenses. You will investigate three different approaches to this problem. You will use `NumPy` and `Scikit-Learn` linear regression models, as well as construct and optimize the sum of squares cost function with gradient descent from scratch."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Table of Contents\n",
"\n",
"- [ 1 - Open the Dataset and State the Problem](#1)\n",
" - [ Exercise 1](#ex01)\n",
"- [ 2 - Linear Regression in Python with `NumPy` and `Scikit-Learn`](#2)\n",
" - [ 2.1 - Linear Regression with `NumPy`](#2.1)\n",
" - [ Exercise 2](#ex02)\n",
" - [ 2.2 - Linear Regression with `Scikit-Learn`](#2.2)\n",
" - [ Exercise 3](#ex03)\n",
" - [ Exercise 4](#ex04)\n",
"- [ 3 - Linear Regression using Gradient Descent](#3)\n",
" - [ Exercise 5](#ex05)\n",
" - [ Exercise 6](#ex06)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Packages\n",
"\n",
"Load the required packages:"
]
},
{
"cell_type": "code",
"execution_count": 34,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": [
"import numpy as np\n",
"# A library for programmatic plot generation.\n",
"import matplotlib.pyplot as plt\n",
"# A library for data manipulation and analysis.\n",
"import pandas as pd\n",
"# LinearRegression from sklearn.\n",
"from sklearn.linear_model import LinearRegression"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Import the unit tests defined for this notebook."
]
},
{
"cell_type": "code",
"execution_count": 35,
"metadata": {},
"outputs": [],
"source": [
"import w2_unittest"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a name='1'></a>\n",
"## 1 - Open the Dataset and State the Problem"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In this lab, you will build a linear regression model for a simple [Kaggle dataset](https://www.kaggle.com/code/devzohaib/simple-linear-regression/notebook), saved in a file `data/tvmarketing.csv`. The dataset has only two fields: TV marketing expenses (`TV`) and sales amount (`Sales`)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a name='ex01'></a>\n",
"### Exercise 1\n",
"\n",
"Use `pandas` function `pd.read_csv` to open the .csv file the from the `path`."
]
},
{
"cell_type": "code",
"execution_count": 36,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": [
"path = \"data/tvmarketing.csv\"\n",
"\n",
"### START CODE HERE ### (~ 1 line of code)\n",
"adv = pd.read_csv(path)\n",
"### END CODE HERE ###"
]
},
{
"cell_type": "code",
"execution_count": 37,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>TV</th>\n",
" <th>Sales</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>230.1</td>\n",
" <td>22.1</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>44.5</td>\n",
" <td>10.4</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>17.2</td>\n",
" <td>9.3</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>151.5</td>\n",
" <td>18.5</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>180.8</td>\n",
" <td>12.9</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" TV Sales\n",
"0 230.1 22.1\n",
"1 44.5 10.4\n",
"2 17.2 9.3\n",
"3 151.5 18.5\n",
"4 180.8 12.9"
]
},
"execution_count": 37,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Print some part of the dataset.\n",
"adv.head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"##### __Expected Output__ \n",
"\n",
"```Python\n",
"\tTV\tSales\n",
"0\t230.1\t22.1\n",
"1\t44.5\t10.4\n",
"2\t17.2\t9.3\n",
"3\t151.5\t18.5\n",
"4\t180.8\t12.9\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 38,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[92m All tests passed\n"
]
}
],
"source": [
"w2_unittest.test_load_data(adv)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"`pandas` has a function to make plots from the DataFrame fields. By default, matplotlib is used at the backend. Let's use it here:"
]
},
{
"cell_type": "code",
"execution_count": 39,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"data": {
"text/plain": [
"<AxesSubplot: xlabel='TV', ylabel='Sales'>"
]
},
"execution_count": 39,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEGCAYAAABiq/5QAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/av/WaAAAACXBIWXMAAAsTAAALEwEAmpwYAAAmqElEQVR4nO2dfWwl13nen7Pc+2V+rFfSrbpRtKTduK3VINXuag0VNWQkJtN6hVryNg50UTRufQEtXK+hMImLLQyrKogYlY147QhBrhNQWDnI0k4cxzYQu7TpqlEctL7cXa4+TFWW2pKVVMW8jiVqV+YuudLpH3eGGl7Ox5mZMzNnZp4fMODlzJ2Z98zMfc6Z97znPUJKCUIIIeVhT9YGEEIISRcKPyGElAwKPyGElAwKPyGElAwKPyGElIy9WRugwg033CAnJiayNoMQQnLF+fPnfyylbA6uz4XwT0xM4Ny5c1mbQQghuUIIseq2nq4eQggpGRR+QggpGRR+QggpGRR+QggpGRR+QggpGRR+QgjRQK/Xw+LiInq9XtamBELhJ4SQmMzNzWF8fBxTU1MYHx/H3Nxc1ib5IvKQlvm2226TjOMnhJhIr9fD+Pg4NjY2ttc1Gg2srq6i2dw1dipVhBDnpZS3Da5ni58QQmKwsrKCarW6Y12lUsHKyko2BilA4SeEkBhMTExgc3Nzx7qtrS2YnGaGwk8IITFoNpuYnZ1Fo9HA2NgYGo0GZmdnM3fz+JGLXD2EEGIyrVYLk5OTWFlZwcTERKDo93o95e8mAVv8hBCigWaziaNHjwYKuQkRQIzqIYSQlEg7AohRPYQQkjGmRABR+AkhJCVMiQCi8BNCSEqYEgHEqB5CCEkYZxRP2AigJGCLnxCSa5JOjhb3+G5RPM4IoCySu1H4CSG5JenQyLjH7/V6aLfb2NjYwPr6OjY2NtBut7dFPqvQToZzEkJySdKhkTqOv7i4iKmpKayvr2+vGxsbw8LCAiYmJhIP7Uw9nFMIcbMQ4lEhxLIQ4gdCiPus9Q8IIV4UQly0lmNJ2UAIKS5Jh0bqOL5fFE+WoZ1JunquAfhNKeUtAG4H8FEhxC3WttNSylut5ZsJ2kAIKShJh0bqOL5fFE+WoZ2JCb+U8iUp5QXr8yUATwO4KanzEULMJKnOy6RDI3Udv9VqYXV1FQsLC1hdXUWr1UrFfj9S8fELISYAPAbg5wH8BoB/A+BVAOfQfyt42WWfewHcCwAHDx48srq6mridhBC9zM3Nod1uo1qtYnNzE7Ozs9vCp4ukE57l+fhePv7EhV8IMQLgLwH8tpTyq0KIGwH8GIAEMAPggJTyw37HYOcuIfnD5Jmp0iLrLJyZ5OoRQlQA/BmAP5ZSfhUApJQ/klK+LqV8A8AfAnhXkjYQQrLBlLw0WWFCFk4vkozqEQBmATwtpfysY/0Bx9c+AOCppGwghGSHKXlpVNHZFxEUv581Sbb4/ymAfw3glwZCNz8thHhSCPEEgF8EMJ2gDYSQjDAlL40Kulvnpr/tcAAXISRRsvZzB5FEX4Qp/RvMx08I0Yqqa0R1ZqqsSKJ1ruttJ6lQWAo/IQRAOJExueMyLEn1RXjF76uS6DWWUhq/HDlyRBJCkuPs2bOy0WjIffv2yUajIc+ePev53bW1NdloNCT6IdkSgGw0GnJtbS1Fi/Vil39sbCyw/Gmg6xoDOCddNJUtfkJKTtgIFNM7LqMQt3Wum6SvMSdiIaTk2CLj7Ii0RcbNJ523ME1Vms2mMf0QSV9jtvgJKTlhRSbrMM0sJi5Jm6SvMcM5CSHbOXUqlQq2traUcupkEaaZRu4fk4h7jTPL1aMDCj8hyVPGePui4yX89PETQgCY5eN2I2xfBPGGPn5CiC9hfepJ+eCL2qmcBRR+QognYQcRJTnoKOtO5SJBHz8hJcbPrx/Wp56WD970vgiTYK4eQsgOglrnYQcRuX1/aGhI+8Au03P/5AEKPyElRGW0blifutv3L1++jAsXLijblETfQBni/sNC4SekhKi05sP61JvNJk6fPr1r/fT0dKDoJtU3UKRkcjqhj5+QEhLGHx/Gp764uIj3vve9uHTp0va6sbExLCws4OjRo7FtCQPj/unjJ4Q4CNOaV/Gp2+6UkZERXLt2bce2oJDLqAnJglw4RUwmpwsKPyElRVdGSqc75ciRI2i326FCLqPE56u4cBj374NbrmbTFubjJ8RMvPLGLy8vy263q5Q/fm1tTc7MzMh6va6UDz9MrnrT8uynDTzy8TNlAyEkMl5pFC5fvuzp03fiTLomhMDHP/5xnDhxwvcNIUzqhlarhcnJydzE/ac1RoGuHkIKQFYhi3HcKW4hpZ/61Ke0nzMvcf9pRiBR+AnJOVmGLMZJoxC187WIqRvCzoIWF4ZzEpJjTAlZjOKiiGt7kVI3LC4uYmpqCuvr69vrgsJgVWA4JyEFxJSQxSjulLgtdxNcOLpcbGlHIFH4CckxboKxubmJl19+ORcpCsKGlJqUfkGni81ZCY6MjKBWq+H06dPJVWpuoT6mLQznJMQbZ8hipVKR1WpV7tu3r3Dhi3Y5TShbmJDSMHQ6HVmr1eTo6KiWMsIjnJM+fkIKQK/Xw9LSEu6+++7M/f1JYEpfhk0SPvkkykgfPyEFptlsYv/+/Ub4+5PAlL4MmyR88mmWkcJPSEEocoqCsGWL0xegsm8SIaWp3j83/49pC338hKhhYoqCtbU15fQNfqiWLU5fQNh9dZVt8Py67h/S9vELIW4G8EUAN6Lf+fEHUsrPCyGuA/BlABMAVgD8qpTyZb9j0cdPiDomxbc7UzJsbm5idnY2cjI4ILhscfzkpvQj6Lx/Wfj4rwH4TSnlLQBuB/BRIcQtAE4B+K6U8h0Avmv9TwjRhAnx7UAyo1GDyqbiJ/dy5ZjSj5DG/UtM+KWUL0kpL1ifLwF4GsBNAO4C8Ij1tUcA3J2UDYSQ7AgrpDpi9IP85H6x90XuI9mFm/9H94K+W+f/AhgD8IpjvXD+P7DPvQDOATh38ODBWH4uQkxHt6/YBKKkT9YRo+/lJ1exx8Q+kjjAw8efhuiPADgP4Lj1/ysD218OOgY7d0mRMWlgkk7W1tbkBz/4wR1Ce/LkSdfv6R4M5VaRdrtduW/fvh3nGRsbk91uN3DfvJKJ8AOoAJgH8BuOdc8AOGB9PgDgmaDjUPhJUUlqBGjWnD17Vtbr9R3l8iqbqiDHpajX2g8v4U/Mxy+EEABmATwtpfysY9M3AHzI+vwhAF9PygZCwpJ2LpisOhSTLKfdqXvlypVd29zKlpZvvYjpnCPjVhvoWAC8G/1a9QkAF63lGIDr0Y/meRbAAoDrgo7FFj9JgyxcLlm0QpMup1sLPqhsafrWi+TKCQJZ+fh1LBR+kjRZugH8OiN1C1SUcoa1w+0c9nmC5tL1O08UO8oi8F5Q+AnxIS0/sxeDIpVUqzxsOaPa4azM6vW6nJmZiSXAYe0oaod5WLyEn9k5CYE5ozaTtiXMsU2ZISusHSbdy6xhdk5CfIjb8aezszTJDt8w5Yxrh64RqGHtMGUErtG4vQaYttDVQ9Iiil9Yt1shjf4GlXIG2ZGWDz3s9Shj2KYXoI+fEP0kJTJZjCB1E3IvO9L2oYe9HkUbgRsVCj8hCZBkp3CaUSl+Qj5oR1Ytakb1hMdL+Nm5S0qHzrS3cdMAm5A+OWwZVKcdNKV8ZYadu4TAPztjFKJ2Cuu2Iw5hO0NVRtqaVD7igttrgGkLXT1EB0m6KMK4FUzrfIxij58P3bTylRmknauHENNIOkxSNXTRtHDDKG8trVYLq6urWFhYwOrq6o5ZtUwrn5O0czGZyt6sDSAkLUyZaMMUO5y0Wi1MTk6G8sk3m03X75lYPkD/NJB5hi1+UhpMyc5oih1udsUZcGW3pgEYV76o00AW9g3Bzf9j2kIfP9GJ7jC/qMdLO9wwyfO5hYOaFE4ZJey2CPl+wDh+QryJKlJZpXLOenTxoD2Dnbn1et0Iwbcp6+hfCj8hHkQVxbzk0k/azm63K0dHR3elYZ6ZmdFyfF2EGc2bdbZWXVD4CXEhjijGFQcdee5VbE1axDqdTqhJV7JE9ZoXvcXPzl1iLGl0rMUJPYwTvRJlgFNUW5OMsun1epiennbdZkoIpxPVDmxTO+C14VYbmLawxV8+0vKdx23ZRUkGFvWccWwNstOtJazSOo4yzWKeMKmDOgqgq4fkhbRfs+NmcgwrDnFcL3Fs9bLTrZJVrXijTrNI0oHCT3JDFh1rabbs4lZsOm31sqVeryvbp3uaRaIPL+HnyF1iHFmM/PQahZrUuWZnZ9Fut1GpVLC1tRXKf6zTVrvfwJmZc8+e3V1/tr/e7bxRRv2SbKHwE+OIK4x5wBSxdKtk33jjjb47wEFQxZtmxZkXTE5LTeEnRmIL49LSEgDg0KFDGVukHxPE0quSBVDoijdpTM8LxIlYiLHo+PGY3OoyCbfr5Hftsr6uWZ/fj6CJbdK03Wsilsw7blUWdu6WDx2RPUXItWIiWV/XTqcja7WaHB0dNfK++gUnpH3twKgekid0jIotwsjLuCSRkC7L6+o2Sti0++p1jZaXl1O/dl7Cz5G7xEjiRvbomgwkj2l5bZu/8IUvaJ/+MIlJVlSvca/Xw3333bdr/dDQkFEjhL1G/V6+fNmcCWrcagO/Bf0c/mNh94uzsMWfT+K2NuMOViqjq8i22S1pmo7Wpe4Wf5hr7JUMrlarGdXitxl8/rN4W0IcVw+AswDGAAwDWAbwAoCPq+yrY6Hw5w9dohml8rD36XQ6mVYcaeM1itZebFdZlhVykL1hUyUDkJ1OJ9L5s0DXtVMlrvBftP7+KwC/A6AC4ImAfR4GsAbgKce6BwC8COCitRxTOT+FP19kKZqDFU6n04kkcnlMy+uXN8e+B3ZlmEWFrGKv6uQoo6Ojslar5Ur0bdIcJR5X+H9gif2fAniPte7xgH3uAHDYRfh/S+WczoXCny+yEk2dFY7bsWq1mlxeXk7AcnWb/ATDq0X8lre8Rdbr9W3RN+UtJk6yujwmTsvCbi/hV+3c/QKAFfRdPY8JIcYBvOq3g5TyMQA/UTw+KRBZTbats+PR2UHXaDQA9FMZHDlyREsnaVhU0jgPdipWKhVUKhUIISCEQK/XM6dz0bL39OnTqNVqGBkZUU59HHdu4LRwdlpHScOdKG61gcoCYK/Cdyawu8W/AuAJ9F1B+1XOxRZ/OuhskaTty5QyGRfT8vKyrNVqmbaSo/jC5+fnYydfS5oiuG28cLoc6/W6rFarmVx3xHT13AhgFsC3rP9vAdBW2G9Q+G8EMIR+ZNBvA3jYZ997AZwDcO7gwYOJX6Cyk8Rk2Vm82uqucLzcVvPz86mVLYrrrNvtugr/zMxM6hWyG3nsPFclqJNd5f7pIq7wfwvAr8Ly66Of4+dJhf12CL/qtsGFLf5kcXtQK5VK7kIZbZJOW1ytVmW9XtdybVT89vPz86Fb6svLy66Cs7y8bISPPI+d56oEdbLnqcW/aP1dcqy7qLDfYIv/gOPzNIAvqZyfwp8sJj2oJjL4FlGpVLRcm6CQV+f2arUqK5WKckvdrcVfr9eNEdaytfjthlTab1pxhf+/AbgewAXr/9sB/GXAPnMAXgKwhX7cfxvAHwF4En0f/zecFYHfQuFPFpNeTaOSdCvWPv78/LyWlmqQ8Hltn5+fVypjHoQ1i36gtHArm0lRParCfxjAXwNYt/7+EMAvqOyrY6HwJ8/gLEppdUbp+DGkOcJWl6AGuTrcto+OjsozZ84onysPwmqC2ykpTChbLOHv74+9AP4RgJ8HUFHdT8dC4U8H54OahmjoEOwsWrY6rk2UFr8t/mHOaYL4kOzwEn7ffPxCiOOeGwFIKb/qt10XzMefDUnmDQ/KWa7K4uIipqamsL6+vr1ubGwMCwsLOHr0qFabnei4NvZ8A87JTpzzDdjb9+7di0uXLu3YN8q1IuXDKx9/0Axc/8JnmwSQivCTbEhyhii3uV795nX1IqvBYjquTdD0i/b2b37zm/jYxz62Q/yjXCtCbHyFX0r5b9MyhKSHCbMX6RLsvM/PG1SBNJtNHDt2DB/5yEd2rE+jciPFRTkfvxDiTiHEvxdC3G8vSRpGksGUoeNeOcujCHar1cLq6ioWFhawurqqZW5Tk/Lw67xWJuF1jU269oXFzfE/uADoAPgigOcB/Ef0QzJnVfbVsbBzVw+6O0J1dBya2Ploah5+E69VVLyusanXPq8gZjjnEwN/RwD8lcq+OhYKvx50jpY07QeqqxLyynFTBLE1BR1TExapEkwSL+FXdfXYPXA/FUL8DIBrAA5EfMkgGaHLr97r9dBut7GxsYH19XVsbGyg3W5n9mquw31lH+P48eM7OpyBbDNYFhGvLKrdblcpe6gp7spc41YbDC4APgngrQCOoz8a9yUAMyr76ljY4teHjhh0tzeH4eFhOT8/n4DF/ujI6R40ctn0Fn9arV9d54nT4s/DiGSTQBRXD4CjAP6u4/9fA/BtAL8L4Dq/fXUuFH696Mi66SaU9Xo9dZdPnFmcbDfVzMyMa66i4eFhI9xYg7gNtEva5ab7PF4NkKCGSVGTuyVVeUcV/gu2wKM/o9b/A/AvAcwA+IrfvjoXCn96qD6A9g8069axjnlb3fLU1+t15bw4Osqg+qMfFGBdCeOC7Euile1Vbr/rkWaLP603qSQr76jC/7jj8+8BeMDx/0W/fXUuFH59+D3MYR/A+fl5OTw8nHnrK4z7yqvFmFWe+jDXPKtkeqa0su1n155C0vR0IiokXZFFFf6nYM20BeB/ArjDuc1vX50LhV8Pfg9zlAcw6Yc2TItL9bt+NqcdKRL2+mWVPtsEv/rgs9vpdBK7V2mWN+lKNarwfwL9bJxfB7AEbOf2+TkAf+23r86Fwh+foIc56gOYVDK3JFtcKjanUQmEveZu99CeFCbpN5WzZ8/Ker0uh4eHlftykuwMTnLi+zTfcIxs8ff3w+0APgBg2LHu7wM4HLSvroXCH5+ghznOA6hbJNNocel0ecWxIWw5s8rzbp9XtcNb5zX0etOp1WqJ3Ju033CSzIQbWfhNWCj88VF5mE2Z/DpLn3IefvSmu6SSGCHu1beRlGsr7T4fo6J6TFko/HpQEZlOpyNrtVrovO868Yvzdv44kvixZFHpmD4KNew10XkNnR26tVot8c5s55tKvV6XMzMzxt4XFSj8BSdM56YJoXJBDFZSJ0+e3OE6GPxfVwWV9TWw789gJZfWeXU8F6oVdxCD7qLPfOYzu8RfdzBBnHtvYgVO4c8pKg+TLn+q2wTdjUYjs8ExThEMCmOMIixeZDVloX1eu6z256TPr/L8hL0mfhW3SkvaS4STDOWM86ZiWu4qGwp/DlF5mHS2UJeXl11FNanoCS8GKzvVMMZarabth2eCHz2NN44wz0/YaxJUcfvdJz8RTureRP0tZf2W6AeFP2eoPkxRWyluP55ut7vrVbper8du8Yf5obpVdioDl9IUyyTwq9yS7GNII++S25ukSiWThZhGedszZYCbGxT+nKH6MMUJCRxsHXc6He0CGndkqn1+L9fB2NiYrNVqu/YL88MzwTdrUovfrvB1uSu83iSD7lNWLrcobzZs8VP4tRDmYQrzA/HreHMTgDghnWF/ECpjDdyiesLkcR/EJN+sbYudNyhtH39SFU6UFr+NCZWyCllVUkFQ+HNIWEFX+YF4ieuZM2d2rR8dHY31uqpjZKqqv1n1Wjn3M7GlllVUT5J5l9J4qzABEyspCr8hOH/Y8/PzgRkgdT9MXmL3ve99T3uoXBw3lJ+Ae7XSg66VSjpmU3yzaZN0JTh4X/MeH58XKPwGMBiuZy/VajXVlo+Xv9y2q16va3td9RLyoLhx3WMN3Pazy2lSiz9LknZXmNgiLjoU/owJikxJW3D8/ONhEmANuk5U8qvH8atHjaAwLR2zqSwvL8szZ86kHsJLkoHCnzFBsejDw8OJxii7sba2Js+cOSNHR0cjuTucAl6tVmWlUgkU87ijOpOItU7ymueplWtSRzfRA4U/Y1Ra/PaoxDR+eM6EbFHePqK+wbhVgPV6PdTgq6guibCd5Sp9MCrny4OQmtjRTeJD4TeAwXA9p4/fFv00fnheoh0mMVvQG4zXW4PKYCzViidKS1plv7Nnz8pqtbptT6VSCS3aeRNSkwchkeikLvwAHgawBsdMXQCuA/AdAM9af/erHKsowi+ld1RPt9uN7HIJi9uPfGRkRJ45cybWoBVVkXO2vOMOvtKNX+ihV3m8RkHnSUjzVlERNbIQ/jsAHB4Q/k8DOGV9PgXgQZVjFUn4vXAbNVupVFJr8UfxdzsF3Pbxq7pfdAy+SoJut7srnt3ZBzOIX2ipSeVSwdRBSCQ6mbh6AEwMCP8zAA5Ynw8AeEblOEUX/rW1tV3unySFX0r3H3kUn7RKVE8UW7IiTIs/SNxNKpcqeeqMJsGYIvyvOD4L5/9+S9GFP2wr0w8/Ifb7P4kWatD5/b6bJao+fhV3jknlIuXDOOG3/n/ZZ997AZwDcO7gwYPJXRkNqMav++3v1uIPK7x+4ZVBE5fo9kkPvj28//3vl9VqNdOZvcKgEtWTR3cOKRemCH/hXD1e7pGwbpOTJ0/uEJC9e/eGEkeVaBk/gdIpYroid/JAHt05pDyYIvyfwc7O3U+rHCdL4Y+SPiBsh6VXOoEwwqgyWcmgG2kw57qOMEZVW0ZGRoyNcAkL3TnEVLyEfw8SQggxB+C/A/gHQogXhBBtAP8ZwJQQ4lkAk9b/xjI3N4fx8XFMTU1hfHwcc3NzO7YvLS1hz56dl7BSqaDb7aJare5av7Ky4nqelZWVXd+vVque33djYmICm5ubyt9/7bXXcPfdd+8o0+Tk5I7ybG1tod1uo9frKR9X1ZatrS1MTEyEOq6pNJtNHD16FM1mM2tTCFHDrTYwbcmixa8SseHll9fR4o/iCvELrzx58mRgP4JOP79Xjnd7iZPnnxCiBjhyNxxBc366iZqbjz/q5NRRfcV+UT1BOdd1d1aura3JmZkZWalUto83NDRE0SckJbyEX/S3mc1tt90mz507l+o5e70exsfHsbGxsb2u0WhgdXUVKysrmJqawvr6+o5tDz74IO65557tV/5er4eVlRVMTEwouQHCfl9nmezzzc3Nod1uo1KpYGtrC7Ozs5icnIxlV6/Xw9LSEgDg0KFDdIkQkhJCiPNSytt2bXCrDUxbsurctd05w8PDO2YL0pHrRhdhQka73e52TiC/NwvnMcPOmctOTkLMAXT1hMcWveHh4V2i59w2WAGkFaqoKsqD3+t0OtpTIOcpEyUhZYHCHxIV0et0Ojv813Dxm+u0Z3AUrIoox/Hbq3b2ciATIWbiJfyJhXPmHbcQS2dIZq/Xw/T0NLa2tnbtqztU0S2sNMg+1XL44RaW6Va2OOcghKQPhd+DINFzEzsAqNVqmJ2d1daB2ev10G63sbGxgfX1dWxsbKDdbmNzcxNXrlzxtE+1HH40m03Mzs6i0WhgbGwMjUbDtWxxzkEIyQC31wDTlqx9/CMjI7JWq+0IQ3Rzb4SZq1YVN3dLo9HYkce+0Wgo+fijhoqqTl7C1AWEmAXo449Gp9ORtVrNNWInDbFTyXujUuGkEXHDqB5CzMJL+BnH74NK3HvSsffAztj6q1evYs+ePTtsGhsbw8LCAo4ePZrI+Qkh+cQrjp8+fh/c/Ph79uzZHowERM/T0uv1sLi4qJQHp9VqYXV1FQsLCzvObUN/OiEkDBR+H9w6Ld2Sm4UlKPmbG3YF8853vlOpw5UQQrygqyeAubk5fPjDH94VQTPo8lFFxX2kepykXUyEkHzj5erZm4UxpuAUTwCuQtpqtXD99dfj+PHjeO2117bXO+PUnfsFCbLtPnIKv32sMALebDYp+ISQSJRW+O0O02q1ip/+9KcQQqDRaGBzcxOzs7NotVrb3z106BDeeOONHftvbW3hwoULeM973oNqtYrNzU20223Mzs5u/z94HIAx74QQA3AL9TFt0R3OGRQi6ZZuYDCs00525nUMr+NIyZh3Qkg6wCOcs5Qtfjd3i5NB18vc3Bymp6e3W/Kf//zncfjwYd9juB3HptVqxU51TAghUSml8AdNDeh0vThTJthMT0/j/PnzsaYXpI+eEJIVpQznbDabOH36NGq1GkZGRlCpVFCtVl3DI90SjUkpcfny5e2wytHRUdfznD59muJOCDGOUgq/03WztbWFhx56CC+88AIWFhawurq6o0N2ZGRklzvnypUrGBkZ2R5Y9dBDD+0S/5GRERw+fDiV8hBCSBhKJ/xO182lS5dw9epVTE9PA4DrCNznn3/e9Tj2+maziWPHjuHatWs7tr/++uuM1CGEGEnphD+J3PGq6YsJIcQESte5GzaO/tChQ9sTj9tUKhUcOnRox/cYqUMIyQula/FHaZ3ff//9qNVqGB4eRr1exyOPPOL6/agJ2wghJE1K1+IH1FvnztG9e/bswalTp3DixAkKOyEk15Q6SZtfXh1dydQIISQrmI9/AK/UyHae/KWlJU4gTggpJKV09ThDOu0Wfbvdxquvvrod33/16lXXxGwM0SSE5J3SCX+v18MnP/nJXYOy9u7di/vuuw9Xr17d3lapVNBoNLajehiiSQgpAqUSfruz1i2x2ubmJoQQO9YNDQ3ha1/7Gvbv388QTUJIYcjExy+EWBFCPCmEuCiESGVqLbdka05OnDixa5atK1eu4Oabb2aIJiGkUGTZufuLUspb3Xqck8BtxK5NvV7HnXfeiUajsWN9o9HA5cuX0zCPEEJSozRRPV6pmKvVKj73uc/tGonr3I8QQopEVsIvAXxbCHFeCHGv2xeEEPcKIc4JIc71er3YJxwcsVupVDA0NIRarYbp6WksLCww3w4hpBRkMoBLCHGTlPJFIcTfAfAdAB+TUj7m9X2dA7h6vR6WlpZw11137fDp24OzAPdJ1wkhJG8YNYBLSvmi9XcNwJ8DeFda5242m9i/fz9qtdqO9c5pEtmZSwgpMqkLvxBiWAgxan8G8MsAnkrThrAZOgkhpEhk0eK/EcD3hBCPA+gC+Asp5X9J0wDmzyeElBkmaaM/nxBSULx8/KUauTtIs9mk4BNCSkdp4vgJIYT0ofATQkjJoPATQkjJoPATQkjJKJXw27Nr6UgBQQgheaU0wu811SIhhJSNUsTxc+J0QkgZMSpXT9q45eLnxOmEkLJSCuFnbh5CCHmTUgg/c/MQQsiblCZlQ6vVwuTkJHPzEEJKT2mEH2BuHkIIAUri6iGEEPImFH5CCCkZFH5CCCkZFH5CCCkZFH5CCCkZhRZ+JmUjhJDdFFb4mZSNEELcKWSSNiZlI4SQkiVpY1I2QgjxppDCz6RshBDiTSGFn0nZCCHEm8Lm6mFSNkIIcaewwg8wKRshhLhRSFcPIYQQbyj8hBBSMij8hBBSMij8hBBSMij8hBBSMnKRskEI0QOwGmHXGwD8WLM5WVKk8hSpLECxylOksgDFKk/YsoxLKXeFNuZC+KMihDjnlqcirxSpPEUqC1Cs8hSpLECxyqOrLHT1EEJIyaDwE0JIySi68P9B1gZopkjlKVJZgGKVp0hlAYpVHi1lKbSPnxBCyG6K3uInhBAyAIWfEEJKRmGFXwjxz4UQzwghnhNCnMranrAIIVaEEE8KIS4KIc5Z664TQnxHCPGs9Xd/1nZ6IYR4WAixJoR4yrHO1X7R53ete/WEEOJwdpbvxqMsDwghXrTuz0UhxDHHtv9gleUZIcQ/y8Zqb4QQNwshHhVCLAshfiCEuM9an7v741OWXN4fIURdCNEVQjxulec/WevfJoT4vmX3l4UQVWt9zfr/OWv7hNKJpJSFWwAMAfhfAN4OoArgcQC3ZG1XyDKsALhhYN2nAZyyPp8C8GDWdvrYfweAwwCeCrIfwDEA3wIgANwO4PtZ269QlgcA/JbLd2+xnrcagLdZz+FQ1mUYsPEAgMPW51EAP7Tszt398SlLLu+PdY1HrM8VAN+3rvmfALjHWt8B8BHr878D0LE+3wPgyyrnKWqL/10AnpNS/m8p5SaALwG4K2ObdHAXgEesz48AuDs7U/yRUj4G4CcDq73svwvAF2Wf/wHgrUKIA6kYqoBHWby4C8CXpJRXpZT/B8Bz6D+PxiClfElKecH6fAnA0wBuQg7vj09ZvDD6/ljX+LL1b8VaJIBfAvAVa/3gvbHv2VcAvFcIIYLOU1ThvwnA847/X4D/w2AiEsC3hRDnhRD3WutulFK+ZH3+GwA3ZmNaZLzsz+v9Omm5Ph52uN1yVRbLNXAI/ZZlru/PQFmAnN4fIcSQEOIigDUA30H/reQVKeU16ytOm7fLY21fB3B90DmKKvxF4N1SysMA3gfgo0KIO5wbZf/dLrexuHm3H8DvA/h7AG4F8BKA38nUmggIIUYA/BmAX5dSvurclrf741KW3N4fKeXrUspbAfws+m8j/1D3OYoq/C8CuNnx/89a63KDlPJF6+8agD9H/wH4kf2Kbf1dy87CSHjZn7v7JaX8kfUDfQPAH+JNd0EuyiKEqKAvlH8spfyqtTqX98etLHm/PwAgpXwFwKMA/gn67jV7qlynzdvlsbbvA/C3QccuqvAvAniH1RNeRb/T4xsZ26SMEGJYCDFqfwbwywCeQr8MH7K+9iEAX8/Gwsh42f8NAL9mRY/cDmDd4XIwkgEf9wfQvz9Avyz3WNEWbwPwDgDdtO3zw/IBzwJ4Wkr5Wcem3N0fr7Lk9f4IIZpCiLdanxsAptDvt3gUwK9YXxu8N/Y9+xUA/9V6W/Mn617spBb0IxF+iL5/7BNZ2xPS9rejH3nwOIAf2Paj77v7LoBnASwAuC5rW33KMIf+K/YW+j7Jtpf96Ecy/J51r54EcFvW9iuU5Y8sW5+wfnwHHN//hFWWZwC8L2v7XcrzbvTdOE8AuGgtx/J4f3zKksv7A+AXACxZdj8F4H5r/dvRr6CeA/CnAGrW+rr1/3PW9rernIcpGwghpGQU1dVDCCHEAwo/IYSUDAo/IYSUDAo/IYSUDAo/IYSUDAo/IQoIIa53ZHr8G0fmRzmY4VEI8etCiN/PylZCgqDwE6KAlPJvpZS3yv5Q+g6A09bnE+gPEHRyD/qx/4QYCYWfkHh8BcCdjvzoEwB+BsBfZWkUIX5Q+AmJgZTyJ+iPmHyfteoeAH8iOTKSGAyFn5D4zOFNdw/dPMR4KPyExOfr6E+AcRjAW6SU57M2iBA/KPyExET2Z0x6FMDDYGuf5AAKPyF6mAPwj0HhJzmA2TkJIaRksMVPCCElg8JPCCElg8JPCCElg8JPCCElg8JPCCElg8JPCCElg8JPCCEl4/8DhE5wlJrCYZwAAAAASUVORK5CYII=\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"adv.plot(x='TV', y='Sales', kind='scatter', c='black')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can use this dataset to solve a simple problem with linear regression: given a TV marketing budget, predict sales."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a name='2'></a>\n",
"## 2 - Linear Regression in Python with `NumPy` and `Scikit-Learn`"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Save the required field of the DataFrame into variables `X` and `Y`:"
]
},
{
"cell_type": "code",
"execution_count": 40,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": [
"X = adv['TV']\n",
"Y = adv['Sales']"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a name='2.1'></a>\n",
"### 2.1 - Linear Regression with `NumPy`"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can use the function `np.polyfit(x, y, deg)` to fit a polynomial of degree `deg` to points $(x, y)$, minimising the sum of squared errors. You can read more in the [documentation](https://numpy.org/doc/stable/reference/generated/numpy.polyfit.html). Taking `deg = 1` you can obtain the slope `m` and the intercept `b` of the linear regression line:"
]
},
{
"cell_type": "code",
"execution_count": 41,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Linear regression with NumPy. Slope: 0.04753664043301978. Intercept: 7.032593549127698\n"
]
}
],
"source": [
"m_numpy, b_numpy = np.polyfit(X, Y, 1)\n",
"\n",
"print(f\"Linear regression with NumPy. Slope: {m_numpy}. Intercept: {b_numpy}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"*Note*: [`NumPy` documentation](https://numpy.org/doc/stable/reference/generated/numpy.polyfit.html) suggests the [`Polynomial.fit` class method](https://numpy.org/doc/stable/reference/generated/numpy.polynomial.polynomial.Polynomial.fit.html#numpy.polynomial.polynomial.Polynomial.fit) as recommended for new code as it is more stable numerically. But in this simple example, you can stick to the `np.polyfit` function for simplicity."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can plot the linear regression line by running the following code. The regression line is red."
]
},
{
"cell_type": "code",
"execution_count": 42,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAe4AAAE9CAYAAADNvYHXAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/av/WaAAAACXBIWXMAAAsTAAALEwEAmpwYAAAxDklEQVR4nO3df5xcZX0v8M+zm11kQpRkgrn82pnQq/RSbAXSGmuJLekPsC/RFryFDjVthZXFq6Re0qLzUuGlS61pC7QC3vUVLLBjULF9ga2xDa1QS9vQJELEcPmlM3uJwZAFAdnFJDvf+8c5s8yPc86cmXnOOc9zzuf9es0rm/n5zDMz53OeH+c5SkRAREREdhhKugBEREQUHoObiIjIIgxuIiIiizC4iYiILMLgJiIisgiDm4iIyCJLki5AGCtXrpRisZh0MYiIiGKxa9eugyJynNdtVgR3sVjEzp07ky4GERFRLJRSNb/b2FVORERkEQY3ERGRRRjcREREFmFwExERWYTBTUREZBEGNxERkUUY3ERERBZhcBMRUapUKhUUi0UMDQ2hWCyiUqkkXSStrFiAhYiIKIxKpYLx8XHMzc0BAGq1GsbHxwEApVIpyaJpwxY3ERGlRrlcXgzthrm5OZTL5YRKpB+Dm4iIUmNmZqan623E4CYiotQYGxvr6XobMbiJiCg1JicnkcvlWq7L5XKYnJxMqET6MbiJiCg1SqUSpqamUCgUoJRCoVDA1NRUaiamAYASkaTL0NWaNWuEp/UkIqKsUErtEpE1XrexxU1EROSy4RhwHsdNREQEe44BZ4ubiIgI9hwDzuAmIiKCPceAM7iJiIhgzzHgDG4iIiLYcww4g5uIiAj2HAPO4CYioszxO+yrVCqhWq2iXq+jWq0aF9oAg5uIiHyYcExzFGVoHPZVq9UgIouHfbU/twnv35OIGH8566yzhIiI4jM9PS25XE4ALF5yuZxMT09bX4ZCodDynI1LoVCI/LXDArBTfDKRS54SEVGHYrGIWq3WcX2hUEC1WrW6DENDQ/DKPqUU6vV6pK8dViJLniqlTlZKfVMptVcp9V2l1JXu9dcopfYppR5yL++IqgxERNQfE45pjqoMYQ77MuH9+4lyjPsIgP8tIqcBWAvgA0qp09zbrheRN7uXr0dYBiIi6oMJxzRHVYYwh32Z8P79RBbcIrJfRHa7f78E4FEAJ0b1ekREpI8JxzRHVYYwh32Z8P59+Q1+67wAKAKYAfBaANcAqALYA+BWAMu7PZ6T04iI4jc9PS2FQkGUUlIoFGKdmGZCGZJ8bSQ5OU0pdQyA+wFMisjfKqVWATgIZ5beJwEcLyJ/6PG4cQDjADA2NnaW1yQBIqK0q1QqKJfLmJmZwdjYGCYnJ408tpj0CpqcFmlwK6VGAPw9gH8Ukb/0uL0I4O9F5PSg5+GsciLKovbTTAJOd62Jq3mRXknNKlcAtgB4tDm0lVLHN93ttwA8ElUZiIhsZstpJileUc4qfxuA3wNwTtuhX59RSn1HKbUHwK8A+KMIy0BEZC2TD0mykbErofVoSVRPLCL/BkB53MTDv4iIQhgbG/NcBMSEQ5Js0z7s0FjmFIB1ww5cq5yIyFBGH5KkSVyt4DQNOzC4iYgMZctpJvsV9mQfOqRp2IFrlRMRUSLiXA886bXHe5XIrHIiImqVlslRusTZCo5q2CGRz9RvZRaTLlw5jYiSNugqWkmfJtJEYU6vqZPuldCi/EzB03oSEfVPx0IotnXVxsH2BWai/EwTWzlNFwY3ESVJxwY6zDmgs8jmJV2j/EwZ3EREA9CxgWaLO32SanFzchoRURc6zs2chWOysyapz5TBTUTUhY4NtA3HZHPWe2+S+kzZVU5EFILNY7Fh2D5RLG04xk1ERIE4Bm8WjnETEVGgNC0JmnYMbiIi0jIBj+LB4CYiIs56twiDm4hIk0FmZSc9o9uGWe/k4OQ0IiINBpmVzRnd1I6T04iIetBP67dcLrcELwDMzc2hXC5H+ljKniVJF4CIyCTtrd9arYbx8XEACGz9DjIr2+8+XodnEbHFTUTUpN/W7yCzsv3uo5SyZvWypMfos4TBTUTUpN+W8yCzsicnJ6GU6rheRPrqLo87RBu9FLVaDSKy2EvB8I6I34m6TbqcddZZA5+UnIgojEKhIAA6LoVCoetjp6enpVAoiFJKCoWCTE9Ph35dr9cEIEqpnso/PT0tuVyu5TlyuVxPZenVIHVG3gDsFJ9M5KxyIqImSc3w1rXkaBJLl/Jc4/pxVjkRUUhJHc+sawEUHUuX9trVzlXX4sXgJiJqUyqVUK1WUa/XUa1WIwnt9nAEoGWHYdAQ7We8mquuxcyvD92kC8e4iShNohyHHvS5+x2vHmR8nzqBY9xEROaIYhy6+XzhK1asAAA899xzPZ87nOPVZuAYNxGRQXSfQrO9e3t2dhbz8/O44447eu7q53i1P1OOVWdwExHFTHc46lwylePV3kw6Vp3BTUSpYkqrKIjucNTZgudZwryZtJ48x7iJKDVsOstW85h0r+PQ7ZI4djtr4h77DxrjZnATUWpkNcBs2mGxVeB361/+BTjuOGDZMm2vx8lpRJQJuid92YLd29FrH974aQC7lUK1VgN+6qeAd787trIwuIkoNbI8IzqORWNMFce8hlKphK3XXovdo6MQAI8COKO5x/rP/1z7a/phcBNRanhN+lJKoVarGTtRLWk2TOYLEvls7+99D1i3DlAK52/ahDMOHVq86cP5PCp33AGIAGecoef1wvBbmcWkC1dOI6KwGit4wT2zFmI8S5ZtkjiTmG6RnJmsVhNZv17EieSWy+Wjo7HUF7hyGhFlTVYnqvUiDXWkbbb3vn3ApZcC3/hG52233AK8//0orl4dW31xchoRZU5WJ6r1Ig11NNC8hv37gfPPB5QCTjqpNbRvvBFYWHDa2ZdfDihlTH0xuIkolbI8US2sNNRRz4vZHDgAXHihE9YnnAB87Wuv3rZ5M3DkiBPWH/oQMNQakabUF4ObiFIpLUt3Rjl5bJA60l2ufp8v1KFws7PA7/6uE9arVgFf/eqrt113HXD4sBPWV10FDA/7vpYx3ym/we9BLwBOBvBNAHsBfBfAle71KwBsB/CE++/ybs/FyWlE1A/bTzUZx+SxfupId7kieZ/PPy+yYYPnBDO59lqRQ4f6Lmsc3ykkMTlNKXU8gONFZLdSahmAXQDeDeD3ATwnIp9WSl3tBvefBD0XJ6cRURaZOnlMd7m0Pd+LLwIf/jCwZUvnbeUy8LGPAUcd1XP5kpDI5DQR2S8iu92/X4JzvPqJAN4F4Db3brfBCXMiImpjymSosK/vdX2YLvCB3uePfwxccYXTDf6617WG9qZNwPy8087+1KesCe1uYhnjVkoVAZwBYAeAVSKy373pGQCrfB4zrpTaqZTa+eyzz8ZRTCIio/QzGSqOBVXClivs4ig9v8+5OWDjRiesly1zDtdq2LgRePllJ6w/8xngNa8J+7bs4deHrusC4Bg43eS/7f7/R223P9/tOTjGTZQ+to8/x2FiYqKnRWTiWlAl7OuEXRwl1PPNz4ts2uQ9Zn3FFSIvvaT1PSYNAWPcUYf2CIB/BPDhpusegzP2DQDHA3is2/MwuInSJQ0rdkXNq46UUjIxMeH7mEhWEQsoX7cdr/adjub3Eer5XnlF5KMf9Q7ryy4TeeEF7e/LFEHBHeXkNAVnDPs5EdnYdP1mALPy6uS0FSLyx0HPxclpROli6qQrEzTO0+1VP0BwHcV9zuhu+vqcDx8GJieBa6/tvG3DBuD664Hly/UW1EBJrZz2NgC/B+AcpdRD7uUdAD4N4NeUUk8A+FX3/0SUIaZOukpa85iwn6A6MmWBkIbQxz0fOeIcT60UMDraGtoXXwwcPOi0s//mbzIR2l35NcVNurCrnKg/po4jx9ml24uk68uvXsLWkYlDEL51euSIyObN3t3gF14ocuBAYmU2AZIa49Z1YXAT9c7EjbjJZTOhTH5jwr2UJ+mdj0ALCyI33OAd1uefL7J/f9IlNAaDmyiDTG3VNvgFTFLBM0h96SpzUIvbuBAOq14Xuekm77A+7zyRp59OuoRGYnATZVAvM3pNkWSrt9/60lnmuN//oDscvo+v10U+/3nvsF6/XqRajeDdpAuDmyiDTG9xe0myzP2+tu4yx9XjMOhOgtfjLx0d9Q7rdetEnnoqkveRVgxuogwyYcy2V0n2EvRbXzb2bIgMvsPRePxFXkENiKxdK/L449G+iRQLCm6e1pMopUKd7tAwSR7O1G99mXYIVlgDHZL31a+iWqtBAGxtunongNMAJ7r/4z+AN7xh8IJSJ79EN+nCFjdRsmzpvk2CjWUW6aPFfffdIiMjHS3rhwE53ZKhGJuAXeVE1C/bJkwlIUyZTXtfoT7Xr39dJJfr7AY/9VT5h09+0sodFlswuImobzZOcotKv+Fraqvc8/1s3y7yutd1hvXq1SI7dnR/PGkRFNyRrVWuE9cqJ0qOaetfJ6WxHOnc3NzidblcLtQ4uPFrs99/P/Ce9wDtp1A++WTgzjuBX/zFZMqVYUmtVU5EKWDr5CvdyuVyS2gDwNzcHMrlctfHGrk2+7//O3DSSc764L/8y6+G9qpVwH33Oe3smRmGtoEY3EQWqlQqKBaLGBoaQrFYRKVSiey1Qp8ooos4yxyFQcK3152fyOrqwQeBU05xwvptbwP27XOuX74c2L7dCetnngHe/nY9r0fR8OtDN+nCMW6iVyUxXqpjhS0Tx3h7MeiSqGHfv/a6qlZF3vjGzjHrpUtFtm3r7zkpcuDkNKL0sHGymI1lbqdjpbEwOz9a6mrPHpHTT+8M69FRkXvuCf88lJig4ObkNCLL2DhZzMYye6lUKiiXy5iZmcHY2BgmJye1L2jTd109+ihwySXA7t2t13/iE8D69cDZZ2stJ0WLk9OIUsTGyWI2ltlLqVRCtVpFvV5HtVqNZBW6nurqiSeAtWudMevTTmsN7a1bgXoduOaaRELb9jkNJmNwE1lG12SxONlY5qR0ravvfQ9Yt84J6ze+Edix49U73n67E9YiwEUXOfdJQOPQuVqtBhFBrVbD+Pg4w1sXvz50ky4c4yZqZePCFzaWOQ5e9dJ+3d/ecINzOkyvk3ls2eKcRtMgOsbps/59ASenEVE/srDxTPI9Bk54e/ppkXPP9Q7rW24xLqybDXrGtDQchTAoBjdRRugMoSxsPJN+j+0t01WA3O0V1IDIjTeKLCzEUq5B6Tpl6CAtdtsxuIkyQHcImbrx1LlzkvR7VErJcYB8xS+sN28WOXIklrLoNOh30dZznOvE4CbKAN0hZOLGU/fOSWLv8eBB+f5b3+oZ1lcDsgSwvmdjkB2spHeoTMDgJsoA3SGkc+Opq5Wse4Mea0A8/7zIhg2eYf0xN6yzGlLtkh7CMAGDmygDdIeQro2nzo2w7p2TyAPihRdELr3UM6w/Cciox3sxoWfDBFmYGBmEwU0UIVM2MFGEkI73pnOHIooWsvbP76WXRCYmPMNaNm0SmZ/33QFhi5saGNxEETGtS8+UnYhmOlvJpp1gpXFbDpAty5Z5h/XGjSIvv9zynH47ICZ8h8gMDG6iiHASTXdRdOHHtXMStKPwxVtvlb9cssQ7rK+4wml59/C8jR0cU3a4KFkMbqKImDjz2jSm9Ur0on2nYxSQT3kFNSD/B5BlPeyQmNg7QuYICm6eHYxoAMViEbVareP6QqGAarUaf4EMFcdZtaIwNDSEYRGUAVzjcfvfANgI4IWm62w74xmZiWcHI4oIT54RThxn1dLqyBHguutQF8FhtIZ2BcCbTzoJxUIBf4DW0AbsO+NZlqTljGVLki4Akc0aAXTllVdidnYWAHD00UcnWSTq18ICcP31wKZNHTd9BcAVAA7C2TGb+vSnAQDj4+OYm5tbvB932szVOGNZ4/NqnLEMgPk7km3Y4ibSYH5+fvHv2dlZLacwTEvrwGj1OnDjjc7pL5csaQ3t888H9u9HZXoamwoFzCqFQqGAqakplEollEolTE1NoVAoQLXd1s6Wz9KWcvajXC637GQBwNzcHMrlMgDL3rvf4LdJF05OI5NFdWyxrRO6jFevi9x8s/ds8PPOc87KpZEtn6Ut5exX0ERSE987ODmNKDpDQ0Pw+h0NMkmJk940EwG2bAEuu6zztvXrgVtvBSIam7bls1y5cuXicE8z08rZr6DPAYBxnxEnpxFFyG8y0iCTlGZmZnq6njyIALff7nSDDw21hPb9AM4+4QRUpqeBe++NLLQBOz7LSqXiGdqAWeUcRNBEUhs+o2YMbqIBRTGzPIqdAcCycbx+iAB33vlqWG/YsHjTfy1Zgp8ZGYEC8MsA/u0HP9AyF6GbqD5LL/1+vo1xXi9pmSUfNCchzs9IC78+dJMuHOMm0+leTCOqdcdNG8fT5q67PMesdw0NyU8nvCZ4XPU+yOsErZ2eiu9HFyb+NsCV04jsW6lKd3lTtzzr3XeLjIx0Bvab3iSyZ0/X9cAblzhWuYvjuzfI5+v32Hw+r72cpjJt+6A1uOF0r7+218cNcmFw06BM3KNuF/WGIxXLs27bJrJ0aWdYn3qqyO7dLXcNcwau5mAzbcPdq0E+Xxt+H1kzcHAD+CKA1wJYCmAvgKcBbOrymFsBHADwSNN11wDYB+Ah9/KOMK/P4KZBmd7ajGPDaXod+Lr3XpFjj+0M61NOEdmxw/dhYVrcjTpOQ3AN+vnavuOSNjqC+yH33xKAvwAwAmBPl8esA3CmR3BfFeY1my8MbhqU6a3NOELVK5wa3aHGbaTvu0/k9a/vDOuTTxZ54IFQT+H1fkdGRiSfz3eEk7U7NU3SsPMRN5N3VnQE93fdsP4KgLe71z0c4nFFBjeZwPQNc1w7FtPT05LP531bnol64AGRE0/sDOtVq0Tuu6+vjazXY7yuM33HLiyTg8g0pu/o6AjuD7ld3F8HoAAUAHwrxOO8grsKYA+crvTlYV6fwU2DMv1HGueOhVE7MQ8+KLJ6dWdYL18usn374t10fX5+z+O1M2PSjl03DOzu2uvI9M984OD2fCCwJMR92oN7FYBhOBPcJgHcGvDYcQA7AewcGxuLtoYoUXFtdEzeuMW5Y5F463L3bmcyWXtYL13qTD7zoGtnw29jnc/njd6xC2L6TqkJ/IaJTO5l0dHiXgVgC4Bt7v9PA/C+EI9rCe6wt7Vf2OJOL7+NzsTEhLEhG5W4diyCQjCyMuzZI3L66Z1hPToqcs89XR+uY2djeno6cGNt8o5dEKN6UAwVZqKiafWmI7i3AfifcMe14ZwO9DshHtfe4j6+6e8/AnBnmNdncKeX3w+qfUPNFoQ+QTtLWltue/eKnHFGR1gvAHJB045CGDrCKWjjbcrGuh+J96BYIOyhgSZtZ3QE93+5/3676bqHujxmK4D9AA7DOXzsfQDuAPAdOGPc9zQHedCFwZ1eYX9Qtm9cTePVutTScnv8cZG3vKWzZQ3Itz7wAckdfXToDWVzGfP5vIyMjAy0kU3r6mBscXfnV0f5fN7YXhYdwX0fgDyA3e7/1wK4P8xjdVwY3OnVSxcWWxDR6rvl9tRTImef7RnWcvvtzmk0pbeA8eoVGB0d9TyUK6ygjbfNOMbdnY11pCO4zwTwAIAX3H8fB/CzYR6r48LgTi+vH5RfgKSxBWHSuGpPLbdaTeScc7zDesuWxbBu1suOQRStSBs33mGZ9D0ylW11NHBwO8+BJQB+BsDpAEbCPk7HhcGdbu0/KO1jrYYyLUi6lufpp0XOPdc7rG+5xTOsm/USxkHd2oNseG3beFN29R3cAH476BL0WJ0XBnf2mLCBjboMJo5Ntr/nuz77WZF3vtM7rG+8UWRhoafnDrujEmYIJY07c0QNgwT3FwIuvsdg674wuClucbSGjZ0N/MMfilxwgXdYb94scuRI308ddmco7HG3aRw+IRIJDm7l3G62NWvWyM6dO5MuBmVIsVhErVbruL5QKKBarVrzGqHNzgIf/CCwdWvnbdddB2zaBCxZEmuRKpUKyuUyZmZm4LedUkqhXq/HWi6iOCildonIGq/bhnp4kt9USv2xUurjjYu+IhKZZWZmpqfr+zE5OYlcLtdyXS6Xw+TkpLbXCPSjHwEbNgBKAStXtob2tdcChw457eyPfCT20AaAUqmEarWKer2OQqHgeZ+xsbGYS0WUvFDBrZT6HIDfAfBBOGuVvwfOeuVEgSqVCorFIoaGhlAsFlGpVJIuUih+gaAzKEqlEqamplAoFKCUQqFQwNTUFEqlkrbX6PDii8CllzphvXw5cPvtr95WLgOvvOKE9cc/DoyMRFeOHiW+k0NkEr8+9OYL3FN4Nv17DEKcZETXhWPcdjJt1nQvbC57h5deEpmY8B6z3rRJZH4+6RKGYsKERaK4QMNx3Dvcf/8TwAkAXgPgyTCP1XFhcNvJxFnTvbA6KF5+WeTKKz3Deu+55zq3k/XCfEet/h5nmI7g/hiAY+EcBrbfvXwyzGN1XBjcdopj1jQ3Sk3m550WtEdY/zUgS23vOaAWYXqFUtVzlDF9BzeAnwfw35r+/14A/wTgrwCsCHqszguD205Rt7ht3yhp2el45RWRj37UM6ynADl2aMjqXg/yF+b31etvkDvC5hgkuHc3AhrAOgA/AHABgE8CuCvosTovDG47RR2sNnfFD1Q3hw6JfOITnmH95Nlny/FtJ/KIuteDkhGmR6uXXi/bd4TTZpDgfrjp75sAXNP0/4eCHqvzwuC2V5R78MYuYBJCzzsdhw+LTE56hrVcfLHIwYOBz2vjzg0F093itnlHOI0GCe5HACxx//6/ANY13xb0WJ0XBjd5CQop07v5Qu10HDnirFTmFdYXXihfufnmjp2iMKdJtb0V5bUzaFoXbxzl0T3GbfOOcJSS+m4NEtxlOGcDuxvAt4HFldb+O4AHgh6r88LgJi/dlsU0OaD8djqKY2MiN9zgHdbnny+yf7+I+G+Q8/m85/MODw8bE2qD8Dvd56Dn6o66jFGVR+escpNa3KbsiCU5fNB3cDuPxVoAvwVgadN1bwRwZrfH6rowuNNtkB9p47G2dQm3bxAmvIIaEDnvPOesXG383nM+n7dinLLfzzzsUECSn71JAdgLU8a4TSmHSLKf5UDBbcKFwW2nsK0BHT9S67r56nX5j/e9zzus1693zncdIOj9mtJa8TPIZx5mKCDpz96m76LXKXWT/u6YtOOT5GfJ4KbYhd046/qRJvljDx2U9brIbbd5h/W6dSJPPRX6NU3auPVqkLKzxa2PSS3bZibt+LDFzeDOlLBfeF0/0qQ2Ql1ft14X2brVO6zXrhV5/PFoXtdgg3zmHOPWx9TJnSbt+Fg7xm3ChcFtn7AbZ50/0iS6iP3K//6VK73D+qyzRPbu1fLavZzbOunuz2aDfuY2zCqfmJiQ4eFhAZyJgRMTEz09Po73023YIamdDdN2fKybVW7KhcFtn7AbZ9N+pL1q3vi9E5CfeIX1m94ksmdPIuUzsX5NLJNOg76/uOonzLBDUt37pu2IJYHBTbHrZePTPDO80Uqx5cf63te/Xl7yCOsnlywR2bUr6eIZ1e3YLM0b5kHrPK7PrNvhlP0MWQ1anrR+J/rB4KZE9PJDtKoVdu+9Isce2xnWgPw8nDHXfD7f8b6T2DCZNNEnKwat8yg/M79Z5Em3uK36/ceEwU0DiSNwTG0ZLrrvPpHjjusIazn5ZPnGxz++WD/5fN5zotTExEQiGybj6zWFTG1xB4Vj0sGp+z2nofXO4Ka+xfWDDuquS8wDD4iceGJnWK9a5QS5B78NUGMIIO4ATXqDHEbzRjafz3v2VphSvjBlinKMe5BA6haOSYadzl4GG77zYTC4qUUvP9C4Wmx+wTY8PKz1dbrasUNk9erOsF6+XGT79q4P72WBkOa6jHpNa1NbH93GWZPe4PYbAoPWud/Mea+6yufzAy1eY8Kwic7tTFp6mRjctKjXDVFcP/ZEW9y7d4ucempnWC9dKrJtm+dD/DbMvba42+s36aCKW9D4qgkbXJNCIKiuwnxvTHov7XS2kk3eQekFg5sW9frjjevHHvdG5e//9E/l0ZGRzrAeHRW5557Ax/YzVug1xu23gTFhQxqXMD0USW5wg8oXd+/FoDs4pnch6+oZMnkHpRcMblrU696ojh97mB/kxMRE9K3PvXtFzjijI6wXALl4dDT0a/U7Vth+vYlBFTdbW9xJBJ9fr00v3xuTh010MX0HJSwGNy3qZ290kB97mB+R132UUj2vNuXpscdE3vKWzpY1IL/TZ0Do6orT3TKwcaNs4xh3UjsWJu/gmMbG30I7BjctintvNEw4ae/aeuopkbPP9gxruf12UQO2dHWVV+dnYXMrw5ZZ5Un3kAw6xk12YXBTizj3RsO0TrW0YGs1kXPO8Q7rLVuck324Bg3eXkMyqL77OdTI6/5pGdczWdJ1POiscrILgzuFbOkKCtrYdWvJdN0gPv20yG/8hndY33JLS1g3i2vcXtdrhXmutMykNZkJvRq2/O5pcAzulPGayGXqnncvs6xDbRB/8AORd77TO6xvvFFkYSF0ueLYAOpspQU9V9KtwaxgcFJcGNwpEtS6MnWsy2tj162l3fIefvhDkQsu8A7rzZtFjhyJtKyD0NkSDnouE1qDRKQPgztFggLPplZW10A7eFDk4ou9w/q660QOH255Ph0TnKIIv7ha3I3yszUYLdYxxYXBnSKmL1gRpHmj53VM6rGA3LV0qXdYX3utyKFDvs+r45CiKLqbszBzPCthZmr9UzoxuFOklxa3SRtUv3BdBsjnvYIaECmXRV55petz6+qF6NYV3W9d6vwcTPpMG+XJSphxHgHFicGdImFbl6ZtUJs3eksBuckvrDdtEpmf7+m5dfVC+G2Y8/m8UXXZj+npacnn8y3vSUf5sxRmnLlPcWJwp0yY8VzTNqg5QK73C+uNG0Vefrnv59bV4vbb2WkOPBPqslfT09MyOjraUf6RkRGjJt+ZzrTfFKVbIsEN4FYABwA80nTdCgDbATzh/rs8zHMxuHszPT0dGGKxmZ8Xueoqz7D+a7flrWOjp3PZTK+uaNvDqdsM/iDduuazFGam9WJRuiUV3OsAnNkW3J8BcLX799UA/izMczG4w+sWYo3x2si88orIRz/qGdZbhodlWUQbvSiXzbQ9nIKGEoJ2PvpdZz7NYWbaHANKr8S6ygEU24L7MQDHu38fD+CxMM/D4A4vkcPFDh0S+cQnPMNaNmwQef55EdG79GecbA+nflvcYXdYTP7siGxlUnD/qOlv1fz/oAuDO7zYDhc7fFhkctI7rC++2DkOOyQbgtHmcOp3jNv2IQIimxkZ3O7/nw947DiAnQB2jo2NRVY5aRNpi/vIEdl10UWeYf1lQM446aTFIOgl6OLsim4v18TEhLWB3It+ZpXbPkRAZDOTgptd5RGbnp6WkZER39DuuSW7sCByww2eYf13gKzyeH6vdciDXjeull238f8wrdAssaEnhCitTAruzWidnPaZMM+TtuAO0xrtt2vWr1u00VIK9TwLCyI33eQZ1v8AyAldWvReq6IFtdTiatmF6Y1otEbJYfMQAZHNEgluAFsB7AdwGMDTAN4HIA/gn+EcDnYvgBVhnitNwR31TN2+Q7BeF/n85z3DWtavF6lWQ42fB138WtBe71cpJRMTE71UbVe9lJ+IKEmJtbh1XWwI7rAtkzDBOkgLtKdu53pd5LbbvMN63TqRp54KVfawLe6gVr/XqUp1d8uGLT+Dm4iSxuCOWNgWctDCKM3BOsiYb9fQr9dFvvhF77Beu1bk8cd7ep/tl37PtR1Hd3mY8gPsKiei5DG4IxYmdLqFhq4Wt99OxP0f+pB3WJ91lsjevaHfa9hZ2WHOud0szglqQT0Co6OjHMclosQxuCMWJnSCQkz3alSN0DwfkJ94hfWb3iSyZ4/2evASNpDjPiTMaydK14k3iIgGxeCOWJjQ6TYxqn3st+/ZvF//ukgu1xnWp54qsmuX5nfeXS+rb3ntrER1nDVnSxORyRjcEQszsSrMxKi+J2Pde6/Iscd2hvXq1SI7dvT9vnSEWy+9B17d8P32PDCYichmDO4IhT2UKezEqNBdw/fdJ3LccZ1hffLJIg88EMn76jc0+20199t9zoVDiMh2DO4I9RIu3SZsdZ2M9cADIiee2BnWq1Y5QZ7Q+2p/j7pCs98Ja3GOlxMRRSEouIdAA5mZmQl9falUwuTkJHK5nO/zjY2NtV7x4IPAKacASgFvexuwb59z/fLlwPbtTnQ/8wzw9rf3XPZKpYJisYihoSEUi0VUKpW+3lezcrmMubm5luvm5uZQLpd7Ll9HXXS5vqHfshMR2YDBPaBew8Ur2BpyuRwmJyeBb38bOPVUJ6zf8hbg+9937rB0KbBtmxPWzz0H/Oqv9l3uSqWC8fFx1Go1iAhqtRrGx8cXw9uE0PTayVmsowD9lp2IyAp+TXGTLiZ3lffaNezX/Xs6IM+fdFJnN/joqMg992gvd7fu5H67vHV3U/czyYxj3ERkO3CMO1r9nsLyfwCyqz2oARGlRO66K9IyBx2e1ngv+Xxe8vm8laHJWeVEZDMGd0zChMXdmzfLjqGhzrAGRLZudZYkjYFfy1jHeuEMTSKiwQQFt3JuN9uaNWtk586dSRcjUGPMuHn8OpfLYWpqCqW3vhX4/d8HvvWtjsd9OJ/HWTfcgNIll8RYWu/yKqXg9X0oFAqoVqsxlo6IKNuUUrtEZI3nbQxuPYrFImq12uL/xwDcCmC91523bAH+4A+cyWcJqlQqKJfLmJmZwdjYWEv5mymlUK/XYy4dEVF2BQU3Z5VrMjMzgxMBbIPTx1xDW2jfcgtQrzud4n/4h4mHNuAcnlatVlGv11GtVlEoFDzvx9nYRETmYHAPav9+4PzzURfB0wDObbrpQwBGhoZQmZ4GLr9ce1gHHYfdj34PvyIiovgwuPtx4ABw4YVOEJ9wAvC1ry3edBWAYQAKwF8DOFKvtxwfrUu347D7USqVMDU1hUKhAKUUCoWCM0ZfKmksORERDYJj3GHNzgIf/CCwdWvnbdddB2zahMqXvoQNGzZgYWGh4y66J3i1j6lH9TpERBQ/jnH36/nngfe+12lZr1zZEtoPX3ABcOiQM2b9kY8AS5agVCr5TuLqtnJYr93eXNaTiCibGNztXnwRuPRSJ6xXrADuuGPxpk8BOApON/gvbtuGype/3PHwMMtttof0FVdc0XO3N5f1JCLKKL8DvE26RL4Ay0sviUxMeC6K8meAHBWwwli7biuH+Z0GNOzzh30dIiKyF3h2MA9zc8DGjU7Letky53At15Zly5CD07L+EwA/8XkKv27po48+evHvfD7fMsHL6yQj4jPPIKjbmxPJiIiyaUnSBYjdgQPAqlWd13/gA8CnPw0ccwwuGxpCmCl77d3SXquRzc/Pt9ynlzHobt3epVKJQU1ElDHZa3Hff/+rf192GfDCC06n+Gc/CxxzDIBw48RexzeHORd12DFoHj9NREReshfc73nPqyPYU1PAa1/bcRevhUhGRkaQz+cDu6XDzPT2eu52w8PD7PYmIiJP2QvukNrHqb/whS/g4MGDi8uDeoXqihUrPJ+rOaibx6b91Ot1hjYREXlicLdpjFPPzs4uXtc+Tt2rl19+ueXQrsYa4VwbnIiIesXgbhNmnNrPc889F/i87bg2OBER9YrB3WaQFcmCWspej+chXURE1CsGd5tBViSbnJyE8jkDmN/j20+tydAmIqIgDO42g3Rfl0olXH755R3hze5vIiLShcHdZtDu65tvvhl33HEHu7+JiCgSPK2nJpVKBeVyGTMzMxgbG8Pk5CTDmoiI+hJ0Ws/sLXkagfalThtn9wLA8CYiIq3YVe6jl/NjD3IIGRERUS/Y4vYQ1IIG0NElPsghZERERL3gGHebSqWCDRs2YGFhoeO2fD6P+fn5ltZ1LpfD0Ucf3bLSWkOhUEC1Wo2yuERElEJBY9zsKndVKhWsXLkSl1xyiWdoA8Ds7KxnlzgAroBGRESxYHDDe33yXszOznIFNCIiigW7ygEUi0XUarXA++RyOczPz8OrvoaHh3HkyJGoikdERBljXFe5UqqqlPqOUuohpVTiB2h3m0TWOD+2306OX9c6ERGRbkl2lf+KiLzZb48iTkHrkOdyOdx2220olUq+p+EMOrc2ERGRThzjhvf65ACglFo8HrtSqfA0nERElLikglsA/JNSapdSatzrDkqpcaXUTqXUzmeffTbSwrSvT57P5zEyMrLYNd58HDcnoRERUZISmZymlDpRRPYppV4PYDuAD4rIv/rdP+61yv0mq/G4bCIiioNxk9NEZJ/77wEAfwfgF5Iohx+uhEZERKaKPbiVUkuVUssafwP4dQCPxF2OIH6T1YImsREREcUhiRb3KgD/ppR6GMCDAP5BRL6RQDl8cRIaERGZKvaTjIjI9wD8XNyv24vGZDOeX5uIiEzDldOIiIgMY9zkNCIiIuoPg5uIiMgiDG4iIiKLMLiJiIgswuAmIiKyCIObiIjIIgxuIiIiizC4iYiILJL54K5UKigWixgaGkKxWESlUkm6SERERL5iX/LUJJVKBePj45ibmwPQet5tLm9KREQmynSLu1wuL4Z2w9zcHMrlckIlIiIiCpbp4OZ5t4mIyDaZDm6ed5uIiGyT6eDmebeJiMg2mQ7uUqmEqakpFAoFKKVQKBQwNTXFiWlERGQsno+biIjIMDwfNxERUUowuImIiCzC4CYiIrIIg5uIiMgiDG4iIiKLMLiJiIgswuAmIiKySKaCm6fwJCIi22XmtJ48hScREaVBZlrcPIUnERGlQWaCm6fwJCKiNMhMcPMUnkRElAaZCW6ewpOIiNIgM8HNU3gSEVEa8LSeREREhuFpPYmIiFKCwU1ERGQRBjcREZFFGNxEREQWYXATERFZhMFNRERkEQY3ERGRRRjcREREFrFiARal1LMAapqebiWAg5qeKy1YJ51YJ61YH51YJ61YH50GqZOCiBzndYMVwa2TUmqn32o0WcU66cQ6acX66MQ6acX66BRVnbCrnIiIyCIMbiIiIotkMbinki6AgVgnnVgnrVgfnVgnrVgfnSKpk8yNcRMREdksiy1uIiIia2UquJVS5yqlHlNKPamUujrp8iRBKVVVSn1HKfWQUmqne90KpdR2pdQT7r/Lky5nlJRStyqlDiilHmm6zrMOlOOv3O/MHqXUmcmVPDo+dXKNUmqf+115SCn1jqbbPuLWyWNKqd9IptTRUUqdrJT6plJqr1Lqu0qpK93rM/s9CaiTTH5PlFKvUUo9qJR62K2Pa93rVyuldrjv+0tKqVH3+qPc/z/p3l7s+8VFJBMXAMMAngJwCoBRAA8DOC3pciVQD1UAK9uu+wyAq92/rwbwZ0mXM+I6WAfgTACPdKsDAO8AsA2AArAWwI6kyx9jnVwD4CqP+57m/n6OArDa/V0NJ/0eNNfH8QDOdP9eBuBx931n9nsSUCeZ/J64n/Ux7t8jAHa4n/2XAVzkXv85ABPu31cA+Jz790UAvtTva2epxf0LAJ4Uke+JyCEAdwJ4V8JlMsW7ANzm/n0bgHcnV5Toici/Aniu7Wq/OngXgNvF8Z8AjlVKHR9LQWPkUyd+3gXgThH5iYh8H8CTcH5fqSEi+0Vkt/v3SwAeBXAiMvw9CagTP6n+nrif9Y/d/464FwFwDoC73OvbvyON785dANYrpVQ/r52l4D4RwP9r+v/TCP7SpZUA+Cel1C6l1Lh73SoR2e/+/QyAVckULVF+dZD1783/crt+b20aQslUnbhdmmfAaVHxe4KOOgEy+j1RSg0rpR4CcADAdji9Cj8SkSPuXZrf82J9uLe/ACDfz+tmKbjJ8UsiciaA8wB8QCm1rvlGcfpxMn2oAetg0S0AfgrAmwHsB/AXiZYmAUqpYwB8FcBGEXmx+basfk886iSz3xMRWRCRNwM4CU5vwk/H8bpZCu59AE5u+v9J7nWZIiL73H8PAPg7OF+2Hza69dx/DyRXwsT41UFmvzci8kN3w1QH8Hm82s2ZiTpRSo3ACaiKiPyte3WmvydedZL17wkAiMiPAHwTwFvhDJMscW9qfs+L9eHe/joAs/28XpaC+78AvMGd8TcKZ3LAPQmXKVZKqaVKqWWNvwH8OoBH4NTDBvduGwDcnUwJE+VXB/cAeK87a3gtgBeaukpTrW2M9rfgfFcAp04ucmfJrgbwBgAPxl2+KLljj1sAPCoif9l0U2a/J351ktXviVLqOKXUse7fRwP4NTjj/t8EcKF7t/bvSOO7cyGAf3F7bXqX9My8OC9wZn4+Dmccopx0eRJ4/6fAmeX5MIDvNuoAzjjLPwN4AsC9AFYkXdaI62ErnC69w3DGoN7nVwdwZo7e5H5nvgNgTdLlj7FO7nDf8x53o3N80/3Lbp08BuC8pMsfQX38Epxu8D0AHnIv78jy9ySgTjL5PQHwswC+7b7vRwB83L3+FDg7KE8C+AqAo9zrX+P+/0n39lP6fW2unEZERGSRLHWVExERWY/BTUREZBEGNxERkUUY3ERERBZhcBMREVmEwU1EUErlm87u9EzT2Z6k/axOSqmNSqlbkiorUdYxuIkIIjIrIm8WZ/nGzwG43v37/XAWK2p2EZzjvokoAQxuIgpyF4DfbDqncBHACQC+lWShiLKMwU1EvkTkOTirPJ3nXnURgC8LV24iSgyDm4i62YpXu8vZTU6UMAY3EXVzN4D1SqkzAeREZFfSBSLKMgY3EQUSkR/DOePRrWBrmyhxDG4iCmMrgJ8Dg5socTw7GBERkUXY4iYiIrIIg5uIiMgiDG4iIiKLMLiJiIgswuAmIiKyCIObiIjIIgxuIiIiizC4iYiILPL/Acm7UnBWp4OjAAAAAElFTkSuQmCC\n",
"text/plain": [
"<Figure size 576x360 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"def plot_linear_regression(X, Y, x_label, y_label, m, b, X_pred=np.array([]), Y_pred=np.array([])):\n",
" fig, ax = plt.subplots(1,1,figsize=(8,5))\n",
" ax.plot(X, Y, 'o', color='black')\n",
" ax.set_xlabel(x_label)\n",
" ax.set_ylabel(y_label)\n",
"\n",
" ax.plot(X, m*X + b, color='red')\n",
" # Plot prediction points (empty arrays by default - the predictions will be calculated later).\n",
" ax.plot(X_pred, Y_pred, 'o', color='blue', markersize=8)\n",
" \n",
"plot_linear_regression(X, Y, 'TV', 'Sales', m_numpy, b_numpy)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a name='ex02'></a>\n",
"### Exercise 2\n",
"\n",
"Make predictions substituting the obtained slope and intercept coefficients into the equation $Y = mX + b$, given an array of $X$ values."
]
},
{
"cell_type": "code",
"execution_count": 46,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": [
"# This is organised as a function only for grading purposes.\n",
"def pred_numpy(m, b, X):\n",
" ### START CODE HERE ### (~ 1 line of code)\n",
" Y = m * X + b\n",
" ### END CODE HERE ###\n",
" \n",
" return Y"
]
},
{
"cell_type": "code",
"execution_count": 47,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"TV marketing expenses:\n",
"[ 50 120 280]\n",
"Predictions of sales using NumPy linear regression:\n",
"[ 9.40942557 12.7369904 20.34285287]\n"
]
}
],
"source": [
"X_pred = np.array([50, 120, 280])\n",
"Y_pred_numpy = pred_numpy(m_numpy, b_numpy, X_pred)\n",
"\n",
"print(f\"TV marketing expenses:\\n{X_pred}\")\n",
"print(f\"Predictions of sales using NumPy linear regression:\\n{Y_pred_numpy}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"##### __Expected Output__ \n",
"\n",
"```Python\n",
"TV marketing expenses:\n",
"[ 50 120 280]\n",
"Predictions of sales using NumPy linear regression:\n",
"[ 9.40942557 12.7369904 20.34285287]\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 48,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[92m All tests passed\n"
]
}
],
"source": [
"w2_unittest.test_pred_numpy(pred_numpy)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now you can add the prediction points to the plot (blue dots)."
]
},
{
"cell_type": "code",
"execution_count": 49,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAe4AAAE9CAYAAADNvYHXAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/av/WaAAAACXBIWXMAAAsTAAALEwEAmpwYAAAx6UlEQVR4nO3dfZQb5X0v8O+z693Ysp1gyxtf3lYyuQm9hLQB3IY0xWlx20B6QtJCbuGIxm0Dm6zTBDeXbQk6NXDSpUncFmgDzt0ckwCrmiS0PdA2TmvaQFPamtoOOATKWyLtxTEYLwQWdh3bq9/9Y0ZrvcyMRtK8PM/M93OOzu5KK+nRI2m+87zMM0pEQERERGboi7sARERE5B+Dm4iIyCAMbiIiIoMwuImIiAzC4CYiIjIIg5uIiMggi+IugB+rVq2SfD4fdzGIiIgisWfPnkMiMuR0mxHBnc/nsXv37riLQUREFAmlVMXtNnaVExERGYTBTUREZBAGNxERkUEY3ERERAZhcBMRERmEwU1ERGQQBjcREZFBGNxERJQopVIJ+XwefX19yOfzKJVKcRcpUEYswEJERORHqVTCyMgIZmdnAQCVSgUjIyMAgEKhEGfRAsMWNxERJUaxWFwI7ZrZ2VkUi8WYShQ8BjcRESXG1NRUR9ebiMFNRESJMTw83NH1JmJwExFRYoyPjyOTyTRcl8lkMD4+HlOJgsfgJiKixCgUCpiYmEAul4NSCrlcDhMTE4mZmAYASkTiLkNba9euFZ7Wk4iI0kIptUdE1jrdxhY3ERGRzYRjwHkcNxEREcw5BpwtbiIiIphzDDiDm4iICOYcA87gJiIigjnHgDO4iYiIYM4x4AxuIiIimHMMOIObiIhSx+2wr0KhgHK5jGq1inK5rF1oAwxuIiJyocMxzWGUoXbYV6VSgYgsHPbV/Ng6vH5HIqL95ZxzzhEiIorO5OSkZDIZAbBwyWQyMjk5aXwZcrlcw2PWLrlcLvTn9gvAbnHJRC55SkRELfL5PCqVSsv1uVwO5XLZ6DL09fXBKfuUUqhWq6E+t1+xLHmqlDpVKfVtpdTjSqnvK6Wusq+/Xim1Xyn1iH15f1hlICKi7uhwTHNYZfBz2JcOr99NmGPcxwD8HxE5A8C5AD6hlDrDvu0mEXmnfflmiGUgIqIu6HBMc1hl8HPYlw6v301owS0iB0Rkr/37DIAnAJwc1vMREVFwdDimOawy+DnsS4fX78pt8DvIC4A8gCkAbwRwPYAygH0Abgewot39OTmNiCh6k5OTksvlRCkluVwu0olpOpQhzudGnJPTlFLLADwIYFxE/kYptRrAIViz9D4L4EQR+V2H+40AGAGA4eHhc5wmCRARJV2pVEKxWMTU1BSGh4cxPj6u5bHFFCyvyWmhBrdSagDA3wP4RxH5c4fb8wD+XkTO9HoczionojRqPs0kYHXX6riaFwUrrlnlCsA2AE/Uh7ZS6sS6f/t1AI+FVQYiIpOZcppJilaYs8rfA+C3AJzfdOjXF5RS31NK7QPwSwB+P8QyEBEZS+dDkkyk7UpoHVoU1gOLyL8BUA438fAvIiIfhoeHHRcB0eGQJNM0DzvUljkFYNywA9cqJyLSlNaHJAUkqlZwkoYdGNxERJoy5TST3fJ7so8gJGnYgWuVExFRLKJcDzzutcc7FcusciIiapSUyVFBibIVHNawQyzvqdvKLDpduHIaEcWt11W04j5NpI78nF4zSEGvhBbmewqe1pOIqHtBLIRiWldtFExfYCbM9zS2ldOCwuAmojgFsYH2cw7oNDJ5Sdcw31MGNxFRD4LYQLPFnTxxtbg5OY2IqI0gzs2chmOy0yau95TBTUTURhAbaBOOyeas987E9Z6yq5yIyAeTx2L9MH2iWNJwjJuIiDxxDF4vHOMmIiJPSVoSNOkY3EREFMgEPIoGg5uIiDjr3SAMbiKigPQyKzvuGd0mzHonCyenEREFoJdZ2ZzRTc04OY2IqAPdtH6LxWJD8ALA7OwsisViqPel9FkUdwGIiHTS3PqtVCoYGRkBAM/Wby+zst3+x+nwLCK2uImI6nTb+u1lVrbb/yiljFm9LO4x+jRhcBMR1em25dzLrOzx8XEopVquF5GuusujDtFaL0WlUoGILPRSMLxD4naibp0u55xzTs8nJSci8iOXywmAlksul2t738nJScnlcqKUklwuJ5OTk76f1+k5AYhSqqPyT05OSiaTaXiMTCbTUVk61UudkTMAu8UlEzmrnIioTlwzvINacjSOpUt5rvHgcVY5EZFPcR3PHNQCKEEsXdppVztXXYsWg5uIqEmhUEC5XEa1WkW5XA4ltJvDEUAgOwy9hmg349VcdS1ibn3oOl04xk1ESRLmOHSvj93teHUv4/vUChzjJiLSRxjj0PXnC1+5ciUA4KWXXur43OEcr9YDx7iJiDQS9Ck0m7u3p6enMTc3h7vuuqvjrn6OV7vT5Vh1BjcRUcSCDscgl0zleLUznY5VZ3ATUaLo0iryEnQ4BtmC51nCnOm0njzHuIkoMUw6y1b9mHSn49DN4jh2O22iHvv3GuNmcBNRYqQ1wEzaYTGV52frX/4FGBoCli8P7Pk4OY2IUiHoSV+mYPd2+JqHN34KwF6lUK5UgLe8BfjQhyIrC4ObiBIjzTOio1g0RldRzGsoFArYfsMN2Ds4CAHwBICz6nus//RPA39ONwxuIkoMp0lfSilUKhVtJ6rFzYTJfF5Cn+39gx8A69YBSuGisTGcdeTIwk2fzmZRuusuQAQ466xgns8Pt5VZdLpw5TQi8qu2ghfsM2shwrNkmSaOM4kFLZQzk1UqIuvXi1iR3HD5+OBgJPUFrpxGRGmT1olqnUhCHfmZ7f3aa8CWLcBttwHT00A2C2zcCIyNAcuW2XfYvx+44grgW99qfZKtW4GPfQz5NWsiqy/OKiei1OHSne0loY7a7Xy89hpw7rnAs88Chw8fv33xYuAtuaP4z9MKWLbjG60PfMstwO/9HtB3fEQ5yvrirHIiSp00T1TzKwl11G4xmy1bWkMbsP5+9slj2LLjjONXbtkCHDtmdYp/6lMNoQ3oU18MbiJKpKQs3Rnm5LFe6ijocnX7eO0OhbvtttbQrjmMJdi6dAw4etQK66uvBvr7XZ9Lm8+U2+B3rxcApwL4NoDHAXwfwFX29SsB7ATwtP1zRbvH4uQ0IuqG6aeajGLyWDd1FHS5QnmdL78ssmGDKMw7zTFbuPT1dV7WKD5TiGNymlLqRAAnishepdRyAHsAfAjAbwN4SUQ+p5S6xg7uP/R6LI5xE1Ea6Tp5LOhyBfZ4r74KfPrTwLZtC1cN4SAOYcj1LkNDwMGDnZQ2GrGMcYvIARHZa/8+A+t49ZMBfBDAHfa/3QErzImIqImuK8F1Ui4/XeA9vc7XXrOmiCsFvOlNDaGNsTFsvPYELF7sfNfFi4HR0fZPoZtIxriVUnkAZwHYBWC1iBywb3oewGqX+4wopXYrpXa/+OKLURSTiEgr3UyGimJBFb/l8rs4Ssevc3YW2LTJCuvly63DtWo2bQJef93qCf/CFzD2mQG85S1oCe/Fi62VSsfGvF6pptz60IO6AFgGq5v8N+y/f9x0+8vtHoNj3ETJY/r4cxRGR0c7WkQmqgVV/D6P38VRfD3e3JzI2JjzQPXGjSIzM67lnZkR2bxZZGjIGtMeGrL+9rhL7OAxxh12aA8A+EcAn6677klYY98AcCKAJ9s9DoObKFmSsGJX2JzqSCklo6OjrvcJZRUxj/K12/Fq3umofx2+Hu/wYZFrr3UO6yuvFHnllcBfly68gjvMyWkK1hj2SyKyqe76LQCm5fjktJUi8gdej8XJaUTJouukKx3UztPtVD+Adx3ptqBKV+/z0aPA+Dhwww2tt23YANx0E7BiRbAF1VBcC7C8B8BvAThfKfWIfXk/gM8B+BWl1NMAftn+m4hSRNdJV3GrHxN241VHuiwQUuP7uOdjx4Abb7TGrAcHG0P7ssuAQ4esdvZXv5qK0G7LrSmu04Vd5UTd0XUcOcou3U7EXV9u9eK3jnQcgnCt02PHRLZsce4Gv+QSkYMHYyuzDhDXGHdQFwY3Ued03IjrXDYdyuQ2JtxJeeLe+fA0Py9y883OYX3RRSIHDsRdQm0wuIlSSNdWbY1bwMQVPL3UV1Bl9mpxaxfCflWrIrfe6hzWF14o8txzcZdQSwxuohTqZEavLuJs9XZbX0GWOerX3+sOh+v9q1WRL3/ZOazXrxcpl0N4NcnC4CZKId1b3E7iLHO3zx10maPqceh1J8Hp/lcMDjqH9bp1Is8+G8rrSCoGN1EK6TBm26k4ewm6rS8TezZEet/hqN3/UqegBkTOPVfkqafCfREJ5hXcPK0nUUK1O92hjuI8nKnb+tLtECy/ejok76//GuVKBQJge93VuwGcAVjR/R//Abz1rb0XlFq5JbpOF7a4ieJlSvdtHEwss0gXLe577xUZGGhpWT8KyJmGDMWYBOwqJ6JumTZhKg5+yqzb6/L1vn7zmyKZTGs3+Omnyz989rNG7rCYgsFNRF0zcZJbWLoNX11b5Y6vZ+dOkTe9qTWs16wR2bWr/f0pEF7BHdpa5UHiWuVE8dFt/eu41JYjnZ2dXbguk8n4GgfXfm32Bx8EPvxhoPkUyqeeCtx9N/DzPx9PuVIsrrXKiSgBTJ18FbRisdgQ2gAwOzuLYrHY9r5ars3+7/8OnHKKtT74L/7i8dBevRp44AGrnT01xdDWEIObyEClUgn5fB59fX3I5/MolUqhPZfvE0W0EWWZw9BL+Ha68xNaXT38MHDaaVZYv+c9wP791vUrVgA7d1ph/fzzwHvfG8zzUTjc+tB1unCMm+i4OMZLg1hhS8cx3k70uiSq39cfeF2VyyJve1vrmPXSpSI7dnT3mBQ6cHIaUXKYOFnMxDI3C2KlMT87P4HU1b59Imee2RrWg4Mi993n/3EoNl7BzclpRIYxcbKYiWV2UiqVUCwWMTU1heHhYYyPjwe+oE3XdfXEE8DllwN79zZef911wPr1wHnnBVpOChcnpxEliImTxUwss5NCoYByuYxqtYpyuRzKKnQd1dXTTwPnnmuNWZ9xRmNob98OVKvA9dfHEtqmz2nQGYObyDBBTRaLkolljkvbuvrBD4B166ywftvbgF27jv/jnXdaYS0CXHqp9T8xqB06V6lUICKoVCoYGRlheAfFrQ9dpwvHuIkambjwhYlljoJTvTRf9zc332ydDtPpZB7btlmn0dRIEOP0af+8gJPTiKgbadh4xvkaPSe8PfecyAUXOIf11q3ahXW9Xs+YloSjEHrF4CZKiSBDKA0bz7hfY3PLdDUg9zoFNSByyy0i8/ORlKtXQZ0ytJcWu+kY3EQpEHQI6brxDHLnJO7XqJSSIUC+4RbWW7aIHDsWSVmC1Otn0dRznAeJwU2UAkGHkI4bz6B3TmJ7jYcOyQ/f/W7HsL4GkEWA8T0bvexgxb1DpQMGN1EKBB1CQW48g2olB71BjzQgXn5ZZMMGx7D+Izus0xpSzeIewtABg5soBYIOoaA2nkFuhIPeOQk9IF55ReSKKxzD+rOADDq8Fh16NnSQhomRXhjcRCHSZQMTRggF8dqC3KEIo4Uc+Ps3MyMyOuoY1jI2JjI357oDwhY31TC4iUKiW5eeLjsR9YJsJet2gpXabRlAti1f7hzWmzaJvP56w2O67YDo8BkiPTC4iULCSTTthdGFH9XOideOwl/dfrv8+aJFzmG9caPV8u7gcWs7OLrscFG8GNxEIdFx5rVudOuV6ETzTscgIH/sFNSA/F9AlnewQ6Jj7wjpwyu4eXYwoh7k83lUKpWW63O5HMrlcvQF0lQUZ9UKQ19fH/pFUARwvcPtXwWwCcArddeZdsYz0hPPDkYUEp48w58ozqoVqGPHgBtvRFUER9EY2iUA7zzlFORzOfwOGkMbMO+MZ2mSlDOWLYq7AEQmqwXQVVddhenpaQDAkiVL4iwSdWt+HrjpJmBsrOWmbwDYCOAQrB2zic99DgAwMjKC2dnZhf/jTpu+amcsq71ftTOWAdB/R7IJW9xEAZibm1v4fXp6OpBTGCaldaC1ahW45Rbr9JeLFjWG9kUXAQcOoDQ5ibFcDtNKIZfLYWJiAoVCAYVCARMTE8jlclBNtzUz5b00pZzdKBaLDTtZADA7O4tisQjAsNfuNvit04WT00hnYR1bbOqELu1VqyK33eY8G/zCC62zcgXIlPfSlHJ2y2siqY6vHZycRhSevr4+OH2PepmkxElvARMBtm0Drryy9bb164HbbwdCGps25b1ctWrVwnBPPd3K2S2v9wGAdu8RJ6cRhchtMlIvk5SmpqY6up4ciAB33ml1g/f1NYT2gwDOO+kklCYngfvvDy20ATPey1Kp5BjagF7l7IXXRFIT3qN6DG6iHoUxszyMnQHAsHG8bogAd999PKw3bFi46b8WLcLbBwagAPwigH/70Y8CmYvQTljvpZNu39/aOK+TpMyS95qTEOV7FAi3PnSdLhzjJt0FvZhGWOuO6zaOF5h77nEcs97T1yc/FfOa4FHVey/P47V2eiI+H23o+N0AV04jMm+lqqDLm7jlWe+9V2RgoDWw3/EOkX372q4HXrtEscpdFJ+9Xt5ft/tms9nAy6kr3bYPgQY3rO71N3Z6v14uDG7qlY571M3C3nAkYnnWHTtEli5tDevTTxfZu7fhX/2cgas+2HTbcHeql/fXhO9H2vQc3AD+CsAbASwF8DiA5wCMtbnP7QAOAnis7rrrAewH8Ih9eb+f52dwU690b21GseHUvQ5c3X+/yAkntIb1aaeJ7Nrlejc/Le5aHSchuHp9f03fcUmaIIL7EftnAcCfARgAsK/NfdYBONshuK/285z1FwY39Ur31mYUoeoUTrXuUO020g88IPLmN7eG9amnijz0kK+HcHq9AwMDks1mW8LJ2J2aOknY+YiazjsrQQT39+2w/gaA99rXPerjfnkGN+lA9w1zVDsWk5OTks1mXVuesXroIZGTT24N69WrRR54oKuNrNN9nK7TfcfOL52DSDe67+gEEdyfsru4vwlAAcgB+I6P+zkFdxnAPlhd6Sv8PD+Dm3ql+5c0yh0LrXZiHn5YZM2a1rBesUJk586Ffwvq/XN7HKedGZ127NphYLfXXEe6v+c9B7fjHYFFPv6nObhXA+iHNcFtHMDtHvcdAbAbwO7h4eFwa4hiFdVGR+eNW5Q7FrG3LvfutSaTNYf10qXW5DMHQe1suG2ss9ms1jt2XnTfKdWB2zCRzr0sQbS4VwPYBmCH/fcZAD7q434Nwe33tuYLW9zJ5bbRGR0d1TZkwxLVjoVXCIZWhn37RM48szWsBwdF7ruv7d2D2NmYnJz03FjrvGPnRaseFE35maioW70FEdw7APxv2OPasE4H+j0f92tucZ9Y9/vvA7jbz/MzuJPL7QvVvKFmCyI4XjtLgbbcHn9c5KyzWsJ6HpCL63YU/AginLw23rpsrLsRew+KAfweGqjTdiaI4P4v++d36657pM19tgM4AOAorMPHPgrgLgDfgzXGfV99kHtdGNzJ5fcLZfrGVTdOrctAWm5PPSXyrne1tqwB+c4nPiGZJUt8byjry5jNZmVgYKCnjWxSVwdji7s9tzrKZrPa9rIEEdwPAMgC2Gv/fS6AB/3cN4gLgzu5OunCYgsiXF233J59VuS88xzDWu680zqNpnQWME69AoODg46HcvnltfE2Gce42zOxjoII7rMBPATgFfvnUwB+2s99g7gwuJPL6QvlFiBJbEHoNK7aUcutUhE5/3znsN62bSGs63WyYxBGK9LEjbdfOn2OdGVaHfUc3NZjYBGAtwM4E8CA3/sFcWFwJ1vzFyrwsVZN6RYkbcvz3HMiF1zgHNZbtzqGdb1OwtirW7uXDa9pG29Kr66DG8BveF287hvkhcGdPjpsYMMug45jk82v+Z4vflHkAx9wDutbbhGZn+/osf3uqPgZQknizhxRTS/B/RWPi+sx2EFfGNwUtShaw9rOBn7hBZGLL3YO6y1bRI4d6/qh/e4M+T3uNonDJ0Qi3sGtrNv1tnbtWtm9e3fcxaAUyefzqFQqLdfncjmUy2VjnsO36Wngk58Etm9vve3GG4GxMWDRokiLVCqVUCwWMTU1BbftlFIK1Wo10nIRRUEptUdE1jrd1tfBg/yaUuoPlFKba5fgikikl6mpqY6u78b4+DgymUzDdZlMBuPj44E9h6cf/xjYsAFQCli1qjG0b7gBOHLEamd/5jORhzYAFAoFlMtlVKtV5HI5x/8ZHh6OuFRE8fMV3EqpLwH4TQCfhLVW+YdhrVdO5KlUKiGfz6Ovrw/5fB6lUinuIvniFghBBkWhUMDExARyuRyUUsjlcpiYmEChUAjsOVq8+ipwxRVWWK9YAdx55/HbikXg8GErrDdvBgYGwitHh2LfySHSiVsfev0F9ik8634ug4+TjAR14Ri3mXSbNd0Jk8veYmZGZHTUecx6bExkbi7uEvqiw4RFoqgggOO4d9k//xPASQAWA3jGz32DuDC4zaTjrOlOGB0Ur78uctVVjmH9+AUXWLeT8fx8Ro3+HKdYEMH9RwBOgHUY2AH78lk/9w3iwuA2UxSzprlRqjM3Z7WgHcL6LwFZanrPATXw0yuUqJ6jlOk6uAH8LID/Uff3RwD8E4C/ALDS675BXhjcZgq7xW36RimQnY7Dh0WuvdYxrCcAOaGvz+heD3Ln5/vV6XeQO8L66CW499YCGsA6AD8CcDGAzwK4x+u+QV4Y3GYKO1hN7orvqW6OHBG57jrHsH7mvPPkxKYTeYTd60Hx8NOj1Umvl+k7wknTS3A/Wvf7rQCur/v7Ea/7BnlhcJsrzD14bRcw8aHjnY6jR0XGxx3DWi67TOTQIc/HNXHnhrwF3eI2eUc4iXoJ7scALLJ//28A6+pv87pvkBcGNznxCindu/mAZQJcJ8BBAebtn9cJsOz4Px07Zq1U5hTWl1wi37jttpadIj+nSTW9FeW0M6hbF28U5Ql6jNvkHeEwxfXZ6iW4i7DOBnYvgO8CCyut/U8AD3ndN8gLg5uctFsWU9eAmpkRGRj4bwFmm/J4VgYG/ltmPn+rc1hfdJHIgQMi4r5BzmazjnXR39+vTaj1wu10n72eqzvsMoZVniBnlevU4tZlRyzO4YOug9u6L84F8OsAltZd9zYAZ7e7b1AXBney9fIlrd3XpC7hzZtFBgaOOmbzYszKZlx3/IoLL7TOytXE7TVns1kjxim7fc/9DgXE+d7rFICd0GWMW5dyiMT7XvYU3DpcGNxm8tsaCOJLalI336pVzg3q2mVo4GXrfNcevF6vLq0VN728536GAuJ+7036LDqdUjfuz45OOz5xvpcMboqc341zUF/SOL/svoOyWhW54w5RmPcM7r6+9s+p08atU72UnS3u4OjUsq2n044PW9wM7lTx+4EP6ksa10ao7fNWqyLbtzck8yoc9G5xD+n7eoPQy3vOMe7g6Dq5U6cdH2PHuHW4MLjN43fjHOSXNI4uYrfyf8ytP/ycc2TzxoOyeLHzzYsXW2PgfnRybuu4uz/r9fqemzCrfHR0VPr7+wWwJgaOjo52dP8oXk+7YYe4djZ02/Exbla5LhcGt3n8bpx1+5J2qn7j9wFAfuKUxu94h8i+fQv3mZkRefvbpSW8Fy+2rp+ZCa58OtavjmUKUq+vL6r68TPsEFf3vm47YnFgcFPkOtn41M8Mr7VSTPmyfuTNb5YZh7B+ZtEikT17XO83M2O1rIeGrDHtoSHr7yBDW0Svbsd6Sd4w91rnUb1n7Q6n7GbIqtfyJPUz0Q0GN8Wiky+iUa2w++8XOeGE1rAG5Gdhjblms9mW1x3HhkmniT5p0Wudh/meuc0ij7vFbdT3PyIMbupJFIGja8twwQMPWM3i5tb1qafKtzZvXqifbDbrOFFqdHQ0lg2T9vWaQLq2uL3CMe7gDPo1J6H1zuCmrkX1hfbqrovNQw+JnHxya1ivXm0FuQO3DVBtCCDqAI17g+xH/UY2m8069lboUj4/ZQpzjLuXQGoXjnGGXZC9DCZ85v1gcFODTr6gUbXY3IKtv78/0Odpa9cukTVrWsN6xQqRnTvb3r2TBULq6zLsNa11bX20G2eNe4PbbQj0WuduM+ed6iqbzfa0eI0OwyZBbmeS0svE4KYFnW6Iovqyx9ri3rtX5PTTW8N66VKRHTsc7+K2Ye60xd1cv3EHVdS8xld12ODqFAJedeXnc6PTa2kWZCtZ5x2UTjC4aUGnX96ovuxRb1T+/k/+RJ4YGGgN68FBkfvu87xvN2OFTmPcbhsYHTakUfHTQxHnBterfFH3XvS6g6N7F3JQPUM676B0gsFNCzrdGw3iy+7nCzk6Ohp+6/Pxx0XOOqslrOcBuWxw0PdzdTtW2Hy9jkEVNVNb3HEEn1uvTSefG52HTYKi+w6KXwxuWtDN3mgvX3Y/XyKn/1FKdbzalKMnnxR517taW9aA/GaXARFUV1zQLQMTN8omjnHHtWOh8w6Obkz8LjRjcNOCqPdG/YRT4F1bzz4rct55jmEtd94pqseWblDlDfK9MLmVYcqs8rh7SHod4yazMLipQZR7o35ap4G0YCsVkfPPdw7rbdusk33Yeg3eTkPSq767OdTI6f+TMq6ns7jruNdZ5WQWBncCmdIV5LWxa9eSabtBfO45kfe9zzmst25tCOt6UY3bB/Vcfh4rKTNpdaZDr4Yp33vqHYM7YZwmcum6593JLGtfG8Qf/UjkAx9wDutbbhGZn/ddrig2gEG20rweK+7WYFowOCkqDO4E8Wpd6TrW5bSxa9fSbngNL7wgcvHFzmG9ZYvIsWOhlrUXQbaEvR5Lh9YgEQWHwZ0gXoFnUiurbaAdOiRy2WXOYX3jjSJHjzY8XhATnMIIv6ha3LXyszUYLtYxRYXBnSC6L1jhpX6j53RM6gmA3LN0qXNY33CDyJEjro8bxCFFYXQ3p2HmeFrCTNf6p2RicCdIJy1unTaobuG6HJAvOwU1IFIsihw+3Paxg+qFaNcV3W1dBvk+6PSe1sqTljDjPAKKEoM7Qfy2LnXboNZv9JYCcqtbWI+NiczNdfTYQfVCuG2Ys9msVnXZjcnJSclmsw2vKYjypynMOHOfosTgThg/47m6bVAzgNzkFtabNom8/nrXjx1Ui9ttZ6c+8HSoy05NTk7K4OBgS/kHBga0mnynO92+U5RssQQ3gNsBHATwWN11KwHsBPC0/XOFn8dicHdmcnLSM8QiMzcncvXVjmH9l3bLO4iNXpDLZjp1RZseTu1m8Htp1zWfpjDTrReLki2u4F4H4Oym4P4CgGvs368B8Hk/j8Xg9q9diNXGa0Nz+LDItdc6hvW2/n5ZHtJGL8xlM00PJ6+hBK+dj27XmU9ymOk2x4CSK7aucgD5puB+EsCJ9u8nAnjSz+MwuP2L5XCxI0dErrvOMaxlwwaRl18WkWCX/oyS6eHUbYvb7w6Lzu8dkal0Cu4f1/2u6v/2ujC4/YvscLGjR0XGx53D+rLLrOOwfTIhGE0Op27HuE0fIiAymZbBbf/9ssd9RwDsBrB7eHg4tMpJmlBb3MeOyZ5LL3UM668DctYppywEQSdBF2VXdHO5RkdHjQ3kTnQzq9z0IQIik+kU3OwqD9nk5KQMDAy4hnatJTszI7J5s8iqVSJKWT83bxaZmWl6wPl5kZtvdgzrvwVktcPjO61D7tWCjqpl1278308rNE1M6AkhSiqdgnsLGienfcHP4yQtuP20RrvtmnXrFq21lGqh/fa3iyxe3JjFixdb18+8Mi9y662OYf0PgJzUpkXvtCqaV0stqpadn96IWmuULCYPERCZLJbgBrAdwAEARwE8B+CjALIA/hnW4WD3A1jp57GSFNxhz9T1E4KbN7eG9kJ4Y1Y247rGK9evFymXfY2fe13cWtBOr1cpJaOjo71Wd4NOyk9EFKfYWtxBXUwIbr8tEz/B2ksL1E+386pVzqFduwzhBZF160SefdZX2f22uOtb/c2cTlUadLes3/IzuIkobgzukPltIXstjFIfrL2M+bYN/WpVFKqewd3X5/91Nl+6Pdd2FN3lfsoPsKuciOLH4A6Zn9BpFxpBtbjddiIe/NSnFpJ5FQ56t7iHvB/fz6xsP+fcrhflBDWvHoHBwUGO4xJR7BjcIfMTOl4hFvRqVLXQvAiQnzgk8+ah22TxG+YdQ3vxYmsMPMq68aqfsA4Jc9qJCurEG0REvWJwh8xP6LSbGNU89tv1bN5vflMkk2lN5NNPF9mzR0Sk/azy5kPCQq6b2ut12lkJ6zhrzpYmIp0xuEPmZ2KVn4lRXU/Guv9+kRNOaA3rNWtEdu1yvEvtOO6hIWtMe2io9TjuIMKtk94Dp274bnseGMxEZDIGd4j8Hsrkd2KU767hBx6w0rY5rE89VeShh0J5Xd2GZret5m67z7lwCBGZjsEdok7Cpd2ErbaTsR56SOTkk1vDevVqK8hjel3NrzGo0Ox2wlqU4+VERGHwCu4+UE+mpqZ8X18oFDA+Po5MJuP6eMPDw41XPPwwcNppgFLAe94D7N9vXb9iBbBzpxXdzz8PvPe9HZe9VCohn8+jr68P+XwepVKpq9dVr1gsYnZ2tuG62dlZFIvFjsvXUhdtrq/ptuxERCZgcPeo03BxCraaTCaD8fFx4LvfBU4/3Qrrd70L+OEPrX9YuhTYscMK65deAn75l7sud6lUwsjICCqVCkQElUoFIyMjC+GtQ2g67eQs1JGHbstORGQEt6a4Thedu8o77Rp26/49E5CXTzmltRt8cFDkvvsCL3e77uRuu7yD7qbuZpIZx7iJyHTgGHe4uj2F5f8CZE9zUAPW6bruuSfUMnsdnlZ7LdlsVrLZrJGhyVnlRGQyBndE/ITFvVu2yK6+vtawBkS2bxepViMpq1vLOIj1whmaRES98QpuZd2ut7Vr18ru3bvjLoan2phx/fh1JpPBxMQECu9+N/Dbvw185zst9/t0Notzbr4Zhcsvj7C0zuVVSsHp85DL5VAulyMsHRFRuiml9ojIWsfbGNzByOfzqFQqC38PA7gdwHqnf962Dfid37Emn8WoVCqhWCxiamoKw8PDDeWvp5RCtVqNuHREROnlFdycVR6QqakpnAxgB6w+5gqaQnvrVqBatTrFf/d3Yw9twDo8rVwuo1qtolwuI5fLOf4fZ2MTEemDwd2rAweAiy5CVQTPAbig7qZPARjo60NpchL4+McDD2uv47C70e3hV0REFB0GdzcOHgQuucQK4pNOAv7u7xZuuhpAPwAF4C8BHKtWG46PDkq747C7USgUMDExgVwuB6UUcrmcNUZfKARYciIi6gXHuP2angY++Ulg+/bW2268ERgbQ+lrX8OGDRswPz/f8i9BT/BqHlMP63mIiCh6HOPu1ssvAx/5iNWyXrWqIbQfvfhi4MgRa8z6M58BFi1CoVBwncTVbuWwTru9uawnEVE6MbibvfoqcMUVVlivXAncddfCTX8M4A2wusF/fscOlL7+9Za7+1luszmkN27c2HG3N5f1JCJKKbcDvHW6hL4Ay8yMyOio46IonwfkDR4rjDVrt3KY22lA/T6+3+chIiJzgWcHczA7C2zaZLWsly+3DteybVu+HBlYLes/BPATl4dw65ZesmTJwu/ZbLZhgpfTSUbEZZ6BV7c3J5IREaXTorgLELmDB4HVq1uv/8QngM99Dli2DFf29cHPlL3mbmmn1cjm5uYa/qeTMeh23d6FQoFBTUSUMulrcT/44PHfr7wSeOUVq1P8i18Eli0D4G+c2On4Zj/novY7Bs3jp4mIyEn6gvvDHz4+gj0xAbzxjS3/4rQQycDAALLZrGe3tJ+Z3k6P3ay/v5/d3kRE5Ch9we1T8zj1V77yFRw6dGhheVCnUF25cqXjY9UHdf3YtJtqtcrQJiIiRwzuJrVx6unp6YXrmsepO/X66683HNpVWyOca4MTEVGnGNxN/IxTu3nppZc8H7cZ1wYnIqJOMbib9LIimVdL2en+PKSLiIg6xeBu0suKZOPj41AuZwBzu3/zqTUZ2kRE5IXB3aSX7utCoYCPf/zjLeHN7m8iIgoKg7tJr93Xt912G+666y52fxMRUSh4Ws+AlEolFItFTE1NYXh4GOPj4wxrIiLqitdpPdO35GkImpc6rZ3dCwDDm4iIAsWuchednB+7l0PIiIiIOsEWtwOvFjSAli7xXg4hIyIi6gTHuJuUSiVs2LAB8/PzLbdls1nMzc01tK4zmQyWLFnSsNJaTS6XQ7lcDrO4RESUQF5j3Owqt5VKJaxatQqXX365Y2gDwPT0tGOXOACugEZERJFgcMN5ffJOTE9PcwU0IiKKBLvKAeTzeVQqFc//yWQymJubg1N99ff349ixY2EVj4iIUka7rnKlVFkp9T2l1CNKqdgP0G43iax2fmy3nRy3rnUiIqKgxdlV/ksi8k63PYooea1DnslkcMcdd6BQKLiehtPr3NpERERB4hg3nNcnBwCl1MLx2KVSiafhJCKi2MUV3ALgn5RSe5RSI07/oJQaUUrtVkrtfvHFF0MtTPP65NlsFgMDAwtd4/XHcXMSGhERxSmWyWlKqZNFZL9S6s0AdgL4pIj8q9v/R71WudtkNR6XTUREUdBucpqI7Ld/HgTwtwB+Lo5yuOFKaEREpKvIg1sptVQptbz2O4BfBfBY1OXw4jZZzWsSGxERURTiaHGvBvBvSqlHATwM4B9E5FsxlMMVJ6EREZGuIj/JiIj8AMDPRP28nahNNuP5tYmISDdcOY2IiEgz2k1OIyIiou4wuImIiAzC4CYiIjIIg5uIiMggDG4iIiKDMLiJiIgMwuAmIiIyCIObiIjIIKkP7lKphHw+j76+PuTzeZRKpbiLRERE5CryJU91UiqVMDIygtnZWQCN593m8qZERKSjVLe4i8XiQmjXzM7OolgsxlQiIiIib6kObp53m4iITJPq4OZ5t4mIyDSpDm6ed5uIiEyT6uAuFAqYmJhALpeDUgq5XA4TExOcmEZERNri+biJiIg0w/NxExERJQSDm4iIyCAMbiIiIoMwuImIiAzC4CYiIjIIg5uIiMggDG4iIiKDpCq4eQpPIiIyXWpO68lTeBIRURKkpsXNU3gSEVESpCa4eQpPIiJKgtQEN0/hSURESZCa4OYpPImIKAlSE9w8hScRESUBT+tJRESkGZ7Wk4iIKCEY3ERERAZhcBMRERmEwU1ERGQQBjcREZFBGNxEREQGYXATEREZhMFNRERkECMWYFFKvQigEtDDrQJwKKDHSgrWSSvWSSPWRyvWSSPWR6te6iQnIkNONxgR3EFSSu12W40mrVgnrVgnjVgfrVgnjVgfrcKqE3aVExERGYTBTUREZJA0BvdE3AXQEOukFeukEeujFeukEeujVSh1kroxbiIiIpOlscVNRERkrFQFt1LqAqXUk0qpZ5RS18RdnjgopcpKqe8ppR5RSu22r1uplNqplHra/rki7nKGSSl1u1LqoFLqsbrrHOtAWf7C/szsU0qdHV/Jw+NSJ9crpfbbn5VHlFLvr7vtM3adPKmUel88pQ6PUupUpdS3lVKPK6W+r5S6yr4+tZ8TjzpJ5edEKbVYKfWwUupRuz5usK9fo5TaZb/urymlBu3r32D//Yx9e77rJxeRVFwA9AN4FsBpAAYBPArgjLjLFUM9lAGsarruCwCusX+/BsDn4y5nyHWwDsDZAB5rVwcA3g9gBwAF4FwAu+Iuf4R1cj2Aqx3+9wz7+/MGAGvs71V/3K8h4Po4EcDZ9u/LATxlv+7Ufk486iSVnxP7vV5m/z4AYJf93n8dwKX29V8CMGr/vhHAl+zfLwXwtW6fO00t7p8D8IyI/EBEjgC4G8AHYy6TLj4I4A779zsAfCi+ooRPRP4VwEtNV7vVwQcB3CmW/wRwglLqxEgKGiGXOnHzQQB3i8hPROSHAJ6B9f1KDBE5ICJ77d9nADwB4GSk+HPiUSduEv05sd/r1+w/B+yLADgfwD329c2fkdpn5x4A65VSqpvnTlNwnwzg/9X9/Ry8P3RJJQD+SSm1Ryk1Yl+3WkQO2L8/D2B1PEWLlVsdpP1z83t21+/tdUMoqaoTu0vzLFgtKn5O0FInQEo/J0qpfqXUIwAOAtgJq1fhxyJyzP6X+te8UB/27a8AyHbzvGkKbrL8goicDeBCAJ9QSq2rv1GsfpxUH2rAOliwFcBbALwTwAEAfxZraWKglFoG4K8BbBKRV+tvS+vnxKFOUvs5EZF5EXkngFNg9Sb8VBTPm6bg3g/g1Lq/T7GvSxUR2W//PAjgb2F92F6odevZPw/GV8LYuNVBaj83IvKCvWGqAvgyjndzpqJOlFIDsAKqJCJ/Y1+d6s+JU52k/XMCACLyYwDfBvBuWMMki+yb6l/zQn3Yt78JwHQ3z5em4P4vAG+1Z/wNwpoccF/MZYqUUmqpUmp57XcAvwrgMVj1sMH+tw0A7o2nhLFyq4P7AHzEnjV8LoBX6rpKE61pjPbXYX1WAKtOLrVnya4B8FYAD0ddvjDZY4/bADwhIn9ed1NqPydudZLWz4lSakgpdYL9+xIAvwJr3P/bAC6x/635M1L77FwC4F/sXpvOxT0zL8oLrJmfT8EahyjGXZ4YXv9psGZ5Pgrg+7U6gDXO8s8AngZwP4CVcZc15HrYDqtL7yisMaiPutUBrJmjt9qfme8BWBt3+SOsk7vs17zP3uicWPf/RbtOngRwYdzlD6E+fgFWN/g+AI/Yl/en+XPiUSep/JwA+GkA37Vf92MANtvXnwZrB+UZAN8A8Ab7+sX238/Yt5/W7XNz5TQiIiKDpKmrnIiIyHgMbiIiIoMwuImIiAzC4CYiIjIIg5uIiMggDG4iglIqW3d2p+frzvYkzWd1UkptUkptjausRGnH4CYiiMi0iLxTrOUbvwTgJvv3j8FarKjepbCO+yaiGDC4icjLPQB+re6cwnkAJwH4TpyFIkozBjcRuRKRl2Ct8nShfdWlAL4uXLmJKDYMbiJqZzuOd5ezm5woZgxuImrnXgDrlVJnA8iIyJ64C0SUZgxuIvIkIq/BOuPR7WBrmyh2DG4i8mM7gJ8Bg5sodjw7GBERkUHY4iYiIjIIg5uIiMggDG4iIiKDMLiJiIgMwuAmIiIyCIObiIjIIAxuIiIigzC4iYiIDPL/ASNn2/5bA32dAAAAAElFTkSuQmCC\n",
"text/plain": [
"<Figure size 576x360 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"plot_linear_regression(X, Y, 'TV', 'Sales', m_numpy, b_numpy, X_pred, Y_pred_numpy)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a name='2.2'></a>\n",
"### 2.2 - Linear Regression with `Scikit-Learn`"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"`Scikit-Learn` is an open-source machine learning library that supports supervised and unsupervised learning. It also provides various tools for model fitting, data preprocessing, model selection, model evaluation, and many other utilities. `Scikit-learn` provides dozens of built-in machine learning algorithms and models, called **estimators**. Each estimator can be fitted to some data using its `fit` method. Full documentation can be found [here](https://scikit-learn.org/stable/)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Create an estimator object for a linear regression model:"
]
},
{
"cell_type": "code",
"execution_count": 50,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": [
"lr_sklearn = LinearRegression()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The estimator can learn from data calling the `fit` function. However, trying to run the following code you will get an error, as the data needs to be reshaped into 2D array:"
]
},
{
"cell_type": "code",
"execution_count": 51,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Shape of X array: (200,)\n",
"Shape of Y array: (200,)\n",
"Expected 2D array, got 1D array instead:\n",
"array=[230.1 44.5 17.2 151.5 180.8 8.7 57.5 120.2 8.6 199.8 66.1 214.7\n",
" 23.8 97.5 204.1 195.4 67.8 281.4 69.2 147.3 218.4 237.4 13.2 228.3\n",
" 62.3 262.9 142.9 240.1 248.8 70.6 292.9 112.9 97.2 265.6 95.7 290.7\n",
" 266.9 74.7 43.1 228. 202.5 177. 293.6 206.9 25.1 175.1 89.7 239.9\n",
" 227.2 66.9 199.8 100.4 216.4 182.6 262.7 198.9 7.3 136.2 210.8 210.7\n",
" 53.5 261.3 239.3 102.7 131.1 69. 31.5 139.3 237.4 216.8 199.1 109.8\n",
" 26.8 129.4 213.4 16.9 27.5 120.5 5.4 116. 76.4 239.8 75.3 68.4\n",
" 213.5 193.2 76.3 110.7 88.3 109.8 134.3 28.6 217.7 250.9 107.4 163.3\n",
" 197.6 184.9 289.7 135.2 222.4 296.4 280.2 187.9 238.2 137.9 25. 90.4\n",
" 13.1 255.4 225.8 241.7 175.7 209.6 78.2 75.1 139.2 76.4 125.7 19.4\n",
" 141.3 18.8 224. 123.1 229.5 87.2 7.8 80.2 220.3 59.6 0.7 265.2\n",
" 8.4 219.8 36.9 48.3 25.6 273.7 43. 184.9 73.4 193.7 220.5 104.6\n",
" 96.2 140.3 240.1 243.2 38. 44.7 280.7 121. 197.6 171.3 187.8 4.1\n",
" 93.9 149.8 11.7 131.7 172.5 85.7 188.4 163.5 117.2 234.5 17.9 206.8\n",
" 215.4 284.3 50. 164.5 19.6 168.4 222.4 276.9 248.4 170.2 276.7 165.6\n",
" 156.6 218.5 56.2 287.6 253.8 205. 139.5 191.1 286. 18.7 39.5 75.5\n",
" 17.2 166.8 149.7 38.2 94.2 177. 283.6 232.1].\n",
"Reshape your data either using array.reshape(-1, 1) if your data has a single feature or array.reshape(1, -1) if it contains a single sample.\n"
]
}
],
"source": [
"print(f\"Shape of X array: {X.shape}\")\n",
"print(f\"Shape of Y array: {Y.shape}\")\n",
"\n",
"try:\n",
" lr_sklearn.fit(X, Y)\n",
"except ValueError as err:\n",
" print(err)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can increase the dimension of the array by one with `reshape` function, or there is another another way to do it:"
]
},
{
"cell_type": "code",
"execution_count": 52,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Shape of new X array: (200, 1)\n",
"Shape of new Y array: (200, 1)\n"
]
}
],
"source": [
"X_sklearn = X[:, np.newaxis]\n",
"Y_sklearn = Y[:, np.newaxis]\n",
"\n",
"print(f\"Shape of new X array: {X_sklearn.shape}\")\n",
"print(f\"Shape of new Y array: {Y_sklearn.shape}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a name='ex03'></a>\n",
"### Exercise 3\n",
"\n",
"Fit the linear regression model passing `X_sklearn` and `Y_sklearn` arrays into the function `lr_sklearn.fit`."
]
},
{
"cell_type": "code",
"execution_count": 53,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"data": {
"text/plain": [
"LinearRegression(copy_X=True, fit_intercept=True, n_jobs=None, normalize=False)"
]
},
"execution_count": 53,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"### START CODE HERE ### (~ 1 line of code)\n",
"lr_sklearn.fit(X_sklearn, Y_sklearn)\n",
"### END CODE HERE ###"
]
},
{
"cell_type": "code",
"execution_count": 54,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Linear regression using Scikit-Learn. Slope: [[0.04753664]]. Intercept: [7.03259355]\n"
]
}
],
"source": [
"m_sklearn = lr_sklearn.coef_\n",
"b_sklearn = lr_sklearn.intercept_\n",
"\n",
"print(f\"Linear regression using Scikit-Learn. Slope: {m_sklearn}. Intercept: {b_sklearn}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"##### __Expected Output__ \n",
"\n",
"```Python\n",
"Linear regression using Scikit-Learn. Slope: [[0.04753664]]. Intercept: [7.03259355]\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 67,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[92m All tests passed\n"
]
}
],
"source": [
"w2_unittest.test_sklearn_fit(lr_sklearn)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Note that you have got the same result as with the `NumPy` function `polyfit`. Now, to make predictions it is convenient to use `Scikit-Learn` function `predict`. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a name='ex04'></a>\n",
"### Exercise 4\n",
"\n",
"\n",
"Increase the dimension of the $X$ array using the function `np.newaxis` (see an example above) and pass the result to the `lr_sklearn.predict` function to make predictions."
]
},
{
"cell_type": "code",
"execution_count": 68,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": [
"# This is organised as a function only for grading purposes.\n",
"def pred_sklearn(X, lr_sklearn):\n",
" ### START CODE HERE ### (~ 2 lines of code)\n",
" X_2D = X[:, np.newaxis]\n",
" Y = lr_sklearn.predict(X_2D)\n",
" ### END CODE HERE ###\n",
" \n",
" return Y"
]
},
{
"cell_type": "code",
"execution_count": 69,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"TV marketing expenses:\n",
"[ 50 120 280]\n",
"Predictions of sales using Scikit_Learn linear regression:\n",
"[[ 9.40942557 12.7369904 20.34285287]]\n"
]
}
],
"source": [
"Y_pred_sklearn = pred_sklearn(X_pred, lr_sklearn)\n",
"\n",
"print(f\"TV marketing expenses:\\n{X_pred}\")\n",
"print(f\"Predictions of sales using Scikit_Learn linear regression:\\n{Y_pred_sklearn.T}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"##### __Expected Output__ \n",
"\n",
"```Python\n",
"TV marketing expenses:\n",
"[ 50 120 280]\n",
"Predictions of sales using Scikit_Learn linear regression:\n",
"[[ 9.40942557 12.7369904 20.34285287]]\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 70,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[92m All tests passed\n"
]
}
],
"source": [
"w2_unittest.test_sklearn_predict(pred_sklearn, lr_sklearn)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The predicted values are also the same."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a name='3'></a>\n",
"## 3 - Linear Regression using Gradient Descent"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Functions to fit the models automatically are convenient to use, but for an in-depth understanding of the model and the maths behind it is good to implement an algorithm by yourself. Let's try to find linear regression coefficients $m$ and $b$, by minimising the difference between original values $y^{(i)}$ and predicted values $\\hat{y}^{(i)}$ with the **loss function** $L\\left(w, b\\right) = \\frac{1}{2}\\left(\\hat{y}^{(i)} - y^{(i)}\\right)^2$ for each of the training examples. Division by $2$ is taken just for scaling purposes, you will see the reason below, calculating partial derivatives.\n",
"\n",
"To compare the resulting vector of the predictions $\\hat{Y}$ with the vector $Y$ of original values $y^{(i)}$, you can take an average of the loss function values for each of the training examples:\n",
"\n",
"$$E\\left(m, b\\right) = \\frac{1}{2n}\\sum_{i=1}^{n} \\left(\\hat{y}^{(i)} - y^{(i)}\\right)^2 = \n",
"\\frac{1}{2n}\\sum_{i=1}^{n} \\left(mx^{(i)}+b - y^{(i)}\\right)^2,\\tag{1}$$\n",
"\n",
"where $n$ is a number of data points. This function is called the sum of squares **cost function**. To use gradient descent algorithm, calculate partial derivatives as:\n",
"\n",
"\\begin{align}\n",
"\\frac{\\partial E }{ \\partial m } &= \n",
"\\frac{1}{n}\\sum_{i=1}^{n} \\left(mx^{(i)}+b - y^{(i)}\\right)x^{(i)},\\\\\n",
"\\frac{\\partial E }{ \\partial b } &= \n",
"\\frac{1}{n}\\sum_{i=1}^{n} \\left(mx^{(i)}+b - y^{(i)}\\right),\n",
"\\tag{2}\\end{align}\n",
"\n",
"and update the parameters iteratively using the expressions\n",
"\n",
"\\begin{align}\n",
"m &= m - \\alpha \\frac{\\partial E }{ \\partial m },\\\\\n",
"b &= b - \\alpha \\frac{\\partial E }{ \\partial b },\n",
"\\tag{3}\\end{align}\n",
"\n",
"where $\\alpha$ is the learning rate."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Original arrays `X` and `Y` have different units. To make gradient descent algorithm efficient, you need to bring them to the same units. A common approach to it is called **normalization**: substract the mean value of the array from each of the elements in the array and divide them by standard deviation (a statistical measure of the amount of dispersion of a set of values). If you are not familiar with mean and standard deviation, do not worry about this for now - this is covered in the next Course of Specialization.\n",
"\n",
"Normalization is not compulsory - gradient descent would work without it. But due to different units of `X` and `Y`, the cost function will be much steeper. Then you would need to take a significantly smaller learning rate $\\alpha$, and the algorithm will require thousands of iterations to converge instead of a few dozens. Normalization helps to increase the efficiency of the gradient descent algorithm.\n",
"\n",
"Normalization is implemented in the following code:"
]
},
{
"cell_type": "code",
"execution_count": 71,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": [
"X_norm = (X - np.mean(X))/np.std(X)\n",
"Y_norm = (Y - np.mean(Y))/np.std(Y)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Define cost function according to the equation $(1)$:"
]
},
{
"cell_type": "code",
"execution_count": 72,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": [
"def E(m, b, X, Y):\n",
" return 1/(2*len(Y))*np.sum((m*X + b - Y)**2)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a name='ex05'></a>\n",
"### Exercise 5\n",
"\n",
"\n",
"Define functions `dEdm` and `dEdb` to calculate partial derivatives according to the equations $(2)$. This can be done using vector form of the input data `X` and `Y`."
]
},
{
"cell_type": "code",
"execution_count": 90,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": [
"def dEdm(m, b, X, Y):\n",
" ### START CODE HERE ### (~ 1 line of code)\n",
" # Use the following line as a hint, replacing all None.\n",
" res = 1/len(X)*np.dot(m*X + b - Y, X)\n",
" ### END CODE HERE ###\n",
" \n",
" return res\n",
" \n",
"\n",
"def dEdb(m, b, X, Y):\n",
" ### START CODE HERE ### (~ 1 line of code)\n",
" # Replace None writing the required expression fully.\n",
" res = 1/len(Y)*np.sum(m*X + b - Y)\n",
" ### END CODE HERE ###\n",
" \n",
" return res\n"
]
},
{
"cell_type": "code",
"execution_count": 91,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"-0.7822244248616067\n",
"5.151434834260726e-16\n",
"0.21777557513839355\n",
"5.000000000000001\n"
]
}
],
"source": [
"print(dEdm(0, 0, X_norm, Y_norm))\n",
"print(dEdb(0, 0, X_norm, Y_norm))\n",
"print(dEdm(1, 5, X_norm, Y_norm))\n",
"print(dEdb(1, 5, X_norm, Y_norm))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"##### __Expected Output__ \n",
"\n",
"```Python\n",
"-0.7822244248616067\n",
"5.098005351200641e-16\n",
"0.21777557513839355\n",
"5.000000000000002\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 92,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[92m All tests passed\n"
]
}
],
"source": [
"w2_unittest.test_partial_derivatives(dEdm, dEdb, X_norm, Y_norm)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a name='ex06'></a>\n",
"### Exercise 6\n",
"\n",
"\n",
"Implement gradient descent using expressions $(3)$:\n",
"\\begin{align}\n",
"m &= m - \\alpha \\frac{\\partial E }{ \\partial m },\\\\\n",
"b &= b - \\alpha \\frac{\\partial E }{ \\partial b },\n",
"\\end{align}\n",
"\n",
"where $\\alpha$ is the `learning_rate`."
]
},
{
"cell_type": "code",
"execution_count": 96,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": [
"def gradient_descent(dEdm, dEdb, m, b, X, Y, learning_rate = 0.001, num_iterations = 1000, print_cost=False):\n",
" for iteration in range(num_iterations):\n",
" ### START CODE HERE ### (~ 2 lines of code)\n",
" m_new = m - learning_rate * dEdm(m, b, X, Y)\n",
" b_new = b - learning_rate * dEdb(m, b, X, Y)\n",
" ### END CODE HERE ###\n",
" m = m_new\n",
" b = b_new\n",
" if print_cost:\n",
" print (f\"Cost after iteration {iteration}: {E(m, b, X, Y)}\")\n",
" \n",
" return m, b"
]
},
{
"cell_type": "code",
"execution_count": 97,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"(0.49460408269589495, -3.4915181856831644e-16)\n",
"(0.9791767513915026, 4.521910375044022)\n"
]
}
],
"source": [
"print(gradient_descent(dEdm, dEdb, 0, 0, X_norm, Y_norm))\n",
"print(gradient_descent(dEdm, dEdb, 1, 5, X_norm, Y_norm, learning_rate = 0.01, num_iterations = 10))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"##### __Expected Output__ \n",
"\n",
"```Python\n",
"(0.49460408269589495, -3.489285249624889e-16)\n",
"(0.9791767513915026, 4.521910375044022)\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 98,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[92m All tests passed\n"
]
}
],
"source": [
"w2_unittest.test_gradient_descent(gradient_descent, dEdm, dEdb, X_norm, Y_norm)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now run the gradient descent method starting from the initial point $\\left(m_0, b_0\\right)=\\left(0, 0\\right)$."
]
},
{
"cell_type": "code",
"execution_count": 99,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Cost after iteration 0: 0.20629997559196597\n",
"Cost after iteration 1: 0.19455197461564464\n",
"Cost after iteration 2: 0.19408205457659178\n",
"Cost after iteration 3: 0.19406325777502967\n",
"Cost after iteration 4: 0.19406250590296714\n",
"Cost after iteration 5: 0.19406247582808467\n",
"Cost after iteration 6: 0.19406247462508938\n",
"Cost after iteration 7: 0.19406247457696957\n",
"Cost after iteration 8: 0.19406247457504477\n",
"Cost after iteration 9: 0.19406247457496775\n",
"Cost after iteration 10: 0.1940624745749647\n",
"Cost after iteration 11: 0.19406247457496456\n",
"Cost after iteration 12: 0.19406247457496456\n",
"Cost after iteration 13: 0.19406247457496456\n",
"Cost after iteration 14: 0.19406247457496456\n",
"Cost after iteration 15: 0.19406247457496456\n",
"Cost after iteration 16: 0.19406247457496456\n",
"Cost after iteration 17: 0.19406247457496456\n",
"Cost after iteration 18: 0.19406247457496456\n",
"Cost after iteration 19: 0.19406247457496456\n",
"Cost after iteration 20: 0.19406247457496456\n",
"Cost after iteration 21: 0.19406247457496456\n",
"Cost after iteration 22: 0.19406247457496456\n",
"Cost after iteration 23: 0.19406247457496456\n",
"Cost after iteration 24: 0.19406247457496456\n",
"Cost after iteration 25: 0.19406247457496456\n",
"Cost after iteration 26: 0.19406247457496456\n",
"Cost after iteration 27: 0.19406247457496456\n",
"Cost after iteration 28: 0.19406247457496456\n",
"Cost after iteration 29: 0.19406247457496456\n",
"Gradient descent result: m_min, b_min = 0.7822244248616068, -6.075140390748858e-16\n"
]
}
],
"source": [
"m_initial = 0; b_initial = 0; num_iterations = 30; learning_rate = 1.2\n",
"m_gd, b_gd = gradient_descent(dEdm, dEdb, m_initial, b_initial, \n",
" X_norm, Y_norm, learning_rate, num_iterations, print_cost=True)\n",
"\n",
"print(f\"Gradient descent result: m_min, b_min = {m_gd}, {b_gd}\") "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Remember, that the initial datasets were normalized. To make the predictions, you need to normalize `X_pred` array, calculate `Y_pred` with the linear regression coefficients `m_gd`, `b_gd` and then **denormalize** the result (perform the reverse process of normalization):"
]
},
{
"cell_type": "code",
"execution_count": 100,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"TV marketing expenses:\n",
"[ 50 120 280]\n",
"Predictions of sales using Scikit_Learn linear regression:\n",
"[[ 9.40942557 12.7369904 20.34285287]]\n",
"Predictions of sales using Gradient Descent:\n",
"[ 9.40942557 12.7369904 20.34285287]\n"
]
}
],
"source": [
"X_pred = np.array([50, 120, 280])\n",
"# Use the same mean and standard deviation of the original training array X\n",
"X_pred_norm = (X_pred - np.mean(X))/np.std(X)\n",
"Y_pred_gd_norm = m_gd * X_pred_norm + b_gd\n",
"# Use the same mean and standard deviation of the original training array Y\n",
"Y_pred_gd = Y_pred_gd_norm * np.std(Y) + np.mean(Y)\n",
"\n",
"print(f\"TV marketing expenses:\\n{X_pred}\")\n",
"print(f\"Predictions of sales using Scikit_Learn linear regression:\\n{Y_pred_sklearn.T}\")\n",
"print(f\"Predictions of sales using Gradient Descent:\\n{Y_pred_gd}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You should have gotten similar results as in the previous sections. \n",
"\n",
"Well done! Now you know how gradient descent algorithm can be applied to train a real model. Re-producing results manually for a simple case should give you extra confidence that you understand what happends under the hood of commonly used functions."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": []
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"collapsed_sections": [],
"name": "C1_W1_Assignment_Solution.ipynb",
"provenance": []
},
"coursera": {
"schema_names": [
"AI4MC1-1"
]
},
"grader_version": "1",
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.8"
},
"toc": {
"base_numbering": 1,
"nav_menu": {},
"number_sections": true,
"sideBar": true,
"skip_h1_title": false,
"title_cell": "Table of Contents",
"title_sidebar": "Contents",
"toc_cell": false,
"toc_position": {},
"toc_section_display": true,
"toc_window_display": false
},
"vscode": {
"interpreter": {
"hash": "478841ab876a4250505273c8a697bbc1b6b194054b009c227dc606f17fb56272"
}
}
},
"nbformat": 4,
"nbformat_minor": 1
}