ipynb • Lines: 1623{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "EAt-K2qgcIou"
},
"source": [
"# Single Perceptron Neural Networks for Linear Regression"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "FZYK-0rin5x7"
},
"source": [
"Welcome to your week 3 programming assignment. Now you are ready to apply matrix multiplication by building your first neural network with a single perceptron.\n",
"\n",
"**After this assignment you will be able to:**\n",
"- Implement a neural network with a single perceptron and one input node for simple linear regression\n",
"- Implement forward propagation using matrix multiplication\n",
"- Implement a neural network with a single perceptron and two input nodes for multiple linear regression\n",
"\n",
"*Note*: Backward propagation with the parameters update requires understanding of Calculus. It is discussed in details in the Course \"Calculus\" (Course 2 in the Specialization \"Mathematics for Machine Learning\"). In this assignment backward propagation and parameters update functions are hidden."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Table of Contents\n",
"\n",
"- [ 1 - Simple Linear Regression](#1)\n",
" - [ 1.1 - Simple Linear Regression Model](#1.1)\n",
" - [ 1.2 - Neural Network Model with a Single Perceptron and One Input Node](#1.2)\n",
" - [ 1.3 - Dataset](#1.3)\n",
" - [ Exercise 1](#ex01)\n",
"- [ 2 - Implementation of the Neural Network Model for Linear Regression](#2)\n",
" - [ 2.1 - Defining the Neural Network Structure](#2.1)\n",
" - [ Exercise 2](#ex02)\n",
" - [ 2.2 - Initialize the Model's Parameters](#2.2)\n",
" - [ Exercise 3](#ex03)\n",
" - [ 2.3 - The Loop](#2.3)\n",
" - [ Exercise 4](#ex04)\n",
" - [ 2.4 - Integrate parts 2.1, 2.2 and 2.3 in nn_model()](#2.4)\n",
" - [ Exercise 5](#ex05)\n",
"- [ 3 - Multiple Linear Regression](#3)\n",
" - [ 3.1 - Multipe Linear Regression Model](#3.1)\n",
" - [ 3.2 - Neural Network Model with a Single Perceptron and Two Input Nodes](#3.2)\n",
" - [ 3.3 - Dataset](#3.3)\n",
" - [ 3.4 - Performance of the Neural Network Model for Multiple Linear Regression](#3.4)\n",
" - [ Exercise 6](#ex06)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "XI8PBrk_2Z4V"
},
"source": [
"## Packages\n",
"\n",
"Let's first import all the packages that you will need during this assignment."
]
},
{
"cell_type": "code",
"execution_count": 56,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": [
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"# A function to create a dataset.\n",
"from sklearn.datasets import make_regression\n",
"# A library for data manipulation and analysis.\n",
"import pandas as pd\n",
"# Some functions defined specifically for this notebook.\n",
"import w3_tools\n",
"\n",
"# Output of plotting commands is displayed inline within the Jupyter notebook.\n",
"%matplotlib inline \n",
"\n",
"# Set a seed so that the results are consistent.\n",
"np.random.seed(3) "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Load the unit tests defined for this notebook."
]
},
{
"cell_type": "code",
"execution_count": 57,
"metadata": {},
"outputs": [],
"source": [
"import w3_unittest"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a name='1'></a>\n",
"## 1 - Simple Linear Regression\n",
"\n",
"**Linear regression** is a linear approach for modelling the relationship between a scalar response (**dependent variable**) and one or more explanatory variables (**independent variables**). The case of one independent variable is called **simple linear regression**; for more than one, it is called **multiple linear regression**. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a name='1.1'></a>\n",
"### 1.1 - Simple Linear Regression Model\n",
"\n",
"Simple linear regression model can be written as\n",
"\n",
"$$\\hat{y} = wx + b,\\tag{1}$$\n",
"\n",
"where $\\hat{y}$ is a prediction of dependent variable $y$ based on independent variable $x$ using a line equation with the slope $w$ and intercept $b$. \n",
"\n",
"Given a set of training data points $(x_1, y_1)$, ..., $(x_m, y_m)$, the aim is to find the \"best\" fitting line - such parameters $w$ and $b$ that the differences between original values $y_i$ and predicted values $\\hat{y}_i = wx_i + b$ are minimum.\n",
"\n",
"You can use a simple neural network model to do that. Vector algebra will be used in the core of the model!"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a name='1.2'></a>\n",
"### 1.2 - Neural Network Model with a Single Perceptron and One Input Node\n",
"\n",
"The simplest neural network model has only one **perceptron**. It takes some inputs and calculates the output value.\n",
"\n",
"The perceptron components are shown in the following scheme:\n",
"\n",
"<img src=\"images/nn_model_linear_regression_simple.png\" style=\"width:400px;\">\n",
"\n",
"The smallest construction block of neural networks is called a **node**. Some nodes store numbers from the input and others store the calculated values. **Input nodes** (here there is only one input node $x$) contain the input to the network which consists of your data. These nodes are set as an **input layer** of the network.\n",
"\n",
"**Weight** ($w$) and **bias** ($b$) are the parameters which will get updated when you will **train** the model. They are initialized to some random value or set to 0 and updated as the training progresses. The bias is analogous to a weight independent of any input node. It makes the model more flexible.\n",
"\n",
"The perceptron output calculation is straightforward: first compute the product of $x$ and weight $w$ and the add the bias:\n",
"\n",
"$$z = w x + b\\tag{2}$$\n",
"\n",
"The **output layer** of the single perceptron has only one node $\\hat{y} = z$.\n",
"\n",
"Putting it all together, mathematically the single perceptron neural network model can be expressed as:\n",
"\n",
"\\begin{align}\n",
"z^{(i)} &= w x^{(i)} + b,\\\\\n",
"\\hat{y}^{(i)} &= z^{(i)},\n",
"\\tag{3}\\end{align}\n",
"\n",
"where $x^{(i)}$ represents the $i$-th training example and $\\hat{y}^{(i)}$ will be the prediction based on that example, $i = 1, \\dots, m$.\n",
"\n",
"If you have $m$ training examples, vector operations will give you a chance to perform the calculations simultaneously for all of them! Organise all training examples as a vector $X$ of size ($1 \\times m$). Then perform scalar multiplication of $X$ ($1 \\times m$) by a scalar $w$, adding $b$, which will be broadcasted to the vector of size ($1 \\times m$):\n",
"\n",
"\\begin{align}\n",
"Z &= w X + b,\\\\\n",
"\\hat{Y} &= Z,\n",
"\\tag{4}\\end{align}\n",
"\n",
"This significantly speeds up the calculations for the larger training sets! This set of calculations is called **forward propagation**."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now, you can compare the resulting vector of the predictions $\\hat{Y}$ ($1 \\times m$) with the original vector of data $Y$. This can be done with the so called **cost function** that measures how close your vector of predictions is to the training data. It evaluates how well the parameters $w$ and $b$ work to solve the problem. There are many different cost functions available depending on the nature of your problem. For your simple neural network you can calculate it as:\n",
"\n",
"$$\\mathcal{L}\\left(w, b\\right) = \\frac{1}{2m}\\sum_{i=1}^{m} \\left(\\hat{y}^{(i)} - y^{(i)}\\right)^2.\\tag{5}$$\n",
"\n",
"The aim is to minimize the cost function during the training, which will minimize the differences between original values $y_i$ and predicted values $\\hat{y}_i$ (division by $2m$ is taken just for scaling purposes).\n",
"\n",
"When your weights were just initialized with some random values, and no training was done yet, you can't expect good results.\n",
"\n",
"The next step is to adjust the weights and bias, in order to minimize the cost function. This process is called **backward propagation** and is done iteratively: you update the parameters with a small change and repeat the process.\n",
"\n",
"*Note*: Backward propagation is not covered in this Course - it will be discussed in the next Course of this Specialization.\n",
"\n",
"The general **methodology** to build a neural network is to:\n",
"1. Define the neural network structure ( # of input units, # of hidden units, etc). \n",
"2. Initialize the model's parameters\n",
"3. Loop:\n",
" - Implement forward propagation (calculate the perceptron output),\n",
" - Implement backward propagation (to get the required corrections for the parameters),\n",
" - Update parameters.\n",
"4. Make predictions.\n",
"\n",
"You often build helper functions to compute steps 1-3 and then merge them into one function you call `nn_model()`. Once you've built `nn_model()` and learnt the right parameters, you can make predictions on new data."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a name='1.3'></a>\n",
"### 1.3 - Dataset\n",
"\n",
"First, let's get the dataset you will work on. The following code will create $m=30$ data points $(x_1, y_1)$, ..., $(x_m, y_m)$ and save them in `NumPy` arrays `X` and `Y` of a shape $(1 \\times m)$."
]
},
{
"cell_type": "code",
"execution_count": 58,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Training dataset X:\n",
"[[ 0.3190391 -1.07296862 0.86540763 -0.17242821 1.14472371 0.50249434\n",
" -2.3015387 -0.68372786 -0.38405435 -0.87785842 -2.06014071 -1.10061918\n",
" -1.09989127 1.13376944 1.74481176 -0.12289023 -0.93576943 1.62434536\n",
" 1.46210794 0.90159072 -0.7612069 0.53035547 -0.52817175 -0.26788808\n",
" 0.58281521 0.04221375 0.90085595 -0.24937038 -0.61175641 -0.3224172 ]]\n",
"Training dataset Y\n",
"[[ -3.01854669 -65.65047675 26.96755728 8.70562603 57.94332628\n",
" -0.69293498 -78.66594473 -12.73881492 -13.26721663 -24.80488085\n",
" -74.24484385 -39.99533724 -22.70174437 73.46766345 55.7257405\n",
" 23.80417646 -13.45481508 25.57952246 75.91238321 50.91155323\n",
" -43.7191551 -1.7025559 -16.44931235 -33.54041234 20.4505961\n",
" 18.35949302 37.69029586 -1.04801683 -4.47915933 -20.89431647]]\n"
]
}
],
"source": [
"m = 30\n",
"\n",
"X, Y = make_regression(n_samples=m, n_features=1, noise=20, random_state=1)\n",
"\n",
"X = X.reshape((1, m))\n",
"Y = Y.reshape((1, m))\n",
"\n",
"print('Training dataset X:')\n",
"print(X)\n",
"print('Training dataset Y')\n",
"print(Y)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Plot the dataset:"
]
},
{
"cell_type": "code",
"execution_count": 59,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"data": {
"text/plain": [
"Text(0, 0.5, '$y$')"
]
},
"execution_count": 59,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYkAAAEGCAYAAACQO2mwAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAYB0lEQVR4nO3df5AkZ13H8ffncgYcfpjEuySXHzuTlBcgiSDcGiOUGvNDYkzlYpWUZy3xStAtEBUoLUycKlFxq1Ao8BdIbYHUUbuSigrkiiKQuxO0qDKEDSSQI8Q7k91NvDPZQInKlodhv/4xvWHvMr07uzvdT/fM51U1NTPdPT3f6dub7zzP0/19FBGYmZl1syV1AGZmVl1OEmZmlstJwszMcjlJmJlZLicJMzPLtTV1AP20bdu2aLVaqcMwM6uV++6776mI2N5t3UAliVarxczMTOowzMxqRdJc3rpKdDdJequkw5IelPRRSc+VdJakA5KOZPdnpo7TzGzYJE8Sks4HfgsYjYjLgdOAPcCtwKGI2Akcyp6bmVmJkieJzFbg+yVtBRrAMWA3sC9bvw+4OU1oZmbDK3mSiIh/B94NzAPHgW9FxN3AORFxPNvmOHB2t9dLGpc0I2lmYWGhrLDNzIZC8iSRjTXsBi4CzgOeJ+m1vb4+IiYjYjQiRrdv7zo4b2ZmG5Q8SQDXAo9GxEJE/B/wMeCVwBOSdgBk908mjNHMbNOmp6dptVps2bKFVqvF9PR06pDWVIUkMQ9cKakhScA1wEPAfmBvts1e4M5E8ZmZbdr09DTj4+PMzc0REczNzTE+Pl75RKEqlAqX9IfALwJPA18GfhV4PnAHMEInkbwmIr652n5GR0fD10mYWRW1Wi3m5p59OUKz2WR2drb8gFaQdF9EjHZbV4WWBBHx9oh4cURcHhG3RMSJiPhGRFwTETuz+1UThJnZZhTdFTQ/P7+u5VVRiSRhZpZSGV1BIyMj61peFU4SZjb02u02i4uLJy1bXFyk3W737T0mJiZoNBonLWs0GkxMTPTtPYrgJGFmQ6+MrqCxsTEmJydpNptIotlsMjk5ydjYWN/eowiVGLjuFw9cm9lGVHlQuQyVH7g2M0uprl1BZXCSMLOhV9euoDK4u8nMbMi5u8nMzDbEScLMzHI5SZiZWS4nCTMzy+UkYWZmuZwkzMwsl5OEmZnlcpIwM7NcThJmZparEklC0hmS/l7S1yU9JOnHJZ0l6YCkI9n9manjNDMbNpVIEsCfA5+OiBcDL6Mzx/WtwKGI2Akcyp6bmVmJkicJSS8EfhL4EEBEfCci/hPYDezLNtsH3JwiPjOzYZY8SQAXAwvAhyV9WdIHJT0POCcijgNk92enDNLMBl/R81zXURWSxFbgFcBfR8TLgW+zjq4lSeOSZiTNLCwsFBWjmQ24Mua5rqPkpcIlnQvcExGt7PlP0EkSPwRcFRHHJe0APhcRL1ptXy4VbmYbNcyz01W6VHhE/AfwmKTlBHAN8DVgP7A3W7YXuDNBeGY2JMqY57oIRXeRbe3r3jbuN4FpSacDjwC/QieB3SHp9cA88JqE8ZnZgBsZGenakhgZGUkQTW+Wu8gWFxcBnukiA/o2q17y7qZ+cneTmW3UqV+40JnnusrTmPari6zS3U1mZlVQx3muy+gic0vCzKym3JIwM7NcExMTNBqNk5Y1Gg0mJib69h5OEmZWa8N8AVwZXWTubjKz2qrjYHMVubvJzAZSu90+KUEALC4u0m63E0U0eJwkzKy26noBXJ04SZhZbeVd6FblC+DqxknCzGqrjLN7hp2ThJnVVh0vgKsbn91kZjbkfHaTmZltiJOEmZnlcpIwM7NcThJmZpbLScLMzHI5SZiZWa7KJAlJp0n6sqRPZs/PknRA0pHs/szUMZqZDZvKJAngzcBDK57fChyKiJ3Aoey5mZmVqBJJQtIFwM8BH1yxeDewL3u8D7i55LDMzIZeJZIE8GfA24ClFcvOiYjjANn92d1eKGlc0oykmYWFhcIDNbNqGeZJh8qQPElIuhF4MiLu28jrI2IyIkYjYnT79u19js7Mqmx50qG5uTkigrm5OcbHx50o+ih5kgBeBdwkaRa4Hbha0hTwhKQdANn9k+lCNLMq8qRDxUueJCLitoi4ICJawB7gHyPitcB+YG+22V7gzkQhmllFedKh4iVPEqt4J3CdpCPAddlzM7NneNKh4lUqSUTE5yLixuzxNyLimojYmd1/M3V8ZlYtnnSoeJVKEmZm6+FJh4rnSYfMzIacJx0yM7MNcZIwM7NcThJmZpbLScLMzHI5SZiZWS4nCTMzy+UkYWa15Oqv5diaOgAzs/Varv66XNxvufor4Avp+swtCTOrnWGp/lqF1pJbEmZWO8NQ/bUqrSW3JMwqpAq/HOtgGKq/VqW15CRhVhGeZa13w1D9tSqtJScJs4qoyi/HOhiG6q9VaS25CqxZRWzZsoVu/x8lsbS0lCAiS+nUMQnotJaKSIauAmtWA1X55VgEj7WsX2VaSxGR9AZcCHwWeAg4DLw5W34WcAA4kt2fuda+du3aFWZ1NTU1FY1GI4Bnbo1GI6amplKHtimD+rkGCTATOd+rVWhJPA38dkS8BLgSeJOkS4FbgUMRsRM4lD03G1iV+eXYZx5rqbfkSSIijkfEl7LH/02nRXE+sBvYl222D7g5SYBmJRobG2N2dpalpSVmZ2d7ShBV78qpylk6tjHJk8RKklrAy4EvAOdExHHoJBLg7JzXjEuakTSzsLBQWqxmVVCH02YHeaxlGFQmSUh6PvAPwFsi4r96fV1ETEbEaESMbt++vbgAzSqoDl05w3BNwyCrRJKQ9H10EsR0RHwsW/yEpB3Z+h3Ak6niM6uqOnTlDOpYy7BIniQkCfgQ8FBEvGfFqv3A3uzxXuDOsmMzq7q6dOVsZKzFqiF5kgBeBdwCXC3p/ux2A/BO4DpJR4DrsudmtoK7cqxoyavARsTnAeWsvqbMWMzqZvkXebvdZn5+npGRESYmJvxL3frGZTnMzIacy3KYmdmGOEmYmVkuJwkzM8vlJGFmZrmcJMwSq3rtJRtuyU+BNRtmVZns3iyPWxJmCdWh9pINNycJs4TqUHvJhpuThFlCdam9ZMPLScIsIddesqpzkjBLyGW0repcu8nMbMi5dpOZmW2Ik4SZmeVykjAzs1yVTxKSrpf0sKSjkm5NHY+Z2TCpdJKQdBrwPuBngUuBX5J0adqozKwb16AaTJVOEsAVwNGIeCQivgPcDuxOHJPZhgzyl+hyDaq5uTkigrm5OW655RYkDdxnHTZVL/B3PvDYiuePAz+2cgNJ48A4+CpVq65BL+TXrQbV8un1g/ZZh82aLQlJByW9rIxgur19l2UnXdgREZMRMRoRo9u3by8pLLP1qVshv/W2etaqNVXlz2qr66W76W3AeyV9WNKOogM6xePAhSueXwAcKzkGs02rUyG/bl1H4+PjqyaKXlrxVfystrY1k0REfCkirgY+CXxa0tslfX/xoQHwRWCnpIsknQ7sAfaX9N42BMoaJ6hTIb+NtHq61aA6VRU/q/UgIta80en2uRx4A/AUnV/4t/Ty2s3egBuAfwX+DWivtu2uXbvCrFdTU1PRaDSCThdmANFoNGJqaqrW77VZkk6Kc/kmadXXTU1NRbPZfGbbOnxW6wBmIu87OG9FfO9L+vN0ungOAO8AbgR+CPhLYHKt15d5c5Kw9Vj+Qjv11mw2C3m/5S9RSdFsNiv7pdmP41KXz2odqyWJNQv8SbocOBxdNpT0UES8pNdWS9Fc4M/WY8uWLXT7+5fE0tJSgoiq4dQzsaBTvtzVaQfXpgr8RcSD3RJE5uc2FZlZQnUaJyiTy5fbSpu6mC4iHulXIGZl84Q/+cbGxpidnWVpaYnZ2VkniCFW9SuuzQrjX8xma/OkQ2ZmQ86TDpmZ2YY4SZiZWS4nCTMzy+UkYUOhyPIbg1wC3MxJwiqrX1++GylYV4V9m1WBz26ySurnVb+tVou5ublnLW82m8zOzm4qziL3bVaW1c5ucpKwSurnl2+R5Tdc2sMGgU+Btdrp5/wLGym/0WtXl0t72KBzkrBK6ueX73rLb6xnnMGlPWzg5ZWHrePNpcIHR7/nX1hP6er1lsp2WWyrOzZTKrxOPCYxWKanp2m328zPzzMyMsLExEQpdZU8zmDDxgPXZuvgM5Zs2FR24FrSuyR9XdJXJH1c0hkr1t0m6aikhyW9OmGYNmQ8zmD2PakHrg8Al0fES+nMY30bgKRLgT3AZcD1wPslnZYsShsqmy0h7iuwbZBsTfnmEXH3iqf3AL+QPd4N3B4RJ4BHJR0FrgD+peQQbUiNjY1taPzj1IsAl8+MWt6nWd2kbkms9Drgruzx+cBjK9Y9ni17FknjkmYkzSwsLBQcom3GMPzCbrfbJ10lDrC4uEi73U4UkdnmFN6SkHQQOLfLqnZE3Jlt0waeBpa/NdRl+64j7BExCUxCZ+B60wFbIYblF3Y/LwI0q4LCWxIRcW1EXN7ltpwg9gI3AmPxvVOtHgcuXLGbC4BjRcdqxRmWX9i+AtsGTeqzm64Hfhe4KSJWfoPsB/ZIeo6ki4CdwL0pYrT+6Ncv7Kp3WfnMKBs0qcck/gp4AXBA0v2SPgAQEYeBO4CvAZ8G3hQR300Xpm1WP35h16Es92bPjDKrGl9MZ6XoR+lvX+RmVozKXkxnw6Mfv7A9KGxWPrckrDbckjArhlsSNhA8KGxWPicJqw0PCpuVz91NZmZDzt1NZma2IU4SZmaWy0nCzMxyOUmYlaDq5UTM8iSdT8JsGAxLBVwbTG5JmBVsWCrg2mBykjArmMuJWJ05SZgVzHNMWJ05SZgVzOVErM6cJMwK5nIiVmcuy2FmNuQqX5ZD0u9ICknbViy7TdJRSQ9LenXK+Kw/fK2AWf0kv05C0oXAdcD8imWXAnuAy4DzgIOSLvEUpvXlawXM6qkKLYn3Am8DVvZ77QZuj4gTEfEocBS4IkVw1h++VsCsnpImCUk3Af8eEQ+csup84LEVzx/PlnXbx7ikGUkzCwsLBUVqm+VrBczqqfDuJkkHgXO7rGoDvwf8TLeXdVnWdYQ9IiaBSegMXG8wTCvYyMhI16lHfa2AWbUV3pKIiGsj4vJTb8AjwEXAA5JmgQuAL0k6l07L4cIVu7kAOFZ0rFacsq4V8OC4WZ9FRCVuwCywLXt8GfAA8Bw6ieQR4LS19rFr166w6pqamopmsxmSotlsxtTUVN/332g0gk6rM4BoNBp9fx+zQQPMRM73amWuk8haE6MR8VT2vA28DngaeEtE3LXWPnydxHBrtVpdu7SazSazs7PlB2RWE6tdJ1GZJNEPThLDbcuWLXT7e5bE0tJSgojM6qHyF9OZ9YML6Zn1n5OEDQwX0jPrPycJGxgupGfWfx6TMDMbch6TMDOzDXGSMDOzXE4SZmaWy0nCzMxyOUmYmVkuJwkzM8vlJGFmZrmcJMzMLJeThJmZ5XKSMDOzXE4SZmaWy0nCzMxyOUmYmVmu5ElC0m9KeljSYUl/umL5bZKOZutenTJGM7NhtTXlm0v6aWA38NKIOCHp7Gz5pcAe4DLgPOCgpEsi4rvpojUzGz6pWxJvBN4ZEScAIuLJbPlu4PaIOBERjwJHgSsSxWhmNrRSJ4lLgJ+Q9AVJ/yTpR7Pl5wOPrdju8WzZs0galzQjaWZhYaHgcM3Mhkvh3U2SDgLndlnVzt7/TOBK4EeBOyRdDKjL9l2n0IuISWASOjPT9SNmMzPrKDxJRMS1eeskvRH4WHTmUL1X0hKwjU7L4cIVm14AHCs0UDMze5bU3U2fAK4GkHQJcDrwFLAf2CPpOZIuAnYC96YK0sxsWKVOEn8DXCzpQeB2YG90HAbuAL4GfBp4k89sGhzT09O0Wi22bNlCq9Vieno6dUhmliPpKbAR8R3gtTnrJoCJciOyok1PTzM+Ps7i4iIAc3NzjI+PAzA2NpYyNDPrInVLwoZMu91+JkEsW1xcpN1uJ4rIzFbjJGGlmp+fX9dyM0vLScJKNTIysq7lZpaWk4SVamJigkajcdKyRqPBxISHn8yqyEnCSjU2Nsbk5CTNZhNJNJtNJicnPWhtVlHqXMc2GEZHR2NmZiZ1GGZmtSLpvogY7bbOLQkzM8vlJGFmZrmcJMzMLJeTRB+53ISZDZqkZTkGictNmNkgckuiT1xuwswGkZNEn7jchJkNIieJPnG5CTMbRE4SfeJyE2Y2iJwk6M9ZSS43YWaDKGlZDkk/AnwAeC7wNPDrEXFvtu424PXAd4HfiojPrLW/jZTlOPWsJOi0APwFb2bDYrWyHKmTxN3AeyPiLkk3AG+LiKskXQp8FLgCOA84CFyy1hSmG0kSrVaLubm5Zy1vNpvMzs6ua19mZnVU5dpNAbwwe/wDwLHs8W7g9og4ERGPAkfpJIy+81lJZmb5Ul9M9xbgM5LeTSdhvTJbfj5wz4rtHs+W9d3IyEjXloTPSjIzK6ElIemgpAe73HYDbwTeGhEXAm8FPrT8si676tovJmlc0oykmYWFhXXH57OSzMzyFd6SiIhr89ZJ+gjw5uzp3wEfzB4/Dly4YtML+F5X1Kn7nwQmoTMmsd74lgen2+028/PzjIyMMDEx4UFrMzPSdzcdA34K+BxwNXAkW74f+FtJ76EzcL0TuLeoIMbGxpwUzMy6SJ0kfg34c0lbgf8FxgEi4rCkO4Cv0Tk19k1rndlkZmb9lzRJRMTngV056yYADwyYmSWU+hRYMzOrMCcJMzPL5SRhZma5kpbl6DdJC8C3gadSx3KKbVQvJqhmXFWMCRzXelQxJqhmXFWJqRkR27utGKgkASBpJq8GSSpVjAmqGVcVYwLHtR5VjAmqGVcVYzqVu5vMzCyXk4SZmeUaxCQxmTqALqoYE1QzrirGBI5rPaoYE1QzrirGdJKBG5MwM7P+GcSWhJmZ9YmThJmZ5ap9kpD0Lklfl/QVSR+XdEbOdrOSvirpfknrm+O0uJiul/SwpKOSbi0ypuz9XiPpsKQlSbmn3ZV8rHqNqexjdZakA5KOZPdn5mxX+LFa67Or4y+y9V+R9Ioi4thAXFdJ+lZ2bO6X9PslxPQ3kp6U9GDO+lTHaq24Sj9WPYuIWt+AnwG2Zo//BPiTnO1mgW1ViQk4Dfg34GLgdOAB4NKC43oJ8CI6pdlHV9muzGO1ZkyJjtWfArdmj29N9XfVy2cHbgDuojNZ15XAF0r4d+slrquAT5bxd7TiPX8SeAXwYM760o9Vj3GVfqx6vdW+JRERd0fE09nTe+hMUJRUjzFdARyNiEci4jvA7XTm9i4yroci4uEi32O9eoyp9GOV7X9f9ngfcHPB75enl8++G/hIdNwDnCFpRwXiKl1E/DPwzVU2SXGseomrsmqfJE7xOjq/EroJ4G5J90kar0BM5wOPrXhe2DzeG5DqWOVJcazOiYjjANn92TnbFX2sevnsKY5Pr+/545IekHSXpMsKjqkXVf5/V7VjBaSfdKgnkg4C53ZZ1Y6IO7Nt2nQmKJrO2c2rIuKYpLOBA5K+nmX3VDH1PI93v+PqQenHaq1ddFlW6LFax276eqy66OWzF3J81tDLe36JTk2g/5F0A/AJOrNMppTiWPWiiscKqEmSiFXmyQaQtBe4Ebgmsg6+Lvs4lt0/KenjdJrLG/7P3IeYep7Hu59x9biPUo9VD0o/VpKekLQjIo5n3RFP5uyjr8eqi14+eyHHZ7NxRcR/rXj8KUnvl7QtIlIWtEtxrNZU0WMFDEB3k6Trgd8FboqIxZxtnifpBcuP6Qwsdz3LoKyYgC8COyVdJOl0YA+dub2TKvtY9SjFsdoP7M0e7wWe1eIp6Vj18tn3A7+cnblzJfCt5a6yAq0Zl6RzJSl7fAWd75tvFBzXWlIcqzVV9Fh1pB453+wNOEqnj/H+7PaBbPl5wKeyxxfTOfviAeAwnW6OpDFlz28A/pXOWSKFxpS938/T+SV1AngC+EwFjtWaMSU6Vj8IHAKOZPdnpTpW3T478AbgDdljAe/L1n+VVc5cKzmu38iOywN0TuB4ZQkxfRQ4Dvxf9nf1+oocq7XiKv1Y9XpzWQ4zM8tV++4mMzMrjpOEmZnlcpIwM7NcThJmZpbLScLMzHI5SZiZWS4nCTMzy+UkYVYwSZ+VdF32+I8l/UXqmMx6VYvaTWY193bgj7IigC8Hbkocj1nPfMW1WQkk/RPwfOCqiPjv1PGY9crdTWYFk/TDwA7ghBOE1Y2ThFmBsjLj03RmRPu2pFcnDslsXZwkzAoiqQF8DPjtiHgIeAfwB0mDMlsnj0mYmVkutyTMzCyXk4SZmeVykjAzs1xOEmZmlstJwszMcjlJmJlZLicJMzPL9f/Au2aw0HQ/2wAAAABJRU5ErkJggg==\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"plt.scatter(X, Y, c=\"black\")\n",
"\n",
"plt.xlabel(\"$x$\")\n",
"plt.ylabel(\"$y$\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a name='ex01'></a>\n",
"### Exercise 1\n",
"\n",
"What is the `shape` of the variables `X` and `Y`? In addition, how many training examples do you have?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<details> \n",
"<summary>\n",
" <font size=\"3\" color=\"darkgreen\"><b>Hint</b></font>\n",
"</summary>\n",
"<p>\n",
"<ul>\n",
" <li>How do you get the shape of a NumPy array?</li>\n",
" <li>The coordinates x1, x2 were saved in the columns of the array X</li>\n",
"</ul>\n",
"</p>"
]
},
{
"cell_type": "code",
"execution_count": 73,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"The shape of X: (1, 30)\n",
"The shape of Y: (1, 30)\n",
"I have m = 30 training examples!\n"
]
}
],
"source": [
"### START CODE HERE ### (~ 3 lines of code)\n",
"# Shape of variable X.\n",
"shape_X = X.shape\n",
"# Shape of variable Y.\n",
"shape_Y = Y.shape\n",
"# Training set size.\n",
"m = shape_Y[1]\n",
"### END CODE HERE ###\n",
"\n",
"print ('The shape of X: ' + str(shape_X))\n",
"print ('The shape of Y: ' + str(shape_Y))\n",
"print ('I have m = %d training examples!' % (m))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"##### __Expected Output__\n",
"\n",
"```Python\n",
"The shape of X: (1, 30)\n",
"The shape of Y: (1, 30)\n",
"I have m = 30 training examples!\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 74,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[92m All tests passed\n"
]
}
],
"source": [
"w3_unittest.test_shapes(shape_X, shape_Y, m)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a name='2'></a>\n",
"## 2 - Implementation of the Neural Network Model for Linear Regression\n",
"\n",
"Let's setup the neural network in a way which will allow to extend this simple case of a model to more complicated structures later."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a name='2.1'></a>\n",
"### 2.1 - Defining the Neural Network Structure"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a name='ex02'></a>\n",
"### Exercise 2\n",
"\n",
"Define two variables:\n",
"- `n_x`: the size of the input layer\n",
"- `n_y`: the size of the output layer"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<details> \n",
"<summary>\n",
" <font size=\"3\" color=\"darkgreen\"><b>Hint</b></font>\n",
"</summary>\n",
"<p>\n",
"<ul>\n",
" Use shapes of X and Y to find n_x and n_y:\n",
" <li>the size of the input layer n_x equals to the size of the input vectors placed in the columns of the array X,</li>\n",
" <li>the outpus for each of the data point will be saved in the columns of the the array Y.</li>\n",
"</ul>\n",
"</p>"
]
},
{
"cell_type": "code",
"execution_count": 75,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": [
"# GRADED FUNCTION: layer_sizes\n",
"\n",
"def layer_sizes(X, Y):\n",
" \"\"\"\n",
" Arguments:\n",
" X -- input dataset of shape (input size, number of examples)\n",
" Y -- labels of shape (output size, number of examples)\n",
" \n",
" Returns:\n",
" n_x -- the size of the input layer\n",
" n_y -- the size of the output layer\n",
" \"\"\"\n",
" ### START CODE HERE ### (~ 2 lines of code)\n",
" # Size of input layer.\n",
" n_x = X.shape[0]\n",
" # Size of output layer.\n",
" n_y = Y.shape[0]\n",
" ### END CODE HERE ###\n",
" return (n_x, n_y)"
]
},
{
"cell_type": "code",
"execution_count": 76,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"The size of the input layer is: n_x = 1\n",
"The size of the output layer is: n_y = 1\n"
]
}
],
"source": [
"(n_x, n_y) = layer_sizes(X, Y)\n",
"print(\"The size of the input layer is: n_x = \" + str(n_x))\n",
"print(\"The size of the output layer is: n_y = \" + str(n_y))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"##### __Expected Output__\n",
"\n",
"```Python\n",
"The size of the input layer is: n_x = 1\n",
"The size of the output layer is: n_y = 1\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 77,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[92m All tests passed\n"
]
}
],
"source": [
"w3_unittest.test_layer_sizes(layer_sizes)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a name='2.2'></a>\n",
"### 2.2 - Initialize the Model's Parameters"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a name='ex03'></a>\n",
"### Exercise 3\n",
"\n",
"Implement the function `initialize_parameters()`.\n",
"\n",
"**Instructions**:\n",
"- Make sure your parameters' sizes are right. Refer to the neural network figure above if needed.\n",
"- You will initialize the weights matrix with random values. \n",
" - Use: `np.random.randn(a,b) * 0.01` to randomly initialize a matrix of shape (a,b).\n",
"- You will initialize the bias vector as zeros. \n",
" - Use: `np.zeros((a,b))` to initialize a matrix of shape (a,b) with zeros."
]
},
{
"cell_type": "code",
"execution_count": 81,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": [
"# GRADED FUNCTION: initialize_parameters\n",
"\n",
"def initialize_parameters(n_x, n_y):\n",
" \"\"\"\n",
" Returns:\n",
" params -- python dictionary containing your parameters:\n",
" W -- weight matrix of shape (n_y, n_x)\n",
" b -- bias value set as a vector of shape (n_y, 1)\n",
" \"\"\"\n",
" \n",
" ### START CODE HERE ### (~ 2 lines of code)\n",
" W = np.zeros((n_y,n_x)) * 0.01\n",
" b = np.zeros((n_y,1))\n",
" ### END CODE HERE ###\n",
" \n",
" assert (W.shape == (n_y, n_x))\n",
" assert (b.shape == (n_y, 1))\n",
" \n",
" parameters = {\"W\": W,\n",
" \"b\": b}\n",
" \n",
" return parameters"
]
},
{
"cell_type": "code",
"execution_count": 82,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"W = [[0.]]\n",
"b = [[0.]]\n"
]
}
],
"source": [
"parameters = initialize_parameters(n_x, n_y)\n",
"print(\"W = \" + str(parameters[\"W\"]))\n",
"print(\"b = \" + str(parameters[\"b\"]))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"##### __Expected Output__ \n",
"Note: the elements of the array W maybe be different due to random initialization. You can try to restart the kernel to get the same values.\n",
"\n",
"```Python\n",
"W = [[0.01788628]]\n",
"b = [[0.]]\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 83,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[92m All tests passed\n"
]
}
],
"source": [
"# Note: \n",
"# Actual values are not checked here in the unit tests (due to random initialization).\n",
"w3_unittest.test_initialize_parameters(initialize_parameters)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a name='2.3'></a>\n",
"### 2.3 - The Loop"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a name='ex04'></a>\n",
"### Exercise 4\n",
"\n",
"Implement `forward_propagation()`.\n",
"\n",
"**Instructions**:\n",
"- Look at the mathematical representation of your model $(4)$ in the section [1.2](#1.2):\n",
"\\begin{align}\n",
"Z &= w X + b\\\\\n",
"\\hat{Y} &= Z,\n",
"\\end{align}\n",
"- The steps you have to implement are:\n",
" 1. Retrieve each parameter from the dictionary \"parameters\" (which is the output of `initialize_parameters()`) by using `parameters[\"..\"]`.\n",
" 2. Implement Forward Propagation. Compute `Z` multiplying arrays `w`, `X` and adding vector `b`. Set the prediction array $A$ equal to $Z$. \n"
]
},
{
"cell_type": "code",
"execution_count": 84,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": [
"# GRADED FUNCTION: forward_propagation\n",
"\n",
"def forward_propagation(X, parameters, n_y):\n",
" \"\"\"\n",
" Argument:\n",
" X -- input data of size (n_x, m)\n",
" parameters -- python dictionary containing your parameters (output of initialization function)\n",
" \n",
" Returns:\n",
" Y_hat -- The output of size (n_y, m)\n",
" \"\"\"\n",
" # Retrieve each parameter from the dictionary \"parameters\".\n",
" ### START CODE HERE ### (~ 2 lines of code)\n",
" W = parameters[\"W\"]\n",
" b = parameters[\"b\"]\n",
" ### END CODE HERE ###\n",
" \n",
" # Implement Forward Propagation to calculate Z.\n",
" ### START CODE HERE ### (~ 2 lines of code)\n",
" Z = np.dot(W, X) + b\n",
" Y_hat = Z\n",
" ### END CODE HERE ###\n",
" \n",
" assert(Y_hat.shape == (n_y, X.shape[1]))\n",
"\n",
" return Y_hat"
]
},
{
"cell_type": "code",
"execution_count": 85,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
" 0. 0. 0. 0. 0. 0.]]\n"
]
}
],
"source": [
"Y_hat = forward_propagation(X, parameters, n_y)\n",
"\n",
"print(Y_hat)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"##### __Expected Output__ \n",
"Note: the elements of the array Y_hat maybe be different depending on the initial parameters. If you would like to get exactly the same output, try to restart the Kernel and rerun the notebook.\n",
"\n",
"```Python\n",
"[[ 0.00570642 -0.01919142 0.01547893 -0.0030841 0.02047485 0.00898776\n",
" -0.04116598 -0.01222935 -0.00686931 -0.01570163 -0.03684826 -0.01968599\n",
" -0.01967297 0.02027892 0.0312082 -0.00219805 -0.01673744 0.0290535\n",
" 0.02615168 0.01612611 -0.01361516 0.00948609 -0.00944703 -0.00479152\n",
" 0.0104244 0.00075505 0.01611297 -0.00446031 -0.01094205 -0.00576685]]\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 86,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[92m All tests passed\n"
]
}
],
"source": [
"w3_unittest.test_forward_propagation(forward_propagation)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Remember that your weights were just initialized with some random values, so the model has not been trained yet. \n",
"\n",
"Define a cost function $(5)$ which will be used to train the model:\n",
"\n",
"$$\\mathcal{L}\\left(w, b\\right) = \\frac{1}{2m}\\sum_{i=1}^{m} \\left(\\hat{y}^{(i)} - y^{(i)}\\right)^2$$"
]
},
{
"cell_type": "code",
"execution_count": 87,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": [
"def compute_cost(Y_hat, Y):\n",
" \"\"\"\n",
" Computes the cost function as a sum of squares\n",
" \n",
" Arguments:\n",
" Y_hat -- The output of the neural network of shape (n_y, number of examples)\n",
" Y -- \"true\" labels vector of shape (n_y, number of examples)\n",
" \n",
" Returns:\n",
" cost -- sum of squares scaled by 1/(2*number of examples)\n",
" \n",
" \"\"\"\n",
" # Number of examples.\n",
" m = Y.shape[1]\n",
"\n",
" # Compute the cost function.\n",
" cost = np.sum((Y_hat - Y)**2)/(2*m)\n",
" \n",
" return cost"
]
},
{
"cell_type": "code",
"execution_count": 88,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"cost = 790.8692082867186\n"
]
}
],
"source": [
"print(\"cost = \" + str(compute_cost(Y_hat, Y)))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You want to minimize the cost value, bringing it as close as possible to $0$, making your vector of predictions as similar to the training data as possible.\n",
"\n",
"To achieve this, backward propagation needs to be performed. It is covered in details in the Course \"Calculus\" (Course 2 in the Specialization \"Mathematics for Machine Learning\"). For now you can use a function `train_nn()` from the uploaded toolbox to get the updated parameters in each step of the loop."
]
},
{
"cell_type": "code",
"execution_count": 89,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"W = [[43.63771442]]\n",
"b = [[0.17797798]]\n"
]
}
],
"source": [
"parameters = w3_tools.train_nn(parameters, Y_hat, X, Y)\n",
"\n",
"print(\"W = \" + str(parameters[\"W\"]))\n",
"print(\"b = \" + str(parameters[\"b\"]))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a name='2.4'></a>\n",
"### 2.4 - Integrate parts 2.1, 2.2 and 2.3 in nn_model()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a name='ex05'></a>\n",
"### Exercise 5\n",
"\n",
"Build your neural network model in `nn_model()`.\n",
"\n",
"**Instructions**: The neural network model has to use the previous functions in the right order."
]
},
{
"cell_type": "code",
"execution_count": 90,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": [
"# GRADED FUNCTION: nn_model\n",
"\n",
"def nn_model(X, Y, num_iterations=10, print_cost=False):\n",
" \"\"\"\n",
" Arguments:\n",
" X -- dataset of shape (n_x, number of examples)\n",
" Y -- labels of shape (n_y, number of examples)\n",
" num_iterations -- number of iterations in the loop\n",
" print_cost -- if True, print the cost every iteration\n",
" \n",
" Returns:\n",
" parameters -- parameters learnt by the model. They can then be used to make predictions.\n",
" \"\"\"\n",
" \n",
" n_x = layer_sizes(X, Y)[0]\n",
" n_y = layer_sizes(X, Y)[1]\n",
" \n",
" # Initialize parameters\n",
" ### START CODE HERE ### (~ 1 line of code)\n",
" parameters = initialize_parameters(n_x, n_y)\n",
" ### END CODE HERE ###\n",
" \n",
" # Loop\n",
" for i in range(0, num_iterations):\n",
" \n",
" ### START CODE HERE ### (~ 2 lines of code)\n",
" # Forward propagation. Inputs: \"X, parameters, n_y\". Outputs: \"Y_hat\".\n",
" Y_hat = forward_propagation(X, parameters, n_y)\n",
" \n",
" # Cost function. Inputs: \"Y_hat, Y\". Outputs: \"cost\".\n",
" cost = compute_cost(Y_hat, Y)\n",
" ### END CODE HERE ###\n",
" \n",
" \n",
" # Parameters update.\n",
" parameters = w3_tools.train_nn(parameters, Y_hat, X, Y) \n",
" \n",
" # Print the cost every iteration.\n",
" if print_cost:\n",
" print (\"Cost after iteration %i: %f\" %(i, cost))\n",
"\n",
" return parameters"
]
},
{
"cell_type": "code",
"execution_count": 91,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Cost after iteration 0: 790.869208\n",
"Cost after iteration 1: 176.498358\n",
"Cost after iteration 2: 143.769946\n",
"Cost after iteration 3: 141.433422\n",
"Cost after iteration 4: 141.248729\n",
"Cost after iteration 5: 141.233727\n",
"Cost after iteration 6: 141.232500\n",
"Cost after iteration 7: 141.232400\n",
"Cost after iteration 8: 141.232391\n",
"Cost after iteration 9: 141.232391\n",
"Cost after iteration 10: 141.232391\n",
"Cost after iteration 11: 141.232391\n",
"Cost after iteration 12: 141.232391\n",
"Cost after iteration 13: 141.232391\n",
"Cost after iteration 14: 141.232391\n",
"W = [[35.71958208]]\n",
"b = [[2.2893077]]\n"
]
}
],
"source": [
"parameters = nn_model(X, Y, num_iterations=15, print_cost=True)\n",
"print(\"W = \" + str(parameters[\"W\"]))\n",
"print(\"b = \" + str(parameters[\"b\"]))\n",
"\n",
"W_simple = parameters[\"W\"]\n",
"b_simple = parameters[\"b\"]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"##### __Expected Output__ \n",
"Note: the actual values can be different!\n",
"\n",
"```Python\n",
"Cost after iteration 0: 791.431703\n",
"Cost after iteration 1: 176.530000\n",
"Cost after iteration 2: 143.772255\n",
"Cost after iteration 3: 141.433606\n",
"Cost after iteration 4: 141.248744\n",
"Cost after iteration 5: 141.233728\n",
"Cost after iteration 6: 141.232500\n",
"Cost after iteration 7: 141.232400\n",
"Cost after iteration 8: 141.232391\n",
"Cost after iteration 9: 141.232391\n",
"Cost after iteration 10: 141.232391\n",
"Cost after iteration 11: 141.232391\n",
"Cost after iteration 12: 141.232391\n",
"Cost after iteration 13: 141.232391\n",
"Cost after iteration 14: 141.232391\n",
"W = [[35.71958208]]\n",
"b = [[2.2893077]]\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 92,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[92m All tests passed\n"
]
}
],
"source": [
"# Note: \n",
"# Actual values are not checked here in the unit tests (due to random initialization).\n",
"w3_unittest.test_nn_model(nn_model)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can see that after a few iterations the cost function does not change anymore (the model converges).\n",
"\n",
"*Note*: This is a very simple model. In reality the models do not converge that quickly.\n",
"\n",
"The final model parameters can be used for making predictions. Let's plot the linear regression line and some predictions. The regression line is red and the predicted points are blue."
]
},
{
"cell_type": "code",
"execution_count": 93,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYkAAAEGCAYAAACQO2mwAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAkf0lEQVR4nO3de5yUdfn/8dcFGrlZv+wrKqk7Y2UC4ilWS+1nKpiHTDpooVuSh9bIMos84JTnNdPykFowaUjtlplaUOYBETyUgaupISuIuruSJAso1pc4udf3j88sDDCzJ2bmvmf2/Xw85jE79z1zz+Woc83nvu7P9TF3R0REJJcBUQcgIiLxpSQhIiJ5KUmIiEheShIiIpKXkoSIiOS1TdQBFNKOO+7oyWQy6jBERMrKU089tczdB+faV1FJIplM0tTUFHUYIiJlxcxa8+2LxekmM/u2mT1vZvPM7Ddm9k4ze5+ZzTCzFzP3O0Qdp4hIfxN5kjCzXYFzgBp3HwEMBMYCFwIz3X1PYGbmsYiIlFDkSSJjG2A7M9sGqAJeA8YAUzP7pwKfiSY0EZH+K/Ik4e7/BH4EtAFLgJXu/iCws7svyTxnCbBTrtebWZ2ZNZlZU3t7e6nCFhHpFyJPEplawxhgD+D9wLvM7Es9fb27p929xt1rBg/OWZwXEZE+ijxJAKOBV9y93d3XAfcAhwCvm9kQgMz90ghjFBHJqbGxkWQyyYABA0gmkzQ2NkYdUkHFIUm0AR8zsyozM2AU0AxMB8ZlnjMOmBZRfCIiOTU2NlJXV0drayvuTmtrK3V1dRWVKCwOrcLN7DLgi8B64O/AmcD2wJ1ANSGRnOTuK7o6Tk1NjWuehIiUSjKZpLV1yykGiUSClpaW0gfUR2b2lLvX5NoXh5EE7n6Juw919xHu/mV3X+Puy919lLvvmbnvMkGIiORSzNNBbW1tvdpejmKRJEREiqHYp4Oqq6t7tb0cKUmISMVKpVKsWrVqk22rVq0ilUoV5Pj19fVUVVVtsq2qqor6+vqCHD8OlCREpGIV+3RQbW0t6XSaRCKBmZFIJEin09TW1hbk+HEQi8J1oahwLSLZKqWwXGyxL1yLiBRDfzgdVGxKEiJSsfrD6SAAXn8dFi8uyqF1uklEpFx1dMBtt8H558PHPw5//GOfDqPTTSIileb55+Gww6CuDvbbD669tihvoyQhIlJO/vtfSKVg//2huRmmTIFZs2Do0KK8XUUtXyoiUtFmzIDx4+Gll2DcuDB6KHL3a40kRETibulS+NKX4JOfhAEDYOZMuP32oicIUJIQEYmvjg649dZwKunOO+Hii+G55+DII0sWgk43iYjE0fz5cNZZ8PjjoUA9aRIMG1byMDSSEBGJk//+F773vVCYnj8ffvELmD07kgQBGkmIiMRHdmH61FPhRz8qSd2hKxpJiIhELVdheurUyBMExCRJmNl7zewuM3vBzJrN7GAze5+ZzTCzFzP3O0Qdp4hIQW1emP7+90temO5OLJIEcCNwv7sPBfYjrHF9ITDT3fcEZmYei4hUhvnz4ROfgK9+FfbZB559Fi6/HN75zqgj20TkScLM3gMcBtwG4O5r3f1NYAwwNfO0qcBnoohPRKSgYlaY7k7kSQL4ANAOTDGzv5vZrWb2LmBnd18CkLnfKdeLzazOzJrMrKm9vb10UYtI2SjmOte98tBDsO++UF8PJ58ML7wAp50GZtHE0wNxSBLbAB8BfubuBwD/Sy9OLbl72t1r3L1mcAyKPCISL8Ve57pHOgvTRx0VEkKMCtPdiUOSWAwsdvc5mcd3EZLG62Y2BCBzvzSi+ESkjBV7nesuxWDG9NaKPEm4+7+AV81sr8ymUcB8YDowLrNtHDAtgvBEpMwVe53rbI2NkEyGq1iT719L4/ArNy1MX3ZZjwrTsTk9Rnwm030TaDSzdwAvA6cREtidZnYG0AacFGF8IlKmqqurc65zXV1dXdD3aWwMSzt0Dlpal7yDuiXfhTMPo3byYSFz9Og44fRY5+in8/QYEMmKelqZTkQq2uZfuhDWuS70MqbJJOTIRSQS0NLSm+Mkcya1RCJBS28O1AtamU5E+q2SrHO9dCltrbl/cPf2rFYpT4/1hJKEiFS82tpaWlpa6OjooKWlpXAJonON6aFDqSb3l3hvz2rlOw1W6NNjPaUkISLSF83NcPjhcOaZMGIE9ddsQ1XVpk+pqgpTInqjvr6eqs0OVFVVRX1vD1QgShIiEmtxutIHgNWrw6Ws++0H8+aFS1xnz6b2vF1Jp0MNwizcp9PQ20FLSU6P9YIK1yISW6UqOvfYzJnwta/BokXh2/+662CnnM0gyooK1yJSliKdCJetvT2s7zB6dHg8YwY0NFREguiOkoSIxFbkV/pkFaa5447QmO+55zYmi35ASUJEYivSK32yC9PDh8Mzz8AVV8B22xX/vWNESUJEYiuSK31yFaYfeSQkin5ISUJEYqvkV/rMnBn6LF1xBXzhC6GV9xln9LilRiXS1U0iIu3tMGEC/OpX8MEPwqRJ/aruoKubRERycQ8rw2UXpv/xj36VILoTly6wIiKl1dwc5jw8+ih8/OMweXK/rTt0RSMJEelfsgvTzz0HP/95vy5Md0cjCRHpP2bOhPHj4cUXw4zpH/8Ydt456qhiTSMJEal87e0wblyoNXR0wIMPhhnTShDdUpIQkcrlDlOmhML0r38NqVQoTB91VNSRlY3YJAkzG2hmfzezP2Uev8/MZpjZi5n7HaKOUUTKyAsvwBFHwOmnw7BhYcb0lVf2uxnTWys2SQL4FtCc9fhCYKa77wnMzDwWEena6tVwySWw777w7LOhX/ejj8Lee0cdWVmKRZIws92ATwG3Zm0eA0zN/D0V+EyJwxKRcvPwwyE5XH75xhnTX/1qv54xvbXi8sndAJwPdGRt29ndlwBk7nP25DWzOjNrMrOm9vb2ogcqIqXVo0WHOgvTo0apMF1gkScJMzseWOruT/Xl9e6edvcad68ZPHhwgaMTkSh1LjrU2tqKu9Pa2kpdXd3GRNFZmB42LBSmL7pIhekCizxJAIcCJ5hZC3AHcKSZNQCvm9kQgMz90uhCFJEodLnoUHZheujQUJiur1dhusAiTxLuPtHdd3P3JDAWeNjdvwRMB8ZlnjYOmBZRiCISkVyLCw0CTmttDTOmVZguusiTRBeuBo4ysxeBozKPRaQf2XxxocOBZ4FLAE46SYXpEojVJ+vus939+Mzfy919lLvvmblfEXV8IlJanYsO/Q8wBZgFbGPGzAsuUGG6RNS7SURiq/aUU/jAo4+y1623sn1HBze95z0MvuEGxp52WtSh9RtKEiISTwsWwFlncfAjj8Chh8LkyXxTdYeSi9XpJhERVq+GSy/dOGP65z9XYTpCGkmISHzMmhUWAlq4UK28Y0IjCRGJ3rJl8JWvwJFHwttva8Z0jChJiEh03OH228NkuMZGzZiOIZ1uEpFoLFgQTi3Nnr2hMK26Q/xoJCEipbVmzcbC9DPP5Jwx3aOmflISGkmISOnMng1nnRUK06ecAtddt0XdobOpX2fPps6mfgC1tbWljrjf00hCRIpv2TI47bTQkG/9enjggVCDyFGY7rKpXwUot1GSRhIiUjzu8MtfwoQJsHIlTJwI3/9+l51aczX162p7OSnHUZJGEiIRKLdfk32yYEG4pPUrX4G99gr1h6uu6raV9+ZN/brbXk7KcZSkJCFSYt0upFPu1qyByy7bWJiePBkee6zHVy51NvXLVlVVRX19fRGCLa2yHCW5e8XcRo4c6SJxl0gkHNjilkgkog5t682a5b7XXu7gfvLJ7kuW9OkwDQ0Nnkgk3Mw8kUh4Q0NDYeOMSFz/3QNNnud7VSMJkRIry1+T3ckuTK9dC/ffH5YT3WWXPh2utraWlpYWOjo6aGlpie35+t4qx1GSkoRIiVXUOXd3mDo1zJhuaAiF6Xnz4Oijo44slmpra0mn0yQSCcyMRCJBOp2OdRKMPEmY2e5mNsvMms3seTP7Vmb7+8xshpm9mLnfIepYRQqhHH9N5rRwIYwaBV/5Cu3vex+fHDyYAVdfTXL48MqprxRBuY2SIk8SwHpggrsPAz4GnG1mw4ELgZnuvicwM/NYpOyV46/JTaxZA5dfDvvsA08/zZzTT2ePxYuZsWRJZRbi+7t8xYqobsA0wprWC4AhmW1DgAXdvVaFa6lUsSnkzp7tPnToJoXpuBZjpecol8K1mSWBA4A5wM7uvgQgc79TntfUmVmTmTW1t7eXLFaRUonFJbPLl8Ppp8Phh4eRxH33bShMV2QhXjaITZIws+2Bu4Fz3f2tnr7O3dPuXuPuNYMHDy5egCIRiXQCVueM6aFD4Ve/2liYPuaYDU+pqEK8bCEWScLMtiUkiEZ3vyez+XUzG5LZPwRYGlV8IlGK7Jf6woUwejSMGwcf/jD8/e9hxvRmRfeKKcRLTpEnCTMz4Dag2d2vy9o1HRiX+XscoVYh0u+U/Jd6dmH6qadg0qQwY3rEiJxPL/tCvHTJQs0iwgDMPg48BvwD6MhsvohQl7gTqAbagJPcfUVXx6qpqfGmpqYiRitSeps3hYPwS70oX8SPPBJaeS9YAGPHwvXX93lCnJQPM3vK3Wty7Yu8C6y7Pw5Ynt2jShmLSBx1JoJUKkVbWxvV1dXU19cXNkEsXw7nnQdTpsAee4TCdFbdQfqvyEcShaSRhEgvuYeC9IQJ8Oab8N3vhlbem9UYpLLFeiQhIhFZuBDGj4eHH4aDDw7dWvfZJ+qoJGYiL1yLSInlKkw//rgShOSkkYRIf5JdmP7iF+GGG1SYli5pJCHSH2TPmF67NhSm77hDCUK6pSQhUiKlXLK0sRGSSRgwwEnu+B8akxeFAvWFF24xY1qkKzrdJFICm8916Oy/BBR8rkNjI9TVQXgro3X59tQNuAHqU9ReqFYZ0ju6BFakBJLJJK2trVtsTyQStLS0FPa9Ek5r25ZTjxIJKPBbSYXo6hJYnW4SKYGS9V965BHa2nL/8FNTVukLJQmREih6/6WswnT1wNfyvFdh3kr6FyUJkRIoWqfUzhnTQ4eGlt4XXED9zwdvMWG6qgrUlFX6QklCpASK0il14UI46ig49VT40Ifg6afh6qupPW0Q6XSoQZiF+3Qa1JRV+kKFa5Fys2YNXHNNGBoMGgRXXx0myA3Qbz7pG/VuEqkUjz4aEsILL8AXvhBmTA8ZEnVUUsH000OkHCxfDmecAZ/4BKxeDX/+M/z2t0oQUnRKEiJxll2YnjoVzj8fnn8ejj026sikn4h9kjCzY8xsgZktMrMLo45HpGRefHFjYfqDHwyF6R/+UGs9SEnFOkmY2UDgFuBYYDhwspkNjzYqkSJbswauuCK07n7ySbjlFvjLX2DffaOOrKT9pyQeYp0kgIOARe7+sruvBe4AxkQck0iXtuqL9LHH4IAD4OKLYcyYUKD++tdh4MDiBdxDnf2nWltbcfcN/aeUKCpb3JPErsCrWY8XZ7aJxFKfv0hXrIAzz4TDDgud+e69N3aF6VQqtaFBYadVq1Yxbtw4JYoKFvcksWWXMthkYoeZ1ZlZk5k1tbe3lygskdzyfZGmUqncL3CHhoZQmL799o2F6eOOK2qcfRnt5Osz9fbbb2tEUcncvcsb8BCwX3fPK8YNOBh4IOvxRGBivuePHDnSRaJkZk74IbPJzcy2fPLChe6jRrmD+0EHuT/zTElibGho8Kqqqk3iq6qq8oaGhi5fl0gkcv6zdd4SiURJ4pfCA5o8z/dqT0YS5wPXm9kUMyv12PdJYE8z28PM3gGMBaaXOAapEKUouvaokd/atXDllZsWpv/6V9hvv4LHk0uvRzsZufpPZSt4R1uJh3zZY/Mb8HngWeASYLuevm5rb8BxwELgJSDV1XM1kpB8+vrrueDv8+ij7sOGhdHDSSe5//OfBX3/nujVaGczDQ0NPnDgQI0kKgxdjCR6+kVtwAjga8AyQgH5yz15bSlvShKST75TJcX4YmtoaPBEIuFm5olEIiSI5cvdzzwz/C+XSLj/6U8Ff9+e2trPolQJV0pnq5IE8DjwGjADuAI4HvgQcBOQ7u71pbwpSUg+W/Preat0dLg3NLgPHuw+cKD7eee5/+c/xX3PbhTiSz5nIpSytbVJYgSZbrE59jV39/pS3pQkJJ9SjiQ2ePFF99Gjw/9mH/1oyQrTPaEvecnWVZLotnDt7vMyB8nlU929XiQOirboTy5r14Y23iNGwNy5G2dMl6gw3RO1tbW0tLTQ0dFBS0vL1q1rIRVtq+ZJuPvLhQpEpJiKsuhPLo8/DvvvD9/7HpxwAjQ3x2bGtEhfaNEhkUJYsQIuuABuvTUsBXfLLfApDbSlPHS16FDcZ1yLxJs7NDaGGdNTpsB554UZ00oQUiG0Mp1IXy1aBOPHw0MPwUEHwYwZsao7iBSCRhIivZVdmJ4zB26+uaQzpkVKSUlCKlIxWnA0NjZy4i678PygQfC979G6336hlffZZ6swLRVLSUJipRBf7sVY9+B3kyezdtw47nr9dbYnzCgdPm8ejbNm9fmYIuVAVzdJbHR+uWc3n6uqqur1parJZJLW1tYtticSCVpaWnoXlDv8+tcsO/VU3tvRwQ2E5mWdEfbpmCIx09XVTUoSEhuF+nIfMGAAuf67NjM6Ojp6HtBLL4XC9IwZzAHOInS43KpjisSQLoGVspCv1XRvW1D3qF13ls1Pcf1m6lS46qpQmP7b3+Dmmzm5unqLBNHVMUUqhZKExEZvv9zz6U0Ljs3rF7u1trLfaadBKgXHHx9mTJ99NldcdVXp2nqIxEm+pk7leFODv/JWyBbUPW1g19n4773gk0MFwlvATxs8uM/HFCk3dNHgTzUJiZXGxkZSqRRtbW1UV1dTX19f1OZzA8wYC1wP/A9wA3ApsEq1BulHuqpJaMa1xEptbW3pOpK+9BKz3/lODlu9mrnAMcAzmV0J1RpEgIhrEmZ2rZm9YGbPmdnvzey9WfsmmtkiM1tgZkdHGKZUmrVrNxSmDwa+s+22HMzGBKFag8hGUReuZwAj3H1fwjrWEwHMbDgwFtib8APvp2amKa2y9f7yF/jIR0Jh+lOfYttFixg5ZQq797GFeDFmdovESaSnm9z9wayHfwNOzPw9BrjD3dcAr5jZIuAg4IkShyiV4o034MILIZ2G6mqYPh0+/Wmg76e4Np/81zmzu/OYIpUg6pFEttOB+zJ/7wq8mrVvcWbbFsyszsyazKypvb29yCFKb0X+S9sdfvOb0Mr7tttgwoTQyjuTILZGKpXaZHY4wKpVq0ilUlt9bJG4KPpIwsweAnbJsSvl7tMyz0kB64HObxDL8fycl2G5expIQ7i6aasDloKJ/Jf2Sy+FVeEefBAOPBDuvx8OOKBghy/U5D+ROCv6SMLdR7v7iBy3zgQxjtAvrdY3Xo+7GNg96zC7Aa8VO1YprMh+aa9bBz/4QZgx/cQTcNNN4b6ACQIKN/lPJM6ivrrpGOAC4AR3z/42mQ6MNbNBZrYHsCcwN4oYpe8K8Uu716er/vKXkAwuuiisDtfcDN/4RlFaefdmZrdI2co3y64UN2ARofbwTOY2KWtfCngJWAAc25PjacZ1vHTOZt78lkgkevT6Xs3AXrHCva7OHdyrq92nTy/sP0wXMWoWtpQ7uphxHXkrjULelCTiZWvbbPQoyXR0uP/61+477+w+cKD7hAnu//53cf6BRCpUV0kiTlc3SYWpra0lnU6T6OMchG5PV738Mhx7LJxySris9ckn4Uc/gu23L9Q/gki/p95NElv51pf4YHU1i8aPh8sug223DbOnx4/XEqIifaT1JKQs5SoMHzFoEE0dHTBxIhx3XFEL0yKiBn8SY52npVKpFG+1tnLT9ttT+5//wIABm8yYFpHiUZKQWKs95RRqBw6Ec8+FZcvCjOlLL1XdQaRElCQkvl5+Gc4+O8yUPvBAuO++gk+IE5GuqSYhsdDYCMlkOJOUTDiNX5wGe+8dJscVaca0iHRPIwmJXGMj1NVBZweP1jajrm001FxF7e9Pgt12izZAkX5MIwmJXCq1MUF0WsW7SLV/u+wSRORdb0UKTCMJiZY7ba2Qq/FvuTVTjbzrrUgRaCQh0Xn5ZTjuOKrZcsIchEnU5UTrS0glUpKQ0lu3Dq6+OhSmH3+c+i+/QGjxtFFVFZRbM1WtLyGVSElCSuuJJ8Ia01kzpmt/eQzptJFIgBkkEmGV0XI7Q6P1JaQSKUlIabz5ZuivdMghsHIlTJsGd9+9oTBdWwstLdDREe7LLUGA1peQyqQkIcXlDr/9bVhjOp2G73wH5s+HE06IOrKC29qutyJxpC6wUjyvvBLWmL7/fqipgcmTw6kmEYmV2HeBNbPvmpmb2Y5Z2yaa2SIzW2BmR0cZn/TSunXwwx/C3nuzbvZsLtthB7ZpaiL5uc9p3oBImYl8noSZ7Q4cBbRlbRsOjAX2Bt4PPGRmH3b3t6OJUnrsiSfgrLPgH/+graaGUfPmseiNNwDNGxApR3EYSVwPnE9YmrLTGOAOd1/j7q8Q1sI+KIrgpIfefDOcWjr0UHjjDfjDHzisvZ1Fq1dv8jTNGxApL5EmCTM7Afinuz+72a5dgVezHi/ObMt1jDozazKzpvb29iJFKnm5w513wrBhoeZw7rmhMD1mjOYNiFSAop9uMrOHgF1y7EoBFwGfzPWyHNtyVtjdPQ2kIRSu+xim9MUrr4RW3vfdByNHwr33blKYrq6uzrn8qOYNiJSPoo8k3H20u4/Y/Aa8DOwBPGtmLcBuwNNmtgth5LB71mF2A14rdqzSQ1mFaR57DG68EebM2eLKpVLMG1BDPZEic/dY3IAWYMfM33sDzwKDCInkZWBgd8cYOXKkS5H99a/u++zjDu6f/az7q692+fSGhgZPJBJuZp5IJLyhoaFgoTQ0NHhV6Oex4VZVVVXQ9xDpD4Amz/O9Gpt5EpnRRI27L8s8TgGnA+uBc939vu6OoXkSRfTmm6GVxuTJsOuucPPNMGZMpCElk8mcp7MSiQQtLS2lD0ikTHU1TyI2SaIQlCSKoLMwfe65sHQpnHMOXH45vPvdUUfGgAEDyPXfr5nR0dERQUQi5Sn2k+kkpl55JTThGzsW3v9+mDsXrr8+FgkC1FBPpBSUJGRL69bBNddsaOXNDTeEwvTIkVFHtgk11BMpPiUJ2dQTT4RkcMEFcPTRYc7Dt74F20Q+OX8LaqgnUnyqSUjw5ptw0UUwaVIoTN90E3zmM1FHJSIloJqE5Lf5jOlzzgmjByUIESEGDf4kQtkzpj/yEfjTn2JXdxCRaGkk0R9lF6Yfeyy2hWkRiZ5GEv3N3/4WWnk/91yYDHfTTbD77t2/TkT6JY0k+ouVK8OppUMOgRUr4Pe/hz/8QQlCRLqkJFHp3OF3vwtrTE+aBN/8pgrTItJjOt1UyVpawujhz38Ohek//jGsNS0i0kMaSVSidevg2mth+HB45JHQSmPOHCUIEek1jSQqTXZh+oQTQrdW1R1EpI80kqgUK1eGNaYPOQSWLw+F6WnTlCBEZKsoSZS77MJ054zp5mYVpkWkIHS6qZxlF6YPOECFaREpuMhHEmb2TTNbYGbPm9k1WdsnmtmizL6jo4wxdjYvTF93XVjrQQlCRAos0pGEmR0BjAH2dfc1ZrZTZvtwYCxhrev3Aw+Z2Yfd/e3ooo2JOXNCYfrZZ+HTnw6FaS2yIyJFEvVIYjxwtbuvAXD3pZntY4A73H2Nu78CLAIOiijGeOicMX3wwbBsGdx9dyhMK0GISBFFnSQ+DPx/M5tjZo+Y2YGZ7bsCr2Y9b3FmW//jDnfdFVp5/+xnG2dMf+5zYBZ1dCJS4Yp+usnMHgJ2ybErlXn/HYCPAQcCd5rZB4Bc3345V0cyszqgDipwbeOWFvjGN+Dee0Nheto0OPDAbl8mIlIoRU8S7j463z4zGw/c42F5vLlm1gHsSBg5ZF/gvxvwWp7jp4E0hJXpChV3pNatgxtvhEsuCaOF664LI4gYLiEqIpUt6tNNfwCOBDCzDwPvAJYB04GxZjbIzPYA9gTmRhVkSc2ZE0YL550Ho0aFU0vf/rYShIhEIupvnl8AvzCzecBaYFxmVPG8md0JzAfWA2dX/JVNK1dCKgU//SkMGRIK05/9rOoOIhKpSEcS7r7W3b/k7iPc/SPu/nDWvnp3/6C77+Xu90UZZ1FlF6Z/+tNQg2hurrjCdGNjI8lkkgEDBpBMJmlsbIw6JBHpgahHEv1bdmF6//0rtjDd2NhIXV0dq1atAqC1tZW6ujoAamtrowxNRLoRdU2if1q/Hn7847DG9KxZ4e8nn6zIBAGQSqU2JIhOq1atIpVKRRSRiPSURhKlNncu1NWFGdPHHx9mTCcSUUdVVG1tbb3aLiLxoZFEqaxcGS5j/djHoL09FKanT6/4BAH5569U3LwWkQqkJFFs7iEhDB8Ot9xSsYXprtTX11NVVbXJtqqqKurr6yOKSER6SkmimFpbw+pwJ54IO+0UVo37yU/gPe+JOrKSqq2tJZ1Ok0gkMDMSiQTpdFpFa5EyYGFaQmWoqanxpqamqMMIhekbb4SLLw6Pr7giLAakCXEiEkNm9pS751xrQN9ahTZ3bmjl/cwz/aYwLSKVS6ebCuWttzYWppcu7VeFaRGpXEoSW6uzMD1sWL8tTItI5VKS2BptbTBmDJx4Iiu23ZYxO+/MgJtvJrnvvmo7ISIVQUmiL9avD+27hw+HmTN5+uSTSS5dyvR//Qt339B2QolCRMqdkkRvdbbPmDABjjgC5s/nc3/9K//+7383eZraTohIJVCS6Km33gqXsX70o6EwfdddGwrTajshIpVKSaI77nDPPaEwffPNGwvTn//8hsK02k6ISKVSkqCLtQ46C9Of/3yXM6bVdkJEKpa7V8xt5MiR3lsNDQ1eVVXlwIbbu7fbzptOOcX9Xe9yr6py//GP3det6/Y4iUTCzcwTiYQ3NDT0OhYRkSgATZ7nezXSthxmtj8wCXgnYZnSr7v73My+icAZwNvAOe7+QHfH60tbjmQySWtr68ZjAGngANCMaRHpF7pqyxH16aZrgMvcfX/g4sxjzGw4MBbYGzgG+KmZDSxGAJ3F5XcDNwJzgJ2AE0EzpkWk34s6STjQeYL//wGvZf4eA9zh7mvc/RVgEXBQMQKorq5mJDAf+AZwCzAMaEokNGNaRPq9qJPEucC1ZvYq8CNgYmb7rsCrWc9bnNm2BTOrM7MmM2tqb2/vdQD19fX8a7vtmA98DDgHeFtFZxERoARdYM3sIWCXHLtSwCjg2+5+t5l9AbgNGA3k+gmfs3ji7mlCGYGamppeF1g61zSoS6Voa2sjUV1NfX291joQESHi9STMbCXwXnd3MzNgpbu/J1O0xt1/kHneA8Cl7v5EV8eLzXoSIiJlJM6F69eAT2T+PhJ4MfP3dGCsmQ0ysz2APYG5EcQnItKvRb3o0FeBG81sG2A1UAfg7s+b2Z2EevJ64Gx3fzu6MEVE+qdIk4S7Pw6MzLOvHlD1WEQkQlGfbhIRkRhTkhARkbyUJEREJC8lCRERySvSeRKFZmbtQGu3TyytHYFlUQcRc/qMuqbPp3v6jLrW3eeTcPfBuXZUVJKIIzNryjdJRQJ9Rl3T59M9fUZd25rPR6ebREQkLyUJERHJS0mi+NJRB1AG9Bl1TZ9P9/QZda3Pn49qEiIikpdGEiIikpeShIiI5KUkUQJmdq2ZvWBmz5nZ783svVHHFDdmdpKZPW9mHWamSxkzzOwYM1tgZovM7MKo44kbM/uFmS01s3lRxxJHZra7mc0ys+bM/1/f6u0xlCRKYwYwwt33BRaycZlW2Wge8Dng0agDiQszG0hYdv1YYDhwspkNjzaq2LkdOCbqIGJsPTDB3YcRVmg+u7f/DSlJlIC7P+ju6zMP/wbsFmU8ceTuze6+IOo4YuYgYJG7v+zua4E7gDERxxQr7v4osCLqOOLK3Ze4+9OZv/8NNAO79uYYShKldzpwX9RBSFnYFXg16/Fievk/uEgnM0sCBwBzevO6qFemqxhm9hCwS45dKXeflnlOijD8ayxlbHHRk89INmE5tumadek1M9seuBs4193f6s1rlSQKxN1Hd7XfzMYBxwOjvJ9OTunuM5ItLAZ2z3q8G2FdeJEeM7NtCQmi0d3v6e3rdbqpBMzsGOAC4AR3XxV1PFI2ngT2NLM9zOwdwFhgesQxSRkxMwNuA5rd/bq+HENJojRuBt4NzDCzZ8xsUtQBxY2ZfdbMFgMHA/ea2QNRxxS1zMUO3wAeIBQc73T356ONKl7M7DfAE8BeZrbYzM6IOqaYORT4MnBk5rvnGTM7rjcHUFsOERHJSyMJERHJS0lCRETyUpIQEZG8lCRERCQvJQkREclLSUJERPJSkhARkbyUJESKLNPP/6jM31ea2U+ijkmkp9S7SaT4LgEuN7OdCF04T4g4HpEe04xrkRIws0eA7YHDM339RcqCTjeJFJmZ7QMMAdYoQUi5UZIQKSIzG0JYP2QM8L9mdnTEIYn0ipKESJGYWRVwD2GN4WbgCuDSSIMS6SXVJEREJC+NJEREJC8lCRERyUtJQkRE8lKSEBGRvJQkREQkLyUJERHJS0lCRETy+j8NbDZfjjQCJAAAAABJRU5ErkJggg==\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"X_pred = np.array([-0.95, 0.2, 1.5])\n",
"\n",
"fig, ax = plt.subplots()\n",
"plt.scatter(X, Y, color = \"black\")\n",
"\n",
"plt.xlabel(\"$x$\")\n",
"plt.ylabel(\"$y$\")\n",
" \n",
"X_line = np.arange(np.min(X[0,:]),np.max(X[0,:])*1.1, 0.1)\n",
"ax.plot(X_line, W_simple[0,0] * X_line + b_simple[0,0], \"r\")\n",
"ax.plot(X_pred, W_simple[0,0] * X_pred + b_simple[0,0], \"bo\")\n",
"plt.plot()\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Not bad for such a small neural network with just a single perceptron and one input node!"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a name='3'></a>\n",
"## 3 - Multiple Linear Regression\n",
"\n",
"Models are not always as simple as the one above. In some cases your output is dependent on more than just one variable. Let's look at the case where the output depends on two input variables."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a name='3.1'></a>\n",
"### 3.1 - Multipe Linear Regression Model\n",
"\n",
"Multiple linear regression model with two independent variables $x_1$, $x_2$ can be written as\n",
"\n",
"$$\\hat{y} = w_1x_1 + w_2x_2 + b = Wx + b,\\tag{6}$$\n",
"\n",
"where $Wx$ is the dot product of the input vector $x = \\begin{bmatrix} x_1 & x_2\\end{bmatrix}$ and parameters vector $W = \\begin{bmatrix} w_1 & w_2\\end{bmatrix}$, scalar parameter $b$ is the intercept. \n",
"\n",
"The goal is the same - find the \"best\" parameters $w_1$, $w_2$ and $b$ such the differences between original values $y_i$ and predicted values $\\hat{y}_i$ are minimum.\n",
"\n",
"You can use a slightly more complicated neural network model to do that. Now matrix multiplication will be in the core of the model!"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a name='3.2'></a>\n",
"### 3.2 - Neural Network Model with a Single Perceptron and Two Input Nodes\n",
"\n",
"Again, you will use only one perceptron, but with two input nodes shown in the following scheme:\n",
"\n",
"<img src=\"images/nn_model_linear_regression_multiple.png\" style=\"width:420px;\">\n",
"\n",
"The perceptron output calculation for every training example $x^{(i)} = \\begin{bmatrix} x_1^{(i)} & x_2^{(i)}\\end{bmatrix}$ can be written with dot product:\n",
"\n",
"$$z^{(i)} = w_1x_1^{(i)} + w_2x_2^{(i)} + b = Wx^{(i)} + b,\\tag{7}$$\n",
"\n",
"where weights are in the vector $W = \\begin{bmatrix} w_1 & w_2\\end{bmatrix}$ and bias $b$ is a scalar. The output layer will have the same single node $\\hat{y}^{(i)} = z^{(i)}$.\n",
"\n",
"Organise all training examples in a matrix $X$ of a shape ($2 \\times m$), putting $x_1^{(i)}$ and $x_2^{(i)}$ into columns. Then matrix multiplication of $W$ ($1 \\times 2$) and $X$ ($2 \\times m$) will give a ($1 \\times m$) vector\n",
"\n",
"$$WX = \n",
"\\begin{bmatrix} w_1 & w_2\\end{bmatrix} \n",
"\\begin{bmatrix} \n",
"x_1^{(1)} & x_1^{(2)} & \\dots & x_1^{(m)} \\\\ \n",
"x_2^{(1)} & x_2^{(2)} & \\dots & x_2^{(m)} \\\\ \\end{bmatrix}\n",
"=\\begin{bmatrix} \n",
"w_1x_1^{(1)} + w_2x_2^{(1)} & \n",
"w_1x_1^{(2)} + w_2x_2^{(2)} & \\dots & \n",
"w_1x_1^{(m)} + w_2x_2^{(m)}\\end{bmatrix}.$$\n",
"\n",
"And the model can be written as\n",
"\n",
"\\begin{align}\n",
"Z &= W X + b,\\\\\n",
"\\hat{Y} &= Z,\n",
"\\tag{8}\\end{align}\n",
"\n",
"where $b$ is broadcasted to the vector of a size ($1 \\times m$). These are the calculations to perform in the forward propagation step. Cost function will remain the same, and there will be no change in methodology and training (that will be discussed in the next Course)!"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a name='3.3'></a>\n",
"### 3.3 - Dataset\n",
"\n",
"Let's build a linear regression model for a Kaggle dataset [House Prices](https://www.kaggle.com/c/house-prices-advanced-regression-techniques), saved in a file `data/house_prices_train.csv`. You will use two fields - ground living area (`GrLivArea`, square feet) and rates of the overall quality of material and finish (`OverallQual`, 1-10) to predict sales price (`SalePrice`, dollars).\n",
"\n",
"To open the dataset you can use `pandas` function `read_csv`:"
]
},
{
"cell_type": "code",
"execution_count": 94,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": [
"df = pd.read_csv('data/house_prices_train.csv')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The data is now saved in the variable `df` as a **DataFrame**, which is the most commonly used `pandas` object. It is a 2-dimensional labeled data structure with columns of potentially different types. You can think of it as a table or a spreadsheet. Full documentation can be found [here](https://pandas.pydata.org/).\n",
"\n",
"Select the required fields and save them in the variables `X_multi`, `Y_multi`:"
]
},
{
"cell_type": "code",
"execution_count": 95,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": [
"X_multi = df[['GrLivArea', 'OverallQual']]\n",
"Y_multi = df['SalePrice']"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Have a look at the data:"
]
},
{
"cell_type": "code",
"execution_count": 96,
"metadata": {
"scrolled": false,
"tags": [
"graded"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"X_multi:\n",
" GrLivArea OverallQual\n",
"0 1710 7\n",
"1 1262 6\n",
"2 1786 7\n",
"3 1717 7\n",
"4 2198 8\n",
"... ... ...\n",
"1455 1647 6\n",
"1456 2073 6\n",
"1457 2340 7\n",
"1458 1078 5\n",
"1459 1256 5\n",
"\n",
"[1460 rows x 2 columns]\n",
"\n",
"Y_multi:\n",
"0 208500\n",
"1 181500\n",
"2 223500\n",
"3 140000\n",
"4 250000\n",
" ... \n",
"1455 175000\n",
"1456 210000\n",
"1457 266500\n",
"1458 142125\n",
"1459 147500\n",
"Name: SalePrice, Length: 1460, dtype: int64\n",
"\n"
]
}
],
"source": [
"print(f\"X_multi:\\n{X_multi}\\n\")\n",
"print(f\"Y_multi:\\n{Y_multi}\\n\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"All of the original arrays have different units. To make training of the neural network efficient, you need to bring them to the same units. A common approach to it is called **normalization**: substract the mean value of the array from each of the elements in the array and divide them by standard deviation (a statistical measure of the amount of dispersion of a set of values). If you are not familiar with mean and standard deviation, do not worry about this for now - this is covered in the third Course of Specialization.\n",
"\n",
"Normalization is implemented in the following code:"
]
},
{
"cell_type": "code",
"execution_count": 97,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": [
"X_multi_norm = (X_multi - np.mean(X_multi))/np.std(X_multi)\n",
"Y_multi_norm = (Y_multi - np.mean(Y_multi))/np.std(Y_multi)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Convert results to the `NumPy` arrays, transpose `X_multi_norm` to get an array of a shape ($2 \\times m$) and reshape `Y_multi_norm` to bring it to the shape ($1 \\times m$):"
]
},
{
"cell_type": "code",
"execution_count": 98,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"The shape of X: (2, 1460)\n",
"The shape of Y: (1, 1460)\n",
"I have m = 1460 training examples!\n"
]
}
],
"source": [
"X_multi_norm = np.array(X_multi_norm).T\n",
"Y_multi_norm = np.array(Y_multi_norm).reshape((1, len(Y_multi_norm)))\n",
"\n",
"print ('The shape of X: ' + str(X_multi_norm.shape))\n",
"print ('The shape of Y: ' + str(Y_multi_norm.shape))\n",
"print ('I have m = %d training examples!' % (X_multi_norm.shape[1]))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 3.4 - Performance of the Neural Network Model for Multiple Linear Regression\n",
"\n",
"The magic is that now you do not need to change anything in your neural network implementation! Go through the code in section [2](#2) and see that if you pass new datasets `X_multi_norm` and `Y_multi_norm`, the input layer size $n_x$ will get equal to $2$ and the rest of the implementation will remain exactly the same. That's the power of the neural networks (and matrix multiplication)!"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a name='ex06'></a>\n",
"### Exercise 6\n",
"\n",
"Run the constructed above neural network model `nn_model()` for `100` iterations, passing the training dataset saved in the arrays `X_multi_norm` and `Y_multi_norm`."
]
},
{
"cell_type": "code",
"execution_count": 99,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Cost after iteration 0: 790.869208\n",
"Cost after iteration 1: 176.498358\n",
"Cost after iteration 2: 143.769946\n",
"Cost after iteration 3: 141.433422\n",
"Cost after iteration 4: 141.248729\n",
"Cost after iteration 5: 141.233727\n",
"Cost after iteration 6: 141.232500\n",
"Cost after iteration 7: 141.232400\n",
"Cost after iteration 8: 141.232391\n",
"Cost after iteration 9: 141.232391\n",
"Cost after iteration 10: 141.232391\n",
"Cost after iteration 11: 141.232391\n",
"Cost after iteration 12: 141.232391\n",
"Cost after iteration 13: 141.232391\n",
"Cost after iteration 14: 141.232391\n",
"Cost after iteration 15: 141.232391\n",
"Cost after iteration 16: 141.232391\n",
"Cost after iteration 17: 141.232391\n",
"Cost after iteration 18: 141.232391\n",
"Cost after iteration 19: 141.232391\n",
"Cost after iteration 20: 141.232391\n",
"Cost after iteration 21: 141.232391\n",
"Cost after iteration 22: 141.232391\n",
"Cost after iteration 23: 141.232391\n",
"Cost after iteration 24: 141.232391\n",
"Cost after iteration 25: 141.232391\n",
"Cost after iteration 26: 141.232391\n",
"Cost after iteration 27: 141.232391\n",
"Cost after iteration 28: 141.232391\n",
"Cost after iteration 29: 141.232391\n",
"Cost after iteration 30: 141.232391\n",
"Cost after iteration 31: 141.232391\n",
"Cost after iteration 32: 141.232391\n",
"Cost after iteration 33: 141.232391\n",
"Cost after iteration 34: 141.232391\n",
"Cost after iteration 35: 141.232391\n",
"Cost after iteration 36: 141.232391\n",
"Cost after iteration 37: 141.232391\n",
"Cost after iteration 38: 141.232391\n",
"Cost after iteration 39: 141.232391\n",
"Cost after iteration 40: 141.232391\n",
"Cost after iteration 41: 141.232391\n",
"Cost after iteration 42: 141.232391\n",
"Cost after iteration 43: 141.232391\n",
"Cost after iteration 44: 141.232391\n",
"Cost after iteration 45: 141.232391\n",
"Cost after iteration 46: 141.232391\n",
"Cost after iteration 47: 141.232391\n",
"Cost after iteration 48: 141.232391\n",
"Cost after iteration 49: 141.232391\n",
"Cost after iteration 50: 141.232391\n",
"Cost after iteration 51: 141.232391\n",
"Cost after iteration 52: 141.232391\n",
"Cost after iteration 53: 141.232391\n",
"Cost after iteration 54: 141.232391\n",
"Cost after iteration 55: 141.232391\n",
"Cost after iteration 56: 141.232391\n",
"Cost after iteration 57: 141.232391\n",
"Cost after iteration 58: 141.232391\n",
"Cost after iteration 59: 141.232391\n",
"Cost after iteration 60: 141.232391\n",
"Cost after iteration 61: 141.232391\n",
"Cost after iteration 62: 141.232391\n",
"Cost after iteration 63: 141.232391\n",
"Cost after iteration 64: 141.232391\n",
"Cost after iteration 65: 141.232391\n",
"Cost after iteration 66: 141.232391\n",
"Cost after iteration 67: 141.232391\n",
"Cost after iteration 68: 141.232391\n",
"Cost after iteration 69: 141.232391\n",
"Cost after iteration 70: 141.232391\n",
"Cost after iteration 71: 141.232391\n",
"Cost after iteration 72: 141.232391\n",
"Cost after iteration 73: 141.232391\n",
"Cost after iteration 74: 141.232391\n",
"Cost after iteration 75: 141.232391\n",
"Cost after iteration 76: 141.232391\n",
"Cost after iteration 77: 141.232391\n",
"Cost after iteration 78: 141.232391\n",
"Cost after iteration 79: 141.232391\n",
"Cost after iteration 80: 141.232391\n",
"Cost after iteration 81: 141.232391\n",
"Cost after iteration 82: 141.232391\n",
"Cost after iteration 83: 141.232391\n",
"Cost after iteration 84: 141.232391\n",
"Cost after iteration 85: 141.232391\n",
"Cost after iteration 86: 141.232391\n",
"Cost after iteration 87: 141.232391\n",
"Cost after iteration 88: 141.232391\n",
"Cost after iteration 89: 141.232391\n",
"Cost after iteration 90: 141.232391\n",
"Cost after iteration 91: 141.232391\n",
"Cost after iteration 92: 141.232391\n",
"Cost after iteration 93: 141.232391\n",
"Cost after iteration 94: 141.232391\n",
"Cost after iteration 95: 141.232391\n",
"Cost after iteration 96: 141.232391\n",
"Cost after iteration 97: 141.232391\n",
"Cost after iteration 98: 141.232391\n",
"Cost after iteration 99: 141.232391\n",
"W = [[35.71958194]]\n",
"b = [[2.28930781]]\n"
]
}
],
"source": [
"# grade-up-to-here\n",
"### START CODE HERE ### (~ 1 line of code)\n",
"parameters_multi = nn_model(X, Y, num_iterations=100, print_cost=True)\n",
"### END CODE HERE ###\n",
"\n",
"print(\"W = \" + str(parameters_multi[\"W\"]))\n",
"print(\"b = \" + str(parameters_multi[\"b\"]))\n",
"\n",
"W_multi = parameters_multi[\"W\"]\n",
"b_multi = parameters_multi[\"b\"]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"##### __Expected Output__ \n",
"Note: the actual values can be different!\n",
"\n",
"```Python\n",
"Cost after iteration 0: 0.489797\n",
"Cost after iteration 1: 0.429192\n",
"Cost after iteration 2: 0.380299\n",
"Cost after iteration 3: 0.340051\n",
"Cost after iteration 4: 0.306705\n",
"Cost after iteration 5: 0.279020\n",
"...\n",
"Cost after iteration 95: 0.142913\n",
"Cost after iteration 96: 0.142913\n",
"Cost after iteration 97: 0.142913\n",
"Cost after iteration 98: 0.142913\n",
"Cost after iteration 99: 0.142913\n",
"W = [[0.36946186 0.5718172 ]]\n",
"b = [[1.35781797e-16]]\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 100,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Test case \"default_check\". Wrong shape of the weights matrix W. \n",
"\tExpected: (1, 2).\n",
"\tGot: (1, 1).\n",
"\u001b[92m 1 Tests passed\n",
"\u001b[91m 1 Tests failed\n"
]
}
],
"source": [
"# Note: \n",
"# Actual values are not checked here in the unit tests (due to random initialization).\n",
"X_multi_norm\n",
"w3_unittest.test_multi(nn_model, X_multi_norm, Y_multi_norm, parameters_multi)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Remember, that the initial datasets were normalized. To make the predictions, you need to normalize the original, calculate predictions with the obtained linear regression coefficients and then **denormalize** the result (perform the reverse process of normalization):"
]
},
{
"cell_type": "code",
"execution_count": 55,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [
{
"data": {
"text/plain": [
"'\\nX_pred_multi = np.array([[1710, 7], [1200, 6], [2200, 8]]).T\\n\\n# Normalize using the same mean and standard deviation of the original training array X_multi.\\nX_multi_mean = np.array(np.mean(X_multi)).reshape((2,1))\\nX_multi_std = np.array(np.std(X_multi)).reshape((2,1))\\nX_pred_multi_norm = (X_pred_multi - X_multi_mean)/ X_multi_std\\n# Make predictions.\\nY_pred_multi_norm = np.matmul(W_multi, X_pred_multi_norm) + b_multi\\n# Denormalize using the same mean and standard deviation of the original training array Y_multi.\\nY_pred_multi = Y_pred_multi_norm * np.std(Y_multi) + np.mean(Y_multi)\\n\\nprint(f\"Ground living area, square feet:\\n{X_pred_multi[0]}\")\\nprint(f\"Rates of the overall quality of material and finish, 1-10:\\n{X_pred_multi[1]}\")\\nprint(f\"Predictions of sales price, $:\\n{np.round(Y_pred_multi)}\")\\n'"
]
},
"execution_count": 55,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"'''\n",
"X_pred_multi = np.array([[1710, 7], [1200, 6], [2200, 8]]).T\n",
"\n",
"# Normalize using the same mean and standard deviation of the original training array X_multi.\n",
"X_multi_mean = np.array(np.mean(X_multi)).reshape((2,1))\n",
"X_multi_std = np.array(np.std(X_multi)).reshape((2,1))\n",
"X_pred_multi_norm = (X_pred_multi - X_multi_mean)/ X_multi_std\n",
"# Make predictions.\n",
"Y_pred_multi_norm = np.matmul(W_multi, X_pred_multi_norm) + b_multi\n",
"# Denormalize using the same mean and standard deviation of the original training array Y_multi.\n",
"Y_pred_multi = Y_pred_multi_norm * np.std(Y_multi) + np.mean(Y_multi)\n",
"\n",
"print(f\"Ground living area, square feet:\\n{X_pred_multi[0]}\")\n",
"print(f\"Rates of the overall quality of material and finish, 1-10:\\n{X_pred_multi[1]}\")\n",
"print(f\"Predictions of sales price, $:\\n{np.round(Y_pred_multi)}\")\n",
"'''"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Congrats on finishing this programming assignment!"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": [
"graded"
]
},
"outputs": [],
"source": []
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"collapsed_sections": [],
"name": "C1_W1_Assignment_Solution.ipynb",
"provenance": []
},
"coursera": {
"schema_names": [
"AI4MC1-1"
]
},
"grader_version": "2",
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.8"
},
"toc": {
"base_numbering": 1,
"nav_menu": {},
"number_sections": true,
"sideBar": true,
"skip_h1_title": false,
"title_cell": "Table of Contents",
"title_sidebar": "Contents",
"toc_cell": false,
"toc_position": {},
"toc_section_display": true,
"toc_window_display": false
}
},
"nbformat": 4,
"nbformat_minor": 1
}