From 6ceea384dcbd5785b3ddd49e1533c3edcf2bf309 Mon Sep 17 00:00:00 2001 From: "Ryan C. Cooper" Date: Tue, 28 Mar 2017 11:53:48 -0400 Subject: [PATCH 1/2] added lecture 16 --- lecture_16/lecture_16.ipynb | 29 +- .../lecture_16-checkpoint.ipynb | 754 +++++ .../lecture_17-checkpoint.ipynb | 6 + lecture_17/gen_examples.m | 5 + lecture_17/lecture_16.ipynb | 2710 +++++++++++++++++ lecture_17/lecture_17.ipynb | 74 + lecture_17/octave-workspace | Bin 0 -> 23760 bytes lecture_17/q1.png | Bin 0 -> 23217 bytes lecture_17/q2.png | Bin 0 -> 25757 bytes lecture_17/sin_data.csv | 2 + lecture_17/xy_data.csv | 2 + 11 files changed, 3575 insertions(+), 7 deletions(-) create mode 100644 lecture_17/.ipynb_checkpoints/lecture_16-checkpoint.ipynb create mode 100644 lecture_17/.ipynb_checkpoints/lecture_17-checkpoint.ipynb create mode 100644 lecture_17/gen_examples.m create mode 100644 lecture_17/lecture_16.ipynb create mode 100644 lecture_17/lecture_17.ipynb create mode 100644 lecture_17/octave-workspace create mode 100644 lecture_17/q1.png create mode 100644 lecture_17/q2.png create mode 100644 lecture_17/sin_data.csv create mode 100644 lecture_17/xy_data.csv diff --git a/lecture_16/lecture_16.ipynb b/lecture_16/lecture_16.ipynb index 7ab08d2..206e84c 100644 --- a/lecture_16/lecture_16.ipynb +++ b/lecture_16/lecture_16.ipynb @@ -20644,7 +20644,7 @@ }, { "cell_type": "code", - "execution_count": 123, + "execution_count": 200, "metadata": { "collapsed": false }, @@ -20793,12 +20793,20 @@ "\n", "\n", "\n", - "\tgnuplot_plot_1a\n", - "\n", + "\n", + "\n", + "\n", + "\t\n", + "\tdata\n", + "\n", "\n", "\n", - "\t \n", - "\t\n", + "\t\n", + "\t\tdata\n", + "\t\n", + "\n", + "\n", + "\t\n", "\t\n", "\t\n", "\t\n", @@ -20806,11 +20814,17 @@ "\t\n", "\t\n", "\t\n", + "\t\n", "\n", "\t\n", - "\tgnuplot_plot_2a\n", + "\tbest-fit\n", + "\n", + "\t\n", + "\t\tbest-fit\n", + "\t\n", + "\n", "\n", - "\t\n", + "\t\n", "\t\n", "\n", "\n", @@ -20851,6 +20865,7 @@ "a=A\\b\n", "\n", "plot(x,y,'o',x,a(1)+a(2)*x)\n", + "legend('data','best-fit','Location','NorthWest')\n", "xlabel('Force (N)')\n", "ylabel('velocity (m/s)')" ] diff --git a/lecture_17/.ipynb_checkpoints/lecture_16-checkpoint.ipynb b/lecture_17/.ipynb_checkpoints/lecture_16-checkpoint.ipynb new file mode 100644 index 0000000..37443ae --- /dev/null +++ b/lecture_17/.ipynb_checkpoints/lecture_16-checkpoint.ipynb @@ -0,0 +1,754 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": 171, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "setdefaults" + ] + }, + { + "cell_type": "code", + "execution_count": 172, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "%plot --format svg" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "![question 1](q1.png)\n", + "\n", + "![question 2](q2.png)\n", + "\n", + "### Project Ideas so far\n", + "\n", + "- Nothing yet...probably something heat transfer related\n", + "\n", + "- Modeling Propulsion or Propagation of Sound Waves\n", + "\n", + "- Low Thrust Orbital Transfer\n", + "\n", + "- Tracking motion of a satellite entering orbit until impact\n", + "\n", + "- What ever you think is best.\n", + "\n", + "- You had heat transfer project as an option; that sounded cool\n", + "\n", + "- Heat transfer through a pipe\n", + "\n", + "- I would prefer to do something with beam/plate mechanics or vibrations than a heat transfer or thermo problem\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Questions from you:\n", + "\n", + "- Is attempting to divide by zero an acceptable project idea?\n", + "\n", + "- Would it be alright if we worked in a group of 4?\n", + "\n", + "- What are acceptable project topics?\n", + "\n", + "- How do the exams look? \n", + "\n", + "- Is there no pdf for the lecture today?\n", + "\n", + "- Thank you for making the formatted lectures available!\n", + "\n", + "- did you do anything cool over spring break?\n", + "\n", + "- Could we have a group of 4? We don't want to have to choose which one of us is on their own.\n", + "\n", + "- In HW 5 should there be 4 vectors as an input?\n", + "\n", + "- Would it be possible for me to join a group of 3? I seem to be the odd man out in two 3 member groups that my friends are in." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Linear Least Squares Regression\n", + "\n", + "When approximating a set of data as a polynomial, there will always be error introduced (except in 2 cases). \n", + "\n", + "For a straight line, the actual data points, $y_{i}$ as a function of the independent variable, $x_{i}$, is:\n", + "\n", + "$y_{i}=a_{0}+a_{1}x_{i}+e_{i}$\n", + "\n", + "where $a_{0}$ and $a_{1}$ are the intercept and slope of the line and $e_{i}$ is the error between the approximate function and the recorded data point. \n", + "\n", + "We make the following assumptions in this analysis:\n", + "\n", + "1. Each x has a fixed value; it is not random and is known without error.\n", + "\n", + "2. The y values are independent random variables and all have the same variance.\n", + "\n", + "3. The y values for a given x must be normally distributed.\n", + "\n", + "The total error is \n", + "\n", + "$\\sum_{i=1}^{n}e_{i}=\\sum_{i=1}^{n}(y_{i}-a_{0}-a_{1}x_{i})$\n", + "\n", + "we don't care about the sign though. One approach has been demonstrated to provide a unique solution is minimizing the sum of squares error or\n", + "\n", + "$S_{r}=\\sum_{i=1}^{n}e_{i}^{2}=\\sum_{i=1}^{n}(y_{i}-a_{0}-a_{1}x_{i})^{2}$\n", + "\n", + "Where, $S_{r}$ is the sum of squares error (SSE). \n", + "\n", + "$\\frac{\\partial S_{r}}{\\partial a_{0}}=-2\\sum(y_{i}-a_{0}-a_{1}x_{i})$\n", + "\n", + "$\\frac{\\partial S_{r}}{\\partial a_{1}}=-2\\sum(y_{i}-a_{0}-a_{1}x_{i})x_{i}$\n", + "\n", + "The minimum $S_{r}$ occurrs when the partial derivatives are 0. \n", + "\n", + "$\\sum y_{i}= \\sum a_{0}+\\sum a_{1}x_{i}$\n", + "\n", + "$\\sum x_{i}y_{i}= \\sum a_{0}x_{i}+\\sum a_{1}x_{i}^{2}$\n", + "\n", + "$\\left[\\begin{array}{c}\n", + "\\sum y_{i}\\\\\n", + "\\sum x_{i}y_{i}\\end{array}\\right]=\n", + "\\left[\\begin{array}{cc}\n", + "n & \\sum x_{i}\\\\\n", + "\\sum x_{i} & \\sum x_{i}^{2}\\end{array}\\right]\n", + "\\left[\\begin{array}{c}\n", + "a_{0}\\\\\n", + "a_{1}\\end{array}\\right]$\n", + "\n", + "\n", + "$b=Ax$\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Example \n", + "\n", + "Find drag coefficient with best-fit line to experimental data\n", + "\n", + "|i | v (m/s) | F (N) |\n", + "|---|---|---|\n", + "|1 | 10 | 25 |\n", + "|2 | 20 | 70 |\n", + "|3 | 30 | 380|\n", + "|4 | 40 | 550|\n", + "|5 | 50 | 610|\n", + "|6 | 60 | 1220|\n", + "|7 | 70 | 830 |\n", + "|8 |80 | 1450|" + ] + }, + { + "cell_type": "code", + "execution_count": 200, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "a =\n", + "\n", + " -234.286\n", + " 19.470\n", + "\n" + ] + }, + { + "data": { + "image/svg+xml": [ + "\n", + "\n", + "Gnuplot\n", + "Produced by GNUPLOT 5.0 patchlevel 3 \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\t\n", + "\t\t-500\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t500\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t1000\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t1500\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t2000\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t10\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t20\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t30\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t40\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t50\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t60\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t70\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t80\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\n", + "\t\n", + "\t\tvelocity (m/s)\n", + "\t\n", + "\n", + "\n", + "\t\n", + "\t\tForce (N)\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\tdata\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\tdata\n", + "\t\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\n", + "\t\n", + "\tbest-fit\n", + "\n", + "\t\n", + "\t\tbest-fit\n", + "\t\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "drag_data=[...\n", + "1 , 10 , 25 \n", + "2 , 20 , 70 \n", + "3 , 30 , 380\n", + "4 , 40 , 550\n", + "5 , 50 , 610\n", + "6 , 60 , 1220\n", + "7 , 70 , 830 \n", + "8 ,80 , 1450];\n", + "x=drag_data(:,2);\n", + "y=drag_data(:,3);\n", + "\n", + "b=[sum(y);sum(x.*y)];\n", + "A=[length(x),sum(x);\n", + " sum(x), sum(x.^2)];\n", + " \n", + "a=A\\b\n", + "\n", + "plot(x,y,'o',x,a(1)+a(2)*x)\n", + "legend('data','best-fit','Location','NorthWest')\n", + "xlabel('Force (N)')\n", + "ylabel('velocity (m/s)')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "How do we know its a \"good\" fit? \n", + "\n", + "Can compare the sum of squares error to the total sum of squares of the dependent variable (here F). \n", + "\n", + "$S_{r}=\\sum(y_{i}-a_{0}-a_{1}x_{i})^{2}$\n", + "\n", + "$S_{t}=\\sum(y_{i}-\\bar{y})^{2}$\n", + "\n", + "Then, we can calculate the *coefficient of determination*, $r^{2}$ or *correlation coefficient*, r. \n", + "\n", + "$r^{2}=\\frac{S_{t}-S_{r}}{S_{t}}$\n", + "\n", + "This represents the relative improvement of assuming that y is a function of x (if the x-values are not random and the y-values are random)\n", + "\n", + "For further information regarding statistical work on regression, look at \n", + "[NIST Statistics Handbook](http://www.itl.nist.gov/div898/handbook/pmd/section4/pmd44.htm)" + ] + }, + { + "cell_type": "code", + "execution_count": 128, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "St = 1.8083e+06\n", + "St = 1.8083e+06\n" + ] + } + ], + "source": [ + "Sr=sum((y-a(1)-a(2)*x).^2);\n", + "St=std(y)^2*(length(y)-1)\n", + "St=sum((y-mean(y)).^2)" + ] + }, + { + "cell_type": "code", + "execution_count": 130, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "r2 = 0.88049\n", + "r = 0.93834\n" + ] + } + ], + "source": [ + "r2=(St-Sr)/St\n", + "r=sqrt(r2)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Limiting cases \n", + "\n", + "#### $r^{2}=0$ $S_{r}=S{t}$\n", + "\n", + "#### $r^{2}=1$ $S_{r}=0$" + ] + }, + { + "cell_type": "code", + "execution_count": 152, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/svg+xml": [ + "\n", + "\n", + "Gnuplot\n", + "Produced by GNUPLOT 5.0 patchlevel 3 \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0.2\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0.4\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0.6\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0.8\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t1\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0.2\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0.4\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0.6\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0.8\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t1\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\tCase 1\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\tCase 1\n", + "\t\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\n", + "\t\n", + "\tCase 2\n", + "\n", + "\t\n", + "\t\tCase 2\n", + "\t\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "gen_examples\n", + "plot(x1(1:50:end),y1(1:50:end),'s',x2,y2,'o')\n", + "legend('Case 1','Case 2','Location','NorthWest')" + ] + }, + { + "cell_type": "code", + "execution_count": 157, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "a1 =\n", + "\n", + " 0.0497269\n", + " 0.0016818\n", + "\n", + "a2 =\n", + "\n", + " 0\n", + " 1\n", + "\n", + "Sr1 = 0.82607\n", + "St1 = 0.82631\n", + "coefficient of determination in Case 1 is 0.000286\n", + "Sr2 = 0\n", + "St2 = 1.0185\n", + "coefficient of determination in Case 2 is 1.000000\n" + ] + } + ], + "source": [ + "b1=[sum(y1);sum(x1.*y1)];\n", + "A1=[length(x1),sum(x1);\n", + " sum(x1), sum(x1.^2)];\n", + " \n", + "a1=A1\\b1\n", + "\n", + "b2=[sum(y2);sum(x2.*y2)];\n", + "A2=[length(x2),sum(x2);\n", + " sum(x2), sum(x2.^2)];\n", + " \n", + "a2=A2\\b2\n", + "\n", + "Sr1=sum((y1-a1(1)-a1(2)*x1).^2)\n", + "St1=sum((y1-mean(y1)).^2)\n", + "fprintf('coefficient of determination in Case 1 is %f\\n',1-Sr1/St1)\n", + "\n", + "Sr2=sum((y2-a2(1)-a2(2)*x2).^2)\n", + "St2=sum((y2-mean(y2)).^2)\n", + "\n", + "fprintf('coefficient of determination in Case 2 is %f\\n',1-Sr2/St2)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "collapsed": true + }, + "source": [ + "# General Regression (Linear and nonlinear)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Octave", + "language": "octave", + "name": "octave" + }, + "language_info": { + "file_extension": ".m", + "help_links": [ + { + "text": "MetaKernel Magics", + "url": "https://github.com/calysto/metakernel/blob/master/metakernel/magics/README.md" + } + ], + "mimetype": "text/x-octave", + "name": "octave", + "version": "0.19.14" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/lecture_17/.ipynb_checkpoints/lecture_17-checkpoint.ipynb b/lecture_17/.ipynb_checkpoints/lecture_17-checkpoint.ipynb new file mode 100644 index 0000000..2fd6442 --- /dev/null +++ b/lecture_17/.ipynb_checkpoints/lecture_17-checkpoint.ipynb @@ -0,0 +1,6 @@ +{ + "cells": [], + "metadata": {}, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/lecture_17/gen_examples.m b/lecture_17/gen_examples.m new file mode 100644 index 0000000..c5d6890 --- /dev/null +++ b/lecture_17/gen_examples.m @@ -0,0 +1,5 @@ +x1=linspace(0,1,1000)'; +y1=0*x1+0.1*rand(length(x1),1); + +x2=linspace(0,1,10)'; +y2=1*x2; diff --git a/lecture_17/lecture_16.ipynb b/lecture_17/lecture_16.ipynb new file mode 100644 index 0000000..3159c42 --- /dev/null +++ b/lecture_17/lecture_16.ipynb @@ -0,0 +1,2710 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": 65, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "setdefaults" + ] + }, + { + "cell_type": "code", + "execution_count": 66, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "%plot --format svg" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "![question 1](q1.png)\n", + "\n", + "![question 2](q2.png)\n", + "\n", + "### Project Ideas so far\n", + "\n", + "- Nothing yet...probably something heat transfer related\n", + "\n", + "- Modeling Propulsion or Propagation of Sound Waves\n", + "\n", + "- Low Thrust Orbital Transfer\n", + "\n", + "- Tracking motion of a satellite entering orbit until impact\n", + "\n", + "- What ever you think is best.\n", + "\n", + "- You had heat transfer project as an option; that sounded cool\n", + "\n", + "- Heat transfer through a pipe\n", + "\n", + "- I would prefer to do something with beam/plate mechanics or vibrations than a heat transfer or thermo problem\n", + "\n", + "- Modeling Couette flow with a pressure gradient using a discretized form of the Navier-Stokes equation and comparing it to the analytical solution\n", + "\n", + "- Software to instruct a robotic arm to orient itself based on input from a gyroscope" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Questions from you:\n", + "\n", + "How was your spring break\n", + "\n", + "Are grades for hw 3 and 4 going to be posted?\n", + "\n", + "For class occasionally switching to doc cam and working through problems by hand might help get ideas across.\n", + "\n", + "Are you assigning those who do not have groups to groups?\n", + "\n", + "How do you graph a standard distribution in Matlab?\n", + "\n", + "What is the longest code you have written?\n", + "\n", + "how to approach probability for hw 5?\n", + "??\n", + "\n", + "Will you be assigning groups to people who do not currently have one? \n", + "\n", + "what are some basic guidelines we should use to brainstorm project ideas?\n", + "\n", + "Are you a fan of bananas?\n", + "\n", + "Going through code isn't the most helpful, because you can easily lose interest. But I am not sure what else you can do.\n", + "\n", + "Has lecture 15 been posted yet? Looking in the repository I can't seem to find it." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Linear Least Squares Regression\n", + "\n", + "When approximating a set of data as a polynomial, there will always be error introduced (except in 2 cases). \n", + "\n", + "For a straight line, the actual data points, $y_{i}$ as a function of the independent variable, $x_{i}$, is:\n", + "\n", + "$y_{i}=a_{0}+a_{1}x_{i}+e_{i}$\n", + "\n", + "where $a_{0}$ and $a_{1}$ are the intercept and slope of the line and $e_{i}$ is the error between the approximate function and the recorded data point. \n", + "\n", + "We make the following assumptions in this analysis:\n", + "\n", + "1. Each x has a fixed value; it is not random and is known without error.\n", + "\n", + "2. The y values are independent random variables and all have the same variance.\n", + "\n", + "3. The y values for a given x must be normally distributed.\n", + "\n", + "The total error is \n", + "\n", + "$\\sum_{i=1}^{n}e_{i}=\\sum_{i=1}^{n}(y_{i}-a_{0}-a_{1}x_{i})$\n", + "\n", + "we don't care about the sign though. One approach has been demonstrated to provide a unique solution is minimizing the sum of squares error or\n", + "\n", + "$S_{r}=\\sum_{i=1}^{n}e_{i}^{2}=\\sum_{i=1}^{n}(y_{i}-a_{0}-a_{1}x_{i})^{2}$\n", + "\n", + "Where, $S_{r}$ is the sum of squares error (SSE). \n", + "\n", + "$\\frac{\\partial S_{r}}{\\partial a_{0}}=-2\\sum(y_{i}-a_{0}-a_{1}x_{i})$\n", + "\n", + "$\\frac{\\partial S_{r}}{\\partial a_{1}}=-2\\sum(y_{i}-a_{0}-a_{1}x_{i})x_{i}$\n", + "\n", + "The minimum $S_{r}$ occurrs when the partial derivatives are 0. \n", + "\n", + "$\\sum y_{i}= \\sum a_{0}+\\sum a_{1}x_{i}$\n", + "\n", + "$\\sum x_{i}y_{i}= \\sum a_{0}x_{i}+\\sum a_{1}x_{i}^{2}$\n", + "\n", + "$\\left[\\begin{array}{c}\n", + "\\sum y_{i}\\\\\n", + "\\sum x_{i}y_{i}\\end{array}\\right]=\n", + "\\left[\\begin{array}{cc}\n", + "n & \\sum x_{i}\\\\\n", + "\\sum x_{i} & \\sum x_{i}^{2}\\end{array}\\right]\n", + "\\left[\\begin{array}{c}\n", + "a_{0}\\\\\n", + "a_{1}\\end{array}\\right]$\n", + "\n", + "\n", + "$b=Ax$\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Example \n", + "\n", + "Find drag coefficient with best-fit line to experimental data\n", + "\n", + "|i | v (m/s) | F (N) |\n", + "|---|---|---|\n", + "|1 | 10 | 25 |\n", + "|2 | 20 | 70 |\n", + "|3 | 30 | 380|\n", + "|4 | 40 | 550|\n", + "|5 | 50 | 610|\n", + "|6 | 60 | 1220|\n", + "|7 | 70 | 830 |\n", + "|8 |80 | 1450|" + ] + }, + { + "cell_type": "code", + "execution_count": 68, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "a =\n", + "\n", + " -234.286\n", + " 19.470\n", + "\n" + ] + }, + { + "data": { + "image/svg+xml": [ + "\n", + "\n", + "Gnuplot\n", + "Produced by GNUPLOT 5.0 patchlevel 3 \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\t\n", + "\t\t-500\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t500\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t1000\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t1500\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t2000\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t10\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t20\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t30\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t40\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t50\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t60\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t70\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t80\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\n", + "\t\n", + "\t\tvelocity (m/s)\n", + "\t\n", + "\n", + "\n", + "\t\n", + "\t\tForce (N)\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\tdata\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\tdata\n", + "\t\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\n", + "\t\n", + "\tbest-fit\n", + "\n", + "\t\n", + "\t\tbest-fit\n", + "\t\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "drag_data=[...\n", + "1 , 10 , 25 \n", + "2 , 20 , 70 \n", + "3 , 30 , 380\n", + "4 , 40 , 550\n", + "5 , 50 , 610\n", + "6 , 60 , 1220\n", + "7 , 70 , 830 \n", + "8 ,80 , 1450];\n", + "x=drag_data(:,2);\n", + "y=drag_data(:,3);\n", + "\n", + "b=[sum(y);sum(x.*y)];\n", + "A=[length(x),sum(x);\n", + " sum(x), sum(x.^2)];\n", + " \n", + "a=A\\b\n", + "\n", + "plot(x,y,'o',x,a(1)+a(2)*x)\n", + "legend('data','best-fit','Location','NorthWest')\n", + "xlabel('Force (N)')\n", + "ylabel('velocity (m/s)')" + ] + }, + { + "cell_type": "code", + "execution_count": 72, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/svg+xml": [ + "\n", + "\n", + "Gnuplot\n", + "Produced by GNUPLOT 5.0 patchlevel 3 \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\t\n", + "\t\t-2000\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t-1500\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t-1000\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t-500\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t10\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t20\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t30\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t40\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t50\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t60\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t70\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t80\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\n", + "\t\n", + "\t\tresiduals (N)\n", + "\t\n", + "\n", + "\n", + "\t\n", + "\t\tvelocity (m/s)\n", + "\t\n", + "\n", + "\n", + "\t\n", + "\t\tModel does not capture measurements\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\tdata\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\tdata\n", + "\t\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "plot(x,y-a(1)-40*x)\n", + "legend('data','best-fit','Location','NorthWest')\n", + "ylabel('residuals (N)')\n", + "xlabel('velocity (m/s)')\n", + "title('Model does not capture measurements')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "How do we know its a \"good\" fit? \n", + "\n", + "Can compare the sum of squares error to the total sum of squares of the dependent variable (here F). \n", + "\n", + "$S_{r}=\\sum(y_{i}-a_{0}-a_{1}x_{i})^{2}$\n", + "\n", + "$S_{t}=\\sum(y_{i}-\\bar{y})^{2}$\n", + "\n", + "Then, we can calculate the *coefficient of determination*, $r^{2}$ or *correlation coefficient*, r. \n", + "\n", + "$r^{2}=\\frac{S_{t}-S_{r}}{S_{t}}$\n", + "\n", + "This represents the relative improvement of assuming that y is a function of x (if the x-values are not random and the y-values are random)\n", + "\n", + "For further information regarding statistical work on regression, look at \n", + "[NIST Statistics Handbook](http://www.itl.nist.gov/div898/handbook/pmd/section4/pmd44.htm)" + ] + }, + { + "cell_type": "code", + "execution_count": 73, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "St = 1.8083e+06\n", + "St = 1.8083e+06\n" + ] + } + ], + "source": [ + "Sr=sum((y-a(1)-a(2)*x).^2);\n", + "St=std(y)^2*(length(y)-1)\n", + "St=sum((y-mean(y)).^2)" + ] + }, + { + "cell_type": "code", + "execution_count": 74, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "r2 = 0.88049\n", + "r = 0.93834\n" + ] + } + ], + "source": [ + "r2=(St-Sr)/St\n", + "r=sqrt(r2)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Limiting cases \n", + "\n", + "#### $r^{2}=0$ $S_{r}=S{t}$\n", + "\n", + "#### $r^{2}=1$ $S_{r}=0$" + ] + }, + { + "cell_type": "code", + "execution_count": 76, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/svg+xml": [ + "\n", + "\n", + "Gnuplot\n", + "Produced by GNUPLOT 5.0 patchlevel 3 \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0.2\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0.4\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0.6\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0.8\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t1\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0.2\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0.4\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0.6\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0.8\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t1\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\tCase 1\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\tCase 1\n", + "\t\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\n", + "\t\n", + "\tCase 2\n", + "\n", + "\t\n", + "\t\tCase 2\n", + "\t\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "gen_examples\n", + "plot(x1(1:50:end),y1(1:50:end),'s',x2,y2,'o')\n", + "legend('Case 1','Case 2','Location','NorthWest')" + ] + }, + { + "cell_type": "code", + "execution_count": 77, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "a1 =\n", + "\n", + " 0.0482496\n", + " 0.0017062\n", + "\n", + "a2 =\n", + "\n", + " 0\n", + " 1\n", + "\n", + "Sr1 = 0.83505\n", + "St1 = 0.83529\n", + "coefficient of determination in Case 1 is 0.000291\n", + "Sr2 = 0\n", + "St2 = 1.0185\n", + "coefficient of determination in Case 2 is 1.000000\n" + ] + } + ], + "source": [ + "b1=[sum(y1);sum(x1.*y1)];\n", + "A1=[length(x1),sum(x1);\n", + " sum(x1), sum(x1.^2)];\n", + " \n", + "a1=A1\\b1\n", + "\n", + "b2=[sum(y2);sum(x2.*y2)];\n", + "A2=[length(x2),sum(x2);\n", + " sum(x2), sum(x2.^2)];\n", + " \n", + "a2=A2\\b2\n", + "\n", + "Sr1=sum((y1-a1(1)-a1(2)*x1).^2)\n", + "St1=sum((y1-mean(y1)).^2)\n", + "fprintf('coefficient of determination in Case 1 is %f\\n',1-Sr1/St1)\n", + "\n", + "Sr2=sum((y2-a2(1)-a2(2)*x2).^2)\n", + "St2=sum((y2-mean(y2)).^2)\n", + "\n", + "fprintf('coefficient of determination in Case 2 is %f\\n',1-Sr2/St2)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "collapsed": true + }, + "source": [ + "# General Linear Regression" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In general, you may want to fit other polynomials besides degree-1 (straight-lines)\n", + "\n", + "$y=a_{0}+a_{1}x+a_{2}x^{2}+\\cdots+a_{m}x^{m}+e$\n", + "\n", + "Now, the solution for $a_{0},~a_{1},...a_{m}$ is the minimization of m+1-dependent linear equations. \n", + "\n", + "Consider the following data:\n", + "\n", + "| x | y |\n", + "|---|---|\n", + "| 0.00 | 21.50 |\n", + "| 2.00 | 20.84 |\n", + "| 4.00 | 23.19 |\n", + "| 6.00 | 22.69 |\n", + "| 8.00 | 30.27 |\n", + "| 10.00 | 40.11 |\n", + "| 12.00 | 43.31 |\n", + "| 14.00 | 54.79 |\n", + "| 16.00 | 70.88 |\n", + "| 18.00 | 89.48 |\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 78, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/svg+xml": [ + "\n", + "\n", + "Gnuplot\n", + "Produced by GNUPLOT 5.0 patchlevel 3 \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\t\n", + "\t\t20\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t30\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t40\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t50\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t60\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t70\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t80\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t90\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t100\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t5\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t10\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t15\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t20\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\n", + "\n", + "\tgnuplot_plot_1a\n", + "\n", + "\n", + "\n", + "\t \n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "load xy_data.csv\n", + "x=xy_data(1,:)';\n", + "y=xy_data(2,:)';\n", + "plot(x,y,'o')" + ] + }, + { + "cell_type": "code", + "execution_count": 46, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "xy_data =\n", + "\n", + " Columns 1 through 7:\n", + "\n", + " 0.00000 2.00000 4.00000 6.00000 8.00000 10.00000 12.00000\n", + " 21.50114 20.84153 23.19201 22.69498 30.26687 40.11075 43.30543\n", + "\n", + " Columns 8 through 11:\n", + "\n", + " 14.00000 16.00000 18.00000 20.00000\n", + " 54.78730 70.88443 89.48368 97.28135\n", + "\n" + ] + } + ], + "source": [ + "xy_data" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The model can be rewritten as \n", + "\n", + "$y=\\left[Z\\right]a+e$\n", + "\n", + "where $a=\\left[\\begin{array}{c}\n", + " a_{0}\\\\\n", + " a_{1}\\\\\n", + " a_{2}\\end{array}\\right]$\n", + " \n", + "$[Z]=\\left[\\begin{array} \n", + "1 & x_{1} & x_{1}^{2} \\\\\n", + "1 & x_{2} & x_{2}^{2} \\\\\n", + "1 & x_{3} & x_{3}^{2} \\\\\n", + "1 & x_{4} & x_{4}^{2} \\\\\n", + "1 & x_{5} & x_{5}^{2} \\\\\n", + "1 & x_{6} & x_{6}^{2} \\\\\n", + "1 & x_{7} & x_{7}^{2} \\\\\n", + "1 & x_{8} & x_{8}^{2} \\\\\n", + "1 & x_{9} & x_{9}^{2} \\\\\n", + "1 & x_{10} & x_{10}^{2} \\end{array}\\right]$\n", + "\n", + "The sum of squares residuals for this model is\n", + "\n", + "$S_{r}=\\sum_{i=1}^{n}\\left(y_{i}-\\sum_{j=0}^{m}a_{j}z_{ji}\\right)$\n", + "\n", + "Minimizing this function results in\n", + "\n", + "$y=[Z]a$\n", + "\n", + "->**A standard Linear Algebra Problem**\n", + "\n", + "*the vector a is unknown, and Z is calculated based upon the assumed function*\n" + ] + }, + { + "cell_type": "code", + "execution_count": 79, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Z =\n", + "\n", + " 1 0 0\n", + " 1 2 4\n", + " 1 4 16\n", + " 1 6 36\n", + " 1 8 64\n", + " 1 10 100\n", + " 1 12 144\n", + " 1 14 196\n", + " 1 16 256\n", + " 1 18 324\n", + " 1 20 400\n", + "\n", + "a =\n", + "\n", + " 21.40341\n", + " -0.81538\n", + " 0.23935\n", + "\n" + ] + } + ], + "source": [ + "Z=[x.^0,x,x.^2]\n", + "\n", + "a=Z\\y" + ] + }, + { + "cell_type": "code", + "execution_count": 80, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/svg+xml": [ + "\n", + "\n", + "Gnuplot\n", + "Produced by GNUPLOT 5.0 patchlevel 3 \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\t\n", + "\t\t20\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t40\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t60\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t80\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t100\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t120\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t5\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t10\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t15\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t20\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\n", + "\n", + "\tgnuplot_plot_1a\n", + "\n", + "\n", + "\n", + "\t \n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\n", + "\t\n", + "\tgnuplot_plot_2a\n", + "\n", + "\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "x_fcn=linspace(min(x),max(x));\n", + "plot(x,y,'o',x_fcn,a(1)+a(2)*x_fcn+a(3)*x_fcn.^2)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### General Coefficient of Determination\n", + "\n", + "$r^{2}=\\frac{S_{t}-S_{r}}{S_{t}}=1-\\frac{S_{r}}{S_t}$" + ] + }, + { + "cell_type": "code", + "execution_count": 49, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "St = 27.923\n", + "Sr = 2.6366\n" + ] + } + ], + "source": [ + "St=std(y)\n", + "Sr=std(y-a(1)-a(2)*x-a(3)*x.^2)" + ] + }, + { + "cell_type": "code", + "execution_count": 81, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "the coefficient of determination for this fit is 0.880485\n", + "the correlation coefficient this fit is 0.938342\n" + ] + } + ], + "source": [ + "r2=1-Sr/St;\n", + "r=sqrt(r2);\n", + "\n", + "fprintf('the coefficient of determination for this fit is %f',r2)\n", + "fprintf('the correlation coefficient this fit is %f',r)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Compare this to a straight line fit" + ] + }, + { + "cell_type": "code", + "execution_count": 83, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "St = 27.923\n", + "Sr = 9.2520\n", + "the coefficient of determination for this fit is 0.668655\n", + "the correlation coefficient this fit is 0.817713\n" + ] + }, + { + "data": { + "image/svg+xml": [ + "\n", + "\n", + "Gnuplot\n", + "Produced by GNUPLOT 5.0 patchlevel 3 \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t20\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t40\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t60\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t80\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t100\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t120\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t5\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t10\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t15\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t20\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\n", + "\n", + "\tgnuplot_plot_1a\n", + "\n", + "\n", + "\n", + "\t \n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\n", + "\t\n", + "\tgnuplot_plot_2a\n", + "\n", + "\t\n", + "\t\n", + "\tgnuplot_plot_3a\n", + "\n", + "\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "Z=[ones(size(x)) x];\n", + "a_line=Z\\y;\n", + "x_fcn=linspace(min(x),max(x));\n", + "plot(x,y,'o',x_fcn,a(1)+a(2)*x_fcn+a(3)*x_fcn.^2,...\n", + "x_fcn,a_line(1)+a_line(2)*x_fcn)\n", + "St=std(y)\n", + "Sr=std(y-a_line(1)-a_line(2)*x)\n", + "r2=1-Sr/St;\n", + "r=sqrt(r2);\n", + "\n", + "fprintf('the coefficient of determination for this fit is %f',r2)\n", + "fprintf('the correlation coefficient this fit is %f',r)" + ] + }, + { + "cell_type": "code", + "execution_count": 85, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/svg+xml": [ + "\n", + "\n", + "Gnuplot\n", + "Produced by GNUPLOT 5.0 patchlevel 3 \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\t\n", + "\t\t-15\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t-10\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t-5\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t5\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t10\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t15\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t5\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t10\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t15\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t20\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\n", + "\t\n", + "\t\tresiduals of straight line\n", + "\t\n", + "\n", + "\n", + "\n", + "\tgnuplot_plot_1a\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "plot(x,y-a_line(1)-a_line(2)*x)\n", + "title('residuals of straight line')" + ] + }, + { + "cell_type": "code", + "execution_count": 86, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/svg+xml": [ + "\n", + "\n", + "Gnuplot\n", + "Produced by GNUPLOT 5.0 patchlevel 3 \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\t\n", + "\t\t-4\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t-2\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t2\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t4\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t6\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t5\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t10\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t15\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t20\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\n", + "\t\n", + "\t\tresiduals of parabola\n", + "\t\n", + "\n", + "\n", + "\n", + "\tgnuplot_plot_1a\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "plot(x,y-a(1)-a(2)*x-a(3)*x.^2)\n", + "title('residuals of parabola')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Warning \n", + "**Coefficient of determination reduction does not always mean a better fit**\n", + "\n", + "Try the function, \n", + "\n", + "$y(x)=a_0+a_{1}x+a_{2}x^{2}+a_{4}x^{4}+a_{5}x^{5}+a_{5}x^{5}+a_{6}x^{6}+a_{7}x^{7}+a_{8}x^{8}$" + ] + }, + { + "cell_type": "code", + "execution_count": 90, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "a_overfit =\n", + "\n", + " 2.1487e+01\n", + " -1.4264e+01\n", + " 1.5240e+01\n", + " -6.0483e+00\n", + " 1.1887e+00\n", + " -1.2651e-01\n", + " 7.4379e-03\n", + " -2.2702e-04\n", + " 2.8063e-06\n", + "\n" + ] + } + ], + "source": [ + "Z=[ones(size(x)) x x.^2 x.^3 x.^4 x.^5 x.^6 x.^7 x.^8];\n", + "a_overfit=Z\\y" + ] + }, + { + "cell_type": "code", + "execution_count": 91, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/svg+xml": [ + "\n", + "\n", + "Gnuplot\n", + "Produced by GNUPLOT 5.0 patchlevel 3 \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t20\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t40\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t60\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t80\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t100\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t120\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t5\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t10\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t15\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t20\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\tdata\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\tdata\n", + "\t\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\n", + "\t\n", + "\tparabola\n", + "\n", + "\t\n", + "\t\tparabola\n", + "\t\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\tn=8-fit\n", + "\n", + "\t\n", + "\t\tn=8-fit\n", + "\t\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "plot(x,y,'o',x_fcn,a(1)+a(2)*x_fcn+a(3)*x_fcn.^2,...\n", + "x_fcn,a_overfit(1)+a_overfit(2)*x_fcn+a_overfit(3)*x_fcn.^2+...\n", + "a_overfit(4)*x_fcn.^3+a_overfit(5)*x_fcn.^4+...\n", + "a_overfit(6)*x_fcn.^5+a_overfit(7)*x_fcn.^6+...\n", + "a_overfit(8)*x_fcn.^7+a_overfit(9)*x_fcn.^8)\n", + "legend('data','parabola','n=8-fit','Location','NorthWest')" + ] + }, + { + "cell_type": "code", + "execution_count": 92, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "St = 27.923\n", + "Sr = 0.77320\n", + "the coefficient of determination for this fit is 0.972309\n", + "the correlation coefficient this fit is 0.986057\n" + ] + } + ], + "source": [ + "St=std(y)\n", + "Sr=std(y-polyval(a_overfit(end:-1:1),x))\n", + "r2=1-Sr/St;\n", + "r=sqrt(r2);\n", + "\n", + "fprintf('the coefficient of determination for this fit is %f',r2)\n", + "fprintf('the correlation coefficient this fit is %f',r)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Linear Regression is only limited by the ability to separate the parameters from the function to achieve\n", + "\n", + "$y=[Z]a$\n", + "\n", + "$Z$ can be any function of the independent variable(s)\n", + "\n", + "**Example**:\n", + "We assume two functions are added together, sin(t) and sin(3t). What are the amplitudes?\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 93, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/svg+xml": [ + "\n", + "\n", + "Gnuplot\n", + "Produced by GNUPLOT 5.0 patchlevel 3 \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\t\n", + "\t\t-1.5\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t-1\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t-0.5\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0.5\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t1\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t1.5\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t2\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t4\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t6\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t8\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t10\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t12\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t14\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\n", + "\n", + "\tgnuplot_plot_1a\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "load sin_data.csv\n", + "t=sin_data(1,:)';\n", + "y=sin_data(2,:)';\n", + "plot(t,y)" + ] + }, + { + "cell_type": "code", + "execution_count": 94, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "a =\n", + "\n", + " 0.99727\n", + " 0.50251\n", + "\n" + ] + } + ], + "source": [ + "Z=[sin(t) sin(3*t)];\n", + "a=Z\\y" + ] + }, + { + "cell_type": "code", + "execution_count": 95, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/svg+xml": [ + "\n", + "\n", + "Gnuplot\n", + "Produced by GNUPLOT 5.0 patchlevel 3 \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\t\n", + "\t\t-1.5\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t-1\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t-0.5\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0.5\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t1\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t1.5\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t2\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t4\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t6\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t8\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t10\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t12\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t14\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\n", + "\n", + "\tgnuplot_plot_1a\n", + "\n", + "\n", + "\n", + "\t \n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\n", + "\t\n", + "\tgnuplot_plot_2a\n", + "\n", + "\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "plot(t,y,'.',t,a(1)*sin(t)+a(2)*sin(3*t))" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Octave", + "language": "octave", + "name": "octave" + }, + "language_info": { + "file_extension": ".m", + "help_links": [ + { + "text": "MetaKernel Magics", + "url": "https://github.com/calysto/metakernel/blob/master/metakernel/magics/README.md" + } + ], + "mimetype": "text/x-octave", + "name": "octave", + "version": "0.19.14" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/lecture_17/lecture_17.ipynb b/lecture_17/lecture_17.ipynb new file mode 100644 index 0000000..a9fec04 --- /dev/null +++ b/lecture_17/lecture_17.ipynb @@ -0,0 +1,74 @@ +{ + "cells": [ + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "setdefaults" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "%plot --format svg" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Questions from last class\n", + "\n", + "### Questions for me\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# General Regression (Linear and nonlinear)\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Octave", + "language": "octave", + "name": "octave" + }, + "language_info": { + "file_extension": ".m", + "help_links": [ + { + "text": "MetaKernel Magics", + "url": "https://github.com/calysto/metakernel/blob/master/metakernel/magics/README.md" + } + ], + "mimetype": "text/x-octave", + "name": "octave", + "version": "0.19.14" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/lecture_17/octave-workspace b/lecture_17/octave-workspace new file mode 100644 index 0000000000000000000000000000000000000000..52eed6b91409ebc41b42f7289e345054fd896bd1 GIT binary patch literal 23760 zcmchxb7>xlQxrusX`bhK zl$7zi_G#_+=ks}<=llGAzkhzc)P9|__t|@`d)@bSU)R0vV~3XhF#~NyCB>}*0|W#F zR*_!;fxkn^pR>pG^>mE>{QdiPAbG|94kN$hN1EXuZW#RMu_cFLFIuIz&t(mJYD1p#HDB|7#VFlQ;iu6$Au+>j@dpQAxzX zrY{3BL^Cnl(n57tT0V?>7Bm%1D#tdVPopIJsvy*~#9msg4mP_sNGattK%=PR*w=Z@ z@KVR4sEMtR@g6Mq=wTZ+2_Ce%Z`O{Bq9?uLdfM^CyXM!jx^}FYIpNlU-R*Em|E{jL zq7D8Y39>(fTHxrZepko43A-08eIYTo0Y{okyJqjJ#kQ9HFVv@3Vf>hJyB$BhUVraQ&5-X5MCvk8-VMt-K2p z2bg`#Z{yu)tRCbV+|IK;|7mFNpLX7Naj)GLpW1oDcm{UkB-(iiy}Z%)YFc@*7P~`A zRa$r_ukLz0K%|LRvd+@z-OYO5Z#mx1SeqJNnbq(gty?R3FJ;%6zqwh=v(x`3b+9*w zw`E@ZZOtXAym9j-Q>~VV@dV2@RFEUS4!UDP-bbcgj10LB=NFBLQ?6@J&na9op0~D7G zCi6m&x$-SCPt@i`%pmi|_l~6v#11&7o@Pnxf?U^E1Bsm=Gij$fu^X011wJBnM7Or# zLt-~wuFLObOYx4*IeB61?*3F{wUs6ROK9DcFu-}SP>)CO*yGvcIXhT8R2zL}&tskCNGmq(9jrI=?U1656{!Ci1jC4*Mil}Y}^g-eaMfq`Mo^WUwKIn_1O9G zzC9Yi&X*@SFj<9wZpt#hAwS$r69X|fuEDYn@!6C|&y&oEj-o>+a z=!w~}fwhBjs9F(ght=+igIIhTz2)N`)(&C%C;9l}XP3zLhbiKHR;(Qk`#uR^?J(K% z^=sA+UiAh0SvzF?x|DQ`*kN`~#`HE~hl!0V?N~dEds$`1+F?!myFbr}KkW5!>*D)E zx~Ma22X#sFEY=_9y*}#aNbIn=dBI244mFz;6InY*esJ5&+QD>rt2^rt$^&NEvUd2G zaXy)~gJFE;7}gHnF3Y!BN}qhqW|(9AAo(AXG<$N`v(4U0&l}sJbm8d&i*XfT)6TJf zOKJZ~O6(c_M`n6PznmTAk9?)vfy&(n`#YUT&zxjU~ zoL^nH*s|GD%-QHZ;{or4nc@78oj1_dJE8M$Y589dWG~>4@k0BVV>TuWn8~U?AY-YJ z*vj-(rCyoC1V$}>pr<6B$n2Qvn?Lv~V(nUY?o~?UZFJwAxXkc61a`Xrbx2a=5lHdX zx%Yl0o8ph1yVO6*!Q`N&t*grt8ulryMTE%ja`sI{ZC&lN+UN9Fk~avgVn=f6qQ)gc ztj?X}KEU4Kgs%RX|4$!cso2SLmoMm^*JmH9Nq+xF+Qy$Z|DbYEoBhWzX7p&+f9$y1fAn``SRiyhSY`v0wJcKC2w4eMe~9K*VK>*$6}20h|7iI-t{JG z$b{HXk%_!1HIV7(S~^Ih04l!)geKZF(DM}-{rrPNaApVmJY5bd?x?}vUrCE@iL~Rb zsSdfm0d}2NOC?R)vGNm7qVQY`=I&bZd*ta3e5y&fs)i0s9BUwIy{#FNDhqX2Z|K61 zGi5pw19_X}NxAF3r)D=|Abba=ACAAHG@Zj1O>N1rdi2&D=K%o^K- zOICefx2_q$*j+rc&-ZXU9nV1K%zNR)cZ=JQy?SExqAi1%gLNh!?8kP(s7Z~?cQB)D z>Z-HwY!@cm+ULlq4q-IbRhOJfZK2~G$_$*6+BmGd0VW$`-i$stjB${N-1wrWijH59 zNoX1~`Ssfz6#dB$@6{a61R5k54tlE` zzaq?-Hh=N`+0@Bf8-9wc?-)kGEp6JWvn@RL#l?$uCyimQjU2y0{7xIM-yR}Nc1?zV z(WhG8`ico*8&X6V#l0%dPuExTJWt*`vwDLlpi=1`*8lLgsT zyiLOIiM_@$?hn#(avaNfK8{P=ls}DS1U_rpE6%Ut_1kY8v)$u~=B3dMye%sYCdg-v zW6Jv$mZg(q=$F!vEu$*N2uL;9-oMetyCN~{(WG24X2s=)zoyu>@wT0vzC~PZJo6#s z=!5kS+Ia!pkB5#e9M4=^?poTN)5hz!_XOtIArrTn6&<`CE?=$-mQG*_M{eLZt3dzyNHUk&7ua4bZ9s{lWo<))Bolng+vdGw2SRW9cvPRB$YlM<6h7J6 zid(sF50|w}WWJ5k&Kf?kgZifwqg&O|o{`px@UT6(IUQ0=^19#S#P7DFe?2EL-(}q= zecacKiu<)&)4C@y^_YJ2%g7o;l@B8RJekpmHGT3}q6U`pWl~=EPG&a!+_U@I(PH$k z_Y|h^!nof*22>+PC~|q!uPKaK-JgoTL1pM)e`)5YM8J#AZ*}PEo?5i$hBR|xNXx@G zt9q17R$9C}Nt&tJtWe%Kt`#X88OI$1r!qAILS||YZ=>-tmH6{?S5Jv99F7s(z1e&! zvwzl+i52pl==YbYjLXQ`b2`6tpj$R;^tz!kO!NCjhrX$8DBD&+{6~ff*{pEZ{!krK z4cpUjM~3ND*tEIjQz82ONrq9qCt&l}$8nHPzF$ugPy zt6Z1HmGZvtzOib7l`Jzlq<7ew6?MD{lSjngWEubJG2en7cJPKg=>9!dXd1J(Z`sLh z-krPwlY`Y2cT8jaPMD1tKc|>l%*j#3+Cd|#dHr}M#|)jPU2$W4H7`p^$xqBlj?wrb^Z2D~ z6;C>8&9JeBa?Gk6)Aw0F>v$7)|I`{IIfHrT{OoD_wKm@A%F)TA56)ngE}zpWDbU8# z)=QBd=`w>^w>#;vkwq8p*r03I1WRWy*-mAP1zkIMVYZJ-hfJ1dx^;|Z9KO{-{a>DG z)zUk-d}K4vH|me0fV(`iH#v2@-j7P&w>2W)e^mr+*DWn zs-zIjV{_R5$e!5j!k+vm!*H2}%P3qX>3?1EPv+n<2A3(g48dgvE+cT6p#MI-{~$ky z_#D#re_nv$kiGvt0~rp&Oa9*1-s!g0vPAsUBd9D;Gk#UU1lRQ>al zVmM^t5Q#$~4uLr2;Sh&I8V+GNWZ@8nLy~@bOl3Ia;1GjD3JxJSWZ)2iLjn!~xX9-s zo{MxY!nw%qx8pR1i)1c>xya=rmWxy_Lb=G~B9esuah#>w&|L|On-*UY6zw%jmrvG{cdYng(%JithC+9~0BcE>|{Q5s0 zeQM|c?;9s}qW^dlE_Vm?Bs!m>uPSTN&e+|En4 z=ym4EICZnK!^(nQe+N2&H%Hj`-lLzh!l0I$^OX`G($8Dt^Y~RE4Z1e;KDKaKl=LRf z*pA-M9;f!N%D-dbK=12_g~mzE`|Tdn`#WLalT`vU-JR)ko*?r@(zw3hr}TL)a4B9j zGcEoZeXc7`|4Lfn{LYO&-yKhdrUpbcy3=uZU?7>DREGy0k0(ye^&o{FFARjtTAGjPnR+2E>myXvL121~iO&jA!$L$9l);#0=>G=JTVKen- zw^RU~M*tRj9vgC4HjvII5H1lO!YMO@=)8h3=xwEh;=EuwzhLO(Or7Si@Hw65b7WQ@ zoAY~l2%T>T7Iu3q)?O1z=N*de`~ij0`G?^aSsh9LFlvV|M2m(8ZJ!-Z?GcU!$*izj zA>q_6;g~4f86Be(LG2TP9kWGVHbq5HJ4N6YS>4HtBdNV25xs0z%I=qu)NYYzSab1% z#mXpZzbHs;waIvq6h-YAjU9W!zqGB5ruK}+twULbGO5wjt}%!{-dVP9OANJd3>tMs zYVLiGp>~di)Mcf{*Slh=y<@TC+OD>)FR|3_akzE+;#ax-an$~Ch+}*aF-oEuXI|3v zdI>U^33hES>AEFA>itFXFM+OK0(SBpAc3yqE12bn%gr@2-YyEsh*Y!0T z>pSN?zW$o7?`urz5FsJSqwCDW&R(Tuf6aMxy?IzoqC;YEB3*Z~H!wKfGjd%bUH?Q} z7Tn~s%`%btK_VWGD)g&QO{D&i2>&r`1Se6yNWvQuT@$w^QU6FnvH0E4=m$yEPm=IW zN+M!MMiTXxBnY$NHRcWVn>Uc3z9iOc_Z#XzZ?Kv~=Op_#)Q{fapyGjq*xWbNpWcAK zoD<$szj}*D3s?YnOa1FD{Fe+*zWwAa^|QBlL!x_1$y@4gZ&AXA|D`fe@wyZ-E4%VP`^yUp?!V1FSS#sf2QEF`rU%P zeks&XQ}F1BL~(mV3ia1_2+#~Dy*uk2_1kxNb7Dz_tnNGNzwc0TI<4x}^LNyb-{IT2 z1GQ@H@2Ee&hwvqK#otrEevg@k#!dJ2-c$d65B?Tc)O+gZ?{Ub~xt;gzJ@xlg7?{oM zIlCD_RaW}O5-67C3ZRlG1F*Vq@m|= z-=Bk~X*534Ak0q~X*5pKzA5cjjxX&U_1WZr;jwwK7u4K@uR}& zG~UuNCS~TtA3M`&+@*u>Zno(({?f7L(uXr=P%wXQ|ILUO%I>Hel|D!=Gu~enwB@66ND=pK1Jl#+bG= zl|L1qX&h&OT`eb>3>wcFSi}E7294_s9R6XvJl!XQ#&-q`{tU;7`V1Q9nPd-#r0a2+ zG~P3@caWIOwgs6q?lYlDvWw)tOd9{0I5*5^TD?Ig#eqz)JEl>OGbtWq;?77m!)H=l z$b`e_Yix*6e8_|+$xai0WKx{S0v`?~vnXC<;pN!93en55C~jmSRqUA}!E}lrS;(DG zJ7>qWEQ%vps3O^oU@pazEVNH#5g?1=N)~=jdZkP-k>X1h1f_?lmTb1AOo!a?KJK?1=PU-Q7HOOkmMXY&xMwe3j9(maZ{c}UP^KaodqHxFqi zOOM_&$)ot22Yz@z%cD4)hpIDYwWht!qj;Q$_OmSG$fLNNhhOJ^Yrp=RNAWoy!}Zpk zyf01m6!|!v4>A1*EM=v5oewfs0wDPmxAQUAXwsRkTlo~f^RfJjwl2>rpW=8vHkmv> zYn7Z&@jM^#rXo5n19ks+)zOAz5sVE*bx;_ z+%JIR9lk*+{ujXWo{zzdzyg{F3J_}5YS{gufaZe&Bs^pjZUM~;1xT~Lc7<>RnjZ?0 zXPa{Mpkg7-6NRX<=K}=I7lmkd+)HLqNb^P^emOleB{_!Xk3tN8T6^PBSRu_Lg&5C= zheDc93L)!unPjj+npcXzr{khUG`|#Kxz`YjxATi=o+$#$W~}!V(R@>ceSQvP#zi#m z6hW&Wuxb7&!ug=-_rJ#!(L7Xye)AX6d{hLM-PrUN(Y#a)N0uH+6w~}v46n#_*0Yxu z(>zrS{^jb$G+z~ikQrj3Vw$&#!Ll7&mtvZ~ijnt{b&I(OKAQp z0ZArq@86ctJXivj4cXU}(0o{eeVGeLh?Ua3SPHG|YhH8YN@;#9#f2{^-apru(mYuT zlf2QssV7TmzAOdHjvVfk(!5y;$D(Kcgc8yGSqiTbHbIusJX(sdGR2@@O{FxSmf}^# zWx^23XkIM?%a(|NXnrk2Ud<2^@?|v7mZ7?CTiCpFWi;QGp`(E%^kp>fmf?3(X~c)n zGMazO(4X$gXdW)Z`2K*W`M4Z>xg%Om^Kv=nef=H#N4cEl=W?vz6UuU$r^~^zDJT7M zny<^jr;85dG;fzft3QF!{9TRpn0fjP2{-> z$^$CUMbwR?X$9p273dXY1GIwjf(r2IhFb;Y2Nf8?-af2?@`MVE9A$YjDzSp{g$jrg zbtIWpL3u+3#D&-jK?UUx6`0Hi&I-ySDlmP_gwxvxR#HAewslx}DOyQ+MJ1Gpx|%3k zN%=)37LR2s3YC;+RAS{gKi!5km6UH(VjVxBRZ`wji7osEuB7~<61#~yn|QgB@{mgG z7hgmmwUY9YN@%cj-l3B6l1gYxnqJu9TS@szCCFfiAg-i5r4o94fmccSN+tT?zLN5m zN|^E|TuJ#$C2miN{JNvJlJb};+?Q6{3N}Q58PU;$4XiuA=;?3fV-RPI^^Ec~TV$6o!!$R7Lqx70UVHTt#_P6}S zChB@hWHsey)mY3=K-H9|RRer_R!#X@HCT42T2oDVTQ#=wGjTQLZ`Ih%%qJAKhVr-? z@XK9dHI&cQK!Y#vYACO(0m~Lu7uHaIR|DO(Z%4dXUqg9b4fu3Kt%mZw8W^o-BeI6_ zz8aWrU^9FT<$pC`+2fu&HIxU|z={uGHIxt5z;-iRA*-Rhum(tfhRi z7A(8mt58dMWi8TGHSAw5uciF57Nqz_D1R;GnYG}jo5Qt~Z`PuG9~<$tly}yGWt)4i z*HZpji7VD3yGtwVy>h7wGRApqeva)v2`F~lMv)O%4h4Kz)u);l-JgQ zh(}l1HFcEV)?o?XA?hg4tpgE*ZXH^6l<$(>F2YXL^y?_^t;1G+L9>qX-#YNi8xD1p z2iM`iX+uH@>nIM0+u z$2-Dq4HV_`JRWEc$%i(Lniq1BMdztA4bB^85zy z)9-}_%J&;Uf{O_F2Fm*zAi*cJ4V3>kfJKvTj|Qp(8X!m5uzExT)dLM6#vv8?2C54h z&@XQrs6J@Gl84ocA67R|ozQ^(2RBf?(17)X9UmOrNOeOa_~qVljZ{B0VmE(*8>xWfC`Iu2U-=0qda8I9n(k6|O#8;vmL16Cu| z9gX0Z#~wFQ{m}>u!k!QLH&Pwa2>x{A8mS&>gdKkap^@s6MmY23OC!}MjbNj(r?HXh zlt%axHht(vBh@QS2zEb15W0!#mL^29<@||FRKGNVt;SlcG<^;a`g`30(G zs>7PWR-bKCnyDUZ#!-H`p_%HkW}G16U!$&>>a%9>;jpKf>a=EDBH~+NSPRu_E#Plg zO=zLItpzvu1=bd--&(*A-^DFd$F+cH|DtyrTd1CEfgLNa542ES*8;Y>ZKu;h^<4|x zS$8vTp*pVxtoS;5uZ8Np7VyifPgMRzkeUE1$^_oY@s@^1&Ks_{P@^H z^ay7n zv{L=pik6&9$5U6fQXScfuV2`LU@O&=tzfI|4jQdgSGGbRU!(5l`BthgTQQU`cv`8> zYz2S&!kYBV^Xs&&7+bW3WV}|YJ6pl0^Wm*jf3`xJU*K+~IfJW5 zy9JN^+Nkbr!wo*+Zln6Q4WyPx7NCvl;5Izqmw(!*9&Q7xj+7ePs4i{;ySwnXuZ`;C zcDQ$FkrH$})yeJPZ;wf~Q@z}d=UptIv{T*O4p!|bEo-OxxgG3ogwvLGs-xSH$d)?} zlHO;2-MbwG3`vEoo$Bg#d}7_xq@C*PcChM2nY2yz>+E*$m;G5g)!Xf;AXoo=-cEIQ zJNV{&-A?s)J6eB@y`KKLo$By*^vnBps>j>$gY0%Vx3^PWPQH7=|LNejX&U`oo94f| zi${ym#YV}Gf%79^kYQ)>VDmVX{OgxuIj1kA={-+qSUqcr|@Z5}^8mmdxThcNx08_KNgtuPZ_DaiCcZ#Xlpu65L1^~sG=bUc$78*=w)b(840CNt*3>EAMR{IZM;-w?8N z9@Chox)Et|Pp8rOOlQRSADB+(HJ#xv`eHdcKRHHlib+S(uo-loGe}<+S)RNZbiVS; zPlthINb+>vGZ}WZ4(rdP^PkDI5$i8oGmF|`7Q=djgy?K)kJ(HmS*_lR+0-ry4FC3_ z3e-M|47<9~*A=Oq6dBfg?6%FJ_L{@+_pznsQoGG%SZ~_dJeS&U9>aH^!}F*e=P{C^ zUkxmh=TUpkWAa1C5eq0$yDBk!_q0=@_Elnb@(rs*?X1jjP4iLRs|uh2c2{e zwSAZ)?zfI!ufTK<+;I5a@h$ZAJm!d?naPtlt~Xto$!v)9?>@4Ro>yT$Sgs2nuzNql z0)R33{cqhy|Mcr|o#0#tIM?~jb$oN3-du+_*V)Z=baS2DTn9JTxy^NKbDi2;hc?%l z&2?mRo!DFlHrIL0bzF0u)?9}**ICVVRCArwTn9DRIn8xUbDh#$hcwq2&2>a`ozPqd zG}rmebv$#O&RmBx*V)W(ow!^FF4uX>b=-2Dwp@oT*ICPT)N-A)Tn8=J zIm>m-a-FhVhb-3_%XP$Zov>U7EZ6zUb-c1)oiP62yXG_;MB~(5^swcSQ)Ah=0J!_u zT^cbf3dwrc?XR`Q;`6cOyJx<+^UA%N}wM$FGdil#>s~tW__J&E2i= zcvJGVAMtmwKup$n`*2qrJy|aMXi5a!wA)^+nRpS`QUkqI2gXA7T=wccI$rQNJYM!_ zWfXjG5BhXQ%oi0p0kb=g`9a7@>gV50UXT)RP|?`#Ow@d7c5AyGL`x^lZmtN0(V&v# ztkhu4dD(MwOk6aa<{2A?5Aa6HhoS1lfluI5thY(oAQsZYMJjJN#bE7*TWv?(Lm_)c zSM@-76kZopEb>!}MO@Cp2|eCqKBW&w^%@1E>R0o~)sasiy=p>(^p7yuJ{9iPdFc#q z3r(}puO7nW>NvSYDIsV-am~B;!7~*6t-H~@)Eo6n`zF*~55ZXVqGoXuFN{&nzjw0j z0habw-nwcQiv<#YzFc2vjrPcT(R{>1cdU?D_`66nYG;?JH^-pBA?B&vDH|-YA8}w* zg+KD5HeL$~zlq%3#VX_U?BU%pDDxv-U5#gk>N@efVREQ1uF@tF+FQeS zJ@~?YIR4~T*84NDC>TFbcdTj<)Eb*Q4#b`#)wA(`tfj+Iuzk+R z&HGNl>ge3Ji_UvM#mg$^$MIO)RmiOf{}cmjk6(6jhXc^@PVM&J`yrSqdGjvs`3%uVqA3Sj;2xINnC=C*d#2Aep(Vy>p@H!`6zo6I` zW!r$zcTsJ680P@KNG6M#3AcQOY7(K zFdV<0F<#0z9MadHDkx~i;iapL>Do>1IJM2;!l-7_HMr=vuxn2^j^@l)k$w?}A$GeT zr@t}8pVz`dmPWp~<953%QXmxPUS}-X>J$y>)NM7_JA$#PY1o!OHqo%@IN?5_%N^$X zrqyq7ibHU@(s&D{CwMSVY26yLaGZ>LBd9nd7UdruC%S8dLtf~D$g&29~Z_4SyM@jJtCc&nPs)+HgxIBdOg@OW2f*&7~hvyR33E~{B1C;DTi$)X)q z1BQ|6yOzPIpSBn<_*9I}K~FRnS;fgO^2H+^iwM1Gw$O~o|7_^zhVB`w?~A?j$8uYR zm)*y`;pRKrx;@4TE_zNDSFhZ_-gA2zEN9tZgwMIB2QEio!_U)SL-zz>PvPjvtE;?l z=yr^|{WAwBoN8_o5xkB`(?9K7R2PDQug>Mgs0U--m&c!Sf_!o1uzabycqq1R>bS9c zu`}d;OtTZ=`NE*DH~*=nC6a?PDw>^r5bvHSJTyEKy^GvBS|-M0qgG^s&6Q4``L6bn==obA~i0@Lt@zmJA_K&Ih+kP(M_ zouZ26o6q6St;FApVQb!Bam zQg8%z?~KtZw2#2q%+zF2nMnO@3Qx=9U7!$m&<7eLMloZAzLAsH^Y-qFU`uBbZW+3W}$4ojYXe_celm_ zooh0Q7LNe8p9fdZkHe&f5J}@BLC{jyvumFl1DX3Z+t>9*qf$H0NnmXpbS=+%I)?k< zlv$=m(L@J$t++PM#xD$rHVrpl^aPeA%T>H@1tY&jW~M{b87TiQ5gRom7=lTTey5vb zapR}jrUxAXNYO3Jo04gd!f3r0FU%s)XP=fZq1^-KvvV{@UH8UCVg2>XK13tJ(PW$J z&tUke-Kf|%E)3-(64M_D=p(K?t5ss-1&D07iyBxR3+0wH9Z~N{w2W0|R=S(wqL1D6 zpP8~q&^R%^d_g>f=eW$TB;B}yf2HpZm=l4dWfrFl+^&OQCz~!u*YsT(4-l4fL*(wSO z<`>q*CD`F$iE!BPN2ELW>yLI_UuVc)-?j0%>oZt426tw?j>4?8ABJ&C_mRGFpMvZ7 zDBPG_bw#Wz0{T(nQ$5$jV9N2Xg?Gq!(i7)ZwR=Qihty2*3r}vsvS^G#q_{h}+y*?5 znDqqH6n}pg7=0hV0}~xJ3hqO!WOmCr*FZGy*<98ouZJ^lhRbd~8-;GMsqd@Wg5bGa zWNLSbGp6aU{C>jz3~b{CwPZhEV0Vc$CNy5bEI$h|o?;Blg0+;A;e(5pbw1SkYa{H` z#bxqZK}cRuy|T)38`d^-tvnj|2m+I{a@DW+z+9(x;nYc1=n?($`LJvl_MCX!6EYwe zN!~4aS<8G-`S4J)NmUFMIkpOCzX^txu;!Z$qrxy#BK*Q)bqm-G41ewy8V2Eol6e6> zb`W+r@?e#vE#`TQKXHT?1eMX5cNRNGLiGF6p2b72z$2|&d{>k&HtjtjwJj? z_YaH4`<{B!Sq%}W`8G00ukbE{SL}{kvNjeapY5ZDT)K;EJD%;IpJfH{Kd+eZUEX*k z8CeqW@dlC+sX6waw0dF@(6wh3Ul<)k*jMkn%fw zx-Ic#h0F;7f`c7qeJS=OR=Dl*aoR}*BREgkAGLqCC*)e&%=gH7pd;gc=BVW{*m|z; z=MG_Kth3dL*xL9MLTf83%|5wfuXaoKIs3;5opaA?>gYIFU#XaX<#Zf2dbBlOH{CkUZTIlCNw(+7vtNv%%LrHX7xvzW640@Q=!)Xt-;n-CL6q4U^OFG<&`L z5hd~YH?~FL`t-k72IY7`?8CSe)5`r=`C-%wtp*vK3e~x8u^|rKdFlt>UkHV~q~ay3 z^InMdZ?+4yi^C2%C7HU60QBrnHBl-FgXrE={FrwE7cHG6tSmiIuH16o+Ry;q+ZszI z+>1uQuyZO8vb<64;ZU(%*c+$ZJu0OiIl}7Xp7pz@hN5tY^NnF5K45Z|x94s5!J^mA z+sx+&qP-w+?dHZ1)cEi3c^wyk-A9q8i9t9NCgm^_vLcav`MFR;QW$j3 z@6m9WXo0(75A#zN#Uiomh}(DK&&Ma+ty#Ii1HH}R#sfz2Zu1Y?Z3ZGQ7|4VlZQ6w68}GVGyjN2Q#5AxcD%Xi z9gAFJ(H|;ePZ3icUurqS61xZcN`6vvz|oy4k#1RpkM1iS@>!Mep<~11m%D4Bb7#=4 zV;aX$kgzu)vpWWiNYKyyWxhyEEj%wB8HTeCQPOebTLNE010H&x3q!N^mdP70#p1`% z(*+}Dg&?_Xh;n4*L%19l(Q65@hmTCX&pTCT15k`>!OhvH1?`c{ykhZX_xW( z?T6xr8LKvo2*A%B8@`?YXaj4Juq%0sqVTZzlh@e{@@mC7X?1&%8?9^~Oea8-?`>xik9T|`P(H|#m-$e3JYv z6T|R!#<{`wW!GYi*)8SSgzwoaXb&|W6ohiW3&(Hob%gKBX9m8@ykPQl(c00EEKnG@ zrq?LY8!n^NW``tsq9t4M+0>6D&$JFN-`Pmw_>;>Q(~Z&iwpvhgsahERPN|oUt8>C# zL>`BC!eiZ6bFA-n^Q)IpFrzD)AW>co^YG1HCca@DYOOejZZ$} zfH4Ob%&pZ4hnrDchLNiW4i8w8b@~qB+oRL=hQD#Z289{=7q-Tu_wB%;I*T3gF?XAz z#SlNtsh#%Ta&!P>uWjwp8W9HRjQ7{#dY+@#(L&+Ap+4F_N+|B{aD|)UDfQ^fHc*S4 ztJ%3J3?tvqi7VaZi`Dl(ulPp%Dz5Q(!LY03$a7peOZp<&wF@)%TNsL8#eYs5(2B&~ zf&0erwBn#PXhgWilt|3kHe(NrqKLog2hLShMx24{i z#~h(2iTmTB5asmbalRYE7A#)3FgOt1uZ|D(n-fKLg)f?ayy1?D`Jc@{ImM&6J49YY zGz#mpHaJyWBX&;zsDEOuCv?}(`(-&)A2u(9bu2e{V{^m%_E@2NFx@5`>6sjk^g3{N?hu?=B6-Q`Od{eFwdfe(I1@0lxrPI|6hKJ>vB$b}S@Q9&d_U7j}dvoHhn?ZrdX6{I8kGce-s}){u zCoiF@>-@CTnM5D$>#bR`#T>E4n-)#z4Z{cTzmZEPhhcZzMdi?v-j&lD{OdbMd*QcM z^}dHD_u(Gi=eqNZKmNAb&Dgaj40%d#H3qG+MCgSDSMQE;L4vsb(c?<4*zfjEHtM_< z9G%y?_f~qLICIkFYwB?*NwJ>~F?NB%Iceo-fKD*~@_Bo_Oq+{2-t>PLlr z+#z-K%ht}zp$I#p>X7oy3lEl;C~bNWhP6XF3j=ON;_O|^Vl}x)*f-7h@wAD7eC_s$ zvt!&ab$Xh#ig`Sqc9^fuZFfhx+S?T+b$U?#G-c)pl}P-RR~jur_)oNo%vje7C$cLd zshFm74^^RQTeqhjM9DfH^R(?Yo;vM(5ZV<3yRanFD+gon_1cUc+jDWaU)>sLdRz<7 zpLawn3J~4bQdMVcVhx3~Hx)-Wc)@<#P$cF=!#ezP*+Dl0GU*zQYgsFPEAhcX$q~6}j6-4UEF23)&ggWL-=oPPx8b{|vvB%JZa? z?XmWr$qDrlJ}_c#cMB%lLNFoJb%S>x4m4}ORQVQ-<8>4Kjf!Hi;cv}=&3)m>Ke)hA zZJZ|(udOi(Xp2Pi@3GZ`CwSuQ*wbM~gP+6COzqh3m>A>_o4vbWY#=&$hxguQ{E+Fr z|GX;seD3(SYZ?Z*;!TNI`Kllr*lx3Yr6q2I<*#=dcBchG#BXW*3{u}%VSV%aZRtSd zR=#$-sT+xBQUVV~{kVsgtW)kYQw&kF`L?tZt2h+m3LFD}!BN7+Nq=QauWOD=#g3VG@dhnKOA9|3EB^U;HG2 z_)T@t+LwQ;ZsH^_-|nnzB!;i@J$h`EAL_?V&zWcziIF!ZWk#&>!H{{ypBxGAlndXr zO=?vbUbQKC`MiA&xlOe)-y*$W`YbKyRhT>G1gftU2q543ThelQ*FG=Y+Ha#%eaa5M z?rS+VpT7;u8E5@xC`6L@PhT1IBpMq8ADJf+oGcieUw%;383m0wh0{Euv3)_M-h-{~ zC^9kr8u~p1QWKqj$Q6fT?&VwU9=9IDCiHZXYGDlW=9i27l_fefV{rFNRa@+r`}8Vj zehfb5mn8pqWsTqdov90cTH*PYXq#Oo zO&)P15QBegJ}kFC6c65iKRWDZ5XQfEv7D(C4nOJ9?YFxLPt^7-Hnns>&m>RR-l;^N zt$8A86muURhxcmcy|#t-0R=1V_h&KU)x^mAJ!jyGvKQukQTYA&VzSztAf#?@UGQCr z_`{%Q;{L++SiN0m-g{wEx13Nr_x&tyWJ(&n8=ZF_O=@C$M;HX7>`#j1xNC9nh<>}_ zP>T;5J0=EJ4hch$MI;Y z$;krU3-c_XQ0T04yWt5!`zi(>Cj8B0MULR@drvW%RN^*$48#39o83=8(n9;qp45Ve zH_>|{%lr<4!27m4#PX3dq|AqGY_NQS$di%_BSwTnW$O#WxAO@9wqPt11Z=T4@x_mC zdNxQ|*OxOqG7w8MY=aV5vyqK*Zn38VPDQ$3_NCq!QXCqb*o0g#=+*E5~(X* z7dtib4~hTtuY!i%ICBFr!COi{Uk^u%TDw4xqZ_{bs(GDf7mpUXX=^SkIz#jAhrbO4 z(b(Ok!X*3%!PBM3l!l3rx~=yB<#s7+q+UL=X>i#S3_j{UL$#Ff&BccYj_{2{y|wLQ zkyT_oolm~(6pw&*)~}P_?d`Dl!JFq@6>-=pVWle576R*B*=O&r-o?&c1yi#}Ti{7t zma>yjG>l?b*PQ$AjFJ$`H+>i4U@-4VjOL(Q2&+%l%PNh-z#jnxYvrO)uN!?UNIw+a z<=dQg5nc5tZETPHxm%=*`{NUr`HxX*cWRue!y_n7*gE93kUvH?Z%x_lrHP5VI$ajJ z$01~UK!jM92kh+D8NS#`^u;@I+4=HzkgANg+anW-s=GPU{ZyY|tc``n%ElNp&3Sxb z@W4pyS=_ZO-<$Y#x#@^T>xbY4XWE->b%5@;{Mi!=3}N>zchAHVo>+PB)|ifI9ylMb zvi0~8JLF|_B?u86V?|t8$|nZLT}zx3Lhd4LvTsE5OLKHT+g`OeFao=bwiLg3<&KhN z4^lSB#lw0N67FS_Z?o%sJ^iu67jqtXoDLzpbcpk%zHLEnIH40%*V*X~LyHrZVPnEz zzVPj3-K6Kpj@(u4=d2ImX&O75!kr+}wKgx>is`y%>RTR;$McI5`c{TR-)is(l?#!GIZ!a^^@a%8cnp7_Q5%k5HzYgH4URR=(fui0{#WSQHGECoGWQ0Q?gb!U13w{`ZxSx0Po@SFe%kCcG z<1()B>U{r3TJ|>dUY5*Mx@wDLmo&!oX(WD*jGeqL&l??8C(>VzI){n&gKIvzOrLBgd2zvk1RvN;<{ z9f?78AMWn3HHV{}d|yzOC+eK<_)NR$gZ$F+Ap?8^@iXs~qw>Zm$hXNZnRGQ60nvIJ zvsT|gLPb)iq-{J-_If6*JsXQrj|X}fwfG|}LMT&?)IokP3qDek8H8!to73%Fw9u&V zV&~z{w^3WMcc|>gAjnuu+0$tnhvx0BU&oMr2VK*Utu{_iaQ@fD7TvEm(6Ddpx#|c% zoYxI96&5ptN#mB=#&d#6x7n{J!@|9ZJ}JI1%Qza!ybGavF=05=CtJ5a-50WB3|3|Q zj)7ufedo7VahQ9`+}G$qB;Kak9MBjT0sDkRowvIq@V#ny&V=l(5F0Z|c-=!aY}}za zD?Ka(pHCK6>A(-cOH!qrQ%!(l(o^1)EbQlyRgQBkBlari-;eEnfdE*!z>|g%t!k^A? zoT`!$fnrQV+w&9?Jy$XD+hVz!jT@0c#nx$%D3b9vy$ zpf6!WH+wg(xaxxfr`h=`Z~dS&T>PBwN+0kJ9lbSmsx!XzN>g#T! z16-R_32&dT`r0y?*g1Wc&hkr<*j%xp{_Ed|DC|kUw~*AKhm;J<8kXP!d++5oM>IW< zB4cn}VZc>9D%n4ZxBMCIjWKAr{5Jy1G8?=$r^mtglT}imUpV%*TiWuDL}2|hn{e$HI`)-I90jAbz`6ndUq@uQW9kdr+Y(( zO68LLCwuzous!5|-%#Ec6-s!y?Kg|bzq0f((!kFk?{oled3?S#t0Wk?SZOmQ#PDlWs4T^#bO6#TuA(l60E2T#uAU)udVI76NpR2*?NoI zqEYoz?3YrRBgSoaU13b}S{e2g}th0;lrAQNAZ^etxAs7AsE)KYJzt%F&R_R-6*WCQU3;(Id+qf+>sh-A|DY<5k4uS*hK7c(s34<( zhK6BUzY8radv!IHzx z+||<3(aqY){Sc!~0uAjYnxc%9mRH8!LV%V5*njh?o-zFm3}1_$XjE|Frw0?q$RFkH zQCV-O#4e|fEjT)%>xXwb|8jxdYp2zdJIa1}ds zF$2_7?w>!HmUHhw2k7fK1qt1cgfujX9W{fQ7Vg$we0Ym4tL}_<>Ydq1^yUC^aB=Um z<9`S_K-zOZ?R+cq@4;R(x`Xy#r?ehQga5f$@JHp}vp;74r!HAYpi6d2&~>7hV8niw z>WL2v3uAv9p4!X)rPZomeB0h@kL>bfy0tr*4iQ*5>-70kwcNHBaqPEo&&Qh~`1Ug< zg_VY$kut)-!CZeYzG`{NXJ z2pW&bN7V}d#EjF5QV9ybF1lcHObu7kUym@OpU=ty(O3hc!F;ILs7R904 zmXw%+U!SmcADroTF-bS#a;NzT6Vmc~7H``2g@U|?!q3w>64(+?raO<0$ zG*0KMlP77mZ)VaRFqVgFb*_>M<2W;=;_5sd*4y_}iSt4a_0bFg$xL@kR~YT}K{3H*_}i z7&qoaS~1(WjNtr7wHEk0Z}0neE*OjjELM<0HgC2GF>rw0z+j$z1O|Ltn{!VfI|Fj9 z2a0M zCG^sFL6BQ)`RIx9`rb#k`(TjkI(DJ{+3>KEBG+pc$q^+8vUAk$^LDuI?}6EFs*S_h zLQq!Q2xQqES52oHKafxuv6_ChS#0!WtoRfY`%({NxYxdo?K@g?#9!7L>?!+ux_{P; zP&E`Xh4(|*F8$e&c3i6|2H>cmF|yxFAJx32n77Zjy~Mf-ZL;4?sp&vKSE&TZLn=WN z&n8x}u@3rIJ>)gn<=qS$1C^#cmK({gbr7!=jH$!?#_DJF;}SG>P3d1xt}D-QmR|hL zaaj4sJa5Bs*ira*luk`Sou*Ht=W)M@Z6+UR7ms`1KWXfCusAtse>?TLmf&iX zmS;u(IWG6S#EqB7V8GtvIo7A|H^q3>EF1oqzxm0lJJ*`tc$cIT!)|dFU5K`Lays%W zHREiEL`_-Bno=~Fh!Z=h310{4mkqO@gpDKUd3G1REy!18+@N#)-QHNDyP49@@mzB{ zvemYx-6XnxLBOLmj5`sFJgR^3O+;s4{TElq6=#JF&C1pAN;@GkFhaKOIA&$ zD+>$#SU5QMON02|-kZ)1WV(^r)W_2%DR*g<5Y`Kh(K>4yEN6^Hur1rIoNmj_LB6LN ztJrww%-;GfPC6C3^f{PVZd0!(ZQj#onuj|DnE{an=S7UV`_fH_(pZ}-5WBFZ*S z*ZC>{A=631UT(P9(Oj(mBh_I!cU{P8 zA0pX&W`j-WbGD1=KZv5fnxPtlr~0ayJkPvYtGdK|x{CU<0=xrb4JRpF410Wcp$o-G zIM%G($IY!$2Nx4t{k=nI^OMk>>UYRZ^7kt>YWq6ru`NQr8`afR9noJd58C>8VRk1k zUXyetx%FckFB~646q}2T!OtUJn_-J4A3rST?4fLm+B)wq8r9Xb)XMzH6aDccMIonrBNvn0nw*niqWVWSrvAGc4-OCyRhyVD z`XH){XS~P-#N~eCAa3Fw3g`XEqoR!l>b_6IlLLEJj4pV$7tPi?>^K$QR16l|%W>_c z|NNtgd;={QHZnJ{{=L;QWSp3euCMh3IUNlzW;s9k zO+DXPeZ+yTzD+ZtD6RCV!)*7L5H`8P-`{!oM;r{wXN6ksG%to4en-NebgaV#8Jq%w zU3Ml|*R(oMljWK(vQ5dmKa3x2M~3X*K+^^mx_^&+=NW;_79r@8xMiD!Q_f^zj5X8V zHxjuU9i+Lidc;<<*zQbf%O2^15inRor?PWX&2MtIl`43+`7KU81Svk6y8N98maQF; zm526Bn33`F9KtN881Smry>Q!7Yg?Gj8gHb@{OVfxNOnH08Ac%?_mdx6U}-n6k{n_y zZ>&={?=KTua@@?)tJdvq6N?x&1xzL^)v}R%)8>BHa`@!-W!zSD(eLnc%)9G+eY^+e zJO+e?pnvTz)N_%)r#{RgqQWl9UhpjHgS9?R4mN@W=5)=0GYWOXsSq{E5t|A2v=n@b zWVMeJ9|e-52HqELFuJZ`=s*k7%;i3QOjQ58m!R&M{O-ea2v9mZCyZ4~2+{`u2f?tQ0au4nh7cgIs_lo&c`(6kZP zm`q#l@~6R0|8%WJOkW50ZRd-vi1{Pce12;_Tf~rcFgYb( z&MVI3DfwqmeBw}Po@4X)jHI_qHSASKmvX(jfG)@a?iuQM&OUA|QP-?sd9JEcm-BZR z$NTg|%hCZMe#0sQ<$-%CQJB6Dp)5Ek$fw=P*i_y0GY;`B*KxiHF5u=TWiKspnnMgikhL`bO;reN)eFxt+IjiIY$J20#($P0&}^^V!WDY_{eyQCK+_VjW``! znPQ)R=jTso+OT>j`C+sKPsvYSRRIG*5ntb^FX}b($BWEspFSqKl&(~dE{K{ucxxeW zyZQV0@V*8Lw@Fr{RoB~!QE$@|)`gMMlbBSYH97gHoG+@m+ggBMl$v&*^{$^hwAh4& z3H@>~AlF&^CU@tP=8XIJF>OhsiTn9EC17$?lle!v$*MjFviAnZdTrbLMw%ZjCuGva zE7Xx(k2dlYIB??^B=ov6}ujMY{$4zx3#KX(!exSJ5ob!cik6CUzJt8=-v zXB`INeRf;-%DTG2ic+?P;|t3a03-PG>_$&#}P!MjkTiq%e7J=4>%0~+e>zPr% zjCHI~SNdwmAW<}Z(1@4cuyUtKkKwG!q+sAnnnL{}*|s<8)m0bnAU9EOrNYZS-D>oP zcgm1;kNe|Aqt3FzZ8Q_n!d(XH3Sm|WLzOqbGqvX+krgXS)SAb0p2}tN+6)ExHEWz) znLa}G{@tQa(XZ3=hg=}`PZE3!mX6^@|GjGCFvP zJ>zO?El#D(wd5ZXiech8JV%g2>wahGjxXBB#ZlP&>-%6}a#BsyAT@kWO~;L`_4JXF zxbmLewEA0b8}zI}l?(z=ZWZ6iPHPlLwi^s}7$iinP{w#|TZlDIMc_xJBdFk)X!~v-xu!g0d)SOLBXXm>XCl3DD zKvb|*p&ogq!v>8t@m@#GXujs-dMRNYJx|f20(Y|o ztrDg-?(>&7xq@F)SZ$WCAm)aLv;tB;;bS3S7NKs%s@XMFAM$yAIk%CCzg0~NG@dR* z6pgy7N4XX+8b(Z(dUkHqn9;ot-)wC21A5<2?OHEF8FU|5HUZ#^j7sOxTV$gt=K|q!pCOjRj09p8c2rB#_?=Q5m7gP>l=~nc7CF}0`>^k z*RNma{Zkp(;U~oA&QTkfI1}K1?U%!OlyQZ3p0_>-|EA-FCUG5NdWM;z#KJd@IUU43 z-{Tzkx*k%B7>IEa2G}jx4E)?(?CQzrW8yb#i#-$lhplH-R2BrgAnomz2K8aJImbbS8Ay`AY98~Og(Kx35!B!i>uFXr#5 zI|CbW;v9kB#O&@1JG~iKdDF3q1sHP>|3`V|gkCm|$ENTKsBJIr@5>pUuebHrF0Zo) z);TH*sWCr#YMfo6@oSCUscX}5y*GA^-(&uz*BYf$i$_$w(G7-?6%~waSR=qE*3op$ zmg(I1Q)+ahXQx#qq6%H=(NnA+ppdnm<|-lsvnS-c=eRI$;uQkZy;X<`(`svtM8V*g zYhBAV*8DQ%Sy^X7Qo)*?13y=_m4La*H!qU-oduil>z33a7mgmKy#TBO4WVX8l!*=> zw^NGM=P z!aw7QytQ7!)t@JWkGp8k8#~b-haB_4QrCX1Olj9Bf^T%Af5ETKA3j8TE5wK)NDCN^ z=yEStp|Erv=k^YeqRO|j(r!IB?w$wcaz2XHvJxHsp-1DEK+ zRzyr=8?;UC1}oTan(n9cTs7rF-Ks(Dh3=naT@Y`UbHyjpaKZlb2gItR@rLq;Td#BR z?xEv)CU=fd(8kJ(2o|>A`!_aQ)MoS8+1lamV={0JnNeM*rV<1e^Lq;|lWa%$ zvkBL1F`AGU>>S}YEB zM>blxtJ6NhMFb7C_obM|!b-%X!OjZl7(fT{cNQLE4%>e6S4&%a_?A^DonJ&OUrRfQ zq^fqyk2+O&fBE8xb6#OQ?fIH>Ui#;zw#e)8LYuwxRu5+%+}G~XCSDt`G`H5xP}mmU zTOX?;oczxB1vDWDaEsE)#hAp?^TEL_x-1!zY24B!XEf?+lOWU%Y@e$MEN--^GhDG`B z$O}C7)YjI%g5Xqj&n%LgbJt~+`u9WiTaA<@R7S@#v-6gh&L+h4nmWC{_mxovBg|Q_n_?Q0OKMZSGABPr*{J5P>GX z;)am@UjewwMn3YScwxJZRoK2(OhaUx&MVS5OlbpPzDa@~~ETj}|8Q$6_9 z_|ATAMMdVu^^1`G)aR-a1vR2pjAZeW9%#NI+uO>J)%Co8xJmq?G92u?c9AQ)$gw%U zw6#L2Z9Egf!e2MQkQU~b5N5;Wjdk!iB=};}s&qp2i<}$_;hUYC4*9LEcx~eGUyC8x zs!~-{H_muxQDqzME4z&eb>-?^q?>2RiezLUtP-R?0GK3IDH*vP=zbhL9GB&X2I zUoU1N-65#2fHy7c$&b~SHQer^8{;o_bdX8rgJw~nK&O+gaV?>PFfxB+%i(5Wtr%TQ z&pUr_yKlqH)Lw7k)shL;7qx0jvek&p9NdlQt1!_Mja-9JAdf`d*Tn1YWxa9RIT)aZ z@C&BPGtPE3=pb)t9kXrYVs~7)R+wc0pooSEkKdlj;jPeo9vbmX@&k)(5Bm+D5Ah8p z%)}H2uc|adlF4~McmqSC>+a6Cmp0{6->jemF`kxiPqyMkd}Bv5eXyuo-Usqjrzssn9eDa58aQvpqoEo_2^*hQB-CbcVTbt4!|= z4rVF3Sh8Mi`!}Ug+1T6nY#x2k)1{nd{>N>NHNToa+dY76=h^qawfL{V)U@W( z%y1=N_E`TIQ!9+j(BRlWvaOyFm_18L;rH%5pQ|O^+h-iosS~*2Md_l}@uuW!a-Kt{ zDI~==4A3bW!IRrXw%8sfa)|>R$5044x~t)LP)8hUeY*xzAUFXiIv4p< zxsdEIzg1qs`*SC$Zh-`G8aT2G&a|T9<)n_}uA3Eku5V^V-aYacqKTo}q0QigV$EIY zuw3g;9Ow=X(d@p@nAnamMWC-KZUQv?~mi@TQ5dBm>gQELmN) zs}=q~O+OHJ^FZngv288$Rihwl!h(*Ds+$-2S{Z;=be`ccsaMfz;w9R!n=h4Vnbl>a z(3d)uV(xUD-|R`{ue~@_%05JDwxnO4IsQ1vwamW35a>Sd6Y+J*bjh!u-a4QSJ6QE# z+Ud218gYAs()c3mFgHR31+ecP^}%`T7OID%M$W^1V1$N(1nc&Eb(V*GX&dFX_Hq^E z0v~&z1z*nMx=T@i?0?mC7l}++l%^Zj1*VYjb#g;<@URNlvc3K7XGy!8aSp1visYi$ zK7#lTX@~A-o2IS9T-C~NlhsOjVyH4d&LY*?94e{;PRf0D?m>!!Bf)okaUqS8f()Bl z-(d1B0&>v~_g8gbwIx#MzBEVcjgzZ>uw-$RpNh6@tNOFTKlXORbx)?iZ7!zgE;XwN zyWR#rlbc(l314-ImH4O;dw9g(6urE`l#2yiW~(F`wH3^rf?ZvNk@=Tqd++1&%PR@B zKR-ReO22)COfN>O;dkh{oW4`M^m_wg+CQS9{ohBw^XYaAGK_U{o8LxMaea1$bm&Qc z)8B|X{+mIiJbUo&{kYmyeKuv76^5+kFIc;d*#R7ErCoplr2 zZF!Dx^QB_E(`&ughAsDDxzCgbJ;VxON6F{m(6;MwqQ5R_dvNMHSw1(*N0l6^x|3jVtY73 zZRjuhoEnT{=0#-HTDd*A0T%v~!g6jVttB}kL^Y(PFnRXxfwaAss_zgau|BG}tI!dA z@ddN#hLRO}#S}a@Iv$ThIcvSF#(iYepDF6S zuSw<0ML4ce+3K4|Ohn}UiDNfBH7u;OH1L9bD4SQ!2}(~-Uv2c7hlEJ+lSEpKU$*F5 zThqU9gdR#kv6H7^ljvb#3Jquy70D~6bR1`i&2#9LQ73CzpReu%f%;@hLP~fTV_abJ z+7^FdrQtXu2+Y#LCHf@X0C?7gJ^qEwoBL`HQ zfFpAI!K8!G!WG)cuBAFI_b){q@|DQ(Gp{4{?8{P4tZ`Q(K+jVBe_uI>YkI^bJ#8Je2M2b&q zNGex6R}PjsQV1PGi7Ss}265Fk();E}RFM}6#vxqd0(TcTBuY~*>Qhv{l9j^0Bu_8d5BGg;Rx-|yce zkz5i%^4XQ#+(cFV$8vPZB2Mz+cO)?mtE+7gIkjOK+J}t_7N6&-Lgi_Lxcmv~{VfD} zGF1MQ`r+1TWCS1m_2aWaWH|T>HsZ@z7}{N2Urm3$Q_gaN*QA4tcAx=o>8qI^7O5`i zLP7|2cf~St!{WTHKXUZgU@-I|CcSUgN2MRlk7Gm^XLDiX5^v<(!Hw$nmZXG%B$o7? zh0Lj&e<^<$G2blCgL>TVNYG@sXS=mL-)`7o$Z5bS|SDVukU;J^Q z6?I-&6V%X=j58T8*_RNDs;QC6=v)D7(lXw%$+^)}U3*DDd}yK?)^Mss1k&Ab2gz29 z2ojI)MD(%)&&t?J>t}lJdxLSg{4`?;rvjX!*gxdK#!xf-EkH!wR~#d)6_dtt!`Gd= zWk|+i2d57e!Y6{VVQNWD{*2&y$uITC54kY{!w$q!XBRG& zjvDR881`?&CIZsq6T7>Hm{uu$d8!IJ|1>)m_<7xCYvpYaxlslE2{h z_HuMQdF)1@QO)Mtg;~f@ZyIJ{2gBi53lh?e*N^i`>Q z&1;+4Z6x1II0Ug5naICwqscJ7c<^bP0|-Hk7=ffWLZ#b}LtFlr|NME{ z5RE9?(C|CJu}@LhRCEFXG8PLrKKzCop)79&5%chl1sZcG&!V3TmecUnn*d}$@Z ziaqaeW&|>$$UoRXhf)v|WPC~wexX+v_9qco%)0YIT!Vj>BY7^<11bv5OWLQ+?wfUR z;9AlflGB>}u>#A7PHIL|w}`eAuMBz|#aU_IzV)fh+ztWJlIqwbQ-&G0Xyy92t6?he z>sg94X5o(B6Ftj_UgB*UKBCm6pb;2>$$q_ltHwfXrFTT3E zd*b6rLZrO!?ct_^6hEF}ZaqHx%j8ir%X{L?)7?`)Hn{kLC1A{cW{g*%3ca1@Yu@)H z-pqz4H*mLYZG(42v9bWJ$2KyH{&RV1SQB|8iNdd`!L6D)lM}x9$R|a0#kB$6VV7ZC zxs4>tg4(nm-me))pwPx`8mX^`U-+mcxcw*#ZI6|@h7G$|tI>K}ZjwK?r)X@r{DwSp zIE{ghEUBwqt;Fyd9h040-#7c#8msjrJlZ;_@R=;oES{_T zu6k>5whX$_63Vjp&U`c+7<~t?`5P~1#Qu}uZ1FWcn@x}@dB~&`KpvQv^eO$x{7)m| zs%oujsP5b8+I0d3RLX^R&P-!voR3=&w<{NrNY(ny*`eE#r7NY|?4s}FE+IOE^yZUSYCdmvqoEcBK@}Q-C%xQUc>oh_R$UqFekWht5Nh z;PR~I=u`%(as0KLznlLv|}Qx`O_Dy#^vM-;`knZe(0ae&q>O+IE_=uzN1GRzzL!)f6tu@ z`LVVR2IKpm!J}acKh({btCPep9|=m}X0}y09x+MOC3gs+g1ikJ{3dp+0lMq^E}#c~ ze`wo%|HEc@DbD||&Q%v`o7=b_viP-EhH>`_X^qKTWg!cD?AW-ubN6!9^seyJV!^yK5>(6NP z#6egW%Toct(``TUR8|k`$QHdY56;uoEM+bFe!TBT1I!4|z8^g-bC#ix+X7~>R5sv7 zZ%0m5PBL87a`-gan_7k;C}@H(_*RcXN!5Y2B*PxzV6uzA+)g z+MJtQ8^BQ(CD*|=@Lvs$g?1LVN96y4?LnXGxpQSsI*g~RENl?iXMeG6f-tiGO8 z&dD9zc`Y*qtr{5O8M;`mfk8b{lV8VY0%LaJBbpPxYw!Jm;^hcxW8gWdK%B=G@!~x# zr}j*P&k1+39NNK{OHo(^Qp#h}C@(iRD3ZmE!Un)pohIo%#LLZo;~!OE78WC?+_g+S zrGwV)aL?|JB){bHce2g5dXfjZP9syJ-MgXL8*_bRtfE5XY8LR zAl@PDJaA7C0XFvmnsK$qVWoVEZ6|}PB(og>^+R{;21hSQ9^hivO`lhB{;%UR>_y8* zRIGxCQpJwpLl}UR-RWEh%R?YM?}&DoltMZeHc@y85P@VY=0k6!kdHnorUKyxOpo+A z9WXYWx=oJ*gFg&dAf$=Kp@Tj@NHdgX?4PN;3V-PtBvruI{IXr&RIOXH?z;+a zl-NViedHtGA%V|(sVd31|B?mYp2Cs0A-GKV5D5-464m^s;3_VlCPhB!>W@S8FGFfSiPV@9 zwj#$}Q)i3sS;{O$)=gcSOjyTxXcWG$`@SxcgxYNF6t=ZRA}gr>?m%5#L8qA$BcLPeV*_v=8*4L)noaZg* z&zj1Si1K*yL-Ez&pto(Ly}?hfFJfx6T+Ocd(>bW#t6WiOd@{b*B^7^R<+Q5)hoCHk zX8Tn_kk;YsBfvfW(T(0*_uf&`Xz-a~-~6ATlbt*$Ovd~_(5MjRW1LWH>&v<>@QEf^ zp~DN%#&UixF)c%!Ms0)u8w&3`W#d|DWqau8s~v%CT;azD4Q9L9GdX-|4q_{F*Eec+ z$Flw_`JMK+8Yj);OeiZJ#NzE+#niAZ;AZK|$`>MRWzo_WS6Rj1z;>n4(u_Gkxa#z} zFebX)>QTA9AXL0g1kWU7bR@fG$Dl+l>8q`P(Wx2>!ns5+TXWTl5XWBjIejzc{1O$T zM+0QOy=ggrRQ|l_hHRnuF5yf7fDs!j61YKYo|B3F2dPSNfM8H+G9Nl>R8w+{(+0rT zALBR-CCA?kF`eD=K&+Jks7*1FkcbUPf!I)5im9c>B>oL4SQi-o0%A2YfD=_6i2*;W z`YrBY`@t9Y`IG7Z2L9W!_IMCZ4VIj7Ar)CTQ+tTDE?6zC+&pBL#$uX@?t@?fg zc-dEj@YW2%Vd%Yx&1x#8i_ zyfs+k0!#`QQbOhPw$~&-_DI86X1dzh$r>jIUh~je_P%=uupcGwqu|;14c%-csoPdC zko}(^F5r1BELAR7_MOsEawWNgVnRta>+`hpmWGd$4dYDQ)Ll;=hhF4y*wj(nyzJ&( zU&)iZr)OJfCp)_0_Vbzoo|}05rf0=&=}S*r>BW%6-PO_FDa=B7)CiLaW8jnX_xqAN zrT$fBd^Pnx;?2|+LNnHPwAwxZWgr%muoaxv>tE1_4ek3zw|MUWfS80K%HJLd+0g%$ z*rgyZUnWBN|AEt>NLyWB%DEyg*ELsC z7k|e7Y6f<@V{c5G5>E4EU*0TnYWppMElWfmyPqihRjqg!luiK@foEVaeytW0d7g)S=%Yvfora?Ogo3+XPAjvqn(Z)z>FWlbZ+}s-) zfV?(NtqX)FF^!})6m(UQhFgj#?O4e0K@7WICL zk{OWP%9z7dRht*(dPHjz<{?oehtS|BLgZ)F*HpvK0)=VQT-r5k9BI|yNUJQEJI$Nb zzth()bdXNy=Fzx;=oP)?Cgzu?9*JW3i*5$#v4KWw8D!N;9~?&u~gFnD=54k>*Al5i3e&D0wkb9d@@s! z%e?wZ?A(i%&UumkaeW-KU8*{m6r6;z;9KHp4LawQ==paUIrEx zdlv9fl08Knu9w%A?`M>XByOp*XgD=~yzx!4-k@mgPZe@Hp7??}{CuB(?SIO)rMt^{ ztB+hc+4-m%rvj7Nl{|*t5B$*WX}mE>iuDvmcJgFN7z~VVdVG(KZI2e0FXmY2C=)Wy zWf0rdv(D~q%KM|jxajY$^`2bzhrmkce?_WNoI0rZx+6tyap|{rXPetw6t5I0MLiB) zI;diznJ{z-c1-T@di>p39`2`SV~gRV>-@8j>h!|_NUFsusvN4MjyT(2KE|rx>3uP; zYpOY8uznEcpxfst6mMuMqjRn@I!HoopPOyrYQ^|T@IQ3p@DW90f<=3++k)nMqKZtU zv^E71vYM%Mt-0#Lb3~2T(bEh3QqZ7LG|;~sUp2XAL?ZS~5dev1u?DQp!=IgmvH4Al z*x7e7E&To0Ci)vDqkY3ysHD}E!Tf1Z70VN+J5wW{_D+HeT7t~%!p;~Ip@$TmZ;>P6 zqmc9V&c|{iNYFbq*$Ce+DPKKPf6GNUY{=BD=cOcBD&w%Rq*a@TZyu1FLytEiMGF>t z+!_t`{H`dSlK~3zGH5=IearF3E~gb=x7{-50FxPAgz{__vfDorM?)d~PG%%Yh1fGS zc8TMcnuXz*{xtn@^)Zpx#RULp77n&3aOGwMfDi0TRzhM`_!2N8#O-iLFe+;!XFV@8 zF5bDmElBx{1?1E47%{E`YINJkYtM2sXi6OPToiuSH$3vu-}zf?zJyCv{&$)0u=bbZ zv(n#}%fkP1u|-syr?FV#^)bfldq(3+62(+j!M?Jg&-j?Vh0_=Z!oXV@i*+SZN1 z(}6B|-{wCgOPyUfv67X_H{}7c#ee~(gBY79h5y;H5`Fr43&N*MSC`x^xWr|Z9=;&< zHOTN+?R!ZYWtYD|3?_AL48B;~X1fdTWBqFW7;LzQ^`II|jdfI8HuNpOSX`{Mr=Nl6 zXB{n@>1%6Q_Hf5=Fc6ZN`}wgsbd&Juq~Yu9VQQKd?Ehf5`&wR5Vjw>KUPF}?$m*5# zdcwUiIVu}ZX4=V(oG#PNq%fDZL-T0Xts81W(*PdkHFFsT;*1abbI_tRV^J@H8~fL{ zV9d|9POGieS@Y5a1tMrOknx6Ki3bHeJ--;yz6luV(pCoGD6qJ51dum;43VLWf|b@7Wwp-WZj8otq;qV{ho8?mxCi}l|(-%8_5 z-iuFUd1WOib0IT&-i`LrdVHLAJsm;XU**M;MgH6le+a;zutKIBAVp7Yre-n#dRS9xht%tFPH+`B$#`O&zmva_0B zM;RkwKAxqVdf8ejWJL64VfOjSa7JNWZ+AW}UpbjkW617E!20bdjZfL$tNQ=D>Qqqi z5R5|^vHYB#(cZ-YM2USV1x_ttS!8~16Q{8+z@I3bO166RHCwz+{dPPqt4#*4x`GuDB$N7a;^zr_U@V)B8LvNQK^XlgK5^LG2~ zXSx{S!?Ew@fMrO~io?aI~(!Y4` z*{} zny+&M&*!6~FbxgE^PNcq@%~+VZc6 zYem~dZyk4)ZzqQ?UStXium(r^!jXeX#)iRwexeS@ld;x!wiEl4Ufp97|@5gb}YYtddU8I8=Xp6LIE!Yj{ zUW&T2xX1arKpj^(Ao%4cr;tY^Ar3U?!KG>?mLO>N^cKwvo z;J`3p(v#v z_1G{7=3g{jz#suly6IYyBxLlqI&WX%<-Tjx~Psn6w7?(NnGqS0?t?}b>CKHE44gamC|GhhFF490OAcQyuPbUOVZPc_sE9;*djexCUJxbY0mcH$ZzeiV6*KnGR z+E{~OtwSkbee8Ju0p4gOUhHOU$WfJ)AqE8ee`Uc})n*dSXKS*Bvw&I;RPN3HpK3k2 zecp^*gU>1>>7u`!+ySx0*KnMd4o%zi6tY@$;yvY^vd>+T=jSj_D^)Cjcl=U&%~~&? zI@SS`9~S7%=`UdcED$#4pPWdCl{UnN{|nBUAvRy5Rj4PPx^Hmu_@6JPH(8{O@uEEXoU_@7*HFI9FRTAyKrI{h@GlLZI@?}OCQ(j*`d`ZAaD``I z#kF+!T|IRK3wH?lHmDYCy-}P#t6sj+^{(kc)aVjWfs!wwa9E|`Ho&Rtxpz5Hvfs)C@bgP4KAg; zE695lq=BYj4iArC_Zw62c8E#ik0_194>2Xz#(EX^omS=T1*;dT0f^ka_VBFW(tsTF zLZ+O|-5YciGO=H_@aEurX=OqP^?v?ADxdoSgg4}DiUol+8+|yq$Pkc#n&So`en+qX z@k3E?Taz~y%BekIJqD#-7RO683M9f$?)orG7@ox{#8H0`u?74=`9w!(DgFAcq|bN- z_^aZkBvRzxnNqTl=seV;`4Tn=)IeIn=aK5rE5^8&tBY$t!b;hI{T1@?mF0SScQ5j|w9(u%zvRhCk}Q3lRw^@DDIUkT8nIb3o<(tqI2} zIM(&#d@nn)i~9Z%P0*y2W|1wJN@ydSc?PRRez{^PqIDjGoLL4m27JP2fms@t%X5SD z{>7N*4J+{0C$>+R?NZZdl)q9TqmJ+CTH*3hbVFANs|%}Empv+Byv!21lrTnWKt)UC z;B0B0T^4Fy9{zTL_fI~W#OIC!k4q+f>$Hgxe#b42kjN(^yBMB$w|eJdf8}`K=3Phj z%m%lV8Iq3<xkB4GbpZ7zo0w10WSXjC{Is?pOp)TZ!=b}m>w=uGt|-ZU zcp?t{f7F0GHqF+sKT!a>4ZesmT8zUrl^5~iTj`k|g5SHL+wTV%vjr+UNoll#hC&@L zcL@h=LYhpF+iEKA}BcMxxof%5Vy$dDE(e`gZ2D zU71n?yN^*2d%D{+lDQc8`)ir0pMbJN6gJ4cK@};&3S`;G$M?q|0XztY?6ha|uKN9XZ+6Vp3K0t<`s0IZ@c{#wCv1Tp1Bia7fO zBnSHQOLIbB4m$J;f0QEwqyl=Dhg;|iw@ByY%y9K(R%HNxZisTRT@alj5H@l1nF$rV zWMVNPR8H#1@2C)XS53BP7v(BNv~lG}PU{((6iX0;Q6IXJ9NnpNz09R=s!!wR`TU&l zx6@vf1C1I}ftyJkF_&k38VsM-qfhHPg92Bu`SLny_K%<#iyobsCeQIw!CBc;odq?OSHkmL zAuQL+Iay8z6pG}nM5kz!ZD{B=8J1G=NE;&c+?(st1P_n$kmp(V z+*(p|xSC=p4CMZa|3m0o&Ld50d-~c8A6W1;UHQrNYAceY9Zq#`O}Fln^`k~+pP-(l zjA~d?-R5#$T1wsqQ(@+Ale&KKW?uYy{>W<>>vFk5X{GN&^^`A-ARYSLkoUpO3QHgx ziK9VeC3P%m^(?+(&Ve)olYtHD<*I#lOw3W29F-*AEmTbch9h}LbuA~knvJsEww2+i zUUl?#;`q0kU{xJpY~Ke+)S`SH63U%&-@~6A90O^7T5RBK-ONj`WRqtFc*L=jfQn#| z4{gJDlE92{=-m6ZMsPca=0|(a`RUF2=P%=6k7Crgb(Wqi4?HS)h>lzI_L7F>SEgVx ztPlMHGCr<)+0LT|W}sIVNOhj2-Z<3E264BZKBv+MupDOZ4)TR!$zPe}Mp2mu^eikJ zUapJZuXZEQ+))$C(<8=kSg)4;Phvh@p}K~8b_sQ_Rf`&+4ne8ZDnlF%#Oc|8gQS)- zX;2%5V<=|AVgkOg=hq4s5uO3t;C+AVzR+yBaX*)0q|DaQS}UDkaLY}Sg^;K?k#27? zaxfr1bAvDiCmDj*ZLYuhzCj|pp|b@)CJkpA%rJkYD|txp`H}PX+Q6;F9?Jm1zUqe| zXC@aQMkfmd#4)g{0{Ex*z?EfZUhT5mbMnIq=+r`;_&ZQbi!E3>hKt_RsP&BXb)WUO z&lTQ^g)#m^4gI#?Z>hBPUPp(jz{A5=e>LIr&YlOfWbN150Y6Qm4S474XPVgU=kvCP z99lB-wm$_v>H5Y}ra8uPn!VwlACx6_1=a7-5z3okze;u+M&4v1{fD@}_Hk`?e&@S9 z_Rcz7^HrbowVG%-6tS7D9b2hTT(rxt&VhR>vYe+rs^0>SHEY{U@5gyU;f#!!{$j}a z?`nuwHu3zBhzTPmL(1>|so!9TWmN{!@@h&x?9Cr+gJ*c1e{x$yXsA9I7}YkTu!{dj zJ6HY>_4fY{B4oLhC0mqr>=Q}|x2PE;#?sg#S+nF?BQdwG6k=rGuZ%RvzDBq-_EKXv zwyaqO*_ZG=x}VSIulT-yc|V-@>%3mC^L)Kt&+|I-csR3!(JQ+a)Ma_UNzptH8uuyo zT_x(g^ZYGCLL@RBJHbmOyJlj(GBin%7Lis19+)g*-~{Xl-=C@yngUICU?*RbTXS z4{}?$dz;o#&a!U6Rao(`pxttMoIMg-b)L>@zJGkMvDY89T(=d@exg5X?_$o6OGxz` z;SZ9ZrAh?o@FFs9*xY8Nmyp0RR*E9`~Z{ucW3$s0>?Nq;vZP1HyVRf zSw{~Kk1xKA4amr)`1Hc!G}^(LtHR$}R5A_7DdIs1h9NHbck&ywok zXS;Pj|LE%DKU@^~tHnun)yhtl7J!(>6?PkmXTf%|)|$BsWodB8Mk_y&rj_lRtaJZ; z;_J`(sjSTe{^d}?$dIP_#g4BTQS9G)XJo1FOFdsVquCdl{4b7~WQua8_R_#WOF1+c zI^rroi2BA?@D%Ni>a9Te__C?8zF` zBUD$S{7GT_mkE(AeR;ESj-u%FSBUmPM&fufCww6ebMLn&KV7sG#P}n&G-<$axGe*V zk84z!S2mTSsEr#x)29=@9Q{VN$0oQkfY59pm<7ui71KIh7FrJ!Y7?xm$_r zT1iw6kFeF`(x$5uCt3N23pZKEjK}J%)3%}wJIsUYRgTxjmUF=Hg5gMLr1El$FRk)R zIiZmGZ2ASlgjtxM-+WK0MQ`2Q0rEYKiYkAtJj#%yws48Q7H}cb-=a~sT#)p z-BJ46x{>Ai`v(S5n`Nn(wSSgG_8o@b)hb%6_RGQA1UPhNB25$3T&7%UP@m=!g#M}@ zx|_>co?Ms?fAkfw5ai#I970`}a(P<(Hh{r%JxBPlnN?q|)PBICwWMnc%nr=Ip6VGp z$6XGGrcnfHk-qxMhGA%k-Z6V>mV^Vq>1iVAp1D{GmoF8i(Ynu^NHi%U`2{OWPp#+G zT<5ROkZ$TQAGKebPVovMbjrEFz-o#?=&q5n$QJCVM|`g0zrT?vL^^(bk`ecMfBwN> zUgW2Rn7S}u^$=ZC$>4+YJ2RoN^-j*$jT9OYCvob%26?5k1KNW6k( zoDXJN#d(Gqq4M|W6}g3}Yg=OW*jBBVrhWQbB=5gvCI(6>b_$qfh%1yx8qh#6fRiJs z(Y^(+cm^B(o0fMn&bx*7{ul_@s9nvc+P!)fS1y&%%B?tCm1^4Ot6`iyL0eefe;f$$ zKsZrVcD!>_Cs1B4+1XpVq*6uYFL~(+6*=C}?&aeAKdKdh&}nEVT21Npyn8vc^R~7* zRa;fTGkAPFsQkgU!_c!3y^q-Tp>L=6cQ+2)_kVthB-C9z%|io{p4c)({8iIwV_}PJ zZKM!hUmy)pdiEQ;?~i%cQ|<0~)|H5%uUw&1A)oA~EN|$)&+j#+U;jXv6&1b!fawu5 zw)>bhzobsaS;oF()vx@D(st)DhJEfeL_O<+br#NPuVz@y?y#Dxu=U>CqQv4p-&5mR zzp#JegSk_D{s!Az&!(U`4-9;aj0^jy`YowzMpGM}vK;@;6A_RhSCA@d|uFi(WS{I}I*RDxc-6t{^sHhI#duX<|*PHNtUP?#$4LuOL z41wH{Pde#tU@jUmZ>0piglPtohdK-RGi950W@4V?2La|-Uh0{#5vo<6ym>RdEjhJO^CANO6(kyLMk@8(VWfl-#J((j;Zeic>llvy&y~^jV?a zWPI7ZGELT*IX*x3;~RFM>0l!z?4h0(92?{rN9EA;H10#kEvWNXkFABzLZ%41&3+E173+ErKeWAbko(ehuHEai#O)|r_CsB&7#3Us8YcPzp9;Z zS4vnud6k#KgqynmAU=7XAnt&!ue;%Iw59}`G^Zuq<4XH{d~P{>rI2eC&%7LfDXmF{ zskZiFL)TsjhJ=WOSy<4!%0J@#ETlH5le6h6be{m3=mN6^&!1mY)z4q?Cqm}~wx43O z)5=yZ65r|-Z(j7NUnv{9H1a@o@UpTs189X?)cDm3>w6W!G=p{nMfm^JmWnNwakc}1 zr04G<8k{Cheoi{4Dli~PwRC0}UmPR6o>qFlnDZ%lt@DA&YMYX2R??dN@R2&{BX!D- z&s_UPBtIBF`23n$XW~zyOa5bw-FrvQ{Ag8istWrSk59zs8|%Bf4_R{qoUtv0gDb_$ zRUD*d9aT^sR*EFdt6XJG<*OssJ+9Z;(S4-z=6lKPfzYkHhV9lT0M6SMZUEsn@2JOc zUS(R&s-IuG96KFG(!Y3x3ksq;3-w%+THt@Ccn<&&@nxCkvIovaAgn?xF6CO`Be zzRrs+%}T4@b(sws!$kkWeeKe-gGsO4BDDqUqmesM{-f!28|m{S27|YeG zd!i5h@4jY434U9UBbc_Huoh z3JvKDszuk?WP7d#BAA&0?h4Fhwe(~c(_STeTI|!sVBa%mIKxz=fav@pipF zDv{^~Firm@W{B(7rbN!)2A9tkQthA@EHJ)>sHig1166wb&Z(xW@oq(F#{e$&-u7|% zJN}}>&J2WWRUNUV4|{x10f33*RIoX(FBqMkx~*Len!oFYX4}HX4qOm~%hBTKa_D72 zmA9+t@Bkr~B3lMMEotcuftaAcB&vw!jBPm!z-wcXbYhJQF*-A-X{Cx7ko2PW0?>?y zJM(GID3tDUX&P;-UYxa(R5xTVCb$m6bT#K09n()ZJE=nCvP@yukt5`1Q2BMW_FNvk zt8SerX%}(=5PAn$(C(q9nG)SAVp)4_MlC%w0Pv;|x)LlLbR;!dO0a@fo&OF9V_imd z(=91zh?(jVlJ38Mc76W()hlw($*gNO$%;0~GYjb;mP14(QVW zoYK)Jj=A^>nRrM_;xxk$XLsl5RUKDxxHs~ecH-<-r_dc2p>UP158Z4#0A^9P3?ea= z4J`6DzcI>RoL0h+%$q0aK*?DW3##IwvH8Jc3Ih0H4;~+l3Wf3D1Jf;?%~8y~6n$D{ zi?^e-w+EVGXV#>Fi+!<3tHatU znO~i43+yq76C1GC^sMBOw}h%Ld2UO=1U^=Afp)HFpO^Sy(3iQPjx0In^#Jxq05`c( zFZHK!_=ax(701s#V6c-LZ+e*He+vLy)RD3o8GKlK)kEI5?s(d`VKWBMz&Oy(NHCh@ zCHjOyQrx9?$?j5fQKbhUQX5G}Mv!IPEIJt<+i_cK?vfv2(#MWM$shhMLQ}K$#Z;lJk*q-<}U5OU`=#t%!gowkVBkJXT9r5-a;^;I zYB0{sq0sQgDlfZ2jDo;TNfkyPY??qcfd?yGVa3NrvXAPgkQz-f#JdY)l;WZzWmOC$ zmLk%ELX&1yAaBw~Zuzn9YVDav`~XBS!7ptKG0d{j?m;g-8MfBwJGVUl5oOtfZD6W| z1J12F1Y?s~=K7Q>(l8Y`!Q?z2H}i-WPCP+k@u0m`kYv+x=5bYTt_*qcRw)L8a||A1 z#F<9!GV_j$#re#HhAP;@7MlY~C!1JEIC`)zNy3mLs#;h)a_KBSiT2=(WGJwO{*~Io zQ$uWETE&`h z;Qm(w6sSx*)NbclM8{sZK0ycYr}XTP4cp0!`5YJy?fiilwpV2>`BP44B66Vi+)~PeI_(Rm!{oSC$3GxY>1E4sP7HY^qk!ip*>$JM;g|kcVmWPOZm(nZ%^UIw;PPl@X+3r}!`N7Cn>?PBV-^ZQmaO*F`370Po!NSn&gQk-t&c@r)k>NjR zm8;pQvZhL4rC^;Tt<1HRr!fFr{tYDE1*-)%;+!eN(}op#Lb43p$^bYjog~>lIZ`90 z+uVqZ{~+0?(@`%s&E*l#0zfF_uED>0OBbiulik7PDSGTW-we@I(+DAK39z;fEtM$q#A7t_EArlN4l=hj1ExJfnZ`H%vt{gznEIXuTg! zmp^rg94j##+H+(0t13jp$+T5+0eg-PG?3sSMP2f&k|=d+Od-L@=u2uFxBSy91$8YE zb&f^H0VueRq?0ev`dzi`rA(wg##~h9eN#T3wK@5BbWV_@H-vsdEps|$i<%Ws6t;5=}3;eh<2+fb@X_yArbfP z!o4NAeLjQ%ngTjINU-n0W(!%e?UJ@+AyOk_1A!>~n170`*^CovzXu^7 zm;S1bs9aertYJf0?R9Zg{3%PE?V7W=%V%B}NVB$pxSeXn6oX}j;xR|!Y%fahH;J~J z`{ng?2Q0NTS&{deOh;F{IP$;glPBZxaD4poDYtjH{zJd&37mtcU7*^A} ziXMZ3{v8RG0nnjV@$0Qw0c!;QjkAZSz^5p}_UI~Q5DzE9r{MxTtiuNjtslKR;40x` l!vlZNfRFV5PyQcZ`x-1E$n*^{o9H8C(AxT11y^mJ{2%owO?3bO literal 0 HcmV?d00001 diff --git a/lecture_17/q2.png b/lecture_17/q2.png new file mode 100644 index 0000000000000000000000000000000000000000..7eea25c09131b64962058b1b1031baf37c985ff1 GIT binary patch literal 25757 zcmdqJ1#n!?ljqs8WHB?N#TGL&Gc#LYF*7rRWid08#mth$%u-SsNotCy88Kbha^FnL)qIB0BW0000dAug;40DzVP03e=_;2)n11d4wC zcmZ<|lu(BJXr7SYLOk#tsTs49M3^Jc>w@I zfP}DsvTORunw&JwQU~3W&G<^pRRaFNGbNC)U(xlFunWv#BJQM&1PjPPyji>kZ91a} zi3O(F0%3~m08z7qOhJ=Ey=Vfs0WET~?rBc5ex8OX5PgAakZ^;%CwqH!N$ zul)+mYD^y+a>))`Fr_Gj*wp_T?iH@h%mQ{F+N|EQRaQ zz&*_SnJv|AYhuv0KVL*eqfi2*Bp23CWTdHjsy*J?n9G)j*W>vLt)@A~h~5!P;yi!v z<9V98i%tCoBf7#TE`!pUFK6$Kwo)nqEs3%>6Hf|F0~a})Za$rciyD{$MJ4E|k)Fn8 zwLPb=uJ&kCMcFs}^#g&4M-O~H+7zBb9CxytA$^_gN^hU~onW6wynla~*fLz{k9a5J zI}G~i^^gv4wt0<8Bpv)Jq9=fO>gaP6I7_O-86OUb`XZh47{kM0(S_2XM5`{FyVpQ9 z-%8Z-`WyMp%O&07+vP&vD}nC|jQ8pT%-cggR40pijco}z=&*B1nTZ(4G}%DtsSZk=z(86jU=@IH9*mXCRCxH_`E^%CfA>&d8k zevQ&k8nI9g_j2LNd?EE-lZcw2XRQsauV)#RyX&C<>8e_E1;XEZH*0YYFy!g%a`Em7 zg&}BTofnttn^i@;U#rozJz6`~;ElTP-n{#03|x7?t@HLqzc-ZdHuY=0b$e{m_z2-s{l%7LCqa41-T~KoAo3vKA~oCal@_ z^MT(z@c!{*v4D-Dg@cz;d7E;C5#wuYoq70Gl9fvzAVKf~S5U_|Yh7Jk_S(@v%R=x2 zNh|@w8SRyy67L{aG9K1o&)Z-uHJ^2eE}Q7;agv8|0BS*b_fK(a(%VIcbwHHWW>4mL}dwOrG@$zglnAy2faEh&vl>P8_Iz)hS%sn(43S*Qmb!V!{qQw0 zRLUwUdCgTQ%{Fl*M=evCuEwY2=iXrmeDF0VnLXY*Y@EmTthQ|dyBchw9YeF?pSx1SBH_5=kJROTX9Vk~YBJe{J>fDIUhTZES!$1a_J-b-z8?w^>ZAIjf@- z^dcw$na*qRpB9vKd^zEQo_qT4k{G7H3s#VA?>^iiea(7SKE?!=ygf+WmX3B~w=i^g zCui&>6U?VR*u3?=rxkzOAb+XhtGPpTnL6cKTiuEb17pha>k!x013Hi^$yD16^W_o<}gxnR8V0hOzQD z(oIzdy+`lrd*#a{&yAu$tW;eF?3&8UfbaR3@CyzTDTT$i_|K+O>zQ$4Hs9qf%*ED~ z-|m~Lc&hTj9ZGs1ykSF*w-2mKq~PdtN=t;a)={r`Ov4{5!s^iQGp;7B#PfRtmyCXT z!s$@S72HV#Sf4XEn^c%d>{0j5_RGnRu#+XN-5FmC4a5a4tYz3`suVNS<=Hqx)I=iD z)q?L+a#|&IS_Ab`oHNVXfhyRbnyCc2?X0>>8v@1$LM|+Tm++;y+v|z- zw@;$1Tkp(w^%!$f2h(n(P8}AL`!Q%PKAr>%`)-E1krQO8DQRMpK4-3|^=9W%!lj*+ zMnpChoq^m6){k+`x5N`m#@Cva_inEVx+c5UOu8nw9}9U-%@I&xsk4C24dd;WmL zGxXw@T1$|3`LJKOVOS(!0KaD=!>TS9m5GMF?qwr|&yeCV{mT-2=dyjvMwA`VN4lFM zFSG}UE+8P{JqHJ9QR0@=_j2wOCh%9JHZttk1ur9C-mvksi0g+?vZ~i;UbIT=)WIUy zsmOWLiowC;NmB`Mf|SjkRbu*IGUXy!Fl6(-rH^t{lAdWDCBu_PQ}*>1oY)|k;P+1; z5$19FFXA;;uPwj36zqvH>Sh?e&kF+o}l2Tr3mX%r1|Nc{+mah8_&GgvBVUMpU_P9iFaL`%c&Q**$T=Bw{N4%9{gh%Z zVO(EcJql6uUcV4OU@n)93mdIB45#RZo*W>)CjWy?JI|Gsm9D{4U~?rE!zkvwu%pbq zAyG*k4^pnQx)L2BWIyP9Lp&W9FSMN{#?_giz^GEaf5Ma^N2_msK>#9|X#y*?IY z#mwO~@>R642}?s)S9MsA@M7j9JL&K>=aNT(P?Q>DsU!=X!FOrof@2c4TfLMs3(6*DYr z`JB?}#5bRu2tUB^O(8!e%*!Rp3JI<$Un6;=ZSItNWKMF*yq~KUE64kWpB*vnr1G4k zE@!LAj@z45Z2Cj-Cth);C65#;<5I%X&HfPJj^7oTrrafj=`37~sibHus+sL=6h~4e za~`gSoGhNE#Z=R121LV1R-Pyjaf|TvXKN{&983?|=Tb$7%*3I1eNI=Qw%e=G%7Ra? zo*6~oiHM&c=CJIH^+?2Yo@>WG@e9ImPoZT?jb46i z1zY>W3QiwyxEGw-F-Y->bKNA;LrEBB(zzR(+e5de?>o$=TAhgdfXPi+=blgsy1JD6 zZxof6PuyZHZB2B+bZ&ZMit{|xp%xEq{Ke@Gs<{@OZ6c8CLE9IQE?s%cF>)6*E_`x!aTy4P7ubD0J z06t+#nLLn$WH zjK7BD{2@n(l90j_-RGGDt-$e|h*Nwv7AiJ`^YiAy^1gJ3Y@eVoS}X^8U`pEINc&3` zNB)G3pZ^ZsP(vT~ma@UNg;3SA3-~n)`BUWnf{9kb5UghE$wR90lTUJt^Ix zgFRKVS3|lrN#)zGUZ9<4e{O<`bhB+GW3&Xr3N1Mwu508Y=uX=q9d7ZSj270ySpELY zwf4-dZhGx8uO=s*9zj^aj!<-Kv@6^4hHW>0eR`YX{HucYpXKl9E@?3(Y15Sc@vPV# z&CfnQWkgnVirbN6&(#Ry z?)Luo0i=KK`=+v#v!RZW76$^y7RW}~V|tg1DotZ9~&hDAsbY`;KE@D;X zMmJKs-Y#4r7yuh|8D$&qr76~P=?(Gta?{(clZj_dT1E<|6l5c$WVX%h`55>>fBOT< zsy>L7SU^dC7DPqH{-Qwr4(9pe;N_JmAke?1ZJVhiHq+wywNpO>|FsIdu+3(*EPLvU zXleK^EVP}m6!fx|F5vCADd6H|3;AXtkhCrmMxPBB_BNK?Ke}GaWNd1$cl`+UA%k z#H}*&1wwD*J)R33IU0U<{N{0fcH<%hNtv+RYrGWfToDM$Z0W^g|LQF zsFz1II+It1y2ifhT3mv2jNm&ap)ic?9+BwRWTzM-b7wptfww<$Q9X5Ysm@+Uh1SYt zknAlLh>wJeYGS`sKHLPbPg?BNS1voL< zN5mM!d1Uo@6bY9PXyJr6iRWWk?6eq z#T^Ht0)FG9wmpH;xskMl)*qFv*soZwqAM@(S`PX~MfAyg#5}d7M|> zBC+N~$Jc`goIgNQ31)ePmQ%0PHBm;lcnHGD6kSmOB?drOgCUZCjC~hVE8vw}Nhq(9 z)363Oyj>@JKMJ}>EWVx?*q11?7TVIyc=(RTDTDm>!iIHR%N3epD9#(%x?se zpbmj#RAZ#aw&$Dp)t!+g1l#YHFGtjMKUId5Ni%EvMnF^s_Oao77m1=3v@S4lY3T-w zz%>cP-rdKOKJZj;A-kZeQ{Setc1S_sGyjaCh-v15R>F~Dwt$rsc{vThaB%Uy%E-U~ zb$_3hIn<}!*- zx_6(y58qr=erW+yQA7}U;hC%0q+ViDP3L*gy&9lcP)h1lt+TyF}g(cpG zaGz~cXG}&0-pPq81JJcT&C{mRb{*Tw$W95GaUdt~^XdWi*7{%ko+L@n>35rPlYs{PdUD<`Wif`iGAjL|WL|fa4VfAl$AZ z8M&mj;j)f=hXkmYo!e07PFJ4%+UBg3Q>h?&dqVK`iUTUo4BZ*Fgm-!-;%d|3*uZ*DcF>{Gs&AIX<}ioQ3RoSn@k+m_2K3Cl_{ zqFW^)NnVy%A2FA)GOPM*7{awwRks6lfNx(617UFrqaDNsJyP!4bRRFHAM`FK_*B}` zlDiiXO{XN>QfcrGZ{%CJ$K!03O>IZ&N5xE5*LTjlsCRGZ{Hg{(mR<{HLgS7q!HEJ~ z%zT=EJzj(WU{2~TOM!mIqxDHR@Sh7E+NE?%Gv`e$qEM^1aq(EF<^_h77(<+a`nLJ) zpKiO6c`jr(ur}NA8B)P9r(VB}yBQXg1eL6Xd*6S085CQ@W*y*r-$bY$$4A$s8o5o^ zf-|>3K0n8OqGLF01$U|s+r3+u$dG=u_2Rp%(wt53m}pZBSzMBE_NW|jZKOUu(qgKj zm(z)eh>`H#-GM+eKPpAmbKgD}(#b)nmsU1uv6x(1$9>-)fJBEkx`>e@ZNYTTyv7zG zBTd3vaPTTMUont8xoTMP*nGYSfq8X(3*p;9@WJeM7rkB?+l3;{s#Q#=mBXI>lEib9 z*=pHws(a7Fud)!b^yoG^F_Bw!!dVE-;)K27`YyesJu?j-aD+IrZTm`Vz4Nl`A9iq+P@qeYvA9Db5daHg_(rn?+8ZhS;SJb`kth1A#QHXDKOAf<$H6=r_s}FJ?BoU_a8+DI zC1kwAx^5P?my0W?q{QjOLi*tBxZ}qux~v@AQ;U|o!%2Vn3f%qdhlx8K)RB2GnNJ#? z3bjfEW$aQmtyxRcoC_Hr3G124Hm;|lCo6NQC)@Y(v_=a>ft=2t+>{-0Dy^j1w{B)? z4jNd&v^=-aJ)YUv9kvX=a^0P8sH=qh!kIsOZ7tmlKrs0s^s^|0Zh{Z;Mq(g;8}RZOB6@w|NzM^m`h2vdx?VaZLpM-#PWbMGjDtKtmeX2DI{1wZd^g|lWF3M2fvQn6--0w|)sICb~L->oV! zB1f|(qjDhwn{7ht=Q4#=F)!D}x^lWh>FE6KKhM>_ohY5%e8N(aQ{fqBI^Ug2J*W^p zddbYl-1E06&gUi#Pt)Dw-o=eKY|s(){yiEHwC~z(hClc6oqE5e`Oecl+M*R0@q=M( zD238Vwr{@nrdj;V}!~dLN5tH+SjBqDR(wzI8D+R_!QpW!x_qFLC*qI!MbCbELw7)FBaKD~+zE_DKJw z&@aU^Yw*&G3-P+@YP3Ri&#`ftw%;_vt`i&T@>sulMNYS4oSPNkRTmj_s)BM+%9=&e4(3 z-vWPXCArOS4XuHg0P?4oHB6PpN7Ah*vBI1A>zvw>&X)XQkkwJSaMWJ4YOGHDH*l-RPj0hk-TSL+(wV_E_{eI3lcoQ(Biw|4XY`H zXcCn2NVByyqkMbEaXaSz=hBox(hR+E?i+48c{Jml1V+#~i&oxA&Pg=RTK`Yi-%7{Gu=A zPg&y8@0q%KHAwD(^vcLbN-P=X4*++vgN1-+4=buZn#Bf$_ zk82PN&3hay!BL#6T~)0ksoA49;r{mfTJ2Tb)m}0n;L%jWuX~A40yBi}I`G%3gGVN4 z&8)}!Y8@-5JvlJCXozjz^QG zp(gSihRgRtN*ufzfC_^6)Qs|DCx8Fmi5TN-fPG+x@<1f%#_qS%9E1{gYtVK?6QK$ptGKb z-{3Llh`_g6P@A>BH&7NchiAKL)?ibF^E}RWduHBW-L>@y@8TUpEh_Xk72A_+pknhi zaq?>qkgOBv@UJAs@G*MuP=!c7FUc(HB2;cF_m|ZRpzGtx1CH2OM^DvfSs~jBMguj) zJ+FJFm|DxdPoYE;f)f4OLIt}eWeQsil#*DF0xA(_yIz(uyh0BAQm|?N(gK(ZK+Q!V z7^Gcx^UWoC$2|P_-qLZr$4SYt=5%l-wK`h5JhiSm{MlQ{cu+<+>!XH;QQg@XSOXL@ zD<4;{QH~l2nV{!B@6B;(>vF_n%OL-$3rxyQ_bL^+@8NAA^O#7~z$b76j|6y zK@wSWA_+UnI6o<9y^FnoLB-5R*G%^GJPGlUF?6EU%T!&CK%{zqXaRB!wRIvs%Kkcj znD7(dK{#(JNq9RfX`~mXr=>px0*=>z!~>MS-;{BVjyU7yj(HDVvGlwWlG*k@G9MP} zro(@<13-d5vV-s%(0etUhufsi4Lu9E182J4PM^r_ZLf!L7%z_)fa0FK%WL5C z9}U7&_3K_C3cs+hgwuohQb0x`{>yC=!NvamyFXvnV^rYYrqgI5?=;GXA|Y<8MkFCr zTtL}rO?`2_cR3#}CNd=tsYq-{w{w5qR*Be^#AYG5dY=!ivQma=)JXS z&4vp`{})9@Rv8W7syqu@>v-$5Tg87|qhCKoF)QEuD*ktM%Ol5{SafU%im+x(HtFNO*Mnh>{g@qGo5GznUes^Hb*_Y zJDW4-h9RwjG4nqG>gb&u*Bps?AvgOK%FRz7#vKsy3h5M-xr$E$M&5Yx$=7qm=n|Un zv4Bt*4Ni15TMdE;05iefQ!-29H(w%bbzyM}KLw2U@m!eRc04~7f_GkS)4{;ipk`XH zyHZjf_jI|9#~ygK39{hSeBv1JKlga5=onj5nIyS^dE@YiQkE?%SHLEaDHiG1FmD_4 zj$;sT}ufkN8d%0EI>c}oB}3wC4L?DkCpn3ph2}4=WC}|8OuiAnl=1;@HSA< zgD6?BorjK2`h%&XOGCSC(1E81_b5D2bq-S(UPsLch_Xz4=HduFvX)DhJs>b(ZaMH9 zr0e1BZ;;W7j^1`!7{*>{D+dV9)Z19VZySi{or+*ocwfU~aqO z*>6F&K&?@)G1b4W^W&i)lwB$^c;j-=4%)>Y2DeUil2Y#?fgq*y*&BF6g5@9r8Ud-M zoJ(An>P=4H7Jll|XuxU#63Afnid+SHTkTWppAd!S$)q0BWrk255mqmZZt#4?A{b2M z+4{>bD#3={407`}Np*A=?riHg9h$lOaSWPOm3$J-3vU0YHF8~e(`(6Zze)n=A031L#L4u-q5`vG>UL56KEr5=BNqhU1(Fc-p#sYM!%9nNx_ z#9Fx>mm_kli<0U%isBeQHTs#P`iK+^U8va1J?=n~Qh-LJ`ViNDj6m=Ua!{mF zdgiCe0qVCOt;ykH}s+pw!3QQm&l6co>az4jaIJEBt1( z9V#3=zV%5rba83i>38D9qoSgcRA&rdqsX>uvXru9)kq`)jnyRIXdrJvWMl5ayW75A z(Cg(g>j@|%H~SGnk5Mf^N7I8cpErW&H^>QBZ-sI#L+I>yv z4NhtWTK9n8>N)%o*L(y|IXFte>5uDU5eKvfF)(E+m8^OWw%E*4UEyL@qS^8QRL7Uk-B7Tk zTBJ-v3h9FWn!|6RrK6C^x#t(~=NEes;B6eWC@zT6!@eDLr{=W{)B;|upG-wx6%^^+ z@eh5rJ6{_KBz0f1-XF7a7LMuOHnA9Yl=u~sT(ZB^!0GtvW19KbWxCj-2Ob)ZlZ1CWXmqOj(n0lD+t z8C7RUL1|Y!7smv=0!l}S%n=S`Y7!iMO$f8jMW7|gv`$4L3i@}b$!B=RoyMZW^R=;(=cC3kyJLjdWyiUIOytBQXuD?BkvwjsQ6$ke*mR}Em z=YNn6mb4_floxo`hjc&)-Xah~pPIoA?mra>0Q-H#^?N#F7As-a{C8^#UH-s3jRdUX zUmK^jx4JI=)b+g^%KXa#jSRm&QiRPXd7P+8kk}ja1Clg#9Ub9^2CRW^RXUG&-->g_ z%sAqO9MnD2bn|fU42?;7NTW5^BP(-2k}uKU)@n%>#W_J8U15Cax4Ae1`}y~5Qga5m zPCDiaMQ`SgZznBC#WnF}6`@w1pvBJ^Kr8a#CT=OP?s{7S*9Wi0M{|KAu(-At+u{5wym=4`CA3E>h>!SuCW!@u&7darA-VtXoT)ZQ+;0INN zp}OzD?Fy<4e1^RXgQ^RLsDAZ?)^h}`SHK%AANFkoqGiPlv`(o4nQ|=*IX&FdQSsbY zQ6QJnG2Nd4#Z7QVNhq`!Kun!&pJ=+5|QA+kY+o^o=(=R5d}DE6BI-p*+TWd zXzPw_*N5v*-JuGl7%{fjY#!nOnnIFr%Bqj-lN9m}!=TV6ifXXW1Y#-|Z-2)Mfq=c* zQ*|Y!f-~HcL){6GXLfLWraJ3deI{=@fm^cqIZWhwG9$mDIu*CXS?UegpQI{RB{?!? zxk*%LBKUIcgjwd$T1?+AtR#W(<|P`f2^Diyf4CUoxd$~{%nhWUsR0pd9IRCOD>g)^}?k0@td2sso|K-|7{Sy7cP0&g-{w3ply`%cIJ;&abb1UKct(|C~==@BcnJ{kgvC z_~~HpRZj)Y^@#ar+l!4)QA^J(;y>YRv>(;x`_DIcpLaPHwhY*R_5lD+{)NrZz5g}) z0R9u4^IyJG;eS9m&zT{@k30uLpN}*4#r?ao!^ivF7)-7I_qIpm-pj%3pBY1p-nt)r z{J~Wm*i{mK-G8I^hb#Jg>&~Oevq`LmALP;%`J{^B*?b(~emPwn0=!^+{&+ex`&FLY zG+h-GO_E(#sQr)hrjv2v(4)N`)NJdHl~smzy9eT$Ct{s;Kys7$Nd*g}2+7?Gfi=8+ zah)2g0|WeEj5ZL!f43#lj&Gv&Ve6tFA<7L>xPE03=$%Nm(}CQ>-HHvp!CW`nEiT@8 zq(1F0kfQ!bx_w%V8WA>d{S1ZPy@ni#;TO^13FE=EM$^Qu@Uc(NxRblO-N7!P>u^{| z-Wlx0PoY#IK&_3VCX0lJ^CH?AzmhNH&t@Cy!_ed$mP%0~GGIHoyE)S6{6ACa!9G{17 z(@7WI@Q2pTxfT6@neR08M}Xoq{i&duCOip(U=A*_9onXrnHwSjSemCuSl}s`{ z!dbyM(8|VvtU}Q6M}M%idpeu;>Rxi4;Lm#cJmq9Si`nO>n&%594)_A4NZc{2$*q2S zlYKyi<$Ex&knG`splX5g3r9)lVo_Jfd#fkn#6%UKXgIT!C55KQ&9iTHk$yY;52+2Q z{X=T584SNd1=KLnli=XoSP5A)4BGF}9rS8d5_o0U5Q|0Wt-g27%qgW?^NIujAx+PW zsFq+X+VLyS&|G!vda@UUs;WYoM-tA;=$c}xO+kD5d#nh!pk5kf_I)j zqqki={Gz+~4W6zu@NoaZMa`@i4&!$dj;{&5>UM?eTE#Y}ek(YXO?+{1819;ImsSV2 z^mTV#^+*D`VP4;FE%iC;JHgd;D+zIze9J~~-^+oK_;S7=3?I(Ni~AealsIO`VG^eH z=g4{5FDn?N6?ZR$myr~nOuFg7WBxMQ=d_DDJe0VJjbCZDWM?XDIrsOCHOI~d3{&Yi zaH9r}w|HkxH~c=b#N#?PmCB^F&{Z=~Swmq0N;w6NW~VJ7O6Rbik;;cCj-Y;}=y)^h zIrs2@?JumJW!3v`k;)+YE{2dD5Z0CPxJ&3MR_{YRQHZ#Epcd+^kr-E!c9CB1} zg6@qWGCSeB=^toRSyeBh{)L@aaJTe@6?%qLlVJO7)QjpF{AxEW>1F5R5)P<1bTqEK zaOz|o{VwZX0o_lCqFK(}c*C`P%PNZ8#|kn5F}`bN7ru-JmGuLUef(pew4>J&pf zx!LVZ5eTOSDrl3LmuCdMoD^{p^66om%)sGybpNO051;jsBNe7!oAM=7V(P$L6%BAe z%A{TEk3b_1slj3wF>XiFptBX(({~Bdw};E+u+c6K{%(d{StzrJ&&*%7kau`35J(fB z+;9v7F@jO9_J0fS2y0V|SZ4FCn0?1`#T<#BTwnIcSpPfjrS-f#uE4DO0u7Pz*?9%5 zgWyV^%g~Qh0W)1v$99Q;9|903@(-+1W1fccMa;NF8TPb>6`@#zaRyH!Y6J};%(kW< zF!o2#uVtfj%uO(;P~gR>c)q@J&9;*1`ylES8aRI1oLLABB#FIQdJgc#AX3h)FGG`Y zhw_DQnc|pr5y|E`S4aawd(^@s12URH-7QHhJ^d;R9|YklDw>!$qoS%K;ZJun>Ha*w zbrSTV1=V$4vqfdB<+lWzUrxYI4^1dTV6m z{;U8-paj_*&_bp&V5%t|Yd0KX{>S};l$MwG@xAo&@#jtD^Ukm`ch6c1OiVKG>6&j6 z;Psgg`K7=t?F>1_}U1^2ix$U?Wxnz@cJy zU1AHkN{%w~RfT|KoBvdw31MDt$o7@Q0TOs%jaf7)=u>#5vEyEw45lv%0&l<|&yF{A`}B%c66a;>tj-AjUGhBUFD$QF302sLE=&SO zU18p7L##+XqBps;j}&Fa8|q2}KCHA&*YUMs|Me%HJiff17h+{>hF!+wMc^O z$fEkaaA{1T(6uZq{t&`Jb3s;;uSNWd5oXd9m^G%` z&xzj%fhx@D-w#QjP*>%%Q_axe$_+^n`J3cV&@a2rfczB50%pB%t^20R$*l&o%!G0O z*BlHu_ZDYvD4LWFc>;L-2>sTVv|%vHVlgNZnU2y@{jQLp3apDB{Nyctmd18MUzig6>BFkM6Sa@-pD*qPjxdJOsTuG1sOd!QO>s2MIydQh`;{nVZ$)FZ``CAvPBikB zB|Ep!vm5m9=UvkFC)`IKq$ZDZp~)S73j`GatGB7;^=QNqrq9EHA|z?~_y>6pxBSC_ z=EEM5IYy1=Skz4-I38%go|=|LIlq5mKOZr^$*+d|_TvL#kVM^=V*H?#lGA@S4T(>%S{YwhFpJj27EC70i?W`-&tLs#Zo?=1Bg(t4sce zWP5f-s_hYB{%!e_P|W>k^glmp;Kly!LEE{_Ep5N-176UXI?`LU`_R`geVs(%(pG}W zp!i*}dGG=jXYcf{(rhk^a#V%a7pE9v=`uOVd6H!Gz9Bj9)3y|{)iggK|5$;SGl5C{ zk1Lb5b2UdSA5$QhMg(5+cXiwd#Dk(#j{z#O);O+pT;vbGdpiOZwqTyq}%QZF-0YTT#9hnxx>Kki}tsD+FPtLJgUA zB(Ln0OU)SyN;|K;Yq^>Iv|A(K9$z+c3m?tU`c{FB2EBCL`$+k#AG))t0FGtYNJDYx z_&kAFE4je>DCvvD51^hwR7sptpd1DV5M}ks@hemR>MWwpR7TY=4UX>71liya4@f`{ z6_W$CqX|=aOhC<>0QfVCrR###a_!1pkQ#QNK;&Feo?~COhDA9%JL~XHgcD;wQj7Jq zs=x$&Z-!bjhn=aI%DJI@6a%|gCLIB|yPTp%I*Cgw&5CT^=@`&suS&z9G?bp<=sL)^U z(0<=r-OAccFUe}cn5wr0NZeM+zQW}y7=j4 zot~9KxRRvqmDPl(^3c6r7J>; zr{Cz4GxJ`ut8vh{cGwF)#-(d|rF7;r27=x0DkYW6<~QSyIgn_aN+wb(pHds0>5M>{ zaY36;scJY7zq2<|w5b)1M~5+Tqjij?cYFIvx7S*dg=JuCx%PFubu#lH2U8$V(#Q!i zL64bm87-h6x%m<7QIK!*1T&z-`o|h2_@Idf9}weMc6zFm(FdZ)r{VSVyT5DjY!2Eu ztT9$bdXAzMB640?QG`0cz$+0O?*_<@3W7M9aOh7h^_8VjZ%~!9rPT%<9%L%J+tV)8 zu{7!I+mCO*vce)J&8=Soa3#suaF-zAKO?e22|*rX6gz~T?X-~kgOyaK@dejIvv3}q z>GEOl5XK#SGS{|!tNK_BbrsFc96-R-$<)hkPbj zAwfd%q~uZKgX5zdKCbSaBm{@u)v$W%V_?P|7?0cx$~c3P6|(Ax^c)JTGn*=1HnSoj z%GcPDF*{ew=z>S$p=R9XrL;U#f$|7UwWg+i%ei*5=WNNkn6Rr6R7QnFG6!6*Uon+u zuz5@tdC6mFs_L7oN;+)gd@^5mf92tq2*e8&@{PytOP_*oS~EX-(?d%uJo*B&oZ+ni1w;q z;Ov`pP;7NQgALdey8_`9fgT3ADnuEvMSzM+1nct5ICFJ()}@Rz}eeA*q#AfYl0 z`O<`v=@EIKNW$e_{cxFDNwokeOki{N)h1U%A=g3TX~bu{Fr9Y)eifkc4D?e zrsoYJo?|R0$C$XGgCJ~Ze}L5jLq`BgZ0`~CE;XL#*$LRv@kmHi{pO|4|IyreMm5=G zdmKeXr78;2QIM!0C?KKt4iW{FPUt8|0#ZT=AiW6)5g`T$N|z=e9fEWbrG#D*Kzi>j zkZ{Ai@7y(O=FXb6X6D2Fbh6fYvY&mP{Xcu3-}!LPg-(8O+C(^;==g>dIN&Ci5$Rt#eIK@VTbl)S zF3yf+76ThT9eyaqp4k>oi$Et@2grJnHjZh!qR2nadrKS}U_(v&{@86?( zuGM00{&-r%9l@iq)7L5>c<*8QywAtb{&`2n#gF@4TwU?xFu6y+PP0ai&;_gC*j==W z)rCFzz?vnEseF9liB!gys$m`u6L!TI&j-UaynSx{;b9TQ^)H2Bv+%k4 zrpfkVeV~&bvE9@H8-VE*Q2ZpzPn|A3C$Ze8{H$H(nUzer-TJu29!D8v@r#|I9Qrg5 z9m}cg+qQldgZbaj9mLZMF{|G$a@H?5vz+;Pjou;7Z9B-+BRgEwR&6NN>=9yQnpLE| zb6MAg-uU{jx2zVc45tZdjK(^WE{BExiaFo!Gv_!tbonCG&nU4CP%$$z8yOjq>tw#R zle{5Ro>DXdu@v>CwC*beEvx3U6U~8<(Diix5ZbDGSD%+Q6@xtEImng!h%TO$!)A9d zyVeQpHGNi#5?EBr#D{fv4Meu`i?JZnpjbq;cV|i)Rzx6PNN>~EUge?M(!phVZt3O{ zKqrUy;aLD7h_2fqit27NRoapyU3~M7efw?NXOcDV$3GFLbyzOVXgAA9mPlbodq=b@ z#Ogh3QLizZth#F3Ix}fAJC_(GRbW1E%BDvKcrT9?*--xues_t6mL>Lq3%!(7+*^S} zjiL4`mlZJ(UDM$!dw_uT^HLS(%x?#xl;nRxD-<3e<`>MWq*f&(u<{R-^WNr2Z1Lro zD5wJ#ds*(#!U|DkTzbVqWgNAo5aJC1mLy{fSPP}n2l5X{e|>q(sqWMt-j0)LK4|r5 zkODS|&QnOS7XEwy?^JMjmzHxB!GW?bl55!=sZj)_Az3J29iHt@aZ;SQpC{a)@nEPt zS~@o*r2KBD_SC55pwKAG;Gj^tSoy}d#MGId@h2g#Dy9#ypKXi0xlG;PngYDan3Q;AE?azxM6Qm z{eWR&Wn|r6BY|&`>vuo+Uwo|pz)^foI-B0TodgZqCl_I*2WQ(67xh|4&7mV0--(Kc zwwqqSNQ3yJbi>`ie#;68l^SWqoAErzA9~4!>ysXjnA`OD`c#>r@Ds7zt0H_%7qR+d z1sT=foXH5G(izJ}p%}E{6==r4=MQu;#Cx!NE^&=Kdl3*S_{L7c^RwEk z8P%>dzj!lPYF~Tl)e6nHUq7xlJA8Vh6WhZ?5pyY2Cn`f!TGRhVbSuYv(TZj7>fqRI zgn${Jv}`xr`<GFOJ z;crd0F0lVhSd7BNIgb{uR#@e%g*qqh8N0>SJ2J zGTEdWRc&}7P3Zo+3lzCYfaK^|33qK99j3ZAE1U0ns^!?PrV2-e08;w8%v%~1cl-7^ifNxzE)AUdnVi6J&6lh zk8xkOJy45ZVzlBvx&tFymcBNu?J%}#1)B~awE!BlU0*gw&Wi?Lys~&_$*Hptt(Lsd zo!ZlO&a0?$mY>*bVw?wg@>W2}|{7g4P+nqsLXNNcY076?Di2*$goOtwLp zGDX9!HaSoYN<~i3bnm-mA*QC%*o&j>vXfD4)?+rxe}h>s%Qp6w>59cJ1YgewsV_z- zmJJ%o4iDT$2oH8#RbQz2WlAO6ObfHHnhVJwCx}JT7Y4I_0E>24(UDyrEl zVY9Xk?q2JP%Xyw6*ETu)Yq9anW7G#rOy&2^`RGjGuLC+ll)>FZRNM&5p*6roqF(As zSw_XsDVQxfGK3~MlFx&wTPc7OHV`uNT{rK3jq*wt1&4-x~Eu8M9SoLO-Q{x)t zk;pDELM@Vt@#3eJPrc5twWhYE=UR7B*?`M{v;`X%>#%JHnoLL;m<7ZTZyUMjp;_)~ z6?H4Yw$JtXPf=HhGuxCUL!?&_X60NKFQ;H4zoEJVn_CJQie|hb#}ydN&HZcvT5!qK z^e52q8oC%sehcG|qJ}r+dsJn(as&`LJFUuQlCY>BHL0D?-ex7Urxy#jF4Ma(@^Zx$ z%UX*$v`PAvpi1(_M-M&78uE+HyQMKlV|#R^Wm?vDi%O(RXi)ZU#QtT6(An8tb+pn_ zvsX}=Xrh*hqV(2k+UE{7XmSoq=IcuHJ1o)kZxW<<`^ z)c2={kI&4EGO!mfunyksU*PGh$2>X{X~2?Fr(hE)KJoT7F}mx>+$ft5K){yQ1mvzzVQ;YLwY_zS$DGWs1?Muc(` zX=Z*1e0#=={0n=PcgE|oy>qh>+IU*~U3)*o zm5rzUSlx8#Os{2lZPnB@6O~A3F27U28o&RWNd@+$i$KfUJbxk0ANiXgaCSX&)e14_ zJj&&+;#qM|4|Nb8a6oNR$Q-ZnM(0IB>qo0L^pX`v1fP7n7Y1dPW-1S(2ni|V^v4gr z=$d0SUAQR^tU8JxhB0xb8>KjSNvHx{vD2!0EL*lsH|$F<=IZKYC3TRlD>B;Mr&69} zoBFo9?PLfPp;aEX+j&pU?nC43ir07s)jjLtRwTrnuKhqv&$t-}b$Pcz-993`q_joA zlRvEyX^Ag{dbID9SB$aBs!(%g>gtLn3?GlaHj;U5Pt7UzLJU@=f1Tr|Qes>7c6$^1 z8BZ4J_g0A=QykloPwcl*U2$WYW9(0EhKK=Idcr~sO{Ag0?l~dU8aXEezF<%R8t#oW z%g_zJy=nB$&`2L@hoU^#qKf^ZkFb~_UahI*trH0^CE*S)Q ztl_I;egAk12Q!Vs91?B8hdoj)n(ybq zgMzLJSwuS5l|lr0B3dKQ1CxwROn}`&W|{YsjbXi?P-BdIgvHT1nH#&+Y$`PixXm6N zL}(KuQ~A}_JS%aB-qiAXl!T#?iNz4mD(00(j`Q2mM~K!fu~|n14>N!(TFJ=HXO4wK zivhnSX?!h9^ioV~n-GNeT93Y+eAAST zgT$UKl2;f!sWLGr4elkmX46n@Wm%=A7^PzVATwf7WkqCGse9!sK~+7ttFqo3EKshw z<9S--I3mU3%+2rx-8ffI8Ev>Tk`&1r)yvV^n`aX3P>|gTT#X8zKIr*c(cYO}; z1pRyI8Hk;Q=&NCs!|@nZjY2c4P-d1b(1S0FBV9_N)i2@H&T*n*#-GIXERT_AA~-$) zP1BP*+}wp)^vB)TBjb@NpQI9U+*j`KN4VtrzO|MF*>_%UyDbaazbq;K%girgV5Elb z2XFU%e^;FKyqeb>hXNoYQ8ElBX32JC@Ux(Bs1wsGSPA}1n*rp*!gr3~L2RIERMfuat}aF`f9Uv4(M2E^vi)e^`8~0OuYkX&G5<W5r2WYe}};S0V?2L4eaT}{SEbiNcj&0GIgHP3*VX8KN0TD-y_^V82C3R@Z%}` zrV=3`r?1i+lKd0@{xv`T_2r2S>oCf%;Gz;ZBnYc5pS{Ptu~B?~W92q4ns_~)2pN@K zWRpKmotQjCj}v|dqe;vGhr8{oL!8w=oF)$00ycu#;<%2ug#FigA9b5n`ffvrj;m|< zrxr(R=K!4no$Syv46$&Q|AM2kuGDra+!QW+;`+!Ar@i{_?0qALf@OGpFdfGIOdvYmX z_`znORT1Zb8)h?)@(5Sw`#8E1f8KxgdUwE`k|}M`Ete+Y-qb1r+r;50_eQWbwYN*O zYQdQ7a0R#!hpTQ|K5~etGV(7dCUaApIf?%a^<*7S)9t8YXBhuQrGU!|`;V>%8% z?0(K5L|mZqKi)`^I6Stg+b{)BZ_QDWy&n0)>pdyq`Yryr<2Fr93A2{BOh@1-bgO*C z*q^j6-_>673}y%LV=K{x&0^#32sanU`Ue-o=XfYOY=bC^Px!${n+%7Ch6#gLpN1`tfur6m!^1Hk_wUHSUG;tXjRS?V_=C&EU|f3z zF5l!{Q3cOd&oi6A(XK|2e0OxffutN}An?huC#Yi8&3~~`1o@FuIl*)(DL`& zU=2vr!Qhf#XSxr!qj`}lGMJBh~w4ioHC${nASBd@5>sHJP9q_4r`w ziSqS=vHFps(*Q@gZ1lzzO1CpSC#D#J1EQ%B=$Vp;@*3ZW7L8XS-GL~CAFw%*Gvis52VO%u}0q@*4nczp; z+!O0zxaNwrz6g&RYQw`RmB`td3ju^XzUu5`Ei1=UQhUAatSk4geYWqoOqRp`+GIO|gRf&61P`u_dP=@mfadg$fH05@VNB*oDbkRdlngS?@vV4@ z_Mc1ls4*R~xc9_Iw$adHT&m?L)#v0?u8(J?+9ki=vS_H=WC~cFe~ABmnJb|GNuczs z<;})mv*~%L52?l14u@$dz`@*NhWKN*kMrX;Rw9|$DX$GK;p=?SkX9*N(}0}?aVq~9 zu3sthL1%*}$rv+Qsta?X_N%$J>@IN6adyaSc@_Pbp!C7rXLmCc0sUrL#vz|Dw@|H3 zR_f1mA9NfdH})=MWU|#TK99TvYO)c z-nW{1=>nX2tpUl8WVuJtu)7M~J{-H2R$k*YwwLEoBT(3B$Dek>pJfQNpx?#5im8;; z_-)xDh0EgHC3kCIPRKyiZ8Xs=a{;_bK(0V)`Hmr7%(rdIII?U@sMCDcs^|A=FZ-Zn zUEZqfc#wYk&Kt$bsI3G7y@|r42y8%b0CD%S+;V&#&Uf-;V_N=@33CEGL4B8({!R%s zpUVkwOK75zp9%!UZPlV>SFTY^JdMtmKa%z*wAl%r5J88<0cGFixpcOK>BXwyhiv3Y zc&O1*gR4*7$uF-J7_zXrx$j3Jk&l3fm$oK$CRM|ytqRvVVoxVp47GLK<-<(|uUUEF zSG$wE@R<{fk9%P7WE^QGpqJ2cp<#Qb{UGA#Y;9@TY2sK8j5`)-IPP5LruLqU5s^Qr zZ9c9QCYq8C)m<8P<@Wx!BJMRmeBYgTyMBk)5!x8caB_*V9n)nfx19_A5e$A`Byopv$yf87S|e)1ph2miI6{|8&j|4&UE2mCtnnjW4_-6!8K OLZPXut5T$78T3DehJKC! literal 0 HcmV?d00001 diff --git a/lecture_17/sin_data.csv b/lecture_17/sin_data.csv new file mode 100644 index 0000000..a5ff272 --- /dev/null +++ b/lecture_17/sin_data.csv @@ -0,0 +1,2 @@ + 0.00000000e+00 1.26933037e-01 2.53866073e-01 3.80799110e-01 5.07732146e-01 6.34665183e-01 7.61598219e-01 8.88531256e-01 1.01546429e+00 1.14239733e+00 1.26933037e+00 1.39626340e+00 1.52319644e+00 1.65012947e+00 1.77706251e+00 1.90399555e+00 2.03092858e+00 2.15786162e+00 2.28479466e+00 2.41172769e+00 2.53866073e+00 2.66559377e+00 2.79252680e+00 2.91945984e+00 3.04639288e+00 3.17332591e+00 3.30025895e+00 3.42719199e+00 3.55412502e+00 3.68105806e+00 3.80799110e+00 3.93492413e+00 4.06185717e+00 4.18879020e+00 4.31572324e+00 4.44265628e+00 4.56958931e+00 4.69652235e+00 4.82345539e+00 4.95038842e+00 5.07732146e+00 5.20425450e+00 5.33118753e+00 5.45812057e+00 5.58505361e+00 5.71198664e+00 5.83891968e+00 5.96585272e+00 6.09278575e+00 6.21971879e+00 6.34665183e+00 6.47358486e+00 6.60051790e+00 6.72745093e+00 6.85438397e+00 6.98131701e+00 7.10825004e+00 7.23518308e+00 7.36211612e+00 7.48904915e+00 7.61598219e+00 7.74291523e+00 7.86984826e+00 7.99678130e+00 8.12371434e+00 8.25064737e+00 8.37758041e+00 8.50451345e+00 8.63144648e+00 8.75837952e+00 8.88531256e+00 9.01224559e+00 9.13917863e+00 9.26611167e+00 9.39304470e+00 9.51997774e+00 9.64691077e+00 9.77384381e+00 9.90077685e+00 1.00277099e+01 1.01546429e+01 1.02815760e+01 1.04085090e+01 1.05354420e+01 1.06623751e+01 1.07893081e+01 1.09162411e+01 1.10431742e+01 1.11701072e+01 1.12970402e+01 1.14239733e+01 1.15509063e+01 1.16778394e+01 1.18047724e+01 1.19317054e+01 1.20586385e+01 1.21855715e+01 1.23125045e+01 1.24394376e+01 1.25663706e+01 + 9.15756288e-02 3.39393873e-01 6.28875306e-01 7.67713096e-01 1.05094584e+00 9.70887288e-01 9.84265740e-01 1.02589034e+00 8.53218113e-01 6.90197665e-01 5.51277193e-01 5.01564914e-01 5.25455797e-01 5.87052838e-01 5.41394658e-01 7.12365594e-01 8.14839678e-01 9.80181855e-01 9.44430709e-01 1.06728057e+00 1.15166322e+00 8.99464065e-01 7.77225453e-01 5.92618124e-01 3.08822183e-01 -1.07884730e-03 -3.46563271e-01 -5.64836023e-01 -8.11931510e-01 -1.05925186e+00 -1.13323611e+00 -1.11986890e+00 -8.88336727e-01 -9.54113139e-01 -6.81378679e-01 -6.02369117e-01 -4.78684439e-01 -5.88160325e-01 -4.93580777e-01 -5.68747320e-01 -7.51641934e-01 -8.14672884e-01 -9.53191554e-01 -9.55337518e-01 -9.85995556e-01 -9.63373597e-01 -1.01511061e+00 -7.56467517e-01 -4.17379564e-01 -1.22340361e-01 2.16273929e-01 5.16909714e-01 7.77031694e-01 1.00653798e+00 9.35718089e-01 1.00660116e+00 1.11177057e+00 9.85485116e-01 8.54344900e-01 6.26444042e-01 6.28124048e-01 4.27764254e-01 5.93991751e-01 4.79248018e-01 7.17522492e-01 7.35927848e-01 9.08802925e-01 9.38646871e-01 1.13125860e+00 1.07247935e+00 1.05198782e+00 9.41647332e-01 6.98801244e-01 4.03193543e-01 1.37009682e-01 -1.43203880e-01 -4.64369445e-01 -6.94978252e-01 -1.03483196e+00 -1.10261288e+00 -1.12892727e+00 -1.03902484e+00 -8.53573083e-01 -7.01815315e-01 -6.84745997e-01 -6.14189417e-01 -4.70090797e-01 -5.95052432e-01 -5.96497000e-01 -5.66861911e-01 -7.18239679e-01 -9.52873043e-01 -9.37512847e-01 -1.15782985e+00 -1.03858206e+00 -1.03182712e+00 -8.45121554e-01 -5.61821980e-01 -2.83427014e-01 -8.27056140e-02 diff --git a/lecture_17/xy_data.csv b/lecture_17/xy_data.csv new file mode 100644 index 0000000..15b919d --- /dev/null +++ b/lecture_17/xy_data.csv @@ -0,0 +1,2 @@ + 0.00000000e+00 2.00000000e+00 4.00000000e+00 6.00000000e+00 8.00000000e+00 1.00000000e+01 1.20000000e+01 1.40000000e+01 1.60000000e+01 1.80000000e+01 2.00000000e+01 + 2.15011412e+01 2.08415256e+01 2.31920098e+01 2.26949773e+01 3.02668745e+01 4.01107461e+01 4.33054255e+01 5.47873036e+01 7.08844291e+01 8.94836828e+01 9.72813533e+01 From 95d9d2d2259954976cafd2611f14593e338ec18b Mon Sep 17 00:00:00 2001 From: "Ryan C. Cooper" Date: Tue, 28 Mar 2017 11:56:23 -0400 Subject: [PATCH 2/2] added lecture 17 --- lecture_17/lecture_16.ipynb | 2710 ----------------------------------- lecture_17/lecture_17.ipynb | 2646 +++++++++++++++++++++++++++++++++- 2 files changed, 2641 insertions(+), 2715 deletions(-) delete mode 100644 lecture_17/lecture_16.ipynb diff --git a/lecture_17/lecture_16.ipynb b/lecture_17/lecture_16.ipynb deleted file mode 100644 index 3159c42..0000000 --- a/lecture_17/lecture_16.ipynb +++ /dev/null @@ -1,2710 +0,0 @@ -{ - "cells": [ - { - "cell_type": "code", - "execution_count": 65, - "metadata": { - "collapsed": true - }, - "outputs": [], - "source": [ - "setdefaults" - ] - }, - { - "cell_type": "code", - "execution_count": 66, - "metadata": { - "collapsed": true - }, - "outputs": [], - "source": [ - "%plot --format svg" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "![question 1](q1.png)\n", - "\n", - "![question 2](q2.png)\n", - "\n", - "### Project Ideas so far\n", - "\n", - "- Nothing yet...probably something heat transfer related\n", - "\n", - "- Modeling Propulsion or Propagation of Sound Waves\n", - "\n", - "- Low Thrust Orbital Transfer\n", - "\n", - "- Tracking motion of a satellite entering orbit until impact\n", - "\n", - "- What ever you think is best.\n", - "\n", - "- You had heat transfer project as an option; that sounded cool\n", - "\n", - "- Heat transfer through a pipe\n", - "\n", - "- I would prefer to do something with beam/plate mechanics or vibrations than a heat transfer or thermo problem\n", - "\n", - "- Modeling Couette flow with a pressure gradient using a discretized form of the Navier-Stokes equation and comparing it to the analytical solution\n", - "\n", - "- Software to instruct a robotic arm to orient itself based on input from a gyroscope" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Questions from you:\n", - "\n", - "How was your spring break\n", - "\n", - "Are grades for hw 3 and 4 going to be posted?\n", - "\n", - "For class occasionally switching to doc cam and working through problems by hand might help get ideas across.\n", - "\n", - "Are you assigning those who do not have groups to groups?\n", - "\n", - "How do you graph a standard distribution in Matlab?\n", - "\n", - "What is the longest code you have written?\n", - "\n", - "how to approach probability for hw 5?\n", - "??\n", - "\n", - "Will you be assigning groups to people who do not currently have one? \n", - "\n", - "what are some basic guidelines we should use to brainstorm project ideas?\n", - "\n", - "Are you a fan of bananas?\n", - "\n", - "Going through code isn't the most helpful, because you can easily lose interest. But I am not sure what else you can do.\n", - "\n", - "Has lecture 15 been posted yet? Looking in the repository I can't seem to find it." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Linear Least Squares Regression\n", - "\n", - "When approximating a set of data as a polynomial, there will always be error introduced (except in 2 cases). \n", - "\n", - "For a straight line, the actual data points, $y_{i}$ as a function of the independent variable, $x_{i}$, is:\n", - "\n", - "$y_{i}=a_{0}+a_{1}x_{i}+e_{i}$\n", - "\n", - "where $a_{0}$ and $a_{1}$ are the intercept and slope of the line and $e_{i}$ is the error between the approximate function and the recorded data point. \n", - "\n", - "We make the following assumptions in this analysis:\n", - "\n", - "1. Each x has a fixed value; it is not random and is known without error.\n", - "\n", - "2. The y values are independent random variables and all have the same variance.\n", - "\n", - "3. The y values for a given x must be normally distributed.\n", - "\n", - "The total error is \n", - "\n", - "$\\sum_{i=1}^{n}e_{i}=\\sum_{i=1}^{n}(y_{i}-a_{0}-a_{1}x_{i})$\n", - "\n", - "we don't care about the sign though. One approach has been demonstrated to provide a unique solution is minimizing the sum of squares error or\n", - "\n", - "$S_{r}=\\sum_{i=1}^{n}e_{i}^{2}=\\sum_{i=1}^{n}(y_{i}-a_{0}-a_{1}x_{i})^{2}$\n", - "\n", - "Where, $S_{r}$ is the sum of squares error (SSE). \n", - "\n", - "$\\frac{\\partial S_{r}}{\\partial a_{0}}=-2\\sum(y_{i}-a_{0}-a_{1}x_{i})$\n", - "\n", - "$\\frac{\\partial S_{r}}{\\partial a_{1}}=-2\\sum(y_{i}-a_{0}-a_{1}x_{i})x_{i}$\n", - "\n", - "The minimum $S_{r}$ occurrs when the partial derivatives are 0. \n", - "\n", - "$\\sum y_{i}= \\sum a_{0}+\\sum a_{1}x_{i}$\n", - "\n", - "$\\sum x_{i}y_{i}= \\sum a_{0}x_{i}+\\sum a_{1}x_{i}^{2}$\n", - "\n", - "$\\left[\\begin{array}{c}\n", - "\\sum y_{i}\\\\\n", - "\\sum x_{i}y_{i}\\end{array}\\right]=\n", - "\\left[\\begin{array}{cc}\n", - "n & \\sum x_{i}\\\\\n", - "\\sum x_{i} & \\sum x_{i}^{2}\\end{array}\\right]\n", - "\\left[\\begin{array}{c}\n", - "a_{0}\\\\\n", - "a_{1}\\end{array}\\right]$\n", - "\n", - "\n", - "$b=Ax$\n", - "\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Example \n", - "\n", - "Find drag coefficient with best-fit line to experimental data\n", - "\n", - "|i | v (m/s) | F (N) |\n", - "|---|---|---|\n", - "|1 | 10 | 25 |\n", - "|2 | 20 | 70 |\n", - "|3 | 30 | 380|\n", - "|4 | 40 | 550|\n", - "|5 | 50 | 610|\n", - "|6 | 60 | 1220|\n", - "|7 | 70 | 830 |\n", - "|8 |80 | 1450|" - ] - }, - { - "cell_type": "code", - "execution_count": 68, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "a =\n", - "\n", - " -234.286\n", - " 19.470\n", - "\n" - ] - }, - { - "data": { - "image/svg+xml": [ - "\n", - "\n", - "Gnuplot\n", - "Produced by GNUPLOT 5.0 patchlevel 3 \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t \n", - "\t \n", - "\t\n", - "\t\n", - "\t \n", - "\t \n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\t\n", - "\t\t\n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\t\t\n", - "\t\t-500\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t0\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t500\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t1000\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t1500\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t2000\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t10\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t20\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t30\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t40\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t50\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t60\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t70\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t80\n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\t\n", - "\n", - "\t\n", - "\t\tvelocity (m/s)\n", - "\t\n", - "\n", - "\n", - "\t\n", - "\t\tForce (N)\n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\t\n", - "\tdata\n", - "\n", - "\n", - "\n", - "\t\n", - "\t\tdata\n", - "\t\n", - "\n", - "\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\n", - "\t\n", - "\tbest-fit\n", - "\n", - "\t\n", - "\t\tbest-fit\n", - "\t\n", - "\n", - "\n", - "\t\n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "" - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "drag_data=[...\n", - "1 , 10 , 25 \n", - "2 , 20 , 70 \n", - "3 , 30 , 380\n", - "4 , 40 , 550\n", - "5 , 50 , 610\n", - "6 , 60 , 1220\n", - "7 , 70 , 830 \n", - "8 ,80 , 1450];\n", - "x=drag_data(:,2);\n", - "y=drag_data(:,3);\n", - "\n", - "b=[sum(y);sum(x.*y)];\n", - "A=[length(x),sum(x);\n", - " sum(x), sum(x.^2)];\n", - " \n", - "a=A\\b\n", - "\n", - "plot(x,y,'o',x,a(1)+a(2)*x)\n", - "legend('data','best-fit','Location','NorthWest')\n", - "xlabel('Force (N)')\n", - "ylabel('velocity (m/s)')" - ] - }, - { - "cell_type": "code", - "execution_count": 72, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "data": { - "image/svg+xml": [ - "\n", - "\n", - "Gnuplot\n", - "Produced by GNUPLOT 5.0 patchlevel 3 \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t \n", - "\t \n", - "\t\n", - "\t\n", - "\t \n", - "\t \n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\t\n", - "\t\t\n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\t\t\n", - "\t\t-2000\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t-1500\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t-1000\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t-500\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t0\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t10\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t20\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t30\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t40\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t50\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t60\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t70\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t80\n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\t\n", - "\n", - "\t\n", - "\t\tresiduals (N)\n", - "\t\n", - "\n", - "\n", - "\t\n", - "\t\tvelocity (m/s)\n", - "\t\n", - "\n", - "\n", - "\t\n", - "\t\tModel does not capture measurements\n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\t\n", - "\tdata\n", - "\n", - "\n", - "\n", - "\t\n", - "\t\tdata\n", - "\t\n", - "\n", - "\n", - "\t\n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "" - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "plot(x,y-a(1)-40*x)\n", - "legend('data','best-fit','Location','NorthWest')\n", - "ylabel('residuals (N)')\n", - "xlabel('velocity (m/s)')\n", - "title('Model does not capture measurements')" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "How do we know its a \"good\" fit? \n", - "\n", - "Can compare the sum of squares error to the total sum of squares of the dependent variable (here F). \n", - "\n", - "$S_{r}=\\sum(y_{i}-a_{0}-a_{1}x_{i})^{2}$\n", - "\n", - "$S_{t}=\\sum(y_{i}-\\bar{y})^{2}$\n", - "\n", - "Then, we can calculate the *coefficient of determination*, $r^{2}$ or *correlation coefficient*, r. \n", - "\n", - "$r^{2}=\\frac{S_{t}-S_{r}}{S_{t}}$\n", - "\n", - "This represents the relative improvement of assuming that y is a function of x (if the x-values are not random and the y-values are random)\n", - "\n", - "For further information regarding statistical work on regression, look at \n", - "[NIST Statistics Handbook](http://www.itl.nist.gov/div898/handbook/pmd/section4/pmd44.htm)" - ] - }, - { - "cell_type": "code", - "execution_count": 73, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "St = 1.8083e+06\n", - "St = 1.8083e+06\n" - ] - } - ], - "source": [ - "Sr=sum((y-a(1)-a(2)*x).^2);\n", - "St=std(y)^2*(length(y)-1)\n", - "St=sum((y-mean(y)).^2)" - ] - }, - { - "cell_type": "code", - "execution_count": 74, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "r2 = 0.88049\n", - "r = 0.93834\n" - ] - } - ], - "source": [ - "r2=(St-Sr)/St\n", - "r=sqrt(r2)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Limiting cases \n", - "\n", - "#### $r^{2}=0$ $S_{r}=S{t}$\n", - "\n", - "#### $r^{2}=1$ $S_{r}=0$" - ] - }, - { - "cell_type": "code", - "execution_count": 76, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "data": { - "image/svg+xml": [ - "\n", - "\n", - "Gnuplot\n", - "Produced by GNUPLOT 5.0 patchlevel 3 \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t \n", - "\t \n", - "\t\n", - "\t\n", - "\t \n", - "\t \n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\t\n", - "\t\t\n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\t\t\n", - "\t\t0\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t0.2\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t0.4\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t0.6\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t0.8\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t1\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t0\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t0.2\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t0.4\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t0.6\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t0.8\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t1\n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\t\n", - "\tCase 1\n", - "\n", - "\n", - "\n", - "\t\n", - "\t\tCase 1\n", - "\t\n", - "\n", - "\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\n", - "\t\n", - "\tCase 2\n", - "\n", - "\t\n", - "\t\tCase 2\n", - "\t\n", - "\n", - "\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "" - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "gen_examples\n", - "plot(x1(1:50:end),y1(1:50:end),'s',x2,y2,'o')\n", - "legend('Case 1','Case 2','Location','NorthWest')" - ] - }, - { - "cell_type": "code", - "execution_count": 77, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "a1 =\n", - "\n", - " 0.0482496\n", - " 0.0017062\n", - "\n", - "a2 =\n", - "\n", - " 0\n", - " 1\n", - "\n", - "Sr1 = 0.83505\n", - "St1 = 0.83529\n", - "coefficient of determination in Case 1 is 0.000291\n", - "Sr2 = 0\n", - "St2 = 1.0185\n", - "coefficient of determination in Case 2 is 1.000000\n" - ] - } - ], - "source": [ - "b1=[sum(y1);sum(x1.*y1)];\n", - "A1=[length(x1),sum(x1);\n", - " sum(x1), sum(x1.^2)];\n", - " \n", - "a1=A1\\b1\n", - "\n", - "b2=[sum(y2);sum(x2.*y2)];\n", - "A2=[length(x2),sum(x2);\n", - " sum(x2), sum(x2.^2)];\n", - " \n", - "a2=A2\\b2\n", - "\n", - "Sr1=sum((y1-a1(1)-a1(2)*x1).^2)\n", - "St1=sum((y1-mean(y1)).^2)\n", - "fprintf('coefficient of determination in Case 1 is %f\\n',1-Sr1/St1)\n", - "\n", - "Sr2=sum((y2-a2(1)-a2(2)*x2).^2)\n", - "St2=sum((y2-mean(y2)).^2)\n", - "\n", - "fprintf('coefficient of determination in Case 2 is %f\\n',1-Sr2/St2)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "collapsed": true - }, - "source": [ - "# General Linear Regression" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "In general, you may want to fit other polynomials besides degree-1 (straight-lines)\n", - "\n", - "$y=a_{0}+a_{1}x+a_{2}x^{2}+\\cdots+a_{m}x^{m}+e$\n", - "\n", - "Now, the solution for $a_{0},~a_{1},...a_{m}$ is the minimization of m+1-dependent linear equations. \n", - "\n", - "Consider the following data:\n", - "\n", - "| x | y |\n", - "|---|---|\n", - "| 0.00 | 21.50 |\n", - "| 2.00 | 20.84 |\n", - "| 4.00 | 23.19 |\n", - "| 6.00 | 22.69 |\n", - "| 8.00 | 30.27 |\n", - "| 10.00 | 40.11 |\n", - "| 12.00 | 43.31 |\n", - "| 14.00 | 54.79 |\n", - "| 16.00 | 70.88 |\n", - "| 18.00 | 89.48 |\n", - "\n" - ] - }, - { - "cell_type": "code", - "execution_count": 78, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "data": { - "image/svg+xml": [ - "\n", - "\n", - "Gnuplot\n", - "Produced by GNUPLOT 5.0 patchlevel 3 \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t \n", - "\t \n", - "\t\n", - "\t\n", - "\t \n", - "\t \n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\t\n", - "\t\t\n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\t\t\n", - "\t\t20\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t30\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t40\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t50\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t60\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t70\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t80\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t90\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t100\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t0\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t5\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t10\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t15\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t20\n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\t\n", - "\n", - "\n", - "\tgnuplot_plot_1a\n", - "\n", - "\n", - "\n", - "\t \n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "" - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "load xy_data.csv\n", - "x=xy_data(1,:)';\n", - "y=xy_data(2,:)';\n", - "plot(x,y,'o')" - ] - }, - { - "cell_type": "code", - "execution_count": 46, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "xy_data =\n", - "\n", - " Columns 1 through 7:\n", - "\n", - " 0.00000 2.00000 4.00000 6.00000 8.00000 10.00000 12.00000\n", - " 21.50114 20.84153 23.19201 22.69498 30.26687 40.11075 43.30543\n", - "\n", - " Columns 8 through 11:\n", - "\n", - " 14.00000 16.00000 18.00000 20.00000\n", - " 54.78730 70.88443 89.48368 97.28135\n", - "\n" - ] - } - ], - "source": [ - "xy_data" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "The model can be rewritten as \n", - "\n", - "$y=\\left[Z\\right]a+e$\n", - "\n", - "where $a=\\left[\\begin{array}{c}\n", - " a_{0}\\\\\n", - " a_{1}\\\\\n", - " a_{2}\\end{array}\\right]$\n", - " \n", - "$[Z]=\\left[\\begin{array} \n", - "1 & x_{1} & x_{1}^{2} \\\\\n", - "1 & x_{2} & x_{2}^{2} \\\\\n", - "1 & x_{3} & x_{3}^{2} \\\\\n", - "1 & x_{4} & x_{4}^{2} \\\\\n", - "1 & x_{5} & x_{5}^{2} \\\\\n", - "1 & x_{6} & x_{6}^{2} \\\\\n", - "1 & x_{7} & x_{7}^{2} \\\\\n", - "1 & x_{8} & x_{8}^{2} \\\\\n", - "1 & x_{9} & x_{9}^{2} \\\\\n", - "1 & x_{10} & x_{10}^{2} \\end{array}\\right]$\n", - "\n", - "The sum of squares residuals for this model is\n", - "\n", - "$S_{r}=\\sum_{i=1}^{n}\\left(y_{i}-\\sum_{j=0}^{m}a_{j}z_{ji}\\right)$\n", - "\n", - "Minimizing this function results in\n", - "\n", - "$y=[Z]a$\n", - "\n", - "->**A standard Linear Algebra Problem**\n", - "\n", - "*the vector a is unknown, and Z is calculated based upon the assumed function*\n" - ] - }, - { - "cell_type": "code", - "execution_count": 79, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Z =\n", - "\n", - " 1 0 0\n", - " 1 2 4\n", - " 1 4 16\n", - " 1 6 36\n", - " 1 8 64\n", - " 1 10 100\n", - " 1 12 144\n", - " 1 14 196\n", - " 1 16 256\n", - " 1 18 324\n", - " 1 20 400\n", - "\n", - "a =\n", - "\n", - " 21.40341\n", - " -0.81538\n", - " 0.23935\n", - "\n" - ] - } - ], - "source": [ - "Z=[x.^0,x,x.^2]\n", - "\n", - "a=Z\\y" - ] - }, - { - "cell_type": "code", - "execution_count": 80, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "data": { - "image/svg+xml": [ - "\n", - "\n", - "Gnuplot\n", - "Produced by GNUPLOT 5.0 patchlevel 3 \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t \n", - "\t \n", - "\t\n", - "\t\n", - "\t \n", - "\t \n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\t\n", - "\t\t\n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\t\t\n", - "\t\t20\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t40\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t60\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t80\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t100\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t120\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t0\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t5\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t10\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t15\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t20\n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\t\n", - "\n", - "\n", - "\tgnuplot_plot_1a\n", - "\n", - "\n", - "\n", - "\t \n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\n", - "\t\n", - "\tgnuplot_plot_2a\n", - "\n", - "\t\n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "" - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "x_fcn=linspace(min(x),max(x));\n", - "plot(x,y,'o',x_fcn,a(1)+a(2)*x_fcn+a(3)*x_fcn.^2)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### General Coefficient of Determination\n", - "\n", - "$r^{2}=\\frac{S_{t}-S_{r}}{S_{t}}=1-\\frac{S_{r}}{S_t}$" - ] - }, - { - "cell_type": "code", - "execution_count": 49, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "St = 27.923\n", - "Sr = 2.6366\n" - ] - } - ], - "source": [ - "St=std(y)\n", - "Sr=std(y-a(1)-a(2)*x-a(3)*x.^2)" - ] - }, - { - "cell_type": "code", - "execution_count": 81, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "the coefficient of determination for this fit is 0.880485\n", - "the correlation coefficient this fit is 0.938342\n" - ] - } - ], - "source": [ - "r2=1-Sr/St;\n", - "r=sqrt(r2);\n", - "\n", - "fprintf('the coefficient of determination for this fit is %f',r2)\n", - "fprintf('the correlation coefficient this fit is %f',r)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Compare this to a straight line fit" - ] - }, - { - "cell_type": "code", - "execution_count": 83, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "St = 27.923\n", - "Sr = 9.2520\n", - "the coefficient of determination for this fit is 0.668655\n", - "the correlation coefficient this fit is 0.817713\n" - ] - }, - { - "data": { - "image/svg+xml": [ - "\n", - "\n", - "Gnuplot\n", - "Produced by GNUPLOT 5.0 patchlevel 3 \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t \n", - "\t \n", - "\t\n", - "\t\n", - "\t \n", - "\t \n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\t\n", - "\t\t\n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\t\t\n", - "\t\t0\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t20\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t40\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t60\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t80\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t100\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t120\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t0\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t5\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t10\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t15\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t20\n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\t\n", - "\n", - "\n", - "\tgnuplot_plot_1a\n", - "\n", - "\n", - "\n", - "\t \n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\n", - "\t\n", - "\tgnuplot_plot_2a\n", - "\n", - "\t\n", - "\t\n", - "\tgnuplot_plot_3a\n", - "\n", - "\t\n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "" - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "Z=[ones(size(x)) x];\n", - "a_line=Z\\y;\n", - "x_fcn=linspace(min(x),max(x));\n", - "plot(x,y,'o',x_fcn,a(1)+a(2)*x_fcn+a(3)*x_fcn.^2,...\n", - "x_fcn,a_line(1)+a_line(2)*x_fcn)\n", - "St=std(y)\n", - "Sr=std(y-a_line(1)-a_line(2)*x)\n", - "r2=1-Sr/St;\n", - "r=sqrt(r2);\n", - "\n", - "fprintf('the coefficient of determination for this fit is %f',r2)\n", - "fprintf('the correlation coefficient this fit is %f',r)" - ] - }, - { - "cell_type": "code", - "execution_count": 85, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "data": { - "image/svg+xml": [ - "\n", - "\n", - "Gnuplot\n", - "Produced by GNUPLOT 5.0 patchlevel 3 \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t \n", - "\t \n", - "\t\n", - "\t\n", - "\t \n", - "\t \n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\t\n", - "\t\t\n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\t\t\n", - "\t\t-15\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t-10\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t-5\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t0\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t5\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t10\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t15\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t0\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t5\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t10\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t15\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t20\n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\t\n", - "\n", - "\t\n", - "\t\tresiduals of straight line\n", - "\t\n", - "\n", - "\n", - "\n", - "\tgnuplot_plot_1a\n", - "\n", - "\n", - "\n", - "\t\n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "" - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "plot(x,y-a_line(1)-a_line(2)*x)\n", - "title('residuals of straight line')" - ] - }, - { - "cell_type": "code", - "execution_count": 86, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "data": { - "image/svg+xml": [ - "\n", - "\n", - "Gnuplot\n", - "Produced by GNUPLOT 5.0 patchlevel 3 \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t \n", - "\t \n", - "\t\n", - "\t\n", - "\t \n", - "\t \n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\t\n", - "\t\t\n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\t\t\n", - "\t\t-4\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t-2\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t0\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t2\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t4\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t6\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t0\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t5\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t10\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t15\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t20\n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\t\n", - "\n", - "\t\n", - "\t\tresiduals of parabola\n", - "\t\n", - "\n", - "\n", - "\n", - "\tgnuplot_plot_1a\n", - "\n", - "\n", - "\n", - "\t\n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "" - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "plot(x,y-a(1)-a(2)*x-a(3)*x.^2)\n", - "title('residuals of parabola')" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Warning \n", - "**Coefficient of determination reduction does not always mean a better fit**\n", - "\n", - "Try the function, \n", - "\n", - "$y(x)=a_0+a_{1}x+a_{2}x^{2}+a_{4}x^{4}+a_{5}x^{5}+a_{5}x^{5}+a_{6}x^{6}+a_{7}x^{7}+a_{8}x^{8}$" - ] - }, - { - "cell_type": "code", - "execution_count": 90, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "a_overfit =\n", - "\n", - " 2.1487e+01\n", - " -1.4264e+01\n", - " 1.5240e+01\n", - " -6.0483e+00\n", - " 1.1887e+00\n", - " -1.2651e-01\n", - " 7.4379e-03\n", - " -2.2702e-04\n", - " 2.8063e-06\n", - "\n" - ] - } - ], - "source": [ - "Z=[ones(size(x)) x x.^2 x.^3 x.^4 x.^5 x.^6 x.^7 x.^8];\n", - "a_overfit=Z\\y" - ] - }, - { - "cell_type": "code", - "execution_count": 91, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "data": { - "image/svg+xml": [ - "\n", - "\n", - "Gnuplot\n", - "Produced by GNUPLOT 5.0 patchlevel 3 \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t \n", - "\t \n", - "\t\n", - "\t\n", - "\t \n", - "\t \n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\t\n", - "\t\t\n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\t\t\n", - "\t\t0\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t20\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t40\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t60\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t80\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t100\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t120\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t0\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t5\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t10\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t15\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t20\n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\t\n", - "\tdata\n", - "\n", - "\n", - "\n", - "\t\n", - "\t\tdata\n", - "\t\n", - "\n", - "\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\n", - "\t\n", - "\tparabola\n", - "\n", - "\t\n", - "\t\tparabola\n", - "\t\n", - "\n", - "\n", - "\t\n", - "\t\n", - "\tn=8-fit\n", - "\n", - "\t\n", - "\t\tn=8-fit\n", - "\t\n", - "\n", - "\n", - "\t\n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "" - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "plot(x,y,'o',x_fcn,a(1)+a(2)*x_fcn+a(3)*x_fcn.^2,...\n", - "x_fcn,a_overfit(1)+a_overfit(2)*x_fcn+a_overfit(3)*x_fcn.^2+...\n", - "a_overfit(4)*x_fcn.^3+a_overfit(5)*x_fcn.^4+...\n", - "a_overfit(6)*x_fcn.^5+a_overfit(7)*x_fcn.^6+...\n", - "a_overfit(8)*x_fcn.^7+a_overfit(9)*x_fcn.^8)\n", - "legend('data','parabola','n=8-fit','Location','NorthWest')" - ] - }, - { - "cell_type": "code", - "execution_count": 92, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "St = 27.923\n", - "Sr = 0.77320\n", - "the coefficient of determination for this fit is 0.972309\n", - "the correlation coefficient this fit is 0.986057\n" - ] - } - ], - "source": [ - "St=std(y)\n", - "Sr=std(y-polyval(a_overfit(end:-1:1),x))\n", - "r2=1-Sr/St;\n", - "r=sqrt(r2);\n", - "\n", - "fprintf('the coefficient of determination for this fit is %f',r2)\n", - "fprintf('the correlation coefficient this fit is %f',r)\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Linear Regression is only limited by the ability to separate the parameters from the function to achieve\n", - "\n", - "$y=[Z]a$\n", - "\n", - "$Z$ can be any function of the independent variable(s)\n", - "\n", - "**Example**:\n", - "We assume two functions are added together, sin(t) and sin(3t). What are the amplitudes?\n", - "\n" - ] - }, - { - "cell_type": "code", - "execution_count": 93, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "data": { - "image/svg+xml": [ - "\n", - "\n", - "Gnuplot\n", - "Produced by GNUPLOT 5.0 patchlevel 3 \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t \n", - "\t \n", - "\t\n", - "\t\n", - "\t \n", - "\t \n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\t\n", - "\t\t\n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\t\t\n", - "\t\t-1.5\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t-1\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t-0.5\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t0\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t0.5\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t1\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t1.5\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t0\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t2\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t4\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t6\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t8\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t10\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t12\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t14\n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\t\n", - "\n", - "\n", - "\tgnuplot_plot_1a\n", - "\n", - "\n", - "\n", - "\t\n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "" - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "load sin_data.csv\n", - "t=sin_data(1,:)';\n", - "y=sin_data(2,:)';\n", - "plot(t,y)" - ] - }, - { - "cell_type": "code", - "execution_count": 94, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "a =\n", - "\n", - " 0.99727\n", - " 0.50251\n", - "\n" - ] - } - ], - "source": [ - "Z=[sin(t) sin(3*t)];\n", - "a=Z\\y" - ] - }, - { - "cell_type": "code", - "execution_count": 95, - "metadata": { - "collapsed": false - }, - "outputs": [ - { - "data": { - "image/svg+xml": [ - "\n", - "\n", - "Gnuplot\n", - "Produced by GNUPLOT 5.0 patchlevel 3 \n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t \n", - "\t \n", - "\t\n", - "\t\n", - "\t \n", - "\t \n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\t\n", - "\t\t\n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\t\t\n", - "\t\t-1.5\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t-1\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t-0.5\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t0\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t0.5\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t1\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t1.5\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t0\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t2\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t4\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t6\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t8\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t10\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t12\n", - "\t\n", - "\n", - "\n", - "\t\t\n", - "\t\t14\n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\t\n", - "\n", - "\n", - "\tgnuplot_plot_1a\n", - "\n", - "\n", - "\n", - "\t \n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\t\n", - "\n", - "\t\n", - "\tgnuplot_plot_2a\n", - "\n", - "\t\n", - "\t\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "" - ], - "text/plain": [ - "" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "plot(t,y,'.',t,a(1)*sin(t)+a(2)*sin(3*t))" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "collapsed": true - }, - "outputs": [], - "source": [] - } - ], - "metadata": { - "kernelspec": { - "display_name": "Octave", - "language": "octave", - "name": "octave" - }, - "language_info": { - "file_extension": ".m", - "help_links": [ - { - "text": "MetaKernel Magics", - "url": "https://github.com/calysto/metakernel/blob/master/metakernel/magics/README.md" - } - ], - "mimetype": "text/x-octave", - "name": "octave", - "version": "0.19.14" - } - }, - "nbformat": 4, - "nbformat_minor": 2 -} diff --git a/lecture_17/lecture_17.ipynb b/lecture_17/lecture_17.ipynb index a9fec04..3159c42 100644 --- a/lecture_17/lecture_17.ipynb +++ b/lecture_17/lecture_17.ipynb @@ -2,7 +2,7 @@ "cells": [ { "cell_type": "code", - "execution_count": 1, + "execution_count": 65, "metadata": { "collapsed": true }, @@ -13,7 +13,7 @@ }, { "cell_type": "code", - "execution_count": 2, + "execution_count": 66, "metadata": { "collapsed": true }, @@ -26,20 +26,2656 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Questions from last class\n", + "![question 1](q1.png)\n", "\n", - "### Questions for me\n", + "![question 2](q2.png)\n", + "\n", + "### Project Ideas so far\n", + "\n", + "- Nothing yet...probably something heat transfer related\n", + "\n", + "- Modeling Propulsion or Propagation of Sound Waves\n", + "\n", + "- Low Thrust Orbital Transfer\n", + "\n", + "- Tracking motion of a satellite entering orbit until impact\n", + "\n", + "- What ever you think is best.\n", + "\n", + "- You had heat transfer project as an option; that sounded cool\n", + "\n", + "- Heat transfer through a pipe\n", + "\n", + "- I would prefer to do something with beam/plate mechanics or vibrations than a heat transfer or thermo problem\n", + "\n", + "- Modeling Couette flow with a pressure gradient using a discretized form of the Navier-Stokes equation and comparing it to the analytical solution\n", + "\n", + "- Software to instruct a robotic arm to orient itself based on input from a gyroscope" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Questions from you:\n", + "\n", + "How was your spring break\n", + "\n", + "Are grades for hw 3 and 4 going to be posted?\n", + "\n", + "For class occasionally switching to doc cam and working through problems by hand might help get ideas across.\n", + "\n", + "Are you assigning those who do not have groups to groups?\n", + "\n", + "How do you graph a standard distribution in Matlab?\n", + "\n", + "What is the longest code you have written?\n", + "\n", + "how to approach probability for hw 5?\n", + "??\n", + "\n", + "Will you be assigning groups to people who do not currently have one? \n", + "\n", + "what are some basic guidelines we should use to brainstorm project ideas?\n", + "\n", + "Are you a fan of bananas?\n", + "\n", + "Going through code isn't the most helpful, because you can easily lose interest. But I am not sure what else you can do.\n", + "\n", + "Has lecture 15 been posted yet? Looking in the repository I can't seem to find it." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Linear Least Squares Regression\n", + "\n", + "When approximating a set of data as a polynomial, there will always be error introduced (except in 2 cases). \n", + "\n", + "For a straight line, the actual data points, $y_{i}$ as a function of the independent variable, $x_{i}$, is:\n", + "\n", + "$y_{i}=a_{0}+a_{1}x_{i}+e_{i}$\n", + "\n", + "where $a_{0}$ and $a_{1}$ are the intercept and slope of the line and $e_{i}$ is the error between the approximate function and the recorded data point. \n", + "\n", + "We make the following assumptions in this analysis:\n", + "\n", + "1. Each x has a fixed value; it is not random and is known without error.\n", + "\n", + "2. The y values are independent random variables and all have the same variance.\n", + "\n", + "3. The y values for a given x must be normally distributed.\n", + "\n", + "The total error is \n", + "\n", + "$\\sum_{i=1}^{n}e_{i}=\\sum_{i=1}^{n}(y_{i}-a_{0}-a_{1}x_{i})$\n", + "\n", + "we don't care about the sign though. One approach has been demonstrated to provide a unique solution is minimizing the sum of squares error or\n", + "\n", + "$S_{r}=\\sum_{i=1}^{n}e_{i}^{2}=\\sum_{i=1}^{n}(y_{i}-a_{0}-a_{1}x_{i})^{2}$\n", + "\n", + "Where, $S_{r}$ is the sum of squares error (SSE). \n", + "\n", + "$\\frac{\\partial S_{r}}{\\partial a_{0}}=-2\\sum(y_{i}-a_{0}-a_{1}x_{i})$\n", + "\n", + "$\\frac{\\partial S_{r}}{\\partial a_{1}}=-2\\sum(y_{i}-a_{0}-a_{1}x_{i})x_{i}$\n", + "\n", + "The minimum $S_{r}$ occurrs when the partial derivatives are 0. \n", + "\n", + "$\\sum y_{i}= \\sum a_{0}+\\sum a_{1}x_{i}$\n", + "\n", + "$\\sum x_{i}y_{i}= \\sum a_{0}x_{i}+\\sum a_{1}x_{i}^{2}$\n", + "\n", + "$\\left[\\begin{array}{c}\n", + "\\sum y_{i}\\\\\n", + "\\sum x_{i}y_{i}\\end{array}\\right]=\n", + "\\left[\\begin{array}{cc}\n", + "n & \\sum x_{i}\\\\\n", + "\\sum x_{i} & \\sum x_{i}^{2}\\end{array}\\right]\n", + "\\left[\\begin{array}{c}\n", + "a_{0}\\\\\n", + "a_{1}\\end{array}\\right]$\n", + "\n", + "\n", + "$b=Ax$\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Example \n", + "\n", + "Find drag coefficient with best-fit line to experimental data\n", + "\n", + "|i | v (m/s) | F (N) |\n", + "|---|---|---|\n", + "|1 | 10 | 25 |\n", + "|2 | 20 | 70 |\n", + "|3 | 30 | 380|\n", + "|4 | 40 | 550|\n", + "|5 | 50 | 610|\n", + "|6 | 60 | 1220|\n", + "|7 | 70 | 830 |\n", + "|8 |80 | 1450|" + ] + }, + { + "cell_type": "code", + "execution_count": 68, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "a =\n", + "\n", + " -234.286\n", + " 19.470\n", + "\n" + ] + }, + { + "data": { + "image/svg+xml": [ + "\n", + "\n", + "Gnuplot\n", + "Produced by GNUPLOT 5.0 patchlevel 3 \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\t\n", + "\t\t-500\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t500\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t1000\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t1500\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t2000\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t10\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t20\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t30\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t40\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t50\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t60\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t70\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t80\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\n", + "\t\n", + "\t\tvelocity (m/s)\n", + "\t\n", + "\n", + "\n", + "\t\n", + "\t\tForce (N)\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\tdata\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\tdata\n", + "\t\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\n", + "\t\n", + "\tbest-fit\n", + "\n", + "\t\n", + "\t\tbest-fit\n", + "\t\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "drag_data=[...\n", + "1 , 10 , 25 \n", + "2 , 20 , 70 \n", + "3 , 30 , 380\n", + "4 , 40 , 550\n", + "5 , 50 , 610\n", + "6 , 60 , 1220\n", + "7 , 70 , 830 \n", + "8 ,80 , 1450];\n", + "x=drag_data(:,2);\n", + "y=drag_data(:,3);\n", + "\n", + "b=[sum(y);sum(x.*y)];\n", + "A=[length(x),sum(x);\n", + " sum(x), sum(x.^2)];\n", + " \n", + "a=A\\b\n", + "\n", + "plot(x,y,'o',x,a(1)+a(2)*x)\n", + "legend('data','best-fit','Location','NorthWest')\n", + "xlabel('Force (N)')\n", + "ylabel('velocity (m/s)')" + ] + }, + { + "cell_type": "code", + "execution_count": 72, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/svg+xml": [ + "\n", + "\n", + "Gnuplot\n", + "Produced by GNUPLOT 5.0 patchlevel 3 \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\t\n", + "\t\t-2000\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t-1500\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t-1000\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t-500\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t10\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t20\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t30\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t40\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t50\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t60\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t70\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t80\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\n", + "\t\n", + "\t\tresiduals (N)\n", + "\t\n", + "\n", + "\n", + "\t\n", + "\t\tvelocity (m/s)\n", + "\t\n", + "\n", + "\n", + "\t\n", + "\t\tModel does not capture measurements\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\tdata\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\tdata\n", + "\t\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "plot(x,y-a(1)-40*x)\n", + "legend('data','best-fit','Location','NorthWest')\n", + "ylabel('residuals (N)')\n", + "xlabel('velocity (m/s)')\n", + "title('Model does not capture measurements')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "How do we know its a \"good\" fit? \n", + "\n", + "Can compare the sum of squares error to the total sum of squares of the dependent variable (here F). \n", + "\n", + "$S_{r}=\\sum(y_{i}-a_{0}-a_{1}x_{i})^{2}$\n", + "\n", + "$S_{t}=\\sum(y_{i}-\\bar{y})^{2}$\n", + "\n", + "Then, we can calculate the *coefficient of determination*, $r^{2}$ or *correlation coefficient*, r. \n", + "\n", + "$r^{2}=\\frac{S_{t}-S_{r}}{S_{t}}$\n", + "\n", + "This represents the relative improvement of assuming that y is a function of x (if the x-values are not random and the y-values are random)\n", + "\n", + "For further information regarding statistical work on regression, look at \n", + "[NIST Statistics Handbook](http://www.itl.nist.gov/div898/handbook/pmd/section4/pmd44.htm)" + ] + }, + { + "cell_type": "code", + "execution_count": 73, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "St = 1.8083e+06\n", + "St = 1.8083e+06\n" + ] + } + ], + "source": [ + "Sr=sum((y-a(1)-a(2)*x).^2);\n", + "St=std(y)^2*(length(y)-1)\n", + "St=sum((y-mean(y)).^2)" + ] + }, + { + "cell_type": "code", + "execution_count": 74, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "r2 = 0.88049\n", + "r = 0.93834\n" + ] + } + ], + "source": [ + "r2=(St-Sr)/St\n", + "r=sqrt(r2)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Limiting cases \n", + "\n", + "#### $r^{2}=0$ $S_{r}=S{t}$\n", + "\n", + "#### $r^{2}=1$ $S_{r}=0$" + ] + }, + { + "cell_type": "code", + "execution_count": 76, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/svg+xml": [ + "\n", + "\n", + "Gnuplot\n", + "Produced by GNUPLOT 5.0 patchlevel 3 \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0.2\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0.4\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0.6\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0.8\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t1\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0.2\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0.4\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0.6\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0.8\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t1\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\tCase 1\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\tCase 1\n", + "\t\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\n", + "\t\n", + "\tCase 2\n", + "\n", + "\t\n", + "\t\tCase 2\n", + "\t\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "gen_examples\n", + "plot(x1(1:50:end),y1(1:50:end),'s',x2,y2,'o')\n", + "legend('Case 1','Case 2','Location','NorthWest')" + ] + }, + { + "cell_type": "code", + "execution_count": 77, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "a1 =\n", + "\n", + " 0.0482496\n", + " 0.0017062\n", + "\n", + "a2 =\n", + "\n", + " 0\n", + " 1\n", + "\n", + "Sr1 = 0.83505\n", + "St1 = 0.83529\n", + "coefficient of determination in Case 1 is 0.000291\n", + "Sr2 = 0\n", + "St2 = 1.0185\n", + "coefficient of determination in Case 2 is 1.000000\n" + ] + } + ], + "source": [ + "b1=[sum(y1);sum(x1.*y1)];\n", + "A1=[length(x1),sum(x1);\n", + " sum(x1), sum(x1.^2)];\n", + " \n", + "a1=A1\\b1\n", + "\n", + "b2=[sum(y2);sum(x2.*y2)];\n", + "A2=[length(x2),sum(x2);\n", + " sum(x2), sum(x2.^2)];\n", + " \n", + "a2=A2\\b2\n", + "\n", + "Sr1=sum((y1-a1(1)-a1(2)*x1).^2)\n", + "St1=sum((y1-mean(y1)).^2)\n", + "fprintf('coefficient of determination in Case 1 is %f\\n',1-Sr1/St1)\n", + "\n", + "Sr2=sum((y2-a2(1)-a2(2)*x2).^2)\n", + "St2=sum((y2-mean(y2)).^2)\n", + "\n", + "fprintf('coefficient of determination in Case 2 is %f\\n',1-Sr2/St2)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "collapsed": true + }, + "source": [ + "# General Linear Regression" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In general, you may want to fit other polynomials besides degree-1 (straight-lines)\n", + "\n", + "$y=a_{0}+a_{1}x+a_{2}x^{2}+\\cdots+a_{m}x^{m}+e$\n", + "\n", + "Now, the solution for $a_{0},~a_{1},...a_{m}$ is the minimization of m+1-dependent linear equations. \n", + "\n", + "Consider the following data:\n", + "\n", + "| x | y |\n", + "|---|---|\n", + "| 0.00 | 21.50 |\n", + "| 2.00 | 20.84 |\n", + "| 4.00 | 23.19 |\n", + "| 6.00 | 22.69 |\n", + "| 8.00 | 30.27 |\n", + "| 10.00 | 40.11 |\n", + "| 12.00 | 43.31 |\n", + "| 14.00 | 54.79 |\n", + "| 16.00 | 70.88 |\n", + "| 18.00 | 89.48 |\n", "\n" ] }, + { + "cell_type": "code", + "execution_count": 78, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/svg+xml": [ + "\n", + "\n", + "Gnuplot\n", + "Produced by GNUPLOT 5.0 patchlevel 3 \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\t\n", + "\t\t20\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t30\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t40\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t50\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t60\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t70\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t80\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t90\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t100\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t5\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t10\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t15\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t20\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\n", + "\n", + "\tgnuplot_plot_1a\n", + "\n", + "\n", + "\n", + "\t \n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "load xy_data.csv\n", + "x=xy_data(1,:)';\n", + "y=xy_data(2,:)';\n", + "plot(x,y,'o')" + ] + }, + { + "cell_type": "code", + "execution_count": 46, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "xy_data =\n", + "\n", + " Columns 1 through 7:\n", + "\n", + " 0.00000 2.00000 4.00000 6.00000 8.00000 10.00000 12.00000\n", + " 21.50114 20.84153 23.19201 22.69498 30.26687 40.11075 43.30543\n", + "\n", + " Columns 8 through 11:\n", + "\n", + " 14.00000 16.00000 18.00000 20.00000\n", + " 54.78730 70.88443 89.48368 97.28135\n", + "\n" + ] + } + ], + "source": [ + "xy_data" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The model can be rewritten as \n", + "\n", + "$y=\\left[Z\\right]a+e$\n", + "\n", + "where $a=\\left[\\begin{array}{c}\n", + " a_{0}\\\\\n", + " a_{1}\\\\\n", + " a_{2}\\end{array}\\right]$\n", + " \n", + "$[Z]=\\left[\\begin{array} \n", + "1 & x_{1} & x_{1}^{2} \\\\\n", + "1 & x_{2} & x_{2}^{2} \\\\\n", + "1 & x_{3} & x_{3}^{2} \\\\\n", + "1 & x_{4} & x_{4}^{2} \\\\\n", + "1 & x_{5} & x_{5}^{2} \\\\\n", + "1 & x_{6} & x_{6}^{2} \\\\\n", + "1 & x_{7} & x_{7}^{2} \\\\\n", + "1 & x_{8} & x_{8}^{2} \\\\\n", + "1 & x_{9} & x_{9}^{2} \\\\\n", + "1 & x_{10} & x_{10}^{2} \\end{array}\\right]$\n", + "\n", + "The sum of squares residuals for this model is\n", + "\n", + "$S_{r}=\\sum_{i=1}^{n}\\left(y_{i}-\\sum_{j=0}^{m}a_{j}z_{ji}\\right)$\n", + "\n", + "Minimizing this function results in\n", + "\n", + "$y=[Z]a$\n", + "\n", + "->**A standard Linear Algebra Problem**\n", + "\n", + "*the vector a is unknown, and Z is calculated based upon the assumed function*\n" + ] + }, + { + "cell_type": "code", + "execution_count": 79, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Z =\n", + "\n", + " 1 0 0\n", + " 1 2 4\n", + " 1 4 16\n", + " 1 6 36\n", + " 1 8 64\n", + " 1 10 100\n", + " 1 12 144\n", + " 1 14 196\n", + " 1 16 256\n", + " 1 18 324\n", + " 1 20 400\n", + "\n", + "a =\n", + "\n", + " 21.40341\n", + " -0.81538\n", + " 0.23935\n", + "\n" + ] + } + ], + "source": [ + "Z=[x.^0,x,x.^2]\n", + "\n", + "a=Z\\y" + ] + }, + { + "cell_type": "code", + "execution_count": 80, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/svg+xml": [ + "\n", + "\n", + "Gnuplot\n", + "Produced by GNUPLOT 5.0 patchlevel 3 \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\t\n", + "\t\t20\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t40\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t60\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t80\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t100\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t120\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t5\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t10\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t15\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t20\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\n", + "\n", + "\tgnuplot_plot_1a\n", + "\n", + "\n", + "\n", + "\t \n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\n", + "\t\n", + "\tgnuplot_plot_2a\n", + "\n", + "\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "x_fcn=linspace(min(x),max(x));\n", + "plot(x,y,'o',x_fcn,a(1)+a(2)*x_fcn+a(3)*x_fcn.^2)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### General Coefficient of Determination\n", + "\n", + "$r^{2}=\\frac{S_{t}-S_{r}}{S_{t}}=1-\\frac{S_{r}}{S_t}$" + ] + }, + { + "cell_type": "code", + "execution_count": 49, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "St = 27.923\n", + "Sr = 2.6366\n" + ] + } + ], + "source": [ + "St=std(y)\n", + "Sr=std(y-a(1)-a(2)*x-a(3)*x.^2)" + ] + }, + { + "cell_type": "code", + "execution_count": 81, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "the coefficient of determination for this fit is 0.880485\n", + "the correlation coefficient this fit is 0.938342\n" + ] + } + ], + "source": [ + "r2=1-Sr/St;\n", + "r=sqrt(r2);\n", + "\n", + "fprintf('the coefficient of determination for this fit is %f',r2)\n", + "fprintf('the correlation coefficient this fit is %f',r)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Compare this to a straight line fit" + ] + }, + { + "cell_type": "code", + "execution_count": 83, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "St = 27.923\n", + "Sr = 9.2520\n", + "the coefficient of determination for this fit is 0.668655\n", + "the correlation coefficient this fit is 0.817713\n" + ] + }, + { + "data": { + "image/svg+xml": [ + "\n", + "\n", + "Gnuplot\n", + "Produced by GNUPLOT 5.0 patchlevel 3 \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t20\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t40\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t60\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t80\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t100\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t120\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t5\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t10\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t15\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t20\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\n", + "\n", + "\tgnuplot_plot_1a\n", + "\n", + "\n", + "\n", + "\t \n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\n", + "\t\n", + "\tgnuplot_plot_2a\n", + "\n", + "\t\n", + "\t\n", + "\tgnuplot_plot_3a\n", + "\n", + "\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "Z=[ones(size(x)) x];\n", + "a_line=Z\\y;\n", + "x_fcn=linspace(min(x),max(x));\n", + "plot(x,y,'o',x_fcn,a(1)+a(2)*x_fcn+a(3)*x_fcn.^2,...\n", + "x_fcn,a_line(1)+a_line(2)*x_fcn)\n", + "St=std(y)\n", + "Sr=std(y-a_line(1)-a_line(2)*x)\n", + "r2=1-Sr/St;\n", + "r=sqrt(r2);\n", + "\n", + "fprintf('the coefficient of determination for this fit is %f',r2)\n", + "fprintf('the correlation coefficient this fit is %f',r)" + ] + }, + { + "cell_type": "code", + "execution_count": 85, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/svg+xml": [ + "\n", + "\n", + "Gnuplot\n", + "Produced by GNUPLOT 5.0 patchlevel 3 \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\t\n", + "\t\t-15\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t-10\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t-5\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t5\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t10\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t15\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t5\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t10\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t15\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t20\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\n", + "\t\n", + "\t\tresiduals of straight line\n", + "\t\n", + "\n", + "\n", + "\n", + "\tgnuplot_plot_1a\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "plot(x,y-a_line(1)-a_line(2)*x)\n", + "title('residuals of straight line')" + ] + }, + { + "cell_type": "code", + "execution_count": 86, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/svg+xml": [ + "\n", + "\n", + "Gnuplot\n", + "Produced by GNUPLOT 5.0 patchlevel 3 \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\t\n", + "\t\t-4\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t-2\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t2\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t4\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t6\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t5\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t10\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t15\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t20\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\n", + "\t\n", + "\t\tresiduals of parabola\n", + "\t\n", + "\n", + "\n", + "\n", + "\tgnuplot_plot_1a\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "plot(x,y-a(1)-a(2)*x-a(3)*x.^2)\n", + "title('residuals of parabola')" + ] + }, { "cell_type": "markdown", "metadata": {}, "source": [ - "# General Regression (Linear and nonlinear)\n", + "## Warning \n", + "**Coefficient of determination reduction does not always mean a better fit**\n", + "\n", + "Try the function, \n", + "\n", + "$y(x)=a_0+a_{1}x+a_{2}x^{2}+a_{4}x^{4}+a_{5}x^{5}+a_{5}x^{5}+a_{6}x^{6}+a_{7}x^{7}+a_{8}x^{8}$" + ] + }, + { + "cell_type": "code", + "execution_count": 90, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "a_overfit =\n", + "\n", + " 2.1487e+01\n", + " -1.4264e+01\n", + " 1.5240e+01\n", + " -6.0483e+00\n", + " 1.1887e+00\n", + " -1.2651e-01\n", + " 7.4379e-03\n", + " -2.2702e-04\n", + " 2.8063e-06\n", + "\n" + ] + } + ], + "source": [ + "Z=[ones(size(x)) x x.^2 x.^3 x.^4 x.^5 x.^6 x.^7 x.^8];\n", + "a_overfit=Z\\y" + ] + }, + { + "cell_type": "code", + "execution_count": 91, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/svg+xml": [ + "\n", + "\n", + "Gnuplot\n", + "Produced by GNUPLOT 5.0 patchlevel 3 \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t20\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t40\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t60\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t80\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t100\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t120\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t5\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t10\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t15\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t20\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\tdata\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\tdata\n", + "\t\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\n", + "\t\n", + "\tparabola\n", + "\n", + "\t\n", + "\t\tparabola\n", + "\t\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\tn=8-fit\n", + "\n", + "\t\n", + "\t\tn=8-fit\n", + "\t\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "plot(x,y,'o',x_fcn,a(1)+a(2)*x_fcn+a(3)*x_fcn.^2,...\n", + "x_fcn,a_overfit(1)+a_overfit(2)*x_fcn+a_overfit(3)*x_fcn.^2+...\n", + "a_overfit(4)*x_fcn.^3+a_overfit(5)*x_fcn.^4+...\n", + "a_overfit(6)*x_fcn.^5+a_overfit(7)*x_fcn.^6+...\n", + "a_overfit(8)*x_fcn.^7+a_overfit(9)*x_fcn.^8)\n", + "legend('data','parabola','n=8-fit','Location','NorthWest')" + ] + }, + { + "cell_type": "code", + "execution_count": 92, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "St = 27.923\n", + "Sr = 0.77320\n", + "the coefficient of determination for this fit is 0.972309\n", + "the correlation coefficient this fit is 0.986057\n" + ] + } + ], + "source": [ + "St=std(y)\n", + "Sr=std(y-polyval(a_overfit(end:-1:1),x))\n", + "r2=1-Sr/St;\n", + "r=sqrt(r2);\n", + "\n", + "fprintf('the coefficient of determination for this fit is %f',r2)\n", + "fprintf('the correlation coefficient this fit is %f',r)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Linear Regression is only limited by the ability to separate the parameters from the function to achieve\n", + "\n", + "$y=[Z]a$\n", + "\n", + "$Z$ can be any function of the independent variable(s)\n", + "\n", + "**Example**:\n", + "We assume two functions are added together, sin(t) and sin(3t). What are the amplitudes?\n", "\n" ] }, + { + "cell_type": "code", + "execution_count": 93, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/svg+xml": [ + "\n", + "\n", + "Gnuplot\n", + "Produced by GNUPLOT 5.0 patchlevel 3 \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\t\n", + "\t\t-1.5\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t-1\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t-0.5\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0.5\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t1\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t1.5\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t2\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t4\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t6\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t8\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t10\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t12\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t14\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\n", + "\n", + "\tgnuplot_plot_1a\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "load sin_data.csv\n", + "t=sin_data(1,:)';\n", + "y=sin_data(2,:)';\n", + "plot(t,y)" + ] + }, + { + "cell_type": "code", + "execution_count": 94, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "a =\n", + "\n", + " 0.99727\n", + " 0.50251\n", + "\n" + ] + } + ], + "source": [ + "Z=[sin(t) sin(3*t)];\n", + "a=Z\\y" + ] + }, + { + "cell_type": "code", + "execution_count": 95, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/svg+xml": [ + "\n", + "\n", + "Gnuplot\n", + "Produced by GNUPLOT 5.0 patchlevel 3 \n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\t\n", + "\t \n", + "\t \n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\t\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\t\n", + "\t\t-1.5\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t-1\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t-0.5\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0.5\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t1\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t1.5\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t0\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t2\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t4\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t6\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t8\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t10\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t12\n", + "\t\n", + "\n", + "\n", + "\t\t\n", + "\t\t14\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\t\n", + "\n", + "\n", + "\tgnuplot_plot_1a\n", + "\n", + "\n", + "\n", + "\t \n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\t\n", + "\n", + "\t\n", + "\tgnuplot_plot_2a\n", + "\n", + "\t\n", + "\t\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "plot(t,y,'.',t,a(1)*sin(t)+a(2)*sin(3*t))" + ] + }, { "cell_type": "code", "execution_count": null,