diff --git a/09_Linear-Algebra/.ipynb_checkpoints/09_Linear-Algebra-checkpoint.ipynb b/09_Linear-Algebra/.ipynb_checkpoints/09_Linear-Algebra-checkpoint.ipynb new file mode 100644 index 0000000..6bc2ad3 --- /dev/null +++ b/09_Linear-Algebra/.ipynb_checkpoints/09_Linear-Algebra-checkpoint.ipynb @@ -0,0 +1,1170 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "# Linear Algebra (Review/Introduction)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "fragment" + } + }, + "source": [ + "Representation of linear equations:\n", + "\n", + "1. $5x_{1}+3x_{2} =1$\n", + "\n", + "2. $x_{1}+2x_{2}+3x_{3} =2$\n", + "\n", + "3. $x_{1}+x_{2}+x_{3} =3$\n", + "\n", + "in matrix form:\n", + "\n", + "$\\left[ \\begin{array}{ccc}\n", + "5 & 3 & 0 \\\\\n", + "1 & 2 & 3 \\\\\n", + "1 & 1 & 1 \\end{array} \\right]\n", + "\\left[\\begin{array}{c} \n", + "x_{1} \\\\ \n", + "x_{2} \\\\\n", + "x_{3}\\end{array}\\right]=\\left[\\begin{array}{c} \n", + "1 \\\\\n", + "2 \\\\\n", + "3\\end{array}\\right]$\n", + "\n", + "$Ax=b$" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "### Vectors " + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "column vector x (length of 3):\n", + "\n", + "$\\left[\\begin{array}{c} \n", + "x_{1} \\\\ \n", + "x_{2} \\\\\n", + "x_{3}\\end{array}\\right]$\n", + "\n", + "row vector y (length of 3):\n", + "\n", + "$\\left[\\begin{array}{ccc} \n", + "y_{1}~ \n", + "y_{2}~ \n", + "y_{3}\\end{array}\\right]$\n", + "\n", + "vector of length N:\n", + "\n", + "$\\left[\\begin{array}{c} \n", + "x_{1} \\\\ \n", + "x_{2} \\\\\n", + "\\vdots \\\\\n", + "x_{N}\\end{array}\\right]$\n", + "\n", + "The $i^{th}$ element of x is $x_{i}$" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "### Matrices" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "fragment" + } + }, + "source": [ + "elements in the matrix are denoted $B_{row~column}$\n", + "\n", + "In general, matrix, B, can be any size, $M \\times N$ (M-rows and N-columns):\n", + "\n", + "$B=\\left[ \\begin{array}{cccc}\n", + "B_{11} & B_{12} &...& B_{1N} \\\\\n", + "B_{21} & B_{22} &...& B_{2N} \\\\\n", + "\\vdots & \\vdots &\\ddots& \\vdots \\\\\n", + "B_{M1} & B_{M2} &...& B_{MN}\\end{array} \\right]$" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "## Multiplication" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "fragment" + } + }, + "source": [ + "A column vector is a $1\\times N$ matrix and a row vector is a $M\\times 1$ matrix\n", + "\n", + "**Multiplying matrices is not commutative**\n", + "\n", + "$A B \\neq B A$\n", + "\n", + "Inner dimensions must agree, output is outer dimensions. \n", + "\n", + "A is $M1 \\times N1$ and B is $M2 \\times N2$\n", + "\n", + "C=AB\n", + "\n", + "Therefore N1=M2 and C is $M1 \\times N2$\n", + "\n", + "If $C'=BA$, then N2=M1 and C is $M2 \\times N1$" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "e.g. \n", + "$A=\\left[ \\begin{array}{cc}\n", + "5 & 3 & 0 \\\\\n", + "1 & 2 & 3 \\end{array} \\right]$\n", + "\n", + "$B=\\left[ \\begin{array}{cccc}\n", + "1 & 2 & 3 & 4 \\\\\n", + "5 & 6 & 7 & 8 \\\\\n", + "9 & 10 & 11 & 12 \\end{array} \\right]$\n", + "\n", + "C=AB\n", + "\n", + "$[2\\times 4] = [2 \\times 3][3 \\times 4]$" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "The rule for multiplying matrices, A and B, is\n", + "\n", + "$C_{ij} = \\sum_{k=1}^{n}A_{ik}B_{kj}$\n", + "\n", + "In the previous example, \n", + "\n", + "$C_{11} = A_{11}B_{11}+A_{12}B_{21}+A_{13}B_{31} = 5*1+3*5+0*9=20$\n", + "\n", + "\n", + "Multiplication is associative:\n", + "\n", + "$(AB)C = A(BC)$\n", + "\n", + "and distributive:\n", + "\n", + "$A(B+C)=AB+AC$" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "collapsed": false, + "slideshow": { + "slide_type": "subslide" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "A =\n", + "\n", + " 5 3 0\n", + " 1 2 3\n", + "\n", + "B =\n", + "\n", + " 1 2 3 4\n", + " 5 6 7 8\n", + " 9 10 11 12\n", + "\n" + ] + } + ], + "source": [ + "A=[5,3,0;1,2,3] \n", + "B=[1,2,3,4;5,6,7,8;9,10,11,12]" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "collapsed": false, + "slideshow": { + "slide_type": "fragment" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "C =\n", + "\n", + " 20 28 0 0\n", + " 0 0 0 0\n", + "\n" + ] + } + ], + "source": [ + "C=zeros(2,4);\n", + "C(1,1)=A(1,1)*B(1,1)+A(1,2)*B(2,1)+A(1,3)*B(3,1);\n", + "C(1,2)=A(1,1)*B(1,2)+A(1,2)*B(2,2)+A(1,3)*B(3,2);\n", + "C\n", + "%C=A*B" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "collapsed": false, + "slideshow": { + "slide_type": "subslide" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "error: operator *: nonconformant arguments (op1 is 3x4, op2 is 2x3)\r\n" + ] + } + ], + "source": [ + "Cp=B*A" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "## Representation of a problem with Matrices and Vectors" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "fragment" + } + }, + "source": [ + "You have:\n", + "\n", + "- a set of known outputs, $y_{1},~y_{2},~...y_{N}$ \n", + "\n", + "- set of equations that relate unknown inputs, $x_{1},~x_{2},~...x_{N}$, " + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "## = vector-matrix format:\n", + "\n", + "$y=Ax=\\left[\\begin{array}{c} \n", + "y_{1} \\\\ \n", + "y_{2} \\\\\n", + "\\vdots \\\\\n", + "y_{N}\\end{array}\\right]\n", + "=\n", + "A\\left[\\begin{array}{c} \n", + "x_{1} \\\\ \n", + "x_{2} \\\\\n", + "\\vdots \\\\\n", + "x_{N}\\end{array}\\right]$\n", + "\n", + "$A=\n", + "\\left[\\begin{array}{cccc} \n", + "| & | & & | \\\\ \n", + "a_{1} & a_{2} & ... & a_{N} \\\\ \n", + "| & | & & | \\end{array}\\right]$\n", + "\n", + "or " + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "$y = a_{1}x_{1} + a_{2}x_{2} +...+a_{N}x_{N}$\n", + "\n", + "- $a_{i}$ is a column vector \n", + "\n", + "- $x_{i}$ is a scalar taken from the $i^{th}$ element of x." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "# Example \n", + "\n", + "Consider 4 masses connected in series to 4 springs with K=10 N/m. What are the final positions of the masses? \n", + "\n", + "![Springs-masses](mass_springs.svg)\n", + "\n", + "The masses haves the following amounts, 1, 2, 3, and 4 kg for masses 1-4. Using a FBD for each mass:\n", + "\n", + "$m_{1}g+k(x_{2}-x_{1})-kx_{1}=0$\n", + "\n", + "$m_{2}g+k(x_{3}-x_{2})-k(x_{2}-x_{1})=0$\n", + "\n", + "$m_{3}g+k(x_{4}-x_{3})-k(x_{3}-x_{2})=0$\n", + "\n", + "$m_{4}g-k(x_{4}-x_{3})=0$" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "in matrix form:\n", + "\n", + "$\\left[ \\begin{array}{cccc}\n", + "2k & -k & 0 & 0 \\\\\n", + "-k & 2k & -k & 0 \\\\\n", + "0 & -k & 2k & -k \\\\\n", + "0 & 0 & -k & k \\end{array} \\right]\n", + "\\left[ \\begin{array}{c}\n", + "x_{1} \\\\\n", + "x_{2} \\\\\n", + "x_{3} \\\\\n", + "x_{4} \\end{array} \\right]=\n", + "\\left[ \\begin{array}{c}\n", + "m_{1}g \\\\\n", + "m_{2}g \\\\\n", + "m_{3}g \\\\\n", + "m_{4}g \\end{array} \\right]$" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "collapsed": false, + "slideshow": { + "slide_type": "subslide" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "K =\n", + "\n", + " 20 -10 0 0\n", + " -10 20 -10 0\n", + " 0 -10 20 -10\n", + " 0 0 -10 10\n", + "\n", + "y =\n", + "\n", + " 9.8100\n", + " 19.6200\n", + " 29.4300\n", + " 39.2400\n", + "\n", + "x =\n", + "\n", + " 9.8100\n", + " 18.6390\n", + " 25.5060\n", + " 29.4300\n", + "\n" + ] + } + ], + "source": [ + "k=10; % N/m\n", + "m1=1; % kg\n", + "m2=2;\n", + "m3=3;\n", + "m4=4;\n", + "g=9.81; % m/s^2\n", + "K=[2*k -k 0 0; -k 2*k -k 0; 0 -k 2*k -k; 0 0 -k k]\n", + "y=[m1*g;m2*g;m3*g;m4*g]\n", + "\n", + "x=K\\y" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Matrix Operations\n", + "\n", + "Identity matrix `eye(M,N)` **Assume M=N unless specfied**\n", + "\n", + "$I_{ij} =1$ if $i=j$ \n", + "\n", + "$I_{ij} =0$ if $i\\neq j$ \n", + "\n", + "AI=A=IA\n", + "\n", + "Diagonal matrix, D, is similar to the identity matrix, but each diagonal element can be any\n", + "number\n", + "\n", + "$D_{ij} =d_{i}$ if $i=j$ \n", + "\n", + "$D_{ij} =0$ if $i\\neq j$ " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Transpose\n", + "\n", + "The transpose of a matrix changes the rows -> columns and columns-> rows\n", + "\n", + "$(A^{T})_{ij} = A_{ji}$\n", + "\n", + "for diagonal matrices, $D^{T}=D$\n", + "\n", + "in general, the transpose has the following qualities:\n", + " \n", + "1. $(A^{T})^{T}=A$\n", + "\n", + "2. $(AB)^{T} = B^{T}A^{T}$\n", + "\n", + "3. $(A+B)^{T} = A^{T} +B^{T}$ \n", + "\n", + "All matrices have a symmetric and antisymmetric part:\n", + "\n", + "$A= 1/2(A+A^{T}) +1/2(A-A^{T})$\n", + "\n", + "If $A=A^{T}$, then A is symmetric, if $A=-A^{T}$, then A is antisymmetric." + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "ans =\n", + "\n", + " 20 38\n", + " 28 44\n", + " 36 50\n", + " 44 56\n", + "\n", + "ans =\n", + "\n", + " 20 38\n", + " 28 44\n", + " 36 50\n", + " 44 56\n", + "\n", + "ans =\n", + "\n", + " 1 1\n", + " 1 1\n", + " 1 1\n", + " 1 1\n", + "\n" + ] + } + ], + "source": [ + "(A*B)'\n", + "\n", + "B'*A' % if this wasnt true, then inner dimensions wouldn't match\n", + "\n", + "B'*A' == (A*B)' % 1=True, 0=False" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "## Square matrix properties\n", + "\n", + "1. Trace\n", + "\n", + "2. Inverse\n", + "\n", + "3. Orthogonal" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "### Trace\n", + "\n", + "The trace of a square matrix is the sum of the diagonal elements\n", + "\n", + "$tr~A=\\sum_{i=1}^{N}A_{ii}$\n", + "\n", + "The trace is invariant, meaning that if you change the basis through a rotation, then the\n", + "trace remains the same. \n", + "\n", + "It also has the following properties:\n", + "\n", + "1. $tr~A=tr~A^{T}$\n", + "\n", + "2. $tr~A+tr~B=tr(A+B)$\n", + "\n", + "3. $tr(kA)=k tr~A$\n", + "\n", + "4. $tr(AB)=tr(BA)$" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "collapsed": false, + "slideshow": { + "slide_type": "subslide" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "id_m =\n", + "\n", + "Diagonal Matrix\n", + "\n", + " 1 0 0\n", + " 0 1 0\n", + " 0 0 1\n", + "\n", + "ans = 3\n" + ] + } + ], + "source": [ + "id_m=eye(3)\n", + "trace(id_m)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "### Inverse\n", + "\n", + "The inverse of a square matrix, $A^{-1}$ is defined such that\n", + "\n", + "$A^{-1}A=I=AA^{-1}$\n", + "\n", + "Not all square matrices have an inverse, they can be *singular* or *non-invertible*\n", + "\n", + "The inverse has the following properties:\n", + "\n", + "1. $(A^{-1})^{-1}=A$\n", + "\n", + "2. $(AB)^{-1}=B^{-1}A^{-1}$\n", + "\n", + "3. $(A^{-1})^{T}=(A^{T})^{-1}$" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": { + "collapsed": false, + "slideshow": { + "slide_type": "slide" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "A =\n", + "\n", + " 0.5762106 0.3533174 0.7172134\n", + " 0.7061664 0.4863733 0.9423436\n", + " 0.4255961 0.0016613 0.3561407\n", + "\n" + ] + } + ], + "source": [ + "A=rand(3,3)" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": { + "collapsed": false, + "slideshow": { + "slide_type": "fragment" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Ainv =\n", + "\n", + " 41.5613 -30.1783 -3.8467\n", + " 36.2130 -24.2201 -8.8415\n", + " -49.8356 36.1767 7.4460\n", + "\n" + ] + } + ], + "source": [ + "Ainv=inv(A)" + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "metadata": { + "collapsed": false, + "slideshow": { + "slide_type": "subslide" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "B =\n", + "\n", + " 0.524529 0.470856 0.708116\n", + " 0.084491 0.730986 0.528292\n", + " 0.303545 0.782522 0.389282\n", + "\n" + ] + } + ], + "source": [ + "B=rand(3,3)" + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "metadata": { + "collapsed": false, + "slideshow": { + "slide_type": "subslide" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "ans =\n", + "\n", + " -182.185 125.738 40.598\n", + " -133.512 97.116 17.079\n", + " 282.422 -200.333 -46.861\n", + "\n", + "ans =\n", + "\n", + " -182.185 125.738 40.598\n", + " -133.512 97.116 17.079\n", + " 282.422 -200.333 -46.861\n", + "\n", + "ans =\n", + "\n", + " 41.5613 36.2130 -49.8356\n", + " -30.1783 -24.2201 36.1767\n", + " -3.8467 -8.8415 7.4460\n", + "\n", + "ans =\n", + "\n", + " 41.5613 36.2130 -49.8356\n", + " -30.1783 -24.2201 36.1767\n", + " -3.8467 -8.8415 7.4460\n", + "\n" + ] + } + ], + "source": [ + "inv(A*B)\n", + "inv(B)*inv(A)\n", + "\n", + "inv(A')\n", + "\n", + "inv(A)'" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "### Orthogonal Matrices\n", + "\n", + "Vectors are *orthogonal* if $x^{T}$ y=0, and a vector is *normalized* if $||x||_{2}=1$. A\n", + "square matrix is *orthogonal* if all its column vectors are orthogonal to each other and\n", + "normalized. The column vectors are then called *orthonormal* and the following results\n", + "\n", + "$U^{T}U=I=UU^{T}$\n", + "\n", + "and \n", + "\n", + "$||Ux||_{2}=||x||_{2}$" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "### Determinant\n", + "\n", + "The **determinant** of a matrix has 3 properties\n", + "\n", + "1\\. The determinant of the identity matrix is one, $|I|=1$\n", + "\n", + "2\\. If you multiply a single row by scalar $t$ then the determinant is $t|A|$:\n", + "\n", + "$t|A|=\\left[ \\begin{array}{cccc}\n", + "tA_{11} & tA_{12} &...& tA_{1N} \\\\\n", + "A_{21} & A_{22} &...& A_{2N} \\\\\n", + "\\vdots & \\vdots &\\ddots& \\vdots \\\\\n", + "A_{M1} & A_{M2} &...& A_{MN}\\end{array} \\right]$" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "## Determinant (con'd)\n", + "3\\. If you switch 2 rows, the determinant changes sign:\n", + "$-|A|=\\left[ \\begin{array}{cccc}\n", + "A_{21} & A_{22} &...& A_{2N} \\\\\n", + "A_{11} & A_{12} &...& A_{1N} \\\\\n", + "\\vdots & \\vdots &\\ddots& \\vdots \\\\\n", + "A_{M1} & A_{M2} &...& A_{MN}\\end{array} \\right]$" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "## Determinant (con'd)\n", + "4\\. inverse of the determinant is the determinant of the inverse:\n", + "\n", + "$|A^{-1}|=\\frac{1}{|A|}=|A|^{-1}$" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "## Calculating the Determinant\n", + "For a $2\\times2$ matrix, \n", + "\n", + "$|A|=\\left|\\left[ \\begin{array}{cc}\n", + "A_{11} & A_{12} \\\\\n", + "A_{21} & A_{22} \\\\\n", + "\\end{array} \\right]\\right| = A_{11}A_{22}-A_{21}A_{12}$\n", + "\n", + "For a $3\\times3$ matrix,\n", + "\n", + "$|A|=\\left|\\left[ \\begin{array}{ccc}\n", + "A_{11} & A_{12} & A_{13} \\\\\n", + "A_{21} & A_{22} & A_{23} \\\\\n", + "A_{31} & A_{32} & A_{33}\\end{array} \\right]\\right|=$\n", + "\n", + "$A_{11}A_{22}A_{33}+A_{12}A_{23}A_{31}+A_{13}A_{21}A_{32}\n", + "-A_{31}A_{22}A_{13}-A_{32}A_{23}A_{11}-A_{33}A_{21}A_{12}$" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "fragment" + } + }, + "source": [ + "For larger matrices, the determinant is more involved" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "### Special Case for determinants\n", + "\n", + "The determinant of a diagonal matrix $|D|=D_{11}D_{22}D_{33}...D_{NN}$. \n", + "\n", + "Similarly, if a matrix is upper triangular (so all values of $A_{ij}=0$ when $j columns and columns-> rows\n", + "\n", + "$(A^{T})_{ij} = A_{ji}$\n", + "\n", + "for diagonal matrices, $D^{T}=D$\n", + "\n", + "in general, the transpose has the following qualities:\n", + " \n", + "1. $(A^{T})^{T}=A$\n", + "\n", + "2. $(AB)^{T} = B^{T}A^{T}$\n", + "\n", + "3. $(A+B)^{T} = A^{T} +B^{T}$ \n", + "\n", + "All matrices have a symmetric and antisymmetric part:\n", + "\n", + "$A= 1/2(A+A^{T}) +1/2(A-A^{T})$\n", + "\n", + "If $A=A^{T}$, then A is symmetric, if $A=-A^{T}$, then A is antisymmetric." + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "ans =\n", + "\n", + " 20 38\n", + " 28 44\n", + " 36 50\n", + " 44 56\n", + "\n", + "ans =\n", + "\n", + " 20 38\n", + " 28 44\n", + " 36 50\n", + " 44 56\n", + "\n", + "ans =\n", + "\n", + " 1 1\n", + " 1 1\n", + " 1 1\n", + " 1 1\n", + "\n" + ] + } + ], + "source": [ + "(A*B)'\n", + "\n", + "B'*A' % if this wasnt true, then inner dimensions wouldn't match\n", + "\n", + "B'*A' == (A*B)' % 1=True, 0=False" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "## Square matrix properties\n", + "\n", + "1. Trace\n", + "\n", + "2. Inverse\n", + "\n", + "3. Orthogonal" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "### Trace\n", + "\n", + "The trace of a square matrix is the sum of the diagonal elements\n", + "\n", + "$tr~A=\\sum_{i=1}^{N}A_{ii}$\n", + "\n", + "The trace is invariant, meaning that if you change the basis through a rotation, then the\n", + "trace remains the same. \n", + "\n", + "It also has the following properties:\n", + "\n", + "1. $tr~A=tr~A^{T}$\n", + "\n", + "2. $tr~A+tr~B=tr(A+B)$\n", + "\n", + "3. $tr(kA)=k tr~A$\n", + "\n", + "4. $tr(AB)=tr(BA)$" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "collapsed": false, + "slideshow": { + "slide_type": "subslide" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "id_m =\n", + "\n", + "Diagonal Matrix\n", + "\n", + " 1 0 0\n", + " 0 1 0\n", + " 0 0 1\n", + "\n", + "ans = 3\n" + ] + } + ], + "source": [ + "id_m=eye(3)\n", + "trace(id_m)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "### Inverse\n", + "\n", + "The inverse of a square matrix, $A^{-1}$ is defined such that\n", + "\n", + "$A^{-1}A=I=AA^{-1}$\n", + "\n", + "Not all square matrices have an inverse, they can be *singular* or *non-invertible*\n", + "\n", + "The inverse has the following properties:\n", + "\n", + "1. $(A^{-1})^{-1}=A$\n", + "\n", + "2. $(AB)^{-1}=B^{-1}A^{-1}$\n", + "\n", + "3. $(A^{-1})^{T}=(A^{T})^{-1}$" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": { + "collapsed": false, + "slideshow": { + "slide_type": "slide" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "A =\n", + "\n", + " 0.5762106 0.3533174 0.7172134\n", + " 0.7061664 0.4863733 0.9423436\n", + " 0.4255961 0.0016613 0.3561407\n", + "\n" + ] + } + ], + "source": [ + "A=rand(3,3)" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": { + "collapsed": false, + "slideshow": { + "slide_type": "fragment" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Ainv =\n", + "\n", + " 41.5613 -30.1783 -3.8467\n", + " 36.2130 -24.2201 -8.8415\n", + " -49.8356 36.1767 7.4460\n", + "\n" + ] + } + ], + "source": [ + "Ainv=inv(A)" + ] + }, + { + "cell_type": "code", + "execution_count": 26, + "metadata": { + "collapsed": false, + "slideshow": { + "slide_type": "subslide" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "B =\n", + "\n", + " 0.524529 0.470856 0.708116\n", + " 0.084491 0.730986 0.528292\n", + " 0.303545 0.782522 0.389282\n", + "\n" + ] + } + ], + "source": [ + "B=rand(3,3)" + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "metadata": { + "collapsed": false, + "slideshow": { + "slide_type": "subslide" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "ans =\n", + "\n", + " -182.185 125.738 40.598\n", + " -133.512 97.116 17.079\n", + " 282.422 -200.333 -46.861\n", + "\n", + "ans =\n", + "\n", + " -182.185 125.738 40.598\n", + " -133.512 97.116 17.079\n", + " 282.422 -200.333 -46.861\n", + "\n", + "ans =\n", + "\n", + " 41.5613 36.2130 -49.8356\n", + " -30.1783 -24.2201 36.1767\n", + " -3.8467 -8.8415 7.4460\n", + "\n", + "ans =\n", + "\n", + " 41.5613 36.2130 -49.8356\n", + " -30.1783 -24.2201 36.1767\n", + " -3.8467 -8.8415 7.4460\n", + "\n" + ] + } + ], + "source": [ + "inv(A*B)\n", + "inv(B)*inv(A)\n", + "\n", + "inv(A')\n", + "\n", + "inv(A)'" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "### Orthogonal Matrices\n", + "\n", + "Vectors are *orthogonal* if $x^{T}$ y=0, and a vector is *normalized* if $||x||_{2}=1$. A\n", + "square matrix is *orthogonal* if all its column vectors are orthogonal to each other and\n", + "normalized. The column vectors are then called *orthonormal* and the following results\n", + "\n", + "$U^{T}U=I=UU^{T}$\n", + "\n", + "and \n", + "\n", + "$||Ux||_{2}=||x||_{2}$" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "### Determinant\n", + "\n", + "The **determinant** of a matrix has 3 properties\n", + "\n", + "1. The determinant of the identity matrix is one, $|I|=1$\n", + "\n", + "2. If you multiply a single row by scalar $t$ then the determinant is $t|A|$:\n", + "\n", + "$t|A|=\\left[ \\begin{array}{cccc}\n", + "tA_{11} & tA_{12} &...& tA_{1N} \\\\\n", + "A_{21} & A_{22} &...& A_{2N} \\\\\n", + "\\vdots & \\vdots &\\ddots& \\vdots \\\\\n", + "A_{M1} & A_{M2} &...& A_{MN}\\end{array} \\right]$\n", + "\n", + "3. If you switch 2 rows, the determinant changes sign:\n", + "\n", + "\n", + "$-|A|=\\left[ \\begin{array}{cccc}\n", + "A_{21} & A_{22} &...& A_{2N} \\\\\n", + "A_{11} & A_{12} &...& A_{1N} \\\\\n", + "\\vdots & \\vdots &\\ddots& \\vdots \\\\\n", + "A_{M1} & A_{M2} &...& A_{MN}\\end{array} \\right]$\n", + "\n", + "4. inverse of the determinant is the determinant of the inverse:\n", + "\n", + "$|A^{-1}|=\\frac{1}{|A|}=|A|^{-1}$\n", + "\n", + "For a $2\\times2$ matrix, \n", + "\n", + "$|A|=\\left|\\left[ \\begin{array}{cc}\n", + "A_{11} & A_{12} \\\\\n", + "A_{21} & A_{22} \\\\\n", + "\\end{array} \\right]\\right| = A_{11}A_{22}-A_{21}A_{12}$\n", + "\n", + "For a $3\\times3$ matrix,\n", + "\n", + "$|A|=\\left|\\left[ \\begin{array}{ccc}\n", + "A_{11} & A_{12} & A_{13} \\\\\n", + "A_{21} & A_{22} & A_{23} \\\\\n", + "A_{31} & A_{32} & A_{33}\\end{array} \\right]\\right|=$\n", + "\n", + "$A_{11}A_{22}A_{33}+A_{12}A_{23}A_{31}+A_{13}A_{21}A_{32}\n", + "-A_{31}A_{22}A_{13}-A_{32}A_{23}A_{11}-A_{33}A_{21}A_{12}$\n", + "\n", + "For larger matrices, the determinant is \n", + "\n", + "$|A|=\n", + "\\frac{1}{n!}\n", + "\\sum_{i_{r}}^{N}\n", + "\\sum_{j_{r}}^{N}\n", + "\\epsilon_{i_{1}...i_{N}}\n", + "\\epsilon_{j_{1}...j_{N}}\n", + "A_{i_{1}j_{1}}\n", + "A_{i_{2}j_{2}}\n", + "...\n", + "A_{i_{N}j_{N}}$\n", + "\n", + "Where the Levi-Cevita symbol is $\\epsilon_{i_{1}i_{2}...i_{N}} = 1$ if it is an even\n", + "permutation and $\\epsilon_{i_{1}i_{2}...i_{N}} = -1$ if it is an odd permutation and\n", + "$\\epsilon_{i_{1}i_{2}...i_{N}} = 0$ if $i_{n}=i_{m}$ e.g. $i_{1}=i_{7}$\n", + "\n", + "![Levi-Cevita rule for even and odd permutations for an $N\\times N$\n", + "determinant](even_odd_perm.svg)\n", + "\n", + "So a $4\\times4$ matrix determinant,\n", + "\n", + "$|A|=\\left|\\left[ \\begin{array}{cccc}\n", + "A_{11} & A_{12} & A_{13} & A_{14} \\\\\n", + "A_{21} & A_{22} & A_{23} & A_{24} \\\\\n", + "A_{31} & A_{32} & A_{33} & A_{34} \\\\\n", + "A_{41} & A_{42} & A_{43} & A_{44} \\end{array} \\right]\\right|$\n", + "\n", + "$=\\epsilon_{1234}\\epsilon_{1234}A_{11}A_{22}A_{33}A_{44}+\n", + "\\epsilon_{2341}\\epsilon_{1234}A_{21}A_{32}A_{43}A_{14}+\n", + "\\epsilon_{3412}\\epsilon_{1234}A_{31}A_{42}A_{13}A_{24}+\n", + "\\epsilon_{4123}\\epsilon_{1234}A_{41}A_{12}A_{23}A_{34}+...\n", + "\\epsilon_{4321}\\epsilon_{4321}A_{44}A_{33}A_{22}A_{11}$\n", + "\n", + "### Special Case for determinants\n", + "\n", + "The determinant of a diagonal matrix $|D|=D_{11}D_{22}D_{33}...D_{NN}$. \n", + "\n", + "Similarly, if a matrix is upper triangular (so all values of $A_{ij}=0$ when $j columns and columns-> rows\n", + "\n", + "$(A^{T})_{ij} = A_{ji}$\n", + "\n", + "for diagonal matrices, $D^{T}=D$\n", + "\n", + "in general, the transpose has the following qualities:\n", + " \n", + "1. $(A^{T})^{T}=A$\n", + "\n", + "2. $(AB)^{T} = B^{T}A^{T}$\n", + "\n", + "3. $(A+B)^{T} = A^{T} +B^{T}$ \n", + "\n", + "All matrices have a symmetric and antisymmetric part:\n", + "\n", + "$A= 1/2(A+A^{T}) +1/2(A-A^{T})$\n", + "\n", + "If $A=A^{T}$, then A is symmetric, if $A=-A^{T}$, then A is antisymmetric." + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "collapsed": false, + "slideshow": { + "slide_type": "subslide" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "ans =\n", + "\n", + " 20 38\n", + " 28 44\n", + " 36 50\n", + " 44 56\n", + "\n", + "ans =\n", + "\n", + " 20 38\n", + " 28 44\n", + " 36 50\n", + " 44 56\n", + "\n", + "ans =\n", + "\n", + " 1 1\n", + " 1 1\n", + " 1 1\n", + " 1 1\n", + "\n" + ] + } + ], + "source": [ + "(A*B)'\n", + "\n", + "B'*A' % if this wasnt true, then inner dimensions wouldn't match\n", + "\n", + "B'*A' == (A*B)' % 1=True, 0=False" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "## Square matrix properties\n", + "\n", + "1. Trace\n", + "\n", + "2. Inverse\n", + "\n", + "3. Orthogonal" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "### Trace\n", + "\n", + "The trace of a square matrix is the sum of the diagonal elements\n", + "\n", + "$tr~A=\\sum_{i=1}^{N}A_{ii}$\n", + "\n", + "The trace is invariant, meaning that if you change the basis through a rotation, then the\n", + "trace remains the same. \n", + "\n", + "It also has the following properties:\n", + "\n", + "1. $tr~A=tr~A^{T}$\n", + "\n", + "2. $tr~A+tr~B=tr(A+B)$\n", + "\n", + "3. $tr(kA)=k tr~A$\n", + "\n", + "4. $tr(AB)=tr(BA)$" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": { + "collapsed": false, + "slideshow": { + "slide_type": "subslide" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "id_m =\n", + "\n", + "Diagonal Matrix\n", + "\n", + " 1 0 0\n", + " 0 4 0\n", + " 0 0 1\n", + "\n", + "ans = 6\n" + ] + } + ], + "source": [ + "id_m=eye(3);\n", + "id_m(2,2)=4\n", + "trace(id_m)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "### Inverse\n", + "\n", + "The inverse of a square matrix, $A^{-1}$ is defined such that\n", + "\n", + "$A^{-1}A=I=AA^{-1}$\n", + "\n", + "Not all square matrices have an inverse, they can be *singular* or *non-invertible*\n", + "\n", + "The inverse has the following properties:\n", + "\n", + "1. $(A^{-1})^{-1}=A$\n", + "\n", + "2. $(AB)^{-1}=B^{-1}A^{-1}$\n", + "\n", + "3. $(A^{-1})^{T}=(A^{T})^{-1}$" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": { + "collapsed": false, + "slideshow": { + "slide_type": "slide" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "A =\n", + "\n", + " 0.71624 0.28713 0.91248\n", + " 0.43423 0.50924 0.89731\n", + " 0.81776 0.15682 0.48825\n", + "\n" + ] + } + ], + "source": [ + "A=rand(3,3)" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": { + "collapsed": false, + "slideshow": { + "slide_type": "fragment" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Ainv =\n", + "\n", + " -1.189272 -0.032028 2.281478\n", + " -5.750167 4.369462 2.716129\n", + " 3.838847 -1.349811 -2.645516\n", + "\n" + ] + } + ], + "source": [ + "Ainv=inv(A)" + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": { + "collapsed": false, + "slideshow": { + "slide_type": "subslide" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "B =\n", + "\n", + " 0.88372 0.43914 0.13476\n", + " 0.43439 0.34519 0.30337\n", + " 0.84604 0.86479 0.21558\n", + "\n" + ] + } + ], + "source": [ + "B=rand(3,3)" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": { + "collapsed": false, + "slideshow": { + "slide_type": "subslide" + } + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "ans =\n", + "\n", + " -1.189272 -5.750167 3.838847\n", + " -0.032028 4.369462 -1.349811\n", + " 2.281478 2.716129 -2.645516\n", + "\n", + "ans =\n", + "\n", + " -1.189272 -5.750167 3.838847\n", + " -0.032028 4.369462 -1.349811\n", + " 2.281478 2.716129 -2.645516\n", + "\n" + ] + } + ], + "source": [ + "%inv(A*B)\n", + "%inv(B)*inv(A)\n", + "\n", + "inv(A')\n", + "\n", + "inv(A)'" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "### Orthogonal Matrices\n", + "\n", + "Vectors are *orthogonal* if $x^{T}$ y=0, and a vector is *normalized* if $||x||_{2}=1$. A\n", + "square matrix is *orthogonal* if all its column vectors are orthogonal to each other and\n", + "normalized. The column vectors are then called *orthonormal* and the following results\n", + "\n", + "$U^{T}U=I=UU^{T}$\n", + "\n", + "and \n", + "\n", + "$||Ux||_{2}=||x||_{2}$" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "### Determinant\n", + "\n", + "The **determinant** of a matrix has 3 properties\n", + "\n", + "1\\. The determinant of the identity matrix is one, $|I|=1$\n", + "\n", + "2\\. If you multiply a single row by scalar $t$ then the determinant is $t|A|$:\n", + "\n", + "$t|A|=\\left[ \\begin{array}{cccc}\n", + "tA_{11} & tA_{12} &...& tA_{1N} \\\\\n", + "A_{21} & A_{22} &...& A_{2N} \\\\\n", + "\\vdots & \\vdots &\\ddots& \\vdots \\\\\n", + "A_{M1} & A_{M2} &...& A_{MN}\\end{array} \\right]$" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "## Determinant (con'd)\n", + "3\\. If you switch 2 rows, the determinant changes sign:\n", + "$-|A|=\\left[ \\begin{array}{cccc}\n", + "A_{21} & A_{22} &...& A_{2N} \\\\\n", + "A_{11} & A_{12} &...& A_{1N} \\\\\n", + "\\vdots & \\vdots &\\ddots& \\vdots \\\\\n", + "A_{M1} & A_{M2} &...& A_{MN}\\end{array} \\right]$" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "## Determinant (con'd)\n", + "4\\. inverse of the determinant is the determinant of the inverse:\n", + "\n", + "$|A^{-1}|=\\frac{1}{|A|}=|A|^{-1}$" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "subslide" + } + }, + "source": [ + "## Calculating the Determinant\n", + "For a $2\\times2$ matrix, \n", + "\n", + "$|A|=\\left|\\left[ \\begin{array}{cc}\n", + "A_{11} & A_{12} \\\\\n", + "A_{21} & A_{22} \\\\\n", + "\\end{array} \\right]\\right| = A_{11}A_{22}-A_{21}A_{12}$\n", + "\n", + "For a $3\\times3$ matrix,\n", + "\n", + "$|A|=\\left|\\left[ \\begin{array}{ccc}\n", + "A_{11} & A_{12} & A_{13} \\\\\n", + "A_{21} & A_{22} & A_{23} \\\\\n", + "A_{31} & A_{32} & A_{33}\\end{array} \\right]\\right|=$\n", + "\n", + "$A_{11}A_{22}A_{33}+A_{12}A_{23}A_{31}+A_{13}A_{21}A_{32}\n", + "-A_{31}A_{22}A_{13}-A_{32}A_{23}A_{11}-A_{33}A_{21}A_{12}$" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "fragment" + } + }, + "source": [ + "For larger matrices, the determinant is more involved" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "### Special Case for determinants\n", + "\n", + "The determinant of a diagonal matrix $|D|=D_{11}D_{22}D_{33}...D_{NN}$. \n", + "\n", + "Similarly, if a matrix is upper triangular (so all values of $A_{ij}=0$ when $j + + + + + + + + + + + + + + + + + + + + + + image/svg+xml + + + + + + + 1 + 2 + 3 + 4 + 5 + 6 + + + + 1 + 2 + 3 + 4 + 5 + 6 + + + + + even+ + odd- + + + diff --git a/09_Linear-Algebra/mass_springs.png b/09_Linear-Algebra/mass_springs.png new file mode 100644 index 0000000..d0dc65c Binary files /dev/null and b/09_Linear-Algebra/mass_springs.png differ diff --git a/09_Linear-Algebra/mass_springs.svg b/09_Linear-Algebra/mass_springs.svg new file mode 100644 index 0000000..2bec8a4 --- /dev/null +++ b/09_Linear-Algebra/mass_springs.svg @@ -0,0 +1,214 @@ + + + + + + + + + + image/svg+xml + + + + + + + + + + + + + + m1 + m2 + m3 + + + m4 + + + + + + + + + diff --git a/09_Linear-Algebra/octave-workspace b/09_Linear-Algebra/octave-workspace new file mode 100644 index 0000000..e69de29