Exercises and solutions

Author: bvanderlei

Applications_EV.ipynb

@@ -23,7 +23,7 @@
     "\n",
     "The eigenvalues of $A$ are solutions to the quadratic equation $\\lambda^2 - (a+d)\\lambda + ad-bc = 0$, which can be written explicitly in terms of $a$, $b$, $c$, and $d$ using the quadratic formula.  The challenges with larger matrices are that the polynomial is more difficult to construct, and the roots cannot be easily found with a formula.\n",
     "\n",
-    "The algorithms we describe in the section are iterative methods.  They generate a sequence of vectors $\\{X^{(1)}, X^{(2)}, X^{(3)}, ... \\}$ that approach a true eigenvector of the matrix under consideration.  An approximation of the corresponding eigenvalue can then be computed by multiplying the approximate eigenvector by $A$. "
+    "The algorithms we describe in this section are iterative methods.  They generate a sequence of vectors $\\{X^{(1)}, X^{(2)}, X^{(3)}, ... \\}$ that approach a true eigenvector of the matrix under consideration.  An approximation of the corresponding eigenvalue can then be computed by multiplying the approximate eigenvector by $A$. "
    ]
   },
   {
@@ -79,7 +79,7 @@
     "\\end{equation}\n",
     "$$\n",
     "\n",
-    "As a matter of practicality, it is common to scale the vectors in the sequence to unit length as the Power Method is applied.  If the vectors in the sequence are not scaled, their magnitudes will grow of $\\lambda_1>1$ or decay if $\\lambda_1<1$.    Since all components of the vectors get divided by the same factor when the vector is scaled, this step doesn't change the ultimate behavior of the sequence.  The scaled sequence of vectors still approaches the direction of the eigenvector. \n",
+    "As a matter of practicality, it is common to scale the vectors in the sequence to unit length as the Power Method is applied.  If the vectors in the sequence are not scaled, their magnitudes will grow if $\\lambda_1>1$ or decay if $\\lambda_1<1$.    Since all components of the vectors get divided by the same factor when the vector is scaled, this step doesn't change the ultimate behavior of the sequence.  The scaled sequence of vectors still approaches the direction of the eigenvector. \n",
     "\n",
     "We choose an arbitrary $X^{(0)}$ and calculate $X^{(20)}$ using the following rule.\n",
     "\n",
@@ -92,7 +92,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
+   "execution_count": 1,
    "metadata": {},
    "outputs": [
     {
@@ -176,7 +176,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Indeed the difference $AX-24X$ small.  Note that in this case, we can even do the calculation with integer multiplication.  Notice that $X$ has 0 in the first entry and the other entries are equal.  If we set these entries to 1, the result is easy to calculate even without the aid of the computer.  (*Remember that we can change the magnitude of an eigenvector and it is still an eigenvector.*) \n",
+    "Indeed the difference $AX-24X$ is small.  Note that in this case, we can even do the calculation with integer multiplication.  Notice that $X$ has 0 in the first entry and the other entries are equal.  If we set these entries to 1, the result is easy to calculate even without the aid of the computer.  (*Remember that we can change the magnitude of an eigenvector and it is still an eigenvector.*) \n",
     "\n",
     "$$\n",
     "\\begin{equation}\n",
@@ -191,12 +191,12 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "In practice, we do not know how many iterations we need to perform in order to get a good approximation of the eigenvector eigenvalue.  Instead we should specify a condition upon which we will be satisfied with the approximation and terminate the iteration.  For example, since $||AX^{(m)}||\\approx \\lambda_1$ and $AX^{(m)}\\approx \\lambda_1X^{(m)}$ we might require that $AX^{(m)} - ||AX^{(m)}||X^{(m)} < \\epsilon$ for some small number $\\epsilon$ known as a tolerance.  This condition ensures that $X^{(m)}$ functions roughly like an eigenvector.  It is also best to include in the code a limit on the number of iterations that will be carried out.  This ensures that the computation will eventually end, even if a satisfactory result has not yet been achieved."
+    "In practice, we do not know how many iterations we need to perform in order to get a good approximation of the eigenvector.  Instead we should specify a condition upon which we will be satisfied with the approximation and terminate the iteration.  For example, since $||AX^{(m)}||\\approx \\lambda_1$ and $AX^{(m)}\\approx \\lambda_1X^{(m)}$ we might require that $AX^{(m)} - ||AX^{(m)}||X^{(m)} < \\epsilon$ for some small number $\\epsilon$ known as a tolerance.  This condition ensures that $X^{(m)}$ functions roughly like an eigenvector.  It is also best to include in the code a limit on the number of iterations that will be carried out.  This ensures that the computation will eventually end, even if a satisfactory result has not yet been achieved."
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 5,
+   "execution_count": 2,
    "metadata": {},
    "outputs": [
     {
@@ -251,7 +251,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "A more common condition to require is that $||X^{(m)} - X^{(m-1})|| < \\epsilon$ for a given tolerance $\\epsilon$.  This condition mere requires that the vectors in the sequence get close to one another, not that they are actually approximate an eigenvector.  "
+    "A more common condition to require is that $||X^{(m)} - X^{(m-1})|| < \\epsilon$ for a given tolerance $\\epsilon$.  This condition merely requires that the vectors in the sequence get close to one another, not that they are actually approximate an eigenvector.  "
    ]
   },
   {
@@ -325,7 +325,7 @@
     "\\end{equation}\n",
     "$$\n",
     "\n",
-    "If $\\lambda$ is an eigenvalue of $A$, then $\\lambda^{-1}$ is an eigenvalue of $A$.  Furthermore the eigenvector of $A$ is also an eigenvector of $A^{-1}$.  The important point here is that if $\\lambda_n$ is the smallest eigenvalue of $A$, then $\\lambda_n^{-1}$ is the *largest* eigenvector of $A^{-1}$.  If we want to approximate the smallest eigenvalue of $A$, we can just apply the Power Method to $A^{-1}$.\n",
+    "If $\\lambda$ is an eigenvalue of $A$, then $\\lambda^{-1}$ is an eigenvalue of $A^{-1}$.  Furthermore the eigenvector of $A$ is also an eigenvector of $A^{-1}$.  The important point here is that if $\\lambda_n$ is the smallest eigenvalue of $A$, then $\\lambda_n^{-1}$ is the *largest* eigenvector of $A^{-1}$.  If we want to approximate the smallest eigenvalue of $A$, we can just apply the Power Method to $A^{-1}$.\n",
     "\n",
     "We demonstrate the calculation for the following $3\\times 3$ matrix.\n",
     "\n",
@@ -346,7 +346,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 7,
+   "execution_count": 4,
    "metadata": {},
    "outputs": [
     {
@@ -358,10 +358,10 @@
       " [ 6.14168469e-05]\n",
       " [-8.94437425e-01]] \n",
       "\n",
-      "Magnitude of the eigenvalue is A inverse is approximately:\n",
+      "Magnitude of the eigenvalue of A inverse is approximately:\n",
       "0.3333371476391265 \n",
       "\n",
-      "Magnitude of the eigenvalue is A is approximately:\n",
+      "Magnitude of the eigenvalue of A is approximately:\n",
       "2.999931351114087 \n",
       "\n"
      ]
@@ -390,9 +390,9 @@
     "    \n",
     "print(\"Eigenvector is approximately:\")\n",
     "print(X,'\\n')\n",
-    "print(\"Magnitude of the eigenvalue is A inverse is approximately:\")\n",
+    "print(\"Magnitude of the eigenvalue of A inverse is approximately:\")\n",
     "print(lag.Magnitude(A_inv@X),'\\n')\n",
-    "print(\"Magnitude of the eigenvalue is A is approximately:\")\n",
+    "print(\"Magnitude of the eigenvalue of A is approximately:\")\n",
     "print(lag.Magnitude(A@X),'\\n')"
    ]
   },
@@ -415,7 +415,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 8,
+   "execution_count": 6,
    "metadata": {},
    "outputs": [
     {
@@ -430,7 +430,7 @@
       "Magnitude of the eigenvalue of A inverse is approximately:\n",
       "0.3333371476391265 \n",
       "\n",
-      "Magnitude of the eigenvalue is A is approximately:\n",
+      "Magnitude of the eigenvalue of A is approximately:\n",
       "2.999931351114087 \n",
       "\n"
      ]
@@ -460,15 +460,15 @@
     "print(X,'\\n')\n",
     "print(\"Magnitude of the eigenvalue of A inverse is approximately:\")\n",
     "print(lag.Magnitude(sla.lu_solve(LU_factorization,X)),'\\n')\n",
-    "print(\"Magnitude of the eigenvalue is A is approximately:\")\n",
+    "print(\"Magnitude of the eigenvalue of A is approximately:\")\n",
     "print(lag.Magnitude(A@X),'\\n')"
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "#### Shifts\n",
+    "### Shifted inverse power method\n",
     "\n",
     "Using a small modification to the Inverse Power Method, we can also approximate eigenvalues that are not the smallest.  For this variation of the method, we need to observe that if we \"shift\" the diagonal entries of a matrix by a scalar $\\mu$, all of the eigenvalues of the matrix are also shifted by $\\mu$.  Let $A$ be an $n\\times n$ matrix with eigenvalue $\\lambda$ and corresponding eigenvector $V$, so that $AV=\\lambda V$.  Then $(A-\\mu I)V = AV - \\mu V = \\lambda V - \\mu V = (\\lambda-\\mu)V$, which means that $V$ is also an eigenvector of the matrix $(A-\\mu I)$ corresponding to the eigenvalue $\\lambda -\\mu$.  \n",
     "\n",
@@ -498,7 +498,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 9,
+   "execution_count": 7,
    "metadata": {},
    "outputs": [
     {
@@ -510,7 +510,7 @@
       " [0.88465174]\n",
       " [0.14744196]] \n",
       "\n",
-      "Eigenvalue is A is approximately:\n",
+      "Eigenvalue of A is approximately:\n",
       "6.0\n"
      ]
     }
@@ -541,7 +541,7 @@
     "\n",
     "print(\"Eigenvector is approximately:\")\n",
     "print(X,'\\n')\n",
-    "print(\"Eigenvalue is A is approximately:\")\n",
+    "print(\"Eigenvalue of A is approximately:\")\n",
     "print(lag.Magnitude(A@X))"
    ]
   },
@@ -551,33 +551,31 @@
    "source": [
     "### Exercises\n",
     "\n",
-    "Let $A$ be the matrix from the Inverse Power Method example.\n",
+    "**Exercise 1:** Let $A$ be the matrix from the Inverse Power Method example.\n",
     "\n",
     "$$\n",
     "\\begin{equation}\n",
     "A = \\left[ \\begin{array}{rrr} 9 & -1 & -3 \\\\ 0 & 6 & 0 \\\\ -6 & 3 & 6 \\end{array}\\right]\n",
     "\\end{equation}\n",
     "$$\n",
     "\n",
-    "- Use the Power Method to approximate the largest eigenvalue $\\lambda_1$.  Verify that the exact value of $\\lambda_1$ is 12."
+    "($a$) Use the Power Method to approximate the largest eigenvalue $\\lambda_1$.  Verify that the exact value of $\\lambda_1$ is 12."
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 10,
+   "execution_count": 8,
    "metadata": {},
    "outputs": [],
    "source": [
-    "A = np.array([[9,-1,-3],[0,6,0],[-6,3,6]])\n",
-    "\n",
     "## Code solution here."
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "- Apply the Inverse Power Method with a shift of $\\mu = 10$.  Explain why the results differ from those in the example."
+    "($b$) Apply the Inverse Power Method with a shift of $\\mu = 10$.  Explain why the results differ from those in the example."
    ]
   },
   {
@@ -593,7 +591,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "- Apply the Inverse Power Method with a shift 0f $\\mu = 7.5$ and the initial vector given below.  Explain why the sequence of vectors approach the eigenvector corresponding to $\\lambda_1$\n",
+    "($c$) Apply the Inverse Power Method with a shift of $\\mu = 7.5$ and the initial vector given below.  Explain why the sequence of vectors approach the eigenvector corresponding to $\\lambda_1$\n",
     "\n",
     "$$\n",
     "\\begin{equation}\n",
@@ -615,15 +613,15 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Let $B$ be the following matrix.\n",
+    "**Exercise 2:*** Let $B$ be the following matrix.\n",
     "\n",
     "$$\n",
     "\\begin{equation}\n",
     "B = \\left[ \\begin{array}{rrrr} -2 & -18 & 6 \\\\ -11 & 3 & 11 \\\\ -27 & 15 & 31 \\end{array}\\right]\n",
     "\\end{equation}\n",
     "$$\n",
     "\n",
-    "- Apply the Power Method and Inverse Power Method with shifts to approximate all eigenvalues of the matrix $B$. (*Note that one of the eigenvalues of this matrix is negative.*)"
+    "($a$) Apply the Power Method and Inverse Power Method with shifts to approximate all eigenvalues of the matrix $B$. (*Note that one of the eigenvalues of this matrix is negative.*)"
    ]
   },
   {
@@ -639,7 +637,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "- Check your results using the $\\texttt{eig}$ function in SciPy."
+    "($b$) Check your results using the $\\texttt{eig}$ function in SciPy."
    ]
   },
   {

Approximating_Eigenvalues.ipynb

@@ -13,7 +13,7 @@
    "source": [
     "In this chapter we shift focus away from solving linear systems, and look closer at the effect of matrix multiplication.  We restrict our attention now to square matrices, which define linear transformations from $\\mathbb{R}^n$ to $\\mathbb{R}^n$.  In this context we will study special values called eigenvalues, and corresponding vectors called eigenvectors, that can be used to analyze the effect of a corresponding matrix.\n",
     "\n",
-    "Given a square $n\\times n$ matrix $A$, a scalar $\\lambda$ is called an **eigenvalue** of $A$ if there exists some nonzero vector $V$ in $\\mathbb{R}^n$ such that $AV=\\lambda V$.  The vector $V$ is the **eigenvector** associated with $\\lambda$.  The equation states that when an eigenvector of $A$ is multiplied by $A$, the result is simply a multiple of the eigenvector.  In general, there may be multiple eigenvalues associated with a given matrix, and we will label them as $\\lambda_1$, $\\lambda_2$, etc., to keep an orderly notation.  We will lable eigenvectors in a similar way in order to track which eigenvectors are associated with which eigenvalues.  \n",
+    "Given a square $n\\times n$ matrix $A$, a scalar $\\lambda$ is called an **eigenvalue** of $A$ if there exists some nonzero vector $V$ in $\\mathbb{R}^n$ such that $AV=\\lambda V$.  The vector $V$ is the **eigenvector** associated with $\\lambda$.  The equation states that when an eigenvector of $A$ is multiplied by $A$, the result is simply a multiple of the eigenvector.  In general, there may be multiple eigenvalues associated with a given matrix, and we will label them as $\\lambda_1$, $\\lambda_2$, etc., to keep an orderly notation.  We will label eigenvectors in a similar way in order to track which eigenvectors are associated with which eigenvalues.  \n",
     "\n",
     "We will visualize examples in $\\mathbb{R}^2$.  "
    ]
@@ -288,13 +288,59 @@
     "$$"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "**Exercise 2:** Find the eigenvalues and the corresponding eigenvectors of the following matrix $R$ that represents the reflection transformation about the line $ x_1 = x_2 $.\n",
+    "\n",
+    "\n",
+    "$$\n",
+    "\\begin{equation}\n",
+    "R = \\left[ \\begin{array}{cc} 0 & 1 \\\\ 1 & 0 \\end{array}\\right]\n",
+    "\\end{equation}\n",
+    "$$"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "## Code solution here"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "**Exercise 3:** Find a matrix that represents a vertical and horizontal stretch by a factor of $2$. Then, find the eigenvalues and the eigenvectors associated with those eigenvalues. (You may have to take a look at the Planar Transformations section  [Planar Transformations](Planar_Transformations.ipynb))."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "## Code solution here"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "**Exercise 4:** Find a matrix that represents reflection about the $x_1-$axis and find its eigenvalues and eigenvectors."
+   ]
+  },
   {
    "cell_type": "code",
-   "execution_count": 5,
+   "execution_count": 3,
    "metadata": {},
    "outputs": [],
    "source": [
-    "## Code solution here."
+    "## Code solution here"
    ]
   }
  ],