Lecture 7: Eigenproblems II: Algorithms (part II) | Highlights and Annotations by Gistr.

This segment discusses the computational trade-offs between inverse iteration and Rayleigh quotient iteration. It explains how Rayleigh quotient iteration, while converging faster, involves a different inverse calculation at each iteration, making LU factorization impractical and increasing the computational cost per iteration to O(n³). The lecture discusses iterative methods for finding eigenvalues and eigenvectors. Power iteration finds the largest eigenvalue. Rayleigh quotient iteration improves convergence but increases computational cost per iteration. Deflation removes known eigenvectors to find others, but numerical errors can be problematic. QR iteration, using QR factorization and conjugation, offers a parallel approach to finding all eigenvalues and eigenvectors, converging to a diagonal matrix of eigenvalues and an orthogonal matrix of eigenvectors. This segment details the iterative scheme used in numerical algorithms to uncover eigenvectors and eigenvalues, focusing on how an initial guess for the eigenvector is refined using an improved estimate of the eigenvalue, leading to the introduction of Rayleigh quotient iteration as a faster converging strategy. This segment breaks down the Rayleigh quotient iteration method, explaining how it leverages the closeness of an estimated eigenvalue (sigma k) to the actual eigenvalue to improve the eigenvector estimation. It highlights the process of shifting the eigenvalue close to zero to amplify the desired component and the subsequent normalization step. This segment introduces and explains the QR algorithm, a powerful method for finding eigenvalues and eigenvectors. The speaker builds intuition by exploring the implications of QR factorization and conjugation with orthogonal matrices. The elegance and efficiency of the algorithm are emphasized, culminating in a description of its convergence properties and the speaker's enthusiastic assessment of its beauty and fundamental nature. This segment presents an algorithm for finding all eigenvectors of a symmetric matrix. It describes a method to iteratively find eigenvectors by projecting out previously found eigenvectors, ensuring that the algorithm converges to the next largest eigenvector. The limitations due to numerical errors and the need for orthogonal eigenvectors are also discussed. This segment details a clever deflation strategy for finding eigenvectors of a matrix. The speaker demonstrates how a similarity transformation using a Householder matrix isolates the first eigenvector and eigenvalue, allowing for recursive application to find subsequent eigenvectors. The preservation of eigenvalues under similarity transformations is highlighted, making this a concise and insightful explanation of a key numerical linear algebra technique. This segment addresses a potential problem with randomly initializing the starting vector (V0) in power iteration. It explains the low probability but possibility of V0 lacking a component parallel to the largest eigenvector, leading to convergence to the second largest eigenvector instead.