Lecture 8: Eigenproblems III (QR iteration, conditioning); singular value decomposition (part I)

This segment delves into the mathematical proof behind the QR iteration method, explaining its effectiveness in revealing the eigenstructure of a matrix. The instructor clarifies the conjugation process and its role in the algorithm, addressing potential confusion from a previous lecture. The instructor specifies the conditions required for the QR iteration method (symmetric matrix with unique eigenvalues) and details the algorithm's iterative process. The explanation includes clarifying the role of diagonal matrices and the algorithm's convergence towards a diagonal matrix containing eigenvalues. This segment focuses on the assumptions made for the proof of QR iteration, specifically the symmetry of the matrix and the uniqueness of its eigenvalues. The instructor discusses how the uniqueness constraint can be relaxed through perturbation, adding depth to the algorithm's applicability. This lecture proves the QR iteration method for finding eigenvalues. It focuses on the symmetric, unique eigenvalue case, showing that if the iteration converges, the resulting matrix is diagonal with eigenvalues on the diagonal. Convergence is intuitively explained, though a rigorous proof is complex. Finally, eigenvalue conditioning is discussed, showing how small matrix perturbations affect eigenvalues and eigenvectors. This segment explores the convergence of the QR iteration method. The instructor uses the power iteration method and Gram-Schmidt orthogonalization to intuitively explain why the algorithm converges to a diagonal matrix whose columns represent the eigenvectors. The discussion also touches upon numerical stability issues. This segment introduces the concept of conditioning in the context of eigenvalue problems. The speaker explains that while rigorous proofs are complex, the fundamental question is how perturbations in the matrix A affect its eigenvalues and eigenvectors. The discussion sets the stage for a deeper dive into the complexities of perturbative analysis and the challenges in providing precise conditioning proofs. This segment presents a detailed proof demonstrating that if the QR iteration converges and the matrix is symmetric with unique eigenvalues, the result is a diagonal matrix. The instructor uses clever algebraic manipulations and properties of orthogonal matrices to reach this conclusion. The speaker outlines the convergence proof for the QR iteration algorithm, emphasizing that while the formal proof is complex, the core idea is that the iteration matrix converges to a matrix of eigenvectors, simplifying the process. The explanation includes a discussion of how the Q matrix approaches the identity matrix as iterations increase, leading to convergence. The segment also briefly introduces Krylov subspace methods as an alternative approach. This segment details the derivation of a formula for eigenvalue perturbation. The speaker introduces the concept of left eigenvectors and uses them, along with a clever manipulation of the perturbation equation, to arrive at a concise expression for the change in eigenvalues due to a small change in the matrix. The derivation involves approximations and assumptions, which are justified intuitively. The final result provides a bound on the perturbation using the Frobenius norm.