Lecture 3: More LU; conditioning and sensitivity (part I) | Highlights and Annotations by Gistr.

The instructor explains the preferred communication channels (Piazza), encourages LaTeX usage for homework submissions, and introduces the concept of LU factorization, relating it to Gaussian elimination and reduced row-echelon form. Homework 0 is due; homework 1 (covering forward/backward error, IEEE floating point, LU factorization) is posted. Linear algebra knowledge is crucial; help will be provided, especially for problem 3. Use LaTeX for homework submissions. Piazza is for questions. LU factorization may require pivoting for non-invertible matrices. Today's lecture covers the sensitivity and conditioning of linear systems, using norms to analyze perturbations and errors. The instructor discusses homework submission policies, extenuating circumstances, and the content covered in the upcoming homework assignments, emphasizing the importance of linear algebra and providing guidance on seeking help.The instructor details the content of homework one, highlighting the inclusion of linear algebra problems to address students' knowledge gaps. The instructor explains the importance of mastering linear algebra for both course success and general mathematical proficiency. The instructor engages students in a discussion about the conditions under which LU factorization exists, introducing the concept of pivoting to address cases where Gaussian elimination might fail due to division by zero. The importance of pivoting for ensuring the success of the LU factorization algorithm is highlighted. This segment delves into the application of norms in solving underdetermined systems of equations, a common problem in machine learning. It explains how minimizing different norms (like the L1 norm) can lead to solutions with desirable properties, such as sparsity, and connects this to applications like compressive sensing in MRI technology, highlighting the practical implications of choosing the right norm. The instructor introduces the concepts of forward and backward error, condition number, and perturbative analysis in the context of solving linear systems. The lecture sets the stage for understanding how sensitive solutions are to small perturbations in the input data. This segment introduces the concept of the taxicab norm (Manhattan distance), illustrating it with the example of a taxicab traveling between two points in a city. It highlights the counterintuitive property that even though a sequence of curves approaches the diagonal of a right triangle, the length of each curve remains constant, emphasizing the importance of understanding convergence and the concept of "smallness." This segment explores the limit of the p-norm as p approaches infinity, demonstrating that it converges to the infinity norm (the maximum absolute value of the elements). It then introduces the concept of unit circles for different norms, setting the stage for visualizing and comparing various norms geometrically.