Lecture 1: Introduction; numerics; error analysis (part I) | Highlights and Annotations by Gistr.

CS 205A (Numerical Analysis) overview: Instructor Justin Solomon covers numerical methods for linear algebra, nonlinear calculus, optimization, differential equations, and more. The course emphasizes practical application and algorithm design, using a variational approach. Grading is based on homework (60%), midterm (15%), and final (25%), with participation credit. Prerequisites are Math 51 and CS 106 (or equivalent). MATLAB or Octave is used. The instructor discusses the official textbook ("Scientific Computing by Heath") and his supplementary lecture notes, clarifying the students' responsibilities regarding both. He explains his rationale for creating the notes and acknowledges the ongoing effort involved in their creation, setting realistic expectations for their completeness. This segment introduces two key roles students should develop: being a client of numerical methods (understanding practical aspects and error handling) and a designer of numerical techniques (creating algorithms for novel problems). The instructor connects these roles to the practical challenges of big data analysis. This segment details the course structure, including homework assignments, office hours (in-person and via GChat), and the importance of seeking help, especially given the math-heavy nature of the course. The instructor emphasizes the availability of support from TAs and encourages students to utilize these resources. This segment showcases a typical mathematical proof used in the course, illustrating the methods and techniques students will be employing throughout the course. The instructor highlights the iterative nature of these proofs, involving expanding and collapsing expressions to achieve desired forms. This segment explains binary number representation, including the concept of fixed-point representation for numbers with decimal points. It illustrates the limitations of fixed-point arithmetic due to the finite number of bits available for storage, leading to truncation errors and the inability to represent certain numbers precisely. This segment describes the floating-point representation of numbers, a more sophisticated approach compared to fixed-point, which uses scientific notation to handle a wider range of magnitudes. It discusses the trade-offs: while floating-point offers greater range, it introduces uneven spacing between numbers and potential precision loss when crossing orders of magnitude. This segment uses a simple code example to demonstrate how seemingly correct mathematical operations can produce unexpected results due to rounding errors inherent in storing numbers as doubles. The example highlights the importance of considering numerical limitations when writing code and introduces the concept of using tolerances to account for these inaccuracies.