13. Caching, the secret behind it all TL;DR: Caching stores frequently accessed data closer to users or processing units to significantly reduce latency, save resources, and boost performance across various systems. The Gist: Topic: Caching Strategies and Real-World Applications Core Concept: Caching is a technique to improve system performance by storing a subset of frequently accessed data in a faster, more accessible location (cache) than its primary storage. This minimizes the need to access slower original sources, leading to quicker data retrieval and reduced computational load. How it works / Key Steps: Content Delivery Networks (CDNs) , : Distribute content (videos, images, web pages) to "Edge servers" or "Points of Presence (PoPs)" located geographically closer to users. DNS systems route user requests to the nearest PoP, serving cached content to minimize latency and originating server load. Utilize algorithms (including ML) to decide which content subset to cache based on region and trends. Employ Time To Live (TTL) to determine how long content remains cached before re-fetching from the originating server. DNS Caching , : Speeds up domain name (e.g., example.com) to IP address resolution. Implemented at multiple layers: user's operating system, web browsers (Chrome, Firefox), and recursive resolvers (ISP-provided, Google DNS, Cloudflare). Resolvers check their local cache first; if a "cache miss" occurs, they recursively query root, Top-Level Domain (TLD), and authoritative name servers. Hardware Caching (L1, L2, L3) : CPU-level caches (L1, L2, L3) store frequently used data and computation results directly on the CPU chip for ultra-fast access. Utilize predictive algorithms for efficient data pre-fetching (e.g., for sequential array access). In-Memory Databases (Redis, Memcached) , : Store data in Random Access Memory (RAM), which offers significantly faster access than disk-based storage. These are often key-value based NoSQL databases, providing simple data storage and retrieval. Key Learnings / Insights: Subset of Data : Caching involves storing only a subset of data, as RAM is more expensive and has limited capacity compared to secondary storage. Volatile vs. Persistent : RAM is volatile (data is lost on power off), while disk storage is persistent. In-memory databases often rely on secondary storage for data persistence. Caching Strategies : Lazy Caching (Cache-Aside) : Data is fetched from the cache first; if not found, it's retrieved from the primary database, stored in the cache, and then returned. Write-Through Caching : Data is written to both the database and the cache simultaneously, ensuring cache freshness but adding overhead. Eviction Policies : Mechanisms to manage limited cache memory by deciding which data to remove when new data needs to be stored. No Eviction : Leads to memory full errors. Least Recently Used (LRU) : Removes the item that hasn't been accessed for the longest time. Least Frequently Used (LFU) : Removes the item that has been accessed the fewest times. Time To Live (TTL) : Automatically invalidates and removes data after a specified duration. Real-World Use Cases: High-Traffic Platforms : Netflix (global streaming), Twitter (trending topics). Compute-Intensive Database Queries : Cache results of complex SQL queries or aggregations (e.g., dashboard data) to reduce database load and API latency. E-commerce Websites : Cache product details, prices, and inventory data to avoid frequent database queries. External API Calls : Cache responses from third-party APIs (e.g., weather data) to reduce billing costs and avoid hitting rate limits. Rate Limiting : Implement request counters using in-memory databases (like Redis) to control the number of requests a client can make within a specific time, preventing abuse and saving server resources. Key Topics and IDs:CDN -> , , Edge Computing -> , DNS Caching -> , Hardware Caching -> In-Memory Databases (Redis/Memcached) -> , , , Lazy Caching (Cache-Aside) -> Write-Through Caching -> Eviction Policies -> LRU (Least Recently Used) -> LFU (Least Frequently Used) -> TTL (Time To Live) -> , Rate Limiting ->