8.1.8. Mapping Schemes

When we started talking about caches in particular, we first looked at direct mapped caches where each memory block can be placed in a unique cache line. So, while talking of caches we said that we divide the memory as well as the caches into same sized blocks. The divisions in memory will be called blocks the divisions in caches will be called lines.

A direct mapped cache is a type of cache memory where each block of main memory is mapped to exactly one cache line. This means that every time a piece of data is needed, the computer knows exactly which cache line to check, making access quicker. Memory and cache are divided into blocks and lines of the same size for this process to work effectively.

Think of a library with specific shelves for different categories of books. If each book category (like fiction, science, history) can only go on one specific shelf, you know exactly where to find any book—no searching through other shelves. This is similar to how a direct mapped cache operates.

To locate data in cache memory address is divided into 3 parts. So, each physical memory address generated by the processor is divided into 3 parts: the tag bits, the index bits, and the word offset.

Each memory address in the cache is divided into three segments: tag bits, index bits, and word offset. The tag bits identify which specific data block is stored, the index bits point to the cache line where data might be found, and the word offset indicates the exact location of the data within that block.

Imagine your home address (for mailing). Your house number is like the index, telling the postman which street (or cache line) to look at. The street name and area code are like the tag, ensuring that the letter gets to the right address. Finally, the specific apartment in a building is analogous to the word offset, pinpointing where exactly to go.

To keep cache and memory consistent a write through scheme can be used so that every write into the cache causes memory to be updated.

In a write-through cache, every time data is written to the cache, the same data is also immediately written to the main memory. This approach helps keep the cache and memory aligned. However, it can slow down performance since writing to main memory is slower than writing to the cache.

Think about writing notes on a whiteboard. If you copy the notes to a notebook right away (like a write-through cache), your information is always up-to-date in two places. But, this can take extra time compared to just writing in one place.

So, the solution to this performance problem of write through caches is to use a write buffer, a queue that holds data while it is writing to memory.

A write buffer is an additional storage area that temporarily holds data that the processor needs to write to memory. This allows the processor to continue executing other instructions without waiting for the memory write operation to complete, improving overall performance.

Imagine you are cooking multiple dishes at home. Instead of waiting for one dish to finish before you start another, you use multiple pots—this is like a write buffer that lets you keep working on other tasks while one is cooking (writing to memory).

In a write allocate policy when we have a write miss; I first bring the block of memory from the main memory into the cache and then write on to this cache line. In a no write allocate policy, when I have a write miss, I do not just allocate a line in cache for this memory block, I directly write on to the main memory.

The write allocate policy means that if data needs to be written and is not yet in the cache (a write miss), it first brings the relevant block of memory into the cache before writing. Conversely, the no write allocate policy skips bringing the block into cache and writes directly to memory instead. The first policy can be faster when data is repeatedly accessed, while the second policy saves time by not interacting with the cache.

If you think of a refrigerator as your cache: the write allocate policy is like placing groceries into an empty fridge before you start cooking. The no write allocate policy is like cooking directly from the grocery bag without putting anything in the fridge. For repetitive use, the first is helpful, but the second can save time if you won’t use the items again soon.

An alternative to write through is write back; where a cache line is written back to memory only when it is replaced.

With write back caches, data is written only to the cache initially and only written back to the main memory when the cache line needs to be replaced. This can save a lot of write operations and improve performance, especially for repetitive writes. However, it requires careful management to ensure data integrity because the memory may not reflect the latest data until a write-back occurs.

Picture a workspace where you jot down ideas on sticky notes. You don’t transfer each idea to a notebook immediately (write back). Instead, you wait until you need to clean up or reorganize your workspace, at which point you copy all the sticky notes at once. This way, you save time by not writing redundantly.

To take advantage of spatial locality a cache must have larger size than one word.

Spatial locality refers to the tendency of programs to access data locations that are close to each other. Thus, having larger cache lines (i.e., not just one word) improves efficiency because when one piece of data is accessed, surrounding data is often loaded too, reducing cache misses.

Consider a student studying. If they are reading a textbook, they often read paragraphs close to each other instead of jumping back and forth across the book (spatial locality). If you read a whole page instead of one line, you grasp more context at once, reducing the need to go back and look at what you missed.

--