8.1.2. Summary

This is a small lecture where we summarize our discussion in the last 3 lectures. So, in order to execute a program a computer needs to fetch its instructions and data from memory and write process data back into memory.

The execution of a program by a computer involves fetching instructions and data from its memory. This means that the CPU retrieves the commands it needs to execute and any necessary information required for carrying out those commands. Once the processing is complete, any changes or output generated must then be written back to the memory.

Think of a chef in a kitchen. Before cooking a dish, the chef needs to get the recipe (instructions) and the ingredients (data) from the pantry (memory). After cooking, the chef might write notes or adjustments on the recipe card (process data back to memory) for future use.

Improvement in operating speed of processor chips has outpaced improvements in memory access times.

This statement highlights a significant issue in computer architecture: the speed at which CPUs can process instructions is much faster than the rate at which memory can be accessed. This creates a bottleneck, meaning that the CPU often has to wait for data to be retrieved from memory, slowing down the overall performance of a program.

Consider a student who is trying to complete a project. If the student has all the resources and tools ready but has to wait for their teacher to provide the additional materials needed, the student cannot work as fast as possible. This delay in receiving the necessary materials represents the slower memory access time.

To fully exploit the capability of a model processor, the computer must have large and fast memory. So, we have lots of programs and all those programs may execute simultaneously on the processor.

For a computer to effectively utilize its powerful processor, it is essential to have a sufficient amount of memory that can operate quickly. This is particularly important when running multiple programs at once, as each program requires a portion of memory to function. As programs grow larger and more complex, the demand for memory increases correspondingly.

Imagine a librarian who can read quickly (processor) but has a limited number of bookshelves (memory). If there are too many books (programs) to display and the shelves are too slow to bring in new books, the librarian can’t work efficiently, even if they read faster.

Developments in semiconductor technology have led to spectacular improvements in the speeds, faster memory chips also suffer from higher cost per bit.

While advancements in technology have resulted in faster memory chips, these improvements often come with a significant increase in the cost associated with memory per unit of storage. Thus, there's a trade-off between speed and affordability, posing challenges for computer system designers.

Think of high-end cameras: they can take stunning pictures (fast memory), but they tend to be expensive. If you want a more budget-friendly camera, you might sacrifice some quality and speed, just like choosing slower, cheaper memory options.

The solution to the above problem of controlling both miss rates and miss penalties at affordable costs lies in having a hierarchy of memories.

Creating a memory hierarchy involves using different types of memory that balance speed and cost. For example, faster memory (like SRAM) can be used for cache, while larger, slower memory (like DRAM) serves as main memory. This arrangement allows efficient data access while managing overall costs.

Imagine you have a filing cabinet (memory hierarchy). The most frequently accessed files are kept in the top drawer (cache), while less frequently used files are stored in deeper drawers (main memory). This organization makes it easier and quicker to find what you need without having to dig through everything every time.

The solution works because of the principle of temporal and spatial locality.

Temporal locality means that if a specific item of data was accessed recently, it will likely be accessed again soon. Spatial locality refers to the idea that data located near recently accessed data is also likely to be needed shortly. These principles inform how data is stored and fetched, enabling more efficient memory use.

Imagine you’re cooking. If you just used salt, you are likely to use it again soon (temporal locality), and the pepper (spatial locality), which is nearby, might be accessed as well. Keeping these items close makes cooking faster.

When we started talking about caches in particular, we first looked at direct mapped caches where each memory block can be placed in a unique cache line.

In direct mapped caches, each block of memory is linked to a specific line in the cache, meaning it has a designated spot. This organization helps the CPU quickly locate data, but it has its limits, as it can lead to conflicts when multiple memory blocks need to occupy the same cache line.

Think of a library where each book is assigned a specific shelf. If a new book needs to go on the same shelf as an existing book, one of them has to be removed, which can lead to problems—just like how cache conflicts can occur.

--