1.6.1. Week 1: Motivation and Asymptotic Complexity

When we study algorithms, the first thing that we need to convince ourselves is that the algorithm is correct and it is doing the job that we expected. So, we look at the strategies for proving the correctness of algorithms.

In algorithm study, the starting point is to ensure the correctness of the algorithm. This means we need to confirm that the algorithm works as intended and produces the expected outcomes. To achieve this, we adopt various strategies to demonstrate that the algorithm successfully solves the problem it's designed for. This might involve tests and formal proofs that show it meets its specifications.

Think of it like a recipe for baking a cake. Before you serve it, you taste it to ensure it has the right flavor and texture. Similarly, proving an algorithm's correctness is like checking if the cake meets your expectations before it's presented.

The other important aspect of an algorithm is its efficiency. How much time does the algorithms take on inputs? Now, of course, we have to factor in the size of the input.

Efficiency is crucial when analyzing algorithms. It refers to how quickly an algorithm can process inputs of varying sizes. This means we need to consider not just the algorithm's speed with small inputs but also how it behaves as the size of the input increases. This aspect is integral to developing algorithms that are practical and usable in real-world scenarios.

Imagine a factory assembly line. For a small order, the line may produce widgets quickly. However, as orders grow larger, a well-designed assembly line may scale up efficiently to handle more items without slowing down too much. This mirrors how efficient algorithms operate with increasing data sizes.

We need a notation or a way of comparing two different algorithms, which operate on the same types of inputs and produce the same types of outputs. This will be achieved through the concept called asymptotic complexity, which measures the running time of an algorithm as inputs grow larger and larger as a function of the inputs.

Asymptotic complexity is a mathematical notation that allows us to compare the efficiency of algorithms without needing to test them under all possible input conditions. It focuses on how the running time of an algorithm increases as the size of the input increases. Common notations include Big O notation, which captures the upper limit of an algorithm's efficiency.

Consider different routes to a destination. Route A may take a consistent amount of time, regardless of traffic, while Route B may vary significantly depending on how congested it is. Asymptotic complexity is like predicting how long it will take to reach your destination as traffic conditions change over time.

An important part of problem solving in any domain and in particular algorithms is the art of modeling the problem at a suitable level of detail. In most algorithms that we will see we need to find a suitable mathematical model.

Modeling a problem involves representing it in a way that makes it easier to analyze and solve. This could involve creating diagrams, equations, or graphs that capture the key aspects of the problem. Once modeled, we can often break the problem into smaller, more manageable sub-problems, which can be solved individually and then combined to solve the overall problem.

Think of constructing a building. Before starting the construction, architects create detailed blueprints (models) of the building's layout. Then, they can tackle one area (like the foundation) at a time, knowing that each part will fit together to make the complete structure.

Over the course of time, many generic techniques have been developed to solve a large number of problems. We will see examples of how these techniques can be applied to very standard problems that we come across repeatedly. Among the techniques are divide and conquer and greedy algorithms.

Various strategies have emerged in the study of algorithms that help solve common types of problems efficiently. The 'divide and conquer' technique involves splitting a problem into individual components, solving those sub-problems, and then combining their solutions. Greedy algorithms, on the other hand, make the locally optimal choice at each step in the hope of finding a global optimum.

In cooking, divide and conquer is like preparing multiple dishes for a meal: you might chop vegetables, cook grains, and grill meat all at once to efficiently create a satisfying meal. Greedy algorithms resemble someone trying to pick the ripest fruits from a tree one by one, hoping that by choosing the best fruit available at each step, they will end with the best selection overall.

--