23.8. Computational Challenges

So, the starting point of dynamic programming is to consider, what we would call inductive definitions. Now, there are many very simple functions which we come across which are inductive. So, we all know that mathematically n factorial is n times n minus 1 factorial. So, we can write an inductive definition of factorial is follows. The base case is when n is 0 and in this case the factorial is 1, so f of 0 is 1. And in general, if we have n greater than 0, then f of n is given by n times f of n minus 1. In other words, we express the factorial of n in terms of the same function applied to a smaller input.

Dynamic programming often starts with inductive definitions, which define a function in terms of itself for smaller inputs. A common example is 'factorial', defined as the product of a number (n) and the factorial of (n-1) with a base case of 0! being 1. This structure shows how to solve complex problems by breaking them into simpler components.

Think of inductive definitions like building a tower with blocks. You start with a single block (the base case) and for every additional level of the tower, you add a block and build upon the previous level. Just as each level is made up of blocks on top of each other, an inductive definition expresses a larger value based on previously defined smaller values.

What such inductive definitions exploit is what is sometimes called the optimal sub structure property. So, what is complicated base means basically is what you expect from an inductive definition that is you solve the original problem by combining solution to sub problems.

The optimal substructure property indicates that a problem can be solved optimally by solving its subproblems. This means that there are smaller problems that lead directly to the solution of the larger problem. In the case of factorial, the solution for n! relies on the solution for (n-1)!. Similarly, in sorting algorithms like insertion sort, the sorted array can be built by adding one element at a time to a previously sorted subarray.

Imagine you are tasked with organizing a series of books in a library. You first sort a small section (the subproblem) and then incorporate new books one by one into that existing sorted section. Each time you do this, you rely on the existing order of the books to maintain the overall organization without starting from scratch.

So, now let us look at a problem that we are seen before in the context of greedy algorithms. So, we will look at the problem called interval scheduling. So, integrated scheduling said that we had a resource which is available over a period of time and now people will come and make bookings for these resources.

The interval scheduling problem involves maximizing the number of non-overlapping intervals (or bookings) from a set of requests that specify start and finish times. It’s a practical problem where you have to manage limited resources among competing demands, ensuring that conflicting requests are handled appropriately.

Think of a hotel management system where two guests want to book the same room at overlapping times. The interval scheduling problem is akin to deciding how to allocate the room to maximize the number of guests accommodated, ensuring that only one guest uses the room during a given time slot.

So, how many sub problems are there? Now, we have N bookings and every possible subset of this is a sub problem, so we have an exponential number of sub problems.

In allocating resources, every potential combination of bookings represents a subset of the problem to examine. A naive approach might involve checking all these subsets, which becomes computationally impractical as the number of bookings grows. Instead, a greedy approach simplifies this by making a locally optimal choice at each step, which can lead to a globally optimal solution quickly, as it avoids checking every subset explicitly.

Imagine trying to fit as many items as possible into a box. Instead of trying every possible combination of items (which would take forever), you could apply a greedy strategy by choosing the largest item that fits first, then the next largest, and so on. This simplifies your decision-making process significantly.

So, the whole problem with this approach is that the inductive solution can give rise to the same problem at different stages. And if we just use recursion as we said before one of the great benefits are having on detective definition is that you can just write a recursive solution which just mirrors the inductive definition of the problem.

When using recursive solutions, the same problem can be solved multiple times across different branches of the recursion tree, leading to inefficiencies. This is particularly problematic for problems with overlapping subproblems, such as in dynamic programming contexts. Therefore, maintaining efficiency requires a method to store results of already computed subproblems to avoid redundant calculations.

Consider a homework assignment where you repeatedly solve the same math problem. Each time you do it, you might take a different amount of time, forgetting what you learned the previous time. Instead, if you jot down the solution, you can complete the assignment much faster by referring back to your notes, rather than re-solving the same problem continuously.

So, the goal of dynamic programming is to avoid this wastefulness. So, there are two techniques that we will see, there is technique called memoization, which is a way to build in some cleverness into recursion.

Dynamic programming aims to optimize problems by storing previously solved subproblems (memoization) instead of recalculating them, thus making previous solutions reusable. This technique significantly reduces computational time and resources by transforming recursive approaches into more iterative, table-filling algorithms.

Think of a chef preparing multiple recipes that each require some of the same ingredients. Instead of constantly measuring and preparing those ingredients anew for each recipe, the chef can prep larger batches once and then divide them between the recipes, saving time and effort.

--