23. Dynamic Programming

Dynamic Programming In the next few lectures, we will look at a very powerful technique for designing algorithms called dynamic programming.

The introduction explains that dynamic programming is a significant method used in algorithm design. It sets the stage for understanding how dynamic programming works and why it's considered powerful. In the upcoming lectures, we will learn how to apply this technique to solve complex problems efficiently.

Think of dynamic programming like planning a road trip. Instead of looking at all possible routes at once, you consider the best way to get to each stop along the way, building on the best possible choices you made to get to each point.

So, the starting point of dynamic programming is to consider, what we would call inductive definitions. Now, there are many very simple functions which we come across which are inductive...

Inductive definitions are foundational to understanding dynamic programming. They allow problems to be solved by expressing them in terms of simpler or smaller instances of the same problem. For example, the factorial function can be defined with a base case and a recursive case. This approach simplifies complex problems into manageable pieces, which is the essence of dynamic programming.

Imagine you're building a Lego structure. You can think of each addition to the structure as recursive: you only add one block on top of another based on what you previously built, mirroring how recursive functions build on simpler forms.

the base case is when n is 0 and in this case the factorial is 1, so f of 0 is 1...

In recursive functions and inductive definitions, a base case is essential. The base case sets the stage for termination of the recursion, preventing infinite loops. For example, factorial function has a base case of f(0) = 1. This ensures that when the function calls itself with smaller values, it eventually reaches a stopping point.

Think of following a recipe, where the base case is the simplest instruction, like preheating the oven. If you skip this step, your dish won't come out right – just as missing the base case can lead to endless function calls.

what such inductive definitions exploit is what is sometimes called the optimal sub structure property...

The optimal substructure property means that optimal solutions to a problem can be constructed from optimal solutions of its subproblems. This concept is fundamental in dynamic programming as it implies that you can solve larger problems by solving smaller ones effectively.

This is similar to solving a jigsaw puzzle. If you find the best way to fit together a few pieces (the subproblems), you can then more easily fit together larger sections and ultimately complete the entire puzzle.

Now, in numerical question like factorial, it does not make sense to say something is optimal...

In the example given, factorial function illustrates how dynamic programming can be applied to numerically recursive problems. Even though factorial doesn't have an 'optimal' solution in its straightforward calculation like sorting does, understanding how to identify subproblems remains crucial.

Using a calculator for complex calculations is like using dynamic programming. While you can do each step manually (recursive calls), these software tools recognize repeated patterns to provide answers more efficiently.

So, one of the attractions are having inductive definitions is that the yield in a very natural way recursive programs...

The text emphasizes the natural fit between inductive definitions and recursive programming. It showcases that by following the inductive logic, one can create recursive code that mimics the mathematical definitions directly, making code easier to read and validate.

It's like following a flow chart: if you stick to the rules laid out, every decision leads to the next logical step, just like following inductive definitions can lead to clear and natural recursive code.

Now, how many sub problems are there? Now, we have N bookings and every possible subset of this is a sub problem...

The discussion highlights the computational challenges posed by the potential exponential growth of subproblems in certain algorithms. When evaluating all subsets of a problem can lead to inefficiencies, it becomes essential to find ways to optimize the process.

Consider finding the best sale on a range of products at a market. If you try to evaluate every possible combination of sales (subproblems), it becomes overwhelming. The solution is to prioritize by categories or types, thus reducing your calculations.

So, the goal of dynamic programming is to avoid this wastefulness...

Dynamic programming is defined as a method to ensure that subproblems are not solved multiple times. By storing the results of previously computed subproblems, dynamic programming can significantly improve efficiency. This is where concepts like memoization come into play.

Think of a recipe that requires repeated ingredients. Instead of re-measuring each time, you prepare larger quantities of ingredients at once and store them, saving both time and effort.

--