22.4.2. Using Heaps for Optimization

So, now, the recursive a solution will say, that how do a figure of what the rest of the tree looks like, well if I have a situation, where I have decided x and y both here. Then, I will kind of tree, this is a unit and make a new letter call x, y and give it the cumulative frequency effects plus x, y of the old two letter. So, construct the new alphabet and which I drop x and y and I add a new composite of hybrid letter x, y; whose frequencies going to be f x plus f y.

In Huffman's encoding algorithm, we start with a set of letters and their corresponding frequencies. When we choose two letters with the lowest frequencies, we merge them into a new letter that represents both. The frequency of this new letter is the sum of the two original frequencies. We repeat this process recursively to create a tree structure where each letter leads to a binary decision - either 0 or 1 - representing its encoded form.

Consider a situation where you have to prioritize tasks based on their urgency. If you merge two similar tasks into one urgent task, you are making a decision based on urgency, just like how Huffman's algorithm merges the two lowest frequency letters to create a new letter. This process continues until all tasks (or letters) are organized into a clear plan (or tree structure), making it easier to decide what to tackle next.

So, let us look at this example that we had earlier, so here the two lowest frequency letters d and e. So, we merge them into the new letter d, e and this is a frequency 0.23, because it is 0.18 plus 0.05. Now, these two are a two lowest letters, so we merge them and we get a new letter c, d, e of cumulative frequency 0.43, which is some of all the frequencies are that two values.

When performing Huffman's algorithm, you begin by identifying the two letters with the lowest frequencies, in this case, 'd' and 'e' with frequencies of 0.18 and 0.05 respectively. When we merge them, we create a new letter, 'd,e', with a cumulative frequency of 0.23 (0.18 + 0.05). We can then continue merging the next lowest frequencies to create a larger composite letter, and repeat this until we have only one letter remaining, with a total cost determined by the frequencies.

Imagine you are combining ingredients for a recipe. If you have two ingredients that are used in very small amounts, say salt and spices, you can measure them together into a small bowl, much like merging frequencies in Huffman's method. The more ingredients you merge, the easier it becomes to see how much of each you need (or how frequencies create a complete picture of the letters in the algorithm).

Now, recursion fiction, I have a k minus 1 letter alphabet, so I have recursively find and optimal encoding of that. Now, before coming to how to adapt the solution, the recursive ends when have a only two letters, for two the optimal solution is to build the tree which exactly two leaves, label 0 and 1 at the path. So, this is the basic case, if I have more than two letters I will recursively construct tree to the smaller thing.

The algorithm reaches a base case when there are only two letters left. In this case, the optimal way is straightforward: simply assign one of these letters a '0' and the other a '1'. This base case ensures that when building the tree structure, we have a clear stopping point, allowing us to work back up the tree to construct the full Huffman tree efficiently from the smaller parts.

Think of a game where you have to choose between two pathways. If you can only choose between two routes, it’s simple - you make your choice right away (like assigning '0' and '1'). This decisiveness reflects the base case of Huffman's algorithm, where the solution is clear and quick to resolve. Once you make that decision, you can create more complex paths leading from that simple choice.

Now, the bottleneck what will make is finding the minimum values. If you use an array, then as we know scan the array instance to find the minimum values, remember that the minimum of values keep changing, I cannot short it one send for all. So, it is an order case can each time, so linear scan and I do this appropriate k these times. So, I get order case two... if I maintain the frequencies is not at as a heap, then the order log k time, I can find the minimum values.

Finding the two minimum values from a list of letters can be challenging, as the values change every time we merge two letters. Instead of using a simple array and scanning for the two lowest frequencies, we can use a heap data structure. Heaps allow us to efficiently retrieve the minimum values in logarithmic time (O(log k)), which significantly optimizes the performance of Huffman's algorithm to O(k log k) instead of O(k^2).

Think of a priority queue at an airport where passengers board the plane based on ticket class. If you have a clear rank of passengers (like using a heap), it’s much easier to always board first-class and business-class passengers efficiently. In contrast, if you were to call passengers randomly from a mixed line (a linear scan), it would take significantly longer, just as using a simple array in Huffman’s algorithm does.

So, recall that, we mention that the beginning that this is going to be a last greedy algorithm. So, y is greedy, well because every time, we make a recursive call, we are deciding to combine two nodes into one and the two letters we are choose always ones with the lowest frequencies.

Huffman’s algorithm is considered a greedy algorithm because it consistently makes a local optimum choice — merging the two letters with the lowest frequencies — at each step. The idea is that this local decision will lead to a globally optimal solution by ensuring that the higher frequency letters are given shorter codes. Greedy algorithms work well in cases like this, where local choices accumulate into a better overall solution.

Imagine you are climbing a mountain. At each point, you choose the shortest and easiest path to climb to make immediate progress - this is your local decision. Throughout your journey (or algorithm), you make these choices and eventually reach the top of the mountain (or optimal solution). This mimics the greedy approach of Huffman's algorithm, where each small step (merging lowest frequencies) leads to the best overall outcome.

--