27.1.1. Mathematical Institute

So, now let us analyze Dijkstra’s algorithm for the single source shortest path problem. Dijkstra's algorithms operate by burning vertices in the analogy we used. We keep track of vertices that have burnt or initially nothing is visited, and initially we do not know any distance to any vertex. We have assumes all distances that at infinity. When we start at the source vertex, let us call it 1 by assumption. So, we set its distance to 0. Then, we repeatedly pick up the first vertex which is not burnt and which has the minimum distance among those which are not burnt, visit, and recompute the distance to all its neighbors.

Dijkstra's algorithm is designed to find the shortest path from a source vertex (point) to other vertices in a graph. Initially, all vertices are unvisited, and the distance to each vertex is considered infinite, except for the starting vertex, which has a distance of 0. As the algorithm runs, it identifies the vertex with the smallest distance that hasn't been visited yet ('burnt') and updates the distances to its neighboring vertices. This process repeats until all vertices have been burnt.

Imagine you are in a city and trying to find the shortest route to each destination starting from your home. Initially, you know your distance to your home (0 miles) but consider all other destinations as far away (infinity). As you drive to a destination, you remember how far it took to get there and update that distance for future reference, making it easier to find the shortest route to all other places.

Before we look at the complexity of these algorithms, we actually first have to verify that it is correct. Dijkstra's algorithms now make these sequences of updates. Its visits vertices in a particular order which is different from breadth-first or depth-first, and we need to justify that this order is actually correct. So, at every point, breadth-first search looks all its neighbors and visits them in the order of their vertex number. Depth-first search will pick the first neighbor, and then explore it further. Dijkstra's algorithm uses the different strategy which is to pick the first, the smallest distance and visited neighbor. We have to justify that this choice which is never under, that is which keeps going forward, and you never go back.

To ensure Dijkstra's algorithm works correctly, we must establish an invariant: at each stage of the algorithm, the vertices that have been burnt have the shortest distance from the source. The method operates greedily, meaning it makes the best local choice (choosing the closest vertex) at that moment with the hope that this choice leads to a globally optimal solution. This distinction from breadth-first and depth-first search is vital in understanding why Dijkstra's approach is valid.

Consider a treasure map where you are trying to decide which route to take to reach the treasure. At each point, you choose the path that looks shortest based on current knowledge. If you consistently choose the path that appears best without reconsidering past choices, you are engaging in a process similar to Dijkstra's. As long as you can trust that the current best path leads to the treasure without adding loops, your strategy will yield the correct destination.

The key to establishing correctness in Dijkstra's algorithm is establishing an invariant. At each iteration, we claim that the burnt vertices are correctly solved. That is at any point if we look at the distances assigned to the vertices that are burnt, then those distances are actually the shortest distance from the source vertex to that burnt vertex. We assume this inductive invariant is true. Now we want to add a new vertex v, such that if we look at the distance to x plus the weight of x v, if this total sum is the smallest among all the vertices which are not burnt.

The invariant in this context refers to the fact that the distances to burnt vertices are always the shortest distances from the source. When adding a new vertex, the algorithm ensures that the path to this new vertex will not be shorter than previously computed paths. This is crucial for maintaining correctness throughout the algorithm's execution.

Imagine you’re climbing a mountain and marking the height you’ve reached at each stop. Each time you check a new path, you compare it to your highest point. If you find a new path that doesn’t take you to a higher altitude, you discard it. Only paths leading to higher altitudes are 'burnt' and remembered. Thus, you ensure that each marked point is indeed the highest you've been, similar to the invariant in Dijkstra's algorithm.

Now that we have established that Dijkstra's algorithm is correct, let us look at the complexity. There are some obvious loops in this. There is a loop of order n for initializing the visited values to false and distances to infinity. Inside this order n loop, we have two loops: one is to scan the outgoing neighbors, and the other is to choose the minimum vertex, which results in an overall complexity of order n squared.

The complexity of Dijkstra’s algorithm mainly stems from two steps: initializing all vertices and scanning through the unvisited vertices to find the minimum distance vertex. The outer loop runs n times, performing operations that often involve scanning n vertices, leading to an O(n^2) time complexity in its initial and naive implementation. This represents a significant performance limitation for larger graphs.

Think of a classroom where students take turns answering questions. Every time a new question comes up, every student has to be asked how well they know the answer, taking time equals to the number of students. If there are 30 students, in the worst case, you may ask all students for each question, leading to a lot of unnecessary questioning time, similar to the inefficient checks in Dijkstra's initial implementation.

To get around the bottleneck, we need to maintain the burnt times in a more sophisticated data structure. It turns out, as we need a data structure in which we can quickly find and remove the minimum element, we can use a heap, which allows us to do updates efficiently, and this can reduce the overall complexity to O(n log n).

Using a heap structure improves the efficiency of selecting the minimum distance vertex and updating the distances of neighboring vertices. Instead of a linear scan to find the minimum, a heap can manage this process in logarithmic time, which is significantly faster as the number of vertices increases.

Imagine you're at an ice cream shop, and there's a long line. If everyone is served first-come, first-served, there's a lot of waiting. Now imagine if instead, there’s a priority seating arrangement where people are served based on their order of arrival but with a twist that allows the chef to quickly pick out the next customer in line. This allows for quicker service overall, akin to how a heap efficiently manages vertex selection in Dijkstra's algorithm.

--