5. Partial Differential Equations

Before delving into Bayes' Theorem, let’s recall a few fundamental probability concepts: • Sample Space (S): Set of all possible outcomes. • Event (E): A subset of the sample space. • Conditional Probability: 𝑃(𝐴∩𝐵) 𝑃(𝐴|𝐵) = , provided 𝑃(𝐵)> 0 𝑃(𝐵)

To understand Bayes' Theorem effectively, we need to revisit some core concepts in probability. The sample space (S) comprises all possible outcomes of a random experiment, like flipping a coin or rolling a die. An event (E) is simply a specific outcome or a collection of outcomes within the sample space.

Conditional probability is focused on the probability of one event occurring given that another event has already occurred, indicated mathematically using the formula P(A|B) = P(A ∩ B) / P(B), assuming P(B) is greater than zero. This concept is crucial for applying Bayes' theorem, as it connects the occurrence of different events.

Imagine you're trying to find a parking spot in a busy city (Sample Space). The act of finding an open space (Event) is part of the overall parking experience. If you know a certain area usually has available spots (condition), that information helps you make better decisions on where to look first. Conditional probability captures this kind of scenario: it’s about modifying your likelihood assessments based on existing knowledge.

Bayes’ Theorem describes the probability of an event based on prior knowledge of conditions that might be related to the event.

Formula: 𝑃(𝐵|𝐴 )⋅𝑃(𝐴 ) 𝑖 𝑖 𝑃(𝐴 |𝐵) = 𝑖 ∑𝑛 𝑃(𝐵|𝐴 )⋅𝑃(𝐴 ) 𝑗=1 𝑗 𝑗 Where: • 𝐴 ,𝐴 ,...,𝐴 : Mutually exclusive and exhaustive events 1 2 𝑛 • 𝐵: An event whose probability depends on the occurrence of 𝐴 𝑖 • 𝑃(𝐴 ): Prior probability 𝑖 • 𝑃(𝐵|𝐴 ): Likelihood 𝑖 • 𝑃(𝐴 |𝐵): Posterior probability 𝑖

Bayes' Theorem provides a mathematical framework for updating probabilities based on new evidence. The theorem states that the probability of event A given event B (posterior probability P(A|B)) can be calculated by considering the likelihood of B occurring given A (P(B|A)), factored by the prior probability of A itself (P(A)). The denominator sums the probabilities of all other mutually exclusive events leading to B.

This allows us to refine our beliefs about A after observing evidence B, revealing how interconnected probabilities can be understood and recalibrated in light of new information.

Consider this in a medical context: If a test is known to detect a disease (Event A) correctly 99% of the time (prior probability), and you know the test’s result was positive (Event B), Bayes' theorem allows you to determine the revised probability that a person truly has the disease after receiving that positive test result. It’s like adjusting your beliefs based on the 'news’ the test provides.

Let: • 𝑃(𝐴 ∩𝐵) = 𝑃(𝐵|𝐴 )⋅𝑃(𝐴 ) 𝑖 𝑖 𝑖 • From the total probability theorem: 𝑛 𝑃(𝐵)= ∑𝑃(𝐵|𝐴 )⋅𝑃(𝐴 ) 𝑗 𝑗 𝑗=1 Now: 𝑃(𝐴 ∩ 𝐵) 𝑃(𝐵|𝐴 )⋅𝑃(𝐴 ) 𝑖 𝑖 𝑖 𝑃(𝐴 |𝐵) = = 𝑖 𝑃(𝐵) ∑𝑛 𝑃(𝐵|𝐴 )⋅𝑃(𝐴 ) 𝑗=1 𝑗 𝑗

The derivation of Bayes’ Theorem starts from the definition of joint probability: P(A ∩ B) = P(B|A) * P(A). This shows how the occurrence of A relates to the occurrence of B. Using the total probability theorem, which accounts for all possible events A, we sum over all mutually exclusive outcomes, giving P(B) as the total probability. This forms the basis for expressing the posterior probability P(A|B), correlating the joint probability of A and B to the individual probabilities and likelihoods involved.

Think of detective work. When investigating a crime (Event B), detectives rely on prior knowledge (Event A explanations) to narrow down their suspect list (prior probability). They gather evidence (using probabilities) about suspects to figure out how likely each person is the actual criminal, just as Bayes’ theorem helps us refine pathologies by using existing evidence.

• Prior Probability 𝑃(𝐴 ): Our belief in event 𝐴 before evidence. 𝑖 𝑖 • Likelihood 𝑃(𝐵|𝐴 ): How probable is the evidence 𝐵 given that 𝐴 is true. 𝑖 𝑖 • Posterior Probability 𝑃(𝐴 |𝐵): Updated belief in 𝐴 after observing 𝐵. 𝑖 𝑖

The interpretation of the terms in Bayes' Theorem is essential for understanding its practical application. The prior probability (P(A)) represents what we believe about the probability of A occurring before we have any evidence. The likelihood (P(B|A)) describes how probable the evidence B would be if A were true. The posterior probability (P(A|B)) then captures our updated belief in A, taking into account the new evidence B. This progressive refinement of understanding based on new information is a hallmark of Bayesian thinking.

Consider a weather forecast. Initially, the probability of rain tomorrow (prior probability) might be set at 30%. If you observe that the sky is visibly cloudy (likelihood), this information updates your belief about rain tomorrow (posterior probability), which might increase to 70%. This process mirrors the systematic update presented by Bayes’ Theorem.

Example: Suppose a disease affects 1% of the population. A diagnostic test has: • True Positive Rate = 99% • False Positive Rate = 5% Find the probability that a person actually has the disease given a positive test. Let: • 𝐷: Has the disease • 𝐷′: Does not have the disease • 𝑇: Test is positive Given: • 𝑃(𝐷) = 0.01, 𝑃(𝐷′)= 0.99 • 𝑃(𝑇|𝐷) = 0.99, 𝑃(𝑇|𝐷′)= 0.05 Now, use Bayes’ Theorem: 𝑃(𝑇|𝐷) ⋅𝑃(𝐷) 𝑃(𝐷|𝑇) = 𝑃(𝑇|𝐷)⋅𝑃(𝐷)+𝑃(𝑇|𝐷′)⋅𝑃(𝐷′) 0.99⋅0.01 0.0099 0.0099 = = = ≈ 0.1667 0.99⋅0.01+ 0.05⋅0.99 0.0099+ 0.0495 0.0594 Conclusion: There’s only a 16.67% chance the person actually has the disease even after testing positive.

This example illustrates how Bayes' Theorem is applied in real-world situations. We start by noting that only 1% of the population has the disease (prior probability). The test is highly accurate for those with the disease (true positive rate of 99%) but has a 5% false positive rate for those without the disease. Using the provided probabilities, we set up the Bayes' Theorem formula to compute the posterior probability. The result shows that despite a positive test result, the likelihood that the individual truly has the disease is only about 16.67%. This highlights the importance of interpreting diagnostic tests cautiously, as initial probabilities can significantly shift results.

Think about the scenarios in our daily lives where we receive 'positive' indications of something being beneficial - like a glowing review of a restaurant. Just because a few people have enjoyed it (test positive) doesn’t guarantee that it will be good for you, particularly if past experiences suggest only a small percentage like it (the prior probability). It’s critical to consider the full context rather than just a positive indication.

Bayes’ Theorem is applied in various real-world and PDE-related contexts:

1. Signal Processing: Estimating noise-reduced signals using Bayes’ filter.
2. Inverse Problems: Reconstructing unknown sources or conditions in a PDE from observed data using Bayesian Inference.
3. Machine Learning: Algorithms like Naive Bayes classifiers and probabilistic models.
4. Structural Reliability: Estimating the likelihood of system failure under uncertainty.
5. Medical Imaging (PDE + Bayes): Inferring organ boundaries or tumor locations from PDE-modeled signals like MRI.

Bayes’ Theorem has numerous applications across various fields, particularly engineering and computational science. In signal processing, it is used to filter out noise, enhancing the clarity of a signal. In addressing inverse problems, Bayes' approach helps to deduce unknown factors in complex systems modeled by partial differential equations (PDEs) based on observed data. This includes important uses in machine learning algorithms, structural reliability assessments for predicting when systems might fail, and medical imaging, which aids in visualizing internal structures through sophisticated modeling that integrates Bayes' Theorem into imaging processes.

Think about how a mechanic diagnoses a car issue. They start with some known facts (prior knowledge) but may perform tests (observed data) to clarify what's wrong (inverse problem). Bayes’ Theorem guides them by weighing various pieces of information, just like how engineers use Bayes’ Theorem to improve systems and make predictions based on uncertain data paths.

In the continuous domain, the formula becomes: 𝑓 (𝑏|𝑎)⋅𝑓 (𝑎) 𝐵|𝐴 𝐴 𝑃(𝐴|𝐵) = 𝑓 (𝑏) 𝐵 Where: • 𝑓 : Conditional density 𝐵|𝐴 • 𝑓 : Prior density 𝐴 • 𝑓 : Marginal density of evidence 𝐵 This is widely used in Bayesian statistics and data assimilation techniques in simulations involving PDEs.

When dealing with continuous random variables, Bayes' Theorem adapts to incorporate probability density functions instead of discrete probabilities. The formula indicates how the conditional density f(B|A) is scaled by the prior density f(A) to yield the posterior density f(A|B). This transformation allows for the same logic of Bayesian updating to apply in contexts where variable outcomes are not confined to finite possibilities, facilitating simulations and analyses in areas involving partial differential equations and Bayesian statistics.

Imagine pouring a drink: the density of liquid represents a continuous probability. As you pour (gather evidence of the outcome), how full your glass is can depend on how much you originally intended to fill (prior belief). Bayes' Theorem in continuous space allows you to 'adjust' your understanding about how likely that glass will remain full or spill based on how much you see pouring as opposed to just considering the original trip to get the drink.

Bayes’ Theorem is a cornerstone of probabilistic inference and is especially powerful in decision-making under uncertainty. It enables us to update prior beliefs in light of new evidence and is vital in fields ranging from machine learning to inverse problems in PDEs. Mastery of this theorem enhances analytical thinking and offers a probabilistic framework to approach real-world and computational challenges in engineering.

In summary, Bayes' Theorem represents a fundamental principle of updating probabilities based on new evidence, crucial in decision-making processes featuring uncertainty. It fosters the ability to integrate prior knowledge with new observations across multiple domains such as machine learning, engineering, and statistics. By mastering this theorem, individuals sharpen their analytical skills and gain a valuable mental tool for solving complex, uncertain situations systematically.

Consider it like a GPS system recalibrating your route based on real-time traffic. Initially, you might have a time estimate based on average speeds (prior belief). However, as you receive new data about traffic jams or open roads (evidence), the GPS updates your estimated time of arrival—similar to how Bayes' Theorem updates beliefs with new information.

--