14. Partial Differential Equations

In many real-world scenarios, we deal with more than one random variable simultaneously. For example, in engineering and science problems, we may study the temperature and pressure in a system at the same time. In such cases, we need a way to describe the relationship between two or more random variables. This is where Joint Probability Distributions become important. Joint probability distributions help us understand the probability structure involving two or more random variables and allow us to compute the probabilities of combinations of events. This topic is foundational for advanced concepts in statistics, data science, machine learning, and stochastic processes.

Joint probability distributions allow us to analyze situations where multiple random variables affect outcomes simultaneously. For instance, imagine researching how the temperature and pressure in a gas storage tank relate. By utilizing a joint probability distribution, we can evaluate how frequently specific combinations of temperature and pressure occur together, enabling us to develop models and predictions in fields such as engineering and statistics.

Think of it like mixing colors. If you have two colors, say blue and yellow, the joint probability distribution would enable you to understand how often you get green (the combination of both colors) at different amounts of blue and yellow. In practice, when dealing with temperature and pressure, knowing how they vary together helps predict when certain outcomes, like a gas leak, might happen.

A random variable is a function that assigns a real number to each outcome in a sample space. • Discrete Random Variable: Takes countable values. • Continuous Random Variable: Takes an uncountable range of values (often intervals of real numbers).

A random variable functions as a systematic way of quantifying outcomes of random phenomena. A discrete random variable might represent the number of students in a classroom – a countable number, while a continuous random variable represents measurements, like time or temperature, which take an infinite range of values. Understanding these types is crucial for formulating probability distributions.

Consider a dice roll. The number you roll (1 through 6) is a discrete random variable because it can only take on whole number values. Now, think of measuring the amount of rain in a week. This measurement can include any value like 0.5 inches, 1.75 inches, etc., which makes it a continuous random variable.

A Joint Probability Distribution describes the probability behavior of two or more random variables simultaneously. • For Discrete Variables: The joint probability mass function (pmf) 𝑃(𝑋 = 𝑥,𝑌 = 𝑦) gives the probability that 𝑋 = 𝑥 and 𝑌 = 𝑦. • For Continuous Variables: The joint probability density function (pdf) 𝑓 (𝑥,𝑦) satisfies: 𝑃((𝑋,𝑌) ∈ 𝐴) = ∬𝑓 (𝑥,𝑦) 𝑑𝑥 𝑑𝑦 where 𝐴 is a region in the 𝑥𝑦-plane.

Joint probability distributions let us investigate the relationships between multiple random variables by defining how they behave together. For discrete variables, the probability mass function (pmf) quantifies specific outcomes. For continuous variables, the probability density function (pdf) does this in a broader range, allowing integration over areas rather than just points.

Imagine you’re assessing the joint likelihood of two students scoring grades in their math and science classes. For example, if one student gets an A in math, what's the chance they also get a B in science? The joint probability tells you this by considering both subjects at once, much like a weather forecast that presents the probability of rain and wind occurring together.

3.2.1 For Discrete Random Variables

1. 𝑃(𝑋 = 𝑥,𝑌 = 𝑦) ≥ 0
2. ∑ ∑ 𝑃(𝑋 = 𝑥,𝑌 = 𝑦) = 1 3.2.2 For Continuous Random Variables
3. 𝑓 (𝑥,𝑦) ≥ 0
4. ∬ 𝑓 (𝑥,𝑦) 𝑑𝑥 𝑑𝑦 = 1

The properties of joint distributions ensure that all assigned probabilities are valid and complete. For discrete variables, the probability values must be non-negative, and their total across all possible outcomes must equal 1. Similarly, for continuous variables, the function should also be non-negative and the total area under the curve (integral) must equal 1. These properties validate our probability models.

Think of it like scoring points in a game. If you cannot score a negative number of points (non-negativity), and if all possible scores add up to exactly 100 points at the end of the game (equal to 1), then your scoring system is valid. Just as with these properties, we want probabilities to be logically sound in our calculations.

To study individual distributions from a joint distribution, we use marginal distributions. 3.3.1 Marginal PMF (Discrete) • Marginal PMF of 𝑋: 𝑃 (𝑥)= ∑𝑃(𝑋 = 𝑥,𝑌 = 𝑦) 𝑋 𝑦 • Marginal PMF of 𝑌: 𝑃 (𝑦)= ∑𝑃(𝑋 = 𝑥,𝑌 = 𝑦) 𝑌 𝑥 3.3.2 Marginal PDF (Continuous) • Marginal PDF of 𝑋: ∞ 𝑓 (𝑥) = ∫ 𝑓 (𝑥,𝑦) 𝑑𝑦 𝑋 𝑋,𝑌 −∞ • Marginal PDF of 𝑌: ∞ 𝑓 (𝑦) = ∫ 𝑓 (𝑥,𝑦) 𝑑𝑥 𝑌 𝑋,𝑌 −∞

Marginal distributions allow us to extract the individual behavior of one variable while ignoring the influence of others. For discrete variables, the marginal probability mass function (PMF) is computed by summing probabilities over the other variables. For continuous variables, we integrate over the range of the other variables to find the marginal probability density function (PDF). This operation simplifies the analysis of one random variable at a time.

Consider a student’s performance data where you have scores in math and science. A marginal distribution would allow you to determine just the performance in math without considering the science scores. It's like looking at the score of an individual game in a season to understand that player's individual performance rather than the entire team's score.

These describe the distribution of one variable given a fixed value of the other. 3.4.1 Conditional PMF 𝑃(𝑋= 𝑥,𝑌 = 𝑦) 𝑃(𝑋 = 𝑥 ∣ 𝑌 = 𝑦) = 𝑃 (𝑦) 𝑌 3.4.2 Conditional PDF 𝑓 (𝑥,𝑦) 𝑓 (𝑥 ∣ 𝑦) = 𝑋|𝑌 𝑓 (𝑦) 𝑌

Conditional distributions focus on how the occurrence of one random variable influences the probabilities of another. The conditional probability mass function (PMF) in discrete cases measures how likely a specific value of one variable is given the value of another. Similarly, the conditional probability density function (PDF) serves the same purpose in continuous cases.

Imagine you're trying to predict a child’s height based on whether their parents are tall or short. The height of the child (random variable) will likely be influenced by the parent's height (fixed value). Conditional distributions help you quantify that relationship, just as a weather app predicts temperature based on the current cloud conditions.

Two random variables 𝑋 and 𝑌 are independent if: • Discrete Case: 𝑃(𝑋 = 𝑥,𝑌 = 𝑦) = 𝑃 (𝑥)⋅𝑃 (𝑦) 𝑋 𝑌 • Continuous Case: 𝑓 (𝑥,𝑦) = 𝑓 (𝑥)⋅𝑓 (𝑦)

Independence of random variables signifies that the occurrence of one variable has no effect on the other. For discrete variables, the joint probability equals the product of their individual probabilities. For continuous variables, the joint density is the product of their individual densities, facilitating simpler calculations as we treat each variable independently.

Imagine flipping a coin and rolling a die. The coin's outcome doesn't influence the die's result, which means they are independent random variables. You’d calculate probabilities of getting heads and a three on the die simultaneously by simply multiplying their individual probabilities, illustrating how independent events work together.

3.6.1 Expectation (Mean) • Discrete: 𝐸[𝑋] = ∑ ∑𝑥 ⋅𝑃(𝑋 = 𝑥,𝑌 = 𝑦) 𝑥 𝑦 • Continuous: 𝐸[𝑋] = ∬𝑥 ⋅𝑓 (𝑥,𝑦) 𝑑𝑥 𝑑𝑦 𝑋,𝑌 3.6.2 Covariance Cov(𝑋,𝑌) = 𝐸[𝑋𝑌]−𝐸[𝑋]𝐸[𝑌] If Cov(𝑋,𝑌) = 0, it implies that 𝑋 and 𝑌 are uncorrelated. However, uncorrelated does not imply independence unless the joint distribution is normal (Gaussian).

Expectation serves as the expected average of a random variable and is commonly referred to as the mean. The formulas differ for discrete and continuous variables as they calculate the weighted outcomes based on their probabilities. Covariance measures the degree to which two random variables change together. If covariance equals zero, it suggests no linear relationship between the two variables, though it doesn't guarantee they are independent without additional context.

If you consider a team's performance where the average score (expectation) is crucial, knowing the expectation helps determine the team's performance over a season. Covariance acts like a friendship level indicator – if two friends tend to share experiences (high covariance), they're likely correlated, but if they have little in common (zero covariance), their interaction level might not affect each other significantly.

Cov(𝑋,𝑌) 𝜌 = 𝑋,𝑌 𝜎 𝜎 𝑋 𝑌 • 𝜌 ∈ [−1,1] • 𝜌 = 1 or −1: perfect linear relationship • 𝜌 = 0: no linear relationship

The correlation coefficient (denoted as ρ) quantifies the degree and direction of linear relationship between two random variables. A value of 1 or -1 indicates a perfect positive or negative linear correlation, respectively, while 0 indicates no linear relationship. This measure is critical in identifying whether two variables move together in predictable patterns.

Consider how height and weight relate to one another. When you plot these measurements, a correlation coefficient close to 1 indicates that as height increases, weight tends to increase consistently. Conversely, if height and shoe size had a correlation near zero, it suggests no meaningful relationship in this context, much like two friends who rarely see each other.

Example 1: Discrete Case Given: 𝑃(𝑋 = 𝑥,𝑌 = 𝑦) = for (𝑥,𝑦) ∈ {(0,0),(0,1),(1,0),(1,1)} 8 • Find marginal distributions • Check independence Solution: 1 1 1 • 𝑃 (0)= 𝑃(0,0)+𝑃(0,1)= + = 𝑋 8 8 4 1 1 1 • 𝑃 (0)= 𝑃(0,0)+ 𝑃(1,0) = + = 𝑌 8 8 4 1 1 1 1 • Since 𝑃(0,0) = ≠ 𝑃 (0)⋅𝑃 (0) = ⋅ = , not independent. 𝑋 𝑌 8 4 4 16 Example 2: Continuous Case Let: 𝑓(𝑥,𝑦) = 4𝑥𝑦 for 0 ≤ 𝑥 ≤ 1, 0 ≤ 𝑦 ≤ 1 • Check validity • Find marginal distributions Solution: • Check total probability: 1 1 ∬𝑓(𝑥,𝑦) 𝑑𝑥 𝑑𝑦 = ∫ ∫ 4𝑥𝑦 𝑑𝑥 𝑑𝑦 = 1 ✓ 0 0 • Marginal of 𝑋: 1 1 𝑓 (𝑥) = ∫ 4𝑥𝑦 𝑑𝑦 = 4𝑥⋅ = 2𝑥 𝑋 2 0 • Marginal of 𝑌: 1 1 𝑓 (𝑦) = ∫ 4𝑥𝑦 𝑑𝑥 = 4𝑦 ⋅ = 2𝑦 𝑌 2 0 • Since 𝑓(𝑥,𝑦) = 𝑓 (𝑥)⋅𝑓 (𝑦), 𝑋 and 𝑌 are independent.

Through example problems, we see practical applications of joint probability distributions. In the discrete case, we derived marginals and checked for independence by verifying if the joint probability matched the product of the marginals. In the continuous case, we checked the joint probability function's validity and calculated marginal distributions, confirming independence between the variables. Working through specific examples solidifies understanding by applying theoretical concepts.

These examples are like scenarios in a sports league where you analyze teams' win-loss records (discrete) compared to player performance metrics (continuous). By checking if winning games relates to a player’s statistics, we can understand how team success interacts with individual performances, thus applying statistical concepts to tangible outcomes.

--