17.3. Independence of Random Variables

Two random variables 𝑋 and 𝑌 are independent if the occurrence of one does not affect the probability distribution of the other.

Independence between two random variables means that knowing the result of one does not change the outcome of the other. For instance, if 𝑋 represents the number rolled on a die and 𝑌 represents the toss of a coin, knowing the result of the die roll gives no information about the coin toss result. This is crucial in probability theory because it allows us to analyze and simplify complicated problems involving multiple variables.

Imagine you're throwing two separate dice. The outcome of the first die does not influence the outcome of the second die. If you roll a 4 on the first die, it doesn't change the chances of rolling a 1, 2, 3, 4, 5, or 6 on the second die. This situation illustrates independence: the results of the two rolls are completely unrelated.

For discrete variables: 𝑃(𝑋 = 𝑥,𝑌 = 𝑦) = 𝑃(𝑋 = 𝑥) ⋅ 𝑃(𝑌 = 𝑦)

The mathematical formula states that for discrete random variables, the probability of both events happening together (i.e., 𝑋 takes on value 𝑥 and 𝑌 takes on value 𝑦) is equal to the product of their individual probabilities. This can be checked for all possible values of 𝑥 and 𝑦, and if the equality holds, then the two variables are independent.

Consider the example of drawing two cards from two separate decks. Let 𝑋 be the outcome of the first card and 𝑌 the outcome of the second. The chance of drawing a king from the first deck and a queen from the second deck can be found by calculating the chance of drawing a king from the first deck (which is 4 out of 52 cards) and multiplying it by the chance of drawing a queen from the second deck (also 4 out of 52 cards). Their independence allows us to multiply these probabilities: P(king) × P(queen) = (4/52) × (4/52).

For continuous variables: 𝑓(𝑥,𝑦) = 𝑓(𝑥)⋅𝑓(𝑦)

For continuous random variables, the independence is expressed through probability density functions (PDFs). The joint PDF of the random variables 𝑋 and 𝑌 can be factored into the product of their individual PDFs. This indicates that the probability of both variables falling within a certain range can also be calculated by simply multiplying the individual probabilities across those ranges.

Imagine you're measuring the height of two unrelated people. Let 𝑋 be the height of person A and 𝑌 of person B. The joint probability density of their heights can be represented by multiplying the individual densities. Thus, if the height guidelines for each are independent, knowing person A's height does not inform us about person B's height and we can calculate probabilities separately and multiply them together.

This means the joint distribution equals the product of the marginal distributions.

In summary, when variables are independent, the joint distribution is equal to the product of their individual distributions. This is a powerful concept as it simplifies the analysis of multiple variables. It allows statisticians and mathematicians to deal with each random variable separately, making calculations more manageable.

Think of a jar filled with different colored marbles where you randomly draw one marble at a time. If each draw is independent, the probability of drawing a red marble does not change regardless of the previous draws. If you multiply the individual probabilities of drawing colored marbles, it reflects all possible outcomes without any influence from previous draws, showcasing independence in choices.

--