3.2.2. Probability Space

A probability space consists of three elements:

• Sample space (S): The set of all possible outcomes.
• Set of events (F): A collection of subsets of S, including S and the empty set.
• Probability function (P): A function that assigns a probability to each event in F.

A probability space establishes a framework for understanding events and their likelihoods in probability theory. The three main components are:

1. Sample Space (S): This is the complete set of all possible outcomes of a probability experiment. For example, if you flip a coin, the sample space is {H, T} — representing heads and tails.

2. Set of Events (F): This includes various events that can occur within the sample space. Events are subsets of the sample space. For instance, the event of flipping heads can be represented as {H}, and you could also have the event of flipping either heads or tails represented as {H, T}.

3. Probability Function (P): This function assigns probabilities to each event in the set of events. The probability must follow the rules laid out by Kolmogorov’s axioms, ensuring that probabilities are non-negative and that their total for a sample space equals one.

Think of a bag of different colored balls. If you have:

• Sample Space (S): The set of colors {Red, Blue, Green, Yellow} represents all outcomes.
• Set of Events (F): You could have events like drawing a Red ball {Red} or drawing any color {Red, Blue, Green, Yellow}.
• Probability Function (P): The chance of drawing any of these balls is calculated based on how many of each color are in the bag, e.g., if there are 2 red balls out of 10 total, then P({Red}) = 2/10 or 0.2.

Let P:F → [0,1] be a probability function. Then it must satisfy:

• Axiom 1 (Non-negativity): P(E) ≥ 0 for every event E ∈ F
• Axiom 2 (Normalization): P(S) = 1
• Axiom 3 (Additivity): If E1, E2, E3,… are mutually exclusive events, then:

∞ P(⋃Ei) = ∑P(Ei) (i=1 to ∞)

Kolmogorov’s axioms are the foundational rules for a valid probability function:

1. Non-negativity: This ensures that the probability assigned to any event cannot be negative. This makes sense intuitively, as probability is a measure of likelihood, and we cannot measure a negative likelihood.

2. Normalization: The total probability of all possible outcomes in the sample space must equal 1. This makes sense because one of the outcomes must occur when you conduct a probability experiment. For instance, when tossing a coin, you must get either heads or tails, so P({H, T}) = 1.

3. Additivity: If you have several mutually exclusive outcomes (events that cannot happen at the same time), then the probability of any of these outcomes occurring is the sum of their separate probabilities. For example, if you roll a die, the probability of rolling a 2 or a 3 is the sum of the individual probabilities of each event occurring, as these two events cannot occur simultaneously.

Imagine you are betting on different races. For two horses to win one specific race (say Horse A and Horse B), the additivity rule applies: if P(Horse A wins) = 0.3 and P(Horse B wins) = 0.4, if you're betting on either one to win the bet, you are looking at the odds of 0.3 + 0.4 = 0.7. Just like the principles of exclusive outcomes, you cannot have both horses win the same race.

Example: Tossing a fair coin Let S = {H, T} Define F = {∅, {H}, {T}, {H, T}} Assign P({H}) = 0.5, P({T}) = 0.5. This satisfies all three axioms:

• Non-negativity: 0.5 ≥ 0
• Normalization: P({H, T}) = 0.5 + 0.5 = 1
• Additivity: P({H} ∪ {T}) = P({H}) + P({T})

In the example of tossing a fair coin, we can clearly define the different elements of a probability space:

• Sample Space (S): The possible outcomes are {H, T} — heads or tails.
• Set of Events (F): Includes the empty set (∅), the outcome of heads ({H}), the outcome of tails ({T}), and the entire sample space ({H, T}).
• Probability Function (P): Each outcome (heads or tails) is equally likely, so we assign P({H}) = 0.5 and P({T}) = 0.5. This meets all three axioms:
  • Non-negativity: Both 0.5 probabilities are greater than or equal to 0.
  • Normalization: The total probabilities sum to 1 (0.5 + 0.5 = 1).
  • Additivity: The probability of getting heads or tails combines as required (P({H} ∪ {T}) = P({H}) + P({T})).

If you think about a simple coin toss, it’s like making a choice between two friends to invite to a game. There’s a fair chance you pick one friend or the other. If both friends are equally likely to get invited, you can think of it as your probability function assigning equal chances (0.5) for either friend to be chosen, ensuring your total friends invited always equals two options.

• Applicable to both finite and infinite sample spaces. • Handles both discrete and continuous cases. • Can model real-world probabilities where outcomes are not equally likely.

The axiomatic definition of probability offers several significant advantages over classical probability definitions, including:

1. Applicability to Finite and Infinite Sample Spaces: It provides flexibility to handle not just simple experiments but also complex scenarios where the outcomes could be infinite, such as the results of continuous measurements.

2. Ability to Handle Discrete and Continuous Cases: Whether you're dealing with a set number of outcomes (like rolling a die) or continuous outcomes (like measuring height), the axiomatic definition is robust enough to accommodate both types.

3. Modeling Real-World Probabilities: Many real-world situations involve uncertainty and complexities where outcomes are not equally likely. The axiomatic framework allows for the assignment of probabilities that reflect actual conditions, such as weather forecasts where the chances of rain might be 70% and no rain 30%, which doesn't conform to the classical equally likely assumption.

Think about weather forecasting: modern forecasts provide probabilities such as a 70% chance of rain. In this case, those probabilities are derived from complex models considering various infinite factors, including atmospheric conditions, historical weather patterns, and current data. The axiomatic approach helps accurately assign these probabilities rather than simply assuming all outcomes are equal, like it often happens in classical models.

The classical definition is a special case of the axiomatic definition where all outcomes are equally likely, and the sample space is finite.

The relationship between the classical and axiomatic definitions of probability is that the classical definition can be seen as a specific scenario within the broader axiomatic framework. In the classical definition:

• All outcomes are equally likely — a key assumption that limits the types of problems it can address.
• The sample space is finite, making it suitable for straightforward problems like calculating the probability of rolling a six on a die.

Therefore, any situation quantified by the classical definition can be expressed using the axiomatic definition, which also incorporates scenarios of unequal likelihood or infinitely many outcomes.

Imagine rolling a fair die: classical definition tells us P(rolling a six) = 1/6 because each side is equal. However, if we include a trick die that has more weight on six, we’d shift to evaluating that through the axiomatic definition, allowing us to adapt our understanding of probability based on unequal likelihoods in a situation that wouldn’t fit classical models.

--