11.3.1. Definition

A moment generating function 𝑀(𝑡) of a random variable 𝑋 is defined as:

𝑀(𝑡) = 𝐸[𝑒^{𝑡𝑋}] provided the expectation exists for 𝑡 in some neighborhood of 0.

The moment generating function (MGF) of a random variable provides a way to capture all the moments of that variable. Mathematically, it is defined as the expected value of the exponential function raised to the power of the random variable multiplied by a number 't'. The notation E[ ] represents the expected value, which means the average outcome if we were to repeat a random experiment many times. This definition states that the MGF is valid as long as we can calculate the expectation for values of 't' near to zero.

Imagine you are investigating the performance of a machine over time. The MGF can be thought of as a ‘snapshot’ that captures all aspects of the machine's performance — its average output, variations, and potential risks — through a single formula. Just like taking a picture can reveal a lot about a moment in time, the MGF summarizes essential information about the random variable's behavior.

Properties of MGFs

1. Existence: If the MGF exists, it uniquely determines the distribution.

2. Derivatives: 𝑑^{𝑟}𝑀(𝑡) 𝑀^{(𝑟)}(0) = 𝐸[𝑋^{𝑟}]

   Hence, the r-th moment of 𝑋 is the r-th derivative of the MGF evaluated at 𝑡 = 0.

3. Additivity: For independent random variables 𝑋 and 𝑌: 𝑀_{𝑋+𝑌}(𝑡) = 𝑀_{𝑋}(𝑡) ⋅ 𝑀_{𝑌}(𝑡)

The properties of moment generating functions highlight their importance in probability theory. 1. Existence: If we can find an MGF for a random variable, it will provide a unique mapping to its underlying probability distribution, meaning no other distribution will have the same MGF. 2. Derivatives: The value of the r-th derivative of the MGF at 't = 0' yields the r-th moment of the distribution. This helps us easily compute moments without working directly with probabilities. 3. Additivity: If you have two independent random variables, the MGF of their sum can be computed simply by multiplying their individual MGFs. This property significantly simplifies calculations involving combined distributions.

Consider two independent factories producing components. The MGF of each factory can represent their production outputs. If you wanted to estimate the total output when both factories operate together, instead of calculating the total production for various scenarios, you can simply multiply their MGFs to get a comprehensive overview. This is akin to finding the overall picture from two separate snapshots.

--