Graphical Modeling

Some Probability Concepts

Probabilities are used in everyday life and determine our decision-making processes. For example, the chance of rain informs one’s plans for the weekend. Each time we model the real-world uncertainty, there will be some underlying random experiment (such as flipping a coin to decide whether to cycle or drive to work). If the experiment (e.g., the toss of a coin) is repeated a very large (in theory infinite) number of times, the event happens roughly a fraction p of the time (e.g., half of the time we will get heads); the larger the number of repetitions, the closer we will get to the true probability of the event.

For a random experiment (e.g., the toss of a fair coin), the sample space is the set of all possible outcomes (e.g., heads and tails), an event (e.g., heads) is a subset of the sample space, and the probability of the event will be a number between 0 and 1 (in the fair coin example, p (heads) = p (tails) = ½).

A random variable, usually denoted by a capital letter, say X, is a variable of which the possible values are the numerical outcomes of a random experiment (in the fair coin example, if X is the number of heads in one toss of the coin, X can take the values 0 and 1).

Statistical inference, in its simplest form, uses an observation to draw conclusions about some unknown quantity. Both the unknown quantity and the observation are represented here by random variables and the modeling objective is to decide whether and in what way the two random variables relate. For the events of getting heads in the toss of a coin, and rain the following day, X represents the number of heads in the toss of a coin (X = 0 or X = 1), and Y is an indicator random variable that it will rain tomorrow (Y = 0 or Y = 1). Then, the observation that the random variable X takes on a specific value (e.g., X = 0) is not expected to affect the random variable Y. In this case, X and Y are independent; conditional on the observed value of X, the probabilities of the values of Y remain unchanged.

...