If there are only two possible outcomes in a trial, it is often observed around us to count the number of one outcome when the trial is repeated several times as follows:

Probabilities can be obtained for various events in the sample space, but what we are more interested in is the events for the number of heads and their probabilities such as tossing a coin two times as follows:

Event of no head	[TT]	P({TT}) = \(\frac{1}{4}\)
Event of one head	{TH, HT}	P({TH, HT}) = \(\frac{2}{4}\)
Event of two heads	{HH}	P({HH}) = \(\frac{1}{4}\)

If \(X\) is the number of heads of the coin, the possible number of \(X\) is 0, 1, and 2. In other words, \(X\) is one-to-one correspondence from each element in the sample space \(S\) to one number as follows:

Sample space	\(X\) = Number of heads when tossing a coin two times
{TT}	0
{TH} {HT}	1
{HH	2

The one-to-one relation in which a single real number is matched to each element in the sample space is called the random variable. Random variables are usually denoted with uppercase letters \(X, Y, Z \) ... and values of the random variables are denoted with lowercase letters \(x, y, z \) ... . When a random variable has finite values or countable numbers as a natural number, the random variable is called the discrete random variable, and when it has an arbitrary real value, the random variable is called the continuous random variable. For example, the number of heads that appear by tossing two coins is a discrete random variable, and the delivery time to home when we order a pizza is a continuous random variable because it can have any positive real value.

If the random variable \(X\) is the number of heads when a coin is tossed twice, the probability of event which has no head, P({TT}) = \(\frac{1}{4}\), can be written as \(P(X=0) = \frac{1}{4}\). Similarly, the probability of events which has one head, P({TH, HT}) = \(\frac{2}{4}\), as \(P(X=1) = \frac{2}{4}\), and the probability of event which has two heads, P({HH}) = \(\frac{1}{4}\)\(\frac{1}{4}\), as \(P(X=2) = \frac{1}{4}\). These probabilities can be summarized in a table as follows:

Sample space	\(X\) = Number of heads	\(P(X=x)\)
{TT}	\(x = 0\)	\(\frac{1}{4}\)
{TH} {HT}	\(x = 1\)	\(\frac{2}{4}\)
{HH}	\(x = 2\)	\(\frac{1}{4}\)

The probability distribution of a discrete random variable \(X\) is a summary of probabilities that \(X\) has a certain value \(x\), \(P(X=x)\), as shown in this table. The above probability distribution can be expressed graphically as in <Figure 3.1>.

In general, if a discrete random variable \(X\) can have values \(x_1 , x_2 , \cdots , x_n\) and their corresponding probabilities are \(p_1 , p_2 , \cdots , p_n\) respectively, it can be summarized as one-to-one relation as follows and it is called the probability distribution of \(X\).

\(X\)	\(x_1\)	\(x_2\)	\(\cdots\)	\(x_n\)	Total
\(P(X=x_i )\)	\(p_1\)	\(p_2\)	\(\cdots\)	\(p_n\)	1

It may also be represented as a mathematical function and it is called the probability density function of \(X\).

Among discrete random variables, the most widely used in reality is the binomial distribution, which will be studied in detail in Section 3.4.

According to the basic properties of probability, the probability distribution function has the following properties.

The probability that a random variable \(X\) has a value between \(a\) and \(b\) is represented as \(P(a \le X \le b)\).

The probability distribution of the random variable \(X\) = 'the number of heads' when a coin tossed twice is as follows:

\(X\)	0	1	2	Total
\(P(X=x)\)	\(\frac{1}{4}\)	\(\frac{2}{4}\)	\(\frac{1}{4}\)	1

This probability distribution is similar to the frequency table of three possible values 0, 1, and 2 with the relative frequency.

Value	Frequency	Relative frequency
0	1	\(\frac{1}{4}\)
1	2	\(\frac{2}{4}\)
2	1	\(\frac{1}{4}\)
Total	4	1

The mean \(\mu\) of these numbers 0, 1, 1, 2 and the variance \(\sigma^2\) are as follows:

Therefore, the standard deviation \(\sigma\) is \(\sqrt{0.5} = 0.707 \).

In general, if a discrete random variable \(X\) can have values as \(x_1 , x_2 , \cdots , x_n\) and their probabilities are \(p_1 , p_2 , \cdots , p_n\) respectively, then the probability distribution is as follows

\(X\)	\(x_1\)	\(x_2\)	\(\cdots\)	\(x_n\)	Total
\(P(X=x_i )\)	\(p_1\)	\(p_2\)	\(\cdots\)	\(p_n\)	1

The mean or expectation of the random variable \(X\) denoted as \(E(X)\) or \( \mu \), the variance denoted as \(V(X)\) or \(\sigma^2 \) and the standard deviation denoted as \(\sigma(X)\) or \(\sigma\) are calculated as follows:

The mean or expectation of the random variable \(X\), \(E(X)\), which can have possible values \(x_1 , x_2 , ... , x_n \) with their probabilities \(p_1 , p_2 , ... , p_n \) can be considered as a weighted average of values using the weight of probabilities. The variance of \(X\), \(V(X)\), is a weighted average of all squared distances from the mean. The standard deviation of \(X\), \(\sigma(X)\), is an average distance from the mean.

If we expand the variance of a discrete random variable, \(V(X) = \sigma^2 = \sum_{i=1}^{n} (x_i - \mu)^2 p_i\), it becomes as follows:

Therefore, the variance of a random variable \(X\) can be calculated using a short cut formula as follows:

Using this short cut formula, the variance of [Example 3.3] can be calculated as follows:

If \(X\) = 'the number of heads' when a coin is tossed twice, the above game is the same as the problem to examine the probability distribution of 1) \(Y = 100X\) and 2) \(Z = 50 + 100X\). The probability distribution of the random variable \(X\) is as follows and \(E(X)\) = 1, \(V(X)\) = 0.5, \(\sigma(X)\) = 0.707.

\(X\)	0	1	2	Total
\(P(X=x)\)	\(\frac{1}{4}\)	\(\frac{2}{4}\)	\(\frac{1}{4}\)	1

1) The probability distribution of \(Y = 100X\) is as follows:

\(Y = 100X\)	0	100	200	Total
\(P(Y = y)\)	\(\frac{1}{4}\)	\(\frac{2}{4}\)	\(\frac{1}{4}\)	1

Therefore, the expectation \(E(Y)\), variance \(V(Y)\) and standard deviation \(\sigma(Y)\) are as follows:

That is, \(E(Y) = 100 E(X)\), \(V(Y) = 100^2 V(X)\), \(\sigma(Y) = 100 \sigma(X)\).

2) he probability distribution of \(Z = 50 + 100X\) is as follows:

\(Z = 50 + 100X\)	50	150	250	Total
\(P(Y = y)\)	\(\frac{1}{4}\)	\(\frac{2}{4}\)	\(\frac{1}{4}\)	1

Therefore, the expectation \(E(Z)\), variance \(V(Z)\), standard deviation \(\sigma(Z)\) are as follows:

That is, \(E(Z) = 50 + 100 E(X)\), \(V(Z) = 100^2 V(X)\), \(\sigma(Z) = 100 \sigma(X)\).

We noticed that if 50 is added to , the variance of \(Z\) is not affected, but if \(X\) is multiplied by 100, the variance of \(Z\) is increased by \(100^2\).

In general, assume that the probability distribution of a discrete random variable \(X\) is as follows and its mean, variance and standard deviation are \(E(X) = \mu\), \(V(X) = \sigma^2\), \(\sigma(X) = \sigma\) respectively.

\(X\)	\(x_1\)	\(x_2\)	\(\cdots\)	\(x_n\)	Total
\(P(X=x )\)	\(p_1\)	\(p_2\)	\(\cdots\)	\(p_n\)	1

The probability distribution of \(Y = aX + b\) is as follows:

\(Y = aX + b\)	\(y_1 = ax_1 + b\)	\(y_2 = ax_2 + b\)	\(\cdots\)	\(y_n = ax_n + b\)	Total
\(P(Y=y )\)	\(p_1\)	\(p_2\)	\(\cdots\)	\(p_n\)	1

Therefore, the expectation \(E(Y)\), variance \(V(Y)\) and standard deviation \(\sigma(Y)\) are as follows:

If the mean, variance, standardard deviation of a random variable \(X\) are \(E(X) = \mu\), \(V(X) = \sigma^2\), \(\sigma(X) = \sigma\) respectively, then the mean, variance and standard deviation of \(Y = aX + b\) are as follows:

This is true even if \(X\) is a continuous random variable.

Using 『Histogram – Frequency Table』 of 『eStatH』, the histogram of pizza delivery time is as <Figure 3.6> if the class interval starts at 0 and the interval size is 10 minutes. It can be seen that the probability that the pizza is delivered between 10 and 20 minutes is 0.47.

Class	Frequency	Relative frequency
0.00 ≤ x < 10.00	1	0.03
10.00 ≤ x < 20.00	14	0.47
20.00 ≤ x < 30.00	13	0.43
30.00 ≤ x < 40.00	2	0.07
Total	30	1.00

If the interval of the histogram starts at 5 and the interval size is 5 minutes, the histogram becomes as <Figure 3.7>. It can be seen that the probability in which the pizza will be delivered between 15 and 20 minutes is 0.30.

Class	Frequency	Relative frequency
0.00 ≤ x < 5.00	1	0.03
10.00 ≤ x < 15.00	5	0.17
15.00 ≤ x < 20.00	9	0.30
20.00 ≤ x < 25.00	8	0.27
25.00 ≤ x < 30.00	5	0.17
30.00 ≤ x < 35.00	1	0.03
35.00 ≤ x < 40.00	1	0.03
Total	30	1.00

In order to obtain the probability of a more detailed interval, a more detailed histogram such as <Figure 3.8> and its frequency table are required as shown below. However, for such a histogram, more data must be collected, and it is cumbersome to redraw the histogram every time to calculate the desired probability.

There are many continuous data around us as the histogram above in which many data are located around the mean, data are symmetrical about the mean, and their shape looks like a bell. In order to easily find the probability for all of this type data, mathematicians searched for a function that can describe these kinds of histograms. This function has the same shape as in <Figure 3.9> and is called the normal distribution function. It is explained in detail in Section 3.5.

If the probability distribution function of a continuous random variable can be expressed as a mathematical function \(f(x)\), it is possible to approximate the desired probability without finding a frequency table and histogram. In general, the probability distribution function \(f(x)\) of a continuous random variable \(X\) has the following properties.

When a product is inspected, if it is a good product, we denote it as G, and if it is a defective product, we denote it as B. When three products are inspected, possible events of the sample space, possible values of the random variable \(X\) which is the number of defective products and their probabilities are as follows:

Sample space	\(X\) = Number of defectives	\(P(X=x)\)
GGG	0	\((0.9)^3\)
GGB GBG BGG	1	\(3 (0.1)(0.9)^2\)
GBB BGB BBG	2	\(3 (0.1)^2(0.9)\)
BBB	3	\( (0.1)^3\)

When the number of defectives is 0, the number of cases is one which is from \({}_3 C_0\). When the number of defectives is 1, the number of cases is three which is from \({}_3 C_1\). When the number of defectives is 2, the number of cases is three which is from \({}_3 C_2\). When the number of defectives is 3, the number of cases is 1 which is from \({}_3 C_3\). Therefore the probability distribution function of the random variable \(X\) can be expressed as follows: $$ P(X=x) = {}_3 C_x (0.1)^x (0.9)^{3-x} \quad (x=0,1,2,3) $$ Similar examples as the above problem, which examines the number of defective products by inspecting the product, are observed frequently around us.

- Toss a coin five times to count the number of heads.
- Count the number of voters in favor of a particular candidate in an election.

Another example can be found at the following instrument as <Figure 3.12> in science museums. This instrument drops a ball from the top and, if the ball hit a bar, it may fall to the left (0 point) or to the right (1 point) with a half chance. Then the ball drops to the next stage bar again and may fall to the left and right with a half chance. We examine the distribution of scores after the second stage when 100 balls are dropped.

What these examples have in common is that there are two possible outcomes of one trial such as {good, defective}, {head, tail}, {approval, opposition}, etc and that this trial is repeated several times. However, the probability of outcomes in each trial is different. The trial with two possible outcomes ('binary') is called the Bernoulli trial, and the outcome of interest among the two is often called 'success' and the rest is called 'failure'. When the Bernoulli trial with probability of 'success' \(p\) is repeated \(n\) times independently, if \(X\) is the random variable of counting the number of 'success', then \(X\) can have values 0, 1, 2, ... , \(n\) and the probability distribution function of \(X\) is as follows: $$ P(X = x) = {}_n C_x p^x (1-p)^{n-x} \quad (x=0,1,2, ... , n) $$ This probability distribution is called the binomial distribution denoted as \(B(n,p)\) and, if we denote \(q=1-p\), the binomial distribution can be expressed as the following table.

\(X\)	0	1	2	\(\cdots\)	\(x\)	\(\cdots\)	n	Total
\(P(X=x)\)	\({}_n C_0 q^n \)	\({}_n C_1 pq^{n-1}\)	\({}_n C_2 p^2 q^{n-2}\)	\(\cdots\)	\({}_n C_x p^x q^{n-x}\)	\(\cdots\)	\({}_n C_n p^n\)	1

Each probability of this binomial distribution is equal to each coefficient of the terms in the expansion of the equation \((q+p)^n\) using the binomial theorem. $$ (q+p)^n = {}_n C_0 q^n + {}_n C_1 pq^{n-1}+ {}_n C_2 p^2 q^{n-2} + \cdots +{}_n C_x p^x q^{n-x}+ \cdots +{}_n C_n p^n $$ The following is a graph of the binomial distribution for various values of \(n\) and \(p\).

If a Bernoulli trial with the probability of 'success' \(p\) and of 'failure' \(q=1-p\) is repeated three times, the random variable \(X\) which counts the number of 'success' is the following binomial distribution.

\(X\)	0	1	2	3	Total
\(P(X=x)\)	\({}_3 C_0 \;q^3 \)	\({}_3 C_1 \;pq^{2}\)	\({}_3 C_2 \;p^2 q\)	\({}_3 C_3 \;p^3\)	1

The random variable has the mean \(E(X)\), the variance \(V(X)\) and the standard deviation \(\sigma(X)\) as follows:

In general, if a random variable \(X\) follows the binomial distribution, \(B(n,p)\), then its probability distribution is as follows:

\(X\)	0	1	2	\(\cdots\)	\(x\)	\(\cdots\)	n	Total
\(P(X=x)\)	\({}_n C_0 q^n \)	\({}_n C_1 pq^{n-1}\)	\({}_n C_2 p^2 q^{n-2}\)	\(\cdots\)	\({}_n C_x p^x q^{n-x}\)	\(\cdots\)	\({}_n C_n p^n\)	1

The mean of \(X\) becomes \(E(X) = np\), the variance becomes \(V(X) = npq\), and the standard deviation becomes \(\sigma(X) = \sqrt{npq}\).

If the value of \(n\) is large, it is not easy to calculate the probability of the binomial distribution even using a calculator or computer. In 『eStatH』, you can find the binomial probability when \(n \le 100\). If \(n\) is more than 100, the probability can be approximated using a normal distribution with mean \(np\) and variance \(np(1-p)\). See details in Section 3.5.

Using the binomial distribution, it can be seen that the mathematical probability of \(\frac{1}{6}\) is a reasonable model. When you roll a dice \(n\) times, the probability that the difference between the relative frequency, \(\frac{X}{n}\), and the mathematical probability, \(\frac{1}{6}\), is less than a small value \(\epsilon\) can be written as follows:

$$ \begin{align} P ( \left | \frac{X}{n} - \frac{1}{6} \right | < \epsilon ) &= P ( \epsilon < \frac{X}{n} - \frac{1}{6} < \epsilon ) \\ &= P ( \frac{1}{6} - \epsilon < \frac{X}{n} < \frac{1}{6} + \epsilon ) \\ &= P \left [ n( \frac{1}{6} - \epsilon )< X < n( \frac{1}{6} + \epsilon ) \right] \end{align} $$

If \(\epsilon\) = 0.1, the above formula becomes as follows: $$ P ( \left | \frac{X}{n} - \frac{1}{6} \right | < 0.1 ) = P \left [ n( \frac{1}{6} - 0.1 ) < X < n( \frac{1}{6} + 0.1 ) \right] $$ If the number of rolls \(n\) is 10, 20, 50, 100, then the above formula becomes as follows and its probability can be calculated using \(B(n, \frac{1}{6})\) in 『eStatH』.

As you noticed here, the probability \( P ( \left | \frac{X}{n} - \frac{1}{6} \right | < 0.1 )\) is close to 1 as \(n\) increases.

This result is true when \(\epsilon\) is smaller than 0.1 such as 0.01, 0.001, ... which means that the relative frequency \(\frac{X}{n}\) is close to the mathematical probability \(\frac{1}{6}\) if \(n\) is sufficiently large. This is called the law of large number.

Among the data of continuous random variables around us, there are many data which have similar shape such as the histogram above. More data are concentrated near the mean, smaller data are located far from the mean, and data are symmetrical with respect to the mean. In order to find easily the probability of any interval on this shape of data, mathematicians found a function that can describe this shape of data. This mathematical function allows you to approximate the desired probability without finding a frequency table or histogram.

This function was first discovered by Abraham de Moivre (1667-1754), and then widely applied to physics and astronomy by German mathematician Karl Friedrich Gauss (1777-1855). This function is called the normal distribution function or the Gaussian distribution function, and the equation and graph are as follows:

$$ f(x) = \frac{1}{\sqrt{ 2\pi} \sigma } e^{ \left[ - \frac{(x-\mu)^2}{2 \sigma^2} \right]} \qquad - ∞ < x < ∞ $$ where \(\mu\) is a constant, \(\sigma\) is a positive constant, \(e\) is the irrational number as

The mean and standard deviation of the normal distribution are \(\mu\) and \(\sigma\) respectively. If a random variable \(X\) follows the normal distribution with the mean \(\mu\) and standard deviation \(\sigma\), that is the variance of \(\sigma^2\), it is denoted as \(N(\mu, \sigma^2 )\).

The normal distribution is the most widely used distribution in reality, and it requires a lot of calculation of the probability of the interval \([a, b]\) of the random variable \(X\). As explained earlier, the probability of the interval \([a, b]\), \(P(a \le X \le b)\), when \(X\) follows the normal distribution \(N(\mu, \sigma^2 )\) is the area of the function \(f(x)\) enclosed between the x-axis and interval \([a, b]\)

Mathematically, this area should be obtained as the following definite integral, but it is only possible to use a computer because the function is impossible to integrate. $$ P(a \le X \le b) = \int_a ^b \frac{1}{\sqrt{ 2\pi} \sigma } exp \left[ - \frac{(x-\mu)^2} {2 \sigma^2} \right] dx $$ The probability of the interval \([\mu － \sigma, \mu + \sigma]\) calculated using the computer is 0.68, the probability of the interval \([\mu － 2 \sigma, \mu + 2 \sigma]\) is 0.95, and the probability of the interval \([\mu － 3 \sigma, \mu + 3 \sigma]\) is 0.997. That is, the normal distribution has most values around the mean, and there are few values that are more than three times the standard deviation to the left and right from the mean.

If a random variable \(X\) follows \(N(\mu, \sigma^2 )\), the transformed variable \(Z = \frac{X - \mu}{\sigma}\) follows a normal distribution with mean 0 and standard deviation 1, \(N(0,1)\). This fact implies that if we can find all probabilities of \(N(0,1)\), we can find the probabilities of any normal distribution. Therefore, \(N(0,1)\) is specifically called the standard normal distribution or \(Z\) distribution. The transformation \(Z = \frac{X - \mu}{\sigma}\) that turns a random variable \(X\) into \(Z\) is called the standardized transformation.

For the standard normal distribution \(N(0,1)\), a table is created by finding the probabilities from the left end to \(z\) for various real values \(z\), \(P(Z \le z)\), which is called the standard normal distribution table. The following table is a part of the standard normal distribution table obtained using 『eStatH』.

In 『eStatH』, it is easy to calculate the probability for the interval \([a, b]\) of any normal distribution, \(P(a \le X \le b)\), and the percentile \(x\) for a given cumulated probability \(p\), that is \(P(X \le x) = p\), as shown in <Figure 3.28>.

It is good to remember probabilities of some intervals in the standard normal distribution that are often used. <Figure 3.29> shows the percentiles of 95%, 97.5%, and 99.5% from the left of the standard normal distribution. <Figure 3.30> shows the 95% and 99% probabilities when both ends are equally excluded..

Using the standard normal distribution table, you can find the probability of a general normal distribution. If \(X\) is the normal distribution with mean \(\mu\) and variance \(\sigma^2\), \(\frac{X-\mu}{\sigma}\) follows the standard normal distribution. Therefore, the probability of the interval \([a, b]\) of \(X\), \(P(a \le X \le b)\), is obtained by finding the probability of the interval \([\frac{a-\mu}{\sigma}, \frac{b-\mu}{\sigma}]\) from the standard normal distribution.

If \(n\) is large (about 50 or more) in case of the binomial distribution, \(B(n,p)\), the probability calculation cannot be obtained even by using 『eStatH』. In this case, the binomial distribution can be approximated using a normal distribution with the mean \(np\), variance \(np(1-p)\). <Figure 3.33> is a graph obtained by approximating the binomial distribution with \(n\) = 50, \(p\) = 0.5 using the normal distribution with the mean \(np=50 \times 0.5 = 25\) and variance \(np=50 \times 0.5 \times 0.5 = 12.5\).