11 Conditional Probabilitym Bayes, Theorem

Dr. Nidhi Handa

 

1.      Introduction

 

2.    Marginal Probability

 

2.      Conditional Probability

 

3.      Multiplication Rule

 

4.      Bayes Theorem

 

5.      Problems

 

6.      Summary

 

Introduction

 

Suppose all 100 employees of a company were asked whether they are in favour of or against paying high salaries to chief executive officers of U.S. companies. Table 3 gives a two way classification of the response of these 100 employees.

 

Table 1. Two way classifications of the responses of the employees

In favour Against
Male 15 45
Female 4 36

 

Table 1 gives the distribution of 100 employees based on two variables or characteristics: gender (male or female) and opinion (in favour or against). Such a table is called a contingency table. In table 1, each box that contains a number is called a cell. Notice that there are four cells in Table 1. Each cell gives frequencies for two characteristics. For example, 15 employees in this group possess two characteristics: They are male and in favour of paying high salaries to CEOs. We can interpret the number in other cells the same way. By adding the row of totals and the column of totals to Table 1, we write Table 2.

 

Table 2 two way classifications of employees

In favour Against Total
Male 15 45 60
Female 4 36 40
Total 19 81 100

 

Suppose one employee is selected at random from these 100 employees. This employee may be classified either on the basis of gender alone or on the basis of opinion. If only one characteristic is considered at a time, the employees selected can be male, a female, in favour, or against. The probability of each of these four characteristics events is called marginal probability or simple probability. These probabilities are called marginal probabilities because they are calculated by dividing the corresponding row margins (totals for the rows) or column margins (totals for the columns by the grand total.

 

MARGINAL PROBABILITY:

 

Marginal probability is the probability of a single event without consideration of any other event .Marginal probability is also called simple probability.

 

Table 3. Listing the marginal probabilities

 

In favour   (A) Against(B) Total
Male (M) 15 45 60
Female (F) 4 36 40
Total 19 81 100

 

P(A)= 19/100=.19

P(B)= 81/100= .81

P(M)= .60

P(F)=  .40

 

Now suppose that one employee is selected at random. Futhermore, assume that it is known that this selected employee is a male. In other words, the event that the employees selected is a male has already occurred. What is the probability that the employee selected (male) is in favour of paying high salaries to CEOs?

 

This probability is called conditional probability i.e. P (in favour / male).

 

CONDITIONAL PROBABILITY:

 

The Probability that an event A will occur, knowing that a certain event B has already occurred is called the conditional probability of A for given B and denoted by P(A / B) .

 

.

p(A / B) = (  ⋂  ) and

( )

p (B / A) =  (  ⋂  )

( )

 

Example: The probability that a regular scheduled flight depart in time is 0.85 and the probability that it arrives in time is 0.80. The probability that it arrives and departs in time 0.75. Find the probability that a plane arrives in time given that it departed on time.

 

Solution:  p (event of departure) = p (D) = 0.85

 

p (event of arrival)    = p (A) = 0.80

p    (A ⋂  ) = 0.75.

 

Required probability is

P (A / D) = ( ⋂ ) /()= 0.75/0.85 = 15 /17= 0.88

Bayés Theorem: Thomas Bayes a renowned British Mathematician introduced the concept of inverse probability.

 

This theory finds the probability of an event by taking into account a given sample information.

 

This is called inverse probability because it consists of finding the probability of a probability.

 

This is called posterior probability as it is calculated after information is taken into account.

 

Bayes theorem is based on the formula for conditional probability.

 

When A1 & A2 are two events which are mutually exclusive (i.e. the two events cannot occur together) & exhaustive (i.e. the two events A1 & A2 are the entire experiment). B is a sample event which intersects each of the A events as shown then part of B which is within A1 shows the area “A1 & B” whereas the part of B within A2 represents the area.

INDEPENDENT EVENTS:

 

The occurrence of one event does not change the probability of the occurrence of other event is called independent events

 

either P(A/B) = P(A) or P(B/A) = P(B)

 

Multiplication Rule:

 

The probability that events A and B happen together is called the joint probability of events A and B can also be denoted by P (  ⋂  ) or P (AB).

 

The probability of event A1 given event B is

P (A1/B) = ( 1 ⋂ ) also P (A/B) = p(A2 ⋂ B)
( ) p(B)
Where p (B) = p ( 1 ⋂  ) + p ( 2 ⋂  )

∴     p ( 1 ⋂  ) = P(A1) x p (B/A1)

p ( 2 ⋂  ) = P(A2) x p (B/A2)

 

GENERALIZED FORM OF BAYE’S THEOREM :

 

Suppose the events A1, A2 ……           form a partition of a sample space S, that is, the

 

events Ai are mutually exclusive and their union is S. Now let B be any other event. Then.

B     =   ⋂  = ( 1 ⋃  2 ⋃ … … . ⋃  ) ⋂

 

=     ( 1 ⋂  ) ⋃( 2 ⋂  ) ⋃ … … . . ⋃(  ⋂  )

 

Where the∩  are also mutually exclusive.

 

Accordingly

P(B)=  ( 1 ∩  ) +   ( 2 ∩  ) … … … … . . +  (    ∩  )

 

Thus by multiplication theorem

P(B) = p(A1) p(B/A1) + P(A2) P(B/A2) +……p(An) P(B/An)——[1]

 

On the other hand, for any i, the conditional probability of Ai given B is defined by

P(Ai/B) =  (   ⋂  )

( )

Here we use (1) to replace P(B) and use P(    ⋂  ) = P(Ai)P(B/Ai) to replace P(A  ∩)

To summarise it if an event B can occur only in combination with one of the n mutually exclusive, & exhaustive events A1, A2……. An then by actual observation, B is found to have occurred, then probability that it was preceded by a particular event A1 is defined by (2) . This is Baye’s theorem.

 

Suppose the probabilities of A1, A2 ……………An as well as conditional probabilities p(A1), ………. ,P(An) are known and we are interested in finding the conditional probability

 

p(A1/B) from the given information. This is possible with the Baye’s Theorem as follows

P(Ai/B)  =  (   ⋂  )

( )

=  ( 1) ( / 1)

( 1)  ( / 1)……..+   (  )   ( /  )

 

 

Example : An item is manufactured by three machines M1, M2 M3. Out of the total manufactured during a specified production period, 50% are manufactured on M1, 30% on M2 and 20% of M3. It is also known that 2% of the item produced by M1 and M2 are defective, while 3% of those manufactured by M3 are defective. All the items are put into one bin. From the bin, one item is drawn at random and is found to be defective. What is the probability that it was made on M1, M2 or M3.

Answer : Let
B : the item is defective.
A1 : the item was made on M1.
A2 : the item was made on M2.
A3 : the item was made on M3.

 

It an item is drawn at random, we know that a prior probability.

 

P(A1) = 0.5,   P(A2) = 0.3, P(A3) = 0.2

 

We also know P(B/A1) = 0.02. P(B/A2) = 0.02, P(B/A3) = 0.3. We are interested in finding the posterior probability, i.e. having known that the item is defective, what is the probability that it was manufactured on M1, M2 or M3.

Example.2 A company has two plants to manufacture, sctoors plant I manufacture 70% of the sctoors and plant II manufactures 30%. At plant I, 80% of sctoors are good standard quality and at plant II, 90% of sctoors are rated standard quality. A sctoor is picted up at random and is found to be of standard quality. What is the chance that it has come from plant I or plant II.

 

Ans :    Let A1  =     The event of drawing a sctoor produced by plant I.

 

A2 =    The event of drawing a scooter produced by plant II

 

E      = Event of drawing a standard quality scooter either by plant I or plant II.

 

P(A1) = 70% = 0.7

 

P(A2) = 30% = 0.3

 

P (E ⋂ 1) = P(E / A1) x P(A1) = 0.8 x 0.7 = 0.56 P(E ⋂ 2) = P(E / A2) x P(A2) = 0.9 x 0.3 = 0.27

 

The probability that a standard quality scooter is chosen at random is manufactured by plant I is given by the Bay’s theorem as :

 

Similarly, we can obtain P(A2/E)

 

= 0.424

 

Example 3. – A box contains 20 cassettes, 4 of which are defective. If 2 cassettes are selected at random (without replacement) from this box, what is the probability that both are defective?

 

Solution: Let us define the following events for this experiment.

G1 = event that the first cassette selected is good
D1 = event that the first cassette selected is defective
G2 = event that the second cassette selected is good
D2 = event that the second cassette selected is defective.

 

We are to calculate the joint probability of D1 and D2, which is given by

 

P(D1 and D2) = P(D1) P(D2 I D1)

 

As we know, there are 4 defective cassettes in 20. Consequently, the probability of selecting a defective cassette at the first selection is

 

P(D1) = 4/20

 

To calculate the probability P(D2/ D1), we know that the first cassette selected is defective because D1 has already occurred. Because the selections are made without replacement, there are 19 total cassettes and 3 of them are defective at the time of the second selection. Therefore,

 

P(D2 / D1)= 3/19

 

Hence, the required probability is

 

P(D1 and D2) = P(D1)  P(D2 / D1) = (4/20) (3/19) = .032

 

The tree diagram in Figure shows the selection procedure and the final four outcomes for this experiment along with their probabilities. The joint probability of D1 and D2 is highlighted in the tree diagram.

 

Conditional probability was discussed already . It is obvious from the formula for joint probability that we know the probability of an even A and the joint probability of events A and B i.e. P (  ⋂  ) , then we can calculate the conditional probability of B given A i.e. P(B/A)..

Example 4.Compute the conditional probability P (in favour / male) for the data on 100 employees given in table.

 

Solution – The probability P (in favour/male) is the conditional probability that a randomly selected employee is in favour given that this employee is a male. It is known that the venet “male” has already occurred. Based on the information that the employee selected is a male, we can infer that the employee seleted must be one of the 60 males and, hence, must belong to the first row of table 3. Therefore, we are concerned only with the first row of that table.

In Favor Against Total
Male 15 45 60
Male who are in favour Total number of males

 

The required conditional probability is calculated as follows.

P(in favour/male) =  ℎ  = 1560 = .25

 

As we can observe from this computation of conditional probability the total number of males (the event that has already occurred) is written in the denominator and the number of males who are in favour (the event whose probability we are to find) is written in the numerator. Note that we are considering the row of the event that has already occurred. The tree diagram in figure II as follows :

Summary

 

The probability of an event is subject to the condition that another event has already occurred is conditional probability. This type of probability opens many spheres for researchers to make inferences/decisions. It involves independent verses dependent probability, mutually exclusive verses mutually non-exclusive events along with complementary events. Bayes’ Theorem involves such type of problem e.g. the following table involves two-way classification of all scientists & engineers (Male & Female)

 

Male Female
Scientists 680 120
Engineers 527 108

 

Find one person selected is an engineer / a scientist given a person is female/ a male given the person is scientist etc.Here conditional probability and dependent probabilities are involved.

 

Learn More:

  •  ‘Fundamentals of Mathematical Statistics’, S.C. Gupta & V.K. Kapoor Public, Sultan Chand & sons, New Delhi.
  • Solution and Problems on Mathematical Statistics’, Dr. B.D. Gupta, Publ., CBS Publishers & Distributors.
  • Probability & Statistics for Engineers’, Miller & Freund & Richard A. Johnson,Pearson Edition.
  • Modern Probability theory and its applications,’ E-Porzen, Wiley, New York.
  • Theory of Statistics’ Schervish Mark J., New York, Springer.
  • Theory and  problems  of  Probability  and  statistics,  Murray    Spiegel,   John  J. Schiller, R. Alu Arinivasan, Tata Mc Graw – Hill Publishing company limited.’
  • Introductory Statistics’(Second Edition) Prem S.Mann, Publ. John Wiley & Sons,1NC , (1995).
  • ‘Business Statistics’(Fourth Edition), JK Sharma, Vikas Publishing HousePvt.,LTD(2014).
  • Discrete Mathematical Structures’,Uma Shanker Gupta,Publ.PEARSON