13 Basic Sampling and Tests of Significance for large samples

Prof. Aslam Mahmood

 

    1) E- Contents

Introduction

A research is generally made to uncover the reality Most of the time prevailing in the larger areas of the society and a Census enumeration can only serve this purpose. However, due to paucity of time, resources and trained manpower the Census enumeration is not always possible. In such situations we take the help of sampling methods through which we study the whole from its part. Theory of sampling equips us in this art. Thus when we carry out a sample survey the results provided by it do not remain to sample only. Theory of sampling provides us the theory of generalizing about the whole (the Universe) from its part (the Sample).

One of the basic principle in this regard is that the sample has to be representative of the whole i.e. Universe or population.The Universe or Population of any study will be demarcated by the research problem itself. For example if the research study is concerned about some health issues of the tribal population living in Rajasthan, all the tribal households in Rajasthan will form the Universe of the study. However if the study is limited to only one tribe say “Bhils” of district of Jaisalmer, all the Bhil households living in Jaisalmer will form the universe. Similarly, Universe could be the agricultural farms of a district, or of a region. Likewise every sample study will have a universe.

A sample survey saves a lot of time and resources of the researchers. But, if the sample does not represents the universe, the whole exercise will go waste. It is therefore, important before the analysis of the sample data to ensure that a sample is drawn from a given universe in such a manner that it is a proper representative of it, as any sample cannot always be representative of the Universe. A sample can be drawn in several different ways. Which method of sampling will make it most representative of the Universe will depend on the structure of the Universe of the study and all possible methods of sampling are discussed in detail in other module.In the present module we will discussed one most important method of sampling which is known as ‘ SimpleRandom Sampling” and discuss the properties of it related to generalization of its Universe from a random sample drawn from it.

Simple random Sampling

It is a method of drawing a sample in which each and every member of the Universe has equal chance of being selected in the sample. This method is useful when the members of the universe are fairly homogeneous. The method of drawing a simple random sample is just like drawing a number in a lottery. A sample drawn in this manner is known as a random sample.Now a days computers are used to generate random numbers used for drawing a random sample. Information about the Universe can be deducted from this sample information with the help of theory of sampling. Detailed discussions on sampling methods are given in modules on sampling .

Sampling Distributions

Sampling distribution is an important concept in the theory of sampling as it provides the basis of most of the tests of significance. It deals with the nature of variations of values generated by a sample in a repeated trial. For example, take a random sample of size “n” drawn from a Universe of a large number of observations and note down its statistics like mean, standard deviations, coefficient of correlations and regressions etc. After noting down these values we replace the sample values to the Universe and draw another sample of same size and note down new values of these statistics. We repeat this exercise again and again over a larger number of times. This will generate a larger set of sample values of such means and standard deviations etc. which when tabulated will give us a distribution for each statistics. These distributions are known as sampling distribution of means, sampling distribution of standard deviations etc. Each sampling distribution has certain theoretical properties relating the parameters of the Universe and the sample statistics. In reality we do not draw large number of samples to note down their means and standard deviations and other characteristics. In practical problems we only draw one or two samples only. The idea of drawing the large number of samples and noting down their means, standard deviations and other statistics is only for conceptual clarityof the theoretical properties of the sampling distribution. These properties help us in formulating different tests of significance relating universe values with the sample values.

Important sampling Distributions

Sampling distribution of means: If a large universe has a population with mean = µ and a standard deviation σ and a large sample of size n> 30 is randomly drawn repeatedly (with replacement) over alarger number of time, the sapling distribution of its means will follow a normal distribution with mean = µ and a standard deviation = σ/√ ,also known as Standard Error. There is no assumption about the distribution of the universe except that it should have a finite standard deviation. Note that as n increases the standard deviation of mean values reduces and approaches to zero for very large values of n i.e. all the mean values of means of very large samples will be identical and equal to the Universe mean µ.

Since the sampling distribution is found to be normally distributed with mean = µ and standard deviation = σ/√ , the properties of normal distribution can be applied to the range with in which all sample mean values will fall.

Thus, we can say that:

68.23 % sample means will fall with in the limit µ – σ/√ to µ + σ/√  ,

95.45 % sample means will fall with in the limit µ – 2 σ/√ to µ + 2 σ/√  and

99.73 % sample means will fall with in the limit µ – 3 σ/√ to µ + 3 σ/√

The above mentioned properties of the sampling distribution give rise to some of important concepts like “Level of Significance” and “ Statistically Significant Difference” etc. and provide the basis of all kinds of “tests of significance”.

Level of Significance

In above example suppose we draw a sample of size n which gives me the sample mean value = m. If somebody who does not know as to where from the sample has been drawn, wants to decide whether the sample has been drawn from the above Universe, he can use the properties mentioned at point number 2 or 3 and makes a hypothesis that the sample is drawn from the above Universe i.e. a Universe with mean µ and standard deviation . Suppose he find the sample mean value falling outside the limit:

µ     – 3 σ/√ to  µ +  3 σ/√

As 99 % sample mean values are likely to fall with in this limit, he may reject his hypothesis. However, there are chances that some of the genuine samples of the above Universe may also fall outside this limit. In those cases he will be committing an error of rejecting a correct hypothesis known as “ Type-I Error”. However, the chances of such error are 1 out of 100. The probability of such an error is known as “The Level of Significance”. Thus we define the “ Level of Significance” or “ Type-I Error” as: “ Probability of rejecting a correct hypothesis”. In the case of the above example the level of significance is 1 %. We also have Type II Error ; defined as probability of accepting a wrong hypothesis.

In some other case , if in place of 99 % limit , we take 95 % limits i.e. µ – 2 σ/√ to µ + 2 σ/√ and reject a hypothesis, it will be rejected at 5 % level of significance.

It is possible that sometimes the sample value may fall between the 99 % and 95 % limits i.e. between (µ    – 3 σ/√ to µ + 3 σ/√ ) and (µ – 2 σ/√ to µ + 2 σ/√ ) in such cases it will be significant at 5% of significant but insignificant at 99 % level of significance.

In most of the statistical analyses, we are not able to conclude with 100 % accuracy. In such cases we have to make a probabilistic conclusion as the next best alternative. Statistical tests are designed in such a manner that with every conclusion some probability of error is also attached. The researcher has a choice to choose the level of error. This give him some freedom of subjectivity in his analysis. Thus, if a hypothesis framed by a researcher is not being proved at a very low level of significance or error ( say at 1 % ), he or she has a freedom to go for a higher level of significance or error (may be at 5 % ) to proof his or her hypothesis, provided the level of significance is specified.

Statistically Significant Difference

Imagine there are two agricultural farms of equal size and have similar conditions for the production of a crop. Total amount of yield of the crop in both the plots will be very close to each other but not identical. Such differences are considered as insignificant and are attributed to the differences of minor factors between two plots due to random factors and are ignored. However, in any scientific analysis, whether a difference between two given values is significant or insignificant should not be left to individual’s subjective decisions. Statistical methods of test of significance provide us the basis to decide as to when a difference should be considered as significant and when not. For a variety of situations these methods give some limiting values beyond which a difference should be considered as significant and below which or equal to which it should not be considered as significant. These limits are however, are flexible and carry certain chance factors associated with them. Like we can arrive at a conclusion using a limit at 5% level of significance. At the same time we can draw a different conclusion by changing the level of significance from 5% to 1%. The test of significance provide us objective criterion with this much subjectivity only at the disposal of the researcher.

In the case of large samples such limitsare used from the sampling distribution of means or other statistics which are normal in the case of means. In the case of small sample the sampling distribution of some of their ratios such as: t=M-m/(s/√n) or F= S12 / S2 2 etc. are used. Once we use some statistical criterion to decide whether a difference is significant or not the conclusion will say that: “the difference is found to be statistically significantat 1 % or 5 % level of significance” or “the difference is found to be statistically insignificant at 1 % or 5 % level of significance”.

Null Hypothesis

In case the value of universe mean is not known and we have some tentative value for it as m, we make a hypothesis that the population mean from where the sample is drawn is m and note down the sample value of mean as X. If the absolute value of the difference between m and X is found to be with in these limits, it is said to be statistically insignificant and is ignored, as it is small enough to be attributed to random factors only and we accept the hypothesis that the value of the universe mean may be m. Such a hypothesis is known as null hypothesis. A null hypothesis is defined as a hypothesis with possible rejection, since we can generally prove the differences between two values by rejecting the hypothesis of their equality. With every null hypothesis is associated an alternative hypothesis which is just opposite to it. When a null hypothesis is rejected its alternative hypothesis is accepted.

One- tail test or Two-tail test

While comparing two situations, we may be interested in two ways: (a) we may be interested only in knowing whether a=b or a≠b. In this case “a” can go to either: “higher than b” or “lower than b”and we take the absolute value of the difference i.e. I a – b I It can be positive or negative. Both types of differences are equally important for us. In such casesto find out the limit (either for large samples or for small samples) we use two- tailed test. Since it can go to any of the two sides. On the other hand; if we are interested in knowing whether a is greater than b only i.e. a > bwe use the limit from “one –tail test”

The difference between one and two tailed test lies only in the level of significance only. In a problem of two-tailed test suppose we choose to go for 5 % level of significance, we get the required value under the column of 5% level of significance. However, if we go for one –tailed test only, we have to use the limiting value for the level of significance double the level of significance for two –tailed test. So, the value of the limit for 5 % level of significance in two –tailed test will correspond to 2.5 % in one tailed test. For 5 % level of significance in one-tailed test we have to see the corresponding value of the limit at 10 % level of significance in two tailed test.

Test of significance of the difference between a large sample mean and the universe mean

Thus if a samples is drawn from a given universe, 99.73% chances are that the difference between the sample mean and the universe will be less than 3σ/√ and 95.45 chances are that the difference will be less than 2σ/√ (derived from the properties of the normal curve). If a difference is found to be with in these limits it is said to be statistically insignificant and is ignored as it is attributed to random factors only. However, if a difference is found to be more than 3σ/√  it is said to be significant at 1% level of significance (as per normal distribution around one percent chances are that we may be wrong) and if it is between 2σ/√ and 3σ/√  it is said to be significant at 5% level of significance (as per normal distribution only around 5 percent chances are that we may be wrong). In case a difference is found to be statistically significant it attributed to some major factors.

Example

A random sample of size 100 is drawn from a large universe and gives the average size of farms equal to 60 acres and a standard deviation of 25 acres. Can the average sizeof farms in the region be equal to 70 acres.

Given :X͞= 60,                µ = 25 and      σ = 40  n= 100

In the present case we do not know the standard deviation of the universe but since the size of the sample is 100 which is large it may be considered as a good estimate of standard deviation of the universe.

Now the difference between estimated value of mean 60 and the proposed value of mean is I60 – 70 I = 10 acres and the 1% level of significance value of 3σ/√                                            = 3 x 40/10 = 12.5 acres.

Whereas the 5% level of significance value of 2σ/√                                        = 2 x 40/10 = 8.0 acres.

Thus the difference between the universe value of 70 and the sample value of 60 acres (i.e. 10 acres) is found to be significant only at 5 % level of significance (as 10 > 8 ). At 1 % level of significance the difference is not found significant (as 10 < 12.5 ).

Test of significance of the difference between twolarge sample means

Similarly, if two large samples are randomly drawn from two universes with mean and standard deviations M1, S1 and M2, S2 respectively and the following results are noted:

Sample	From Universe I	From Universe II
Mean	m1	m2
Standard Deviation	S1	S2
Size	n1	n2

Equality of two universe means M1 and M2 can be tested by testing the value of the absolute value of the difference between the observed sample means: I m1 and m2I. In case the difference of the sample means is found to be insignificant the inference would be that the two universes may have equal means. On the other hand if the difference is found to be significant the inference would be that the population means M1 and M2 can’t be the same.

It is found that the sampling distribution of the differences of sample means would also be normal with mean = I M1 – M2 I and standard error = √  21/  1 + 22/  2 . The difference between the sample means will be significant at 1 % level of significance if it is less than or equal to 3 x √  21/  1 + 22/  2. For 5 % level of significance the value will be 2 x √  21/  1 + 22/  2 .

Example

Two random samples of rural settlements are randomly drawn from two regions. Size of samples, their average population and its standard deviations are given below. Test the null hypothesis that the average size of rural settlements in both the regions are equal.

Sample	From Region I	From Region II
Average Population	700 persons	730 persons
Standard Deviation	85 persons	95 persons
Size	100 rural settlements	150 rural settlements

IM1 – M2 I = I700 -730 I = 30 persons

Standard Error = √852/100 + 952/150 = √132.42 = 11.51 persons

3 x standard Error = 3 x 11.51 =34.53 persons

2 x standard Error = 2 x 11.51 = 23.02 or 23 persons

Thus we note that the difference is found to be insignificant at 1 % level of significance i.e. the two universes may have equal average population ( as the difference of 30 persons < the 1 % permissible limit of 34. 53 persons) or in other words the two universe do not differ in terms of their average population. However, if we increase the level of significance to 5 % level, we can conclude that the two universes will not have equal average population (the difference of 30 persons is greater than the 5 % permissible limit of 23 persons or in other words the two region differ in terms of their average population size.