2 Different Setups,Functions and Plug-in Estimators

Mr Taranga Mukherjee

1 A short recap

So far we have learnt that data can be non normal in real practice. Traditional parametric methods then fail to describe the features of the data set. If forcefully the normality assump-tions are made, the results can lead to severe error, especially in small sample cases. Then an alternative inferential method is adopted, which does not require any particular distri-butional assumption unlike parametric methods. These methods are called Non Parametric methods.

2 The set up

Here the data at hand is x = (x1; x2; :::; xn), where xi 2 ( 1; 1) is the observed value of a random variable Xi; i = 1; 2; ::; n. F (x) = P (X1 x1; :::; Xn xn) is the distribution func-tion(DF) from which x is generated. The components of X are not in general independent, but we assume independence. i.e, F (x) = Qni=1 Fi(xi) for marginal DFs Fi; i = 1; ::n. The pair (X; F ) generates the induced probability space (X ; B; P ), where X is the sample space, B is the eld generated by the all possible subsets of X , andP is the induced probability measure. Suppose p(x) is the generalised density of X. In any situation, p(x) is either par-tially or completely unknown. In statistical inference, we try to give some idea about p(x) or some suitable function or functional of it.

3 Different inferential set up

Depending on the knowledge of p(x)(i.e. whether completely or partly unknown), inferential set ups are either-i. Parametric,ii. Nonparametric and iii. Semiparametric

4 Classification

Suppose p(x) is partially known. We can write p(x) = p(x; ); 2 , where p is functionally known. (numerical or abstract valued) is an unknown quantity indexing p, often called a labelling parameter. is the space of all possible values of . It is called the parameter space.

Example of parametric set up

Example of non parametric set up

1 If  is completely abstract valued, the set up is nonparametric.Suppose Xi; i = 1; 2; ::; n are iid observations from an unknown DF F (x), where The functional form of F is unknown but known to be continuous. Then = F is unknown and abstract valued. = is the class of all absolutely continuous DFs. Thus the set up is nonparametric.

2 Consider two independent samples fXi; i = 1; ::; mg and fYj; j = 1; ::; ng, where Xi are iid observations from an unknown density f(x) Yj are iid observations from an unknown density g(x), where f and g are both unknown but known to be continuous. Then = (f; g) is unknown and abstract valued. Thus the set up is nonparametric.

5 Concept of functional

In a nonparametric set up, the basic assumption is the continuity of the underlying dis-tribution F. The unknown quantity of interest (parametric function in parametric set up) is de ned in terms of a functional. If F is the class of all absolutely continuous distribu-tions, then a functional F = (F ) is a real valued function de ned for every F 2 F Then F : F ! R is a mapping from the abstract space to real line. In nonparametric infer-ence, the objective is to learn the value of F = (F ) for a known functional based on iid observations from an unknown DF F.

Example of functionals

Suppose Xi; i = 1; 2; ::; n are iid observations from an unknown DF F (x), where F is unknown but known to be continuous. Few possible functionals are:

1. F = EF I(X a), for some known a, where F is the class of all absolutely continuous DFs.

2. F = EF (X), where F = fF : EF (jXj) < 1g

3. F = EF (X2), where F = fF : EF (X2) < 1g

Linear functional

Each of the functionals can be expressed as EF  (X), for some.

For examples 2 and 3, (X) = X and X2.

For example 1, (X) = I(X a), where I( ) is the indicator function. Then ( F1 + (1 )F2) = F1 + (1 ) F2 for any 0 < < 1. The above functionals are called linear functionals.

Nonlinear functionals

Suppose Xi; i = 1; 2; ::; n are iid observations from an unknown DF F (x), where F is unknown but known to be continuous. Then the following are also valid functionals:

1. F = fEF (X)g2, where F = fF : EF (jXj) < 1g

2. F = VarF(X), where, F = fF : EF (X2) < 1g

The above functionals are nonlinear functionals as ( F1(1 )F2) 6= F1 + (1 ) F2 for 0 < < 1.

Further examples

Suppose Xi; i = 1; 2; ::; n are iid observations from an unknown DF F (x), where F is un-known but known to be continuous. Then the following are also nonlinear functionals:

1. F = F 1(12 ), the median , where, F is the class of all absolutely continuous DFs.F  1( 3 )  F  1( 1 )

2. F = 4 4 , the quartile deviation, where, F is the class of all absolutely continu-2 ous DFs.

A critical example

Suppose F is class of all absolutely continuous DF with nite expectation. Then the question is what will be the type of the kernel F = fEF (X)g2?

Take Fi 2 F; i = 1; 2 Then for any 2 (0; 1),

E F1+(1 )F2 (X) =  EF1 (X) + (1 )EF2 (X).

But fE F1+(1 )F2 (X)g2 6=  fEF1 (X)g2 + (1 )fEF2 (X)g2.

Therefore,  F is not a linear functional.

Plug in estimator: Analogue to statistic.Suppose Xi; i = 1; 2; ::; n are iid observations from an unknown DF F (x), where F is unknown but known to be continuous. Fn is the empirical DF based on the data. (Fn) is the plug-in estimator corresponding to the functional (F)

1. EFn  (X) is the plug-in estimator corresponding to the functional EF  (X).

2. Fn 1(12 ), the sample median is the plug-in estimator corresponding to F 1(12 ).

A misleading example

Suppose F is class of all absolutely continuous DF with symmetry at the origin. Then what will be the type of the functional F = P (X > 0)?

At a rst look it appears as a linear functional. But symmetry at the origin gives F = 1 . 2