14 Pre-Testing of Questionnaire

V. Premala Priyadharsini

Questionnaires are the most widely used survey data collection methods. The main reasons why they are mostly used are they are very easy, flexible, reliable and the most convenient among all. If the questionnaire is designed properly according to the objectives, it is considered to be one of the best methods of data collection.

Questionnaires give their best when the researcher uses standardised questions and when he/she is very much confident about the interpretation of the questions asked. Hence, it can be said that they are best used for descriptive or explanatory research.

Questionnaires when used along with any other collection method, they help in collecting for in-depth information and more complete study. It is also known as multi-method approach. For e.g., to know the attitude of customers, a questionnaire can be used. To understand their attitude towards something, an in-depth interview can be used.

TYPES OF QUESTIONNAIRES:

A design of the questionnaire completely depends on the researcher and the type of study involved. The type of questionnaire required is based on the amount of the involvement of the respondent in completing the questionnaire.

There are mainly two types of questionnaires:

1.Self- administered questionnaires:

These questionnaires are completed by respondents themselves. They can be given by hand to an individual and collect them later or can be sent through an E-mail or even a post.

Nowadays, many apps and websites have been developed to conduct a survey using questionnaire where the respondents can fill in the questionnaire using their smart phones itself.

2.Interviewer- administered questionnaires:

These types of questionnaires are filled up by the interviewee or the researcher after knowing the respondents response. They are more reliable and complete when compared to self-administered questionnaires. These type of questionnaires can be administered and filled up through telephone interview or a proper structured interview schedule.

The type of questionnaire to be chosen depends on various factors. Some of the factors include:

1. Type or characteristics of the sample group:

The selected respondents should be able to answer the questionnaire by themselves if you want to use self-administered questionnaires. In rural places if a survey should be conducted then the use of interviewer administered questionnaires can be used.

2. Particularity:

The selected type of questionnaire should give valid answers but not distorted ones. The answers given to these questions should be useful to the surveyor in some way or the other.

3. Sample size of the study:

For a larger sample size of a study, self-administered questionnaires are preferred as the larger number of population is included and time constraint will be there.

4. Type of questions:

If any kind of complicated questions or scientific questions are present, the questionnaire filling would be little difficult to the respondent, hence , the interviewer would be filling the questionnaire.

5.  Total number of questions:

If the total questionnaire is small enough, it is better preferred to the interviewer to fill it. Else, if it is huge, then it is the respondent who will fill it as the time taken to interview and fill it will be a huge process.

The type of questionnaire one chooses increases the reliability of questionnaire, and also the respondent rate will increase. The resources available should be considered while selecting a questionnaire. They are:

     i.  Time constraint

ii.  Finance

iii. Number of field workers

iv.  Ease of data entry or coding and interpretation.

DESIGNING A QUESTIONNAIRE:

Each question in a questionnaire should be administered according to the objective of the study and the information required achieving that objective. Three main principles should be kept in mind while designing a questionnaire. They are:

a) Adopting questions from other standardised questionnaires.

b) Adapting the questions according to the objectives of your study.

c) Developing or designing new questions as required.

Adopt and adapt helps to increase the reliability of the questionnaire and also they help to compare the results of your study with an existing study.

But, to meet your own objectives of a study, you require or you should design new questions such that the results will be unique from other studies.

TYPES OF QUESTIONS:

There are two types of questions which can be used in a questionnaire. Both types of questions have their own advantages and disadvantages.

1.  Open- ended questions:

These types of questions allow the respondent to write their own response in the space provided.

2.  Close- ended questions:

These types of questions will force the respondent to choose an answer among the options provided. There are six types of close ended questions:

a.  List- the respondent should select or choose from a list of options. They can be more than 4 or 5 options and can select how many options as they want.

b. Category- in this type, only one option should be chosen.

c. Ranking- the respondent should be able to rank or arrange the options given in a particular order.

d.  Rating/scale- a rating scale can be used. Eg. Hedonic scale.

e.  Quantity- the type of questions which asks about the amount specifically.

f.  Grid- the responses to 2 or more similar questions can be recorded at the same time.

INTRODUCING A QUESTIONNAIRE:

A clear concise introduction about the questionnaire can be given so that it explains to the respondent about the questionnaire and about the topic of survey. This helps the respondent to have a small idea about the survey and the purpose of filling the questionnaire. This introduction should include:

I. Clear and unbiased title

II. A subtitle which speaks about the nature of the title III. A logo or design which attracts the respondent

PILOT TESTING:

Pilot testing is also known as pre-testing. It is very important as it increases the use of any tool which is used to collect the primary data. A pilot testing is done by giving the questionnaire to smaller set of people and the responses are collected and interpreted. These interpretations should be checked with the required answers and objectives whether they are meeting the criteria or not.

Pilot testing helps us to:

Know the clarity of the questions.

Time taken by the respondent or the interviewer to complete a questionnaire. To filter out unnecessary questions.

To know any ambiguity

To know how the respondent will understand the questions.

To know how attractive and appealing a questionnaire is to the respondent.

RELIABILITY OF A QUESTIONNAIRE:

Reliability can be defined as the ability of a measure to produce the same or highly similar results on repeated administrations. The reliability of a questionnaire can be tested in three stages:

1.Test re-tests – this helps to correlate the data collected from same questionnaire at nearly equal circumstances. Therefore, questionnaire should be given twice to the respondents.

2.Internal consistency – this helps to correlate the response of each question with other questions. It hence helps in knowing the internal consistency of a questionnaire.

3. Final approach is alternative form – this helps in comparing the responses with the responses taken when the same question is asked in another form. They are usually known as check questions.

According to Rogers, there are 4 steps to increase the reliability,

1.Increase in the number of questions or items according to the required responses. The respondents should not feel bored.

2.When the administration procedure of the questions to the respondents is similar, the reliability increases.

3.Scoring errors should be reduced.

4. The items written or given should be checked clearly so that there is no ambiguity.

VALIDITY OF A QUESTIONNAIRE:

Validity of a measure is the extent to which the tool measures what you intend to measure. To test the validity of a questionnaire, three types of tests are done:

Content validity- It checks whether the questionnaire contains the questions related to the topic of our study and should be relevant to the study.

 Construct validity-It is by showing that the results obtained are nearly or equal to the predictions.

Criterion related validity-  This can be achieved in two ways:

Concurrent validity- the questionnaire’s results are compared with same type of established one and correlate.

Predictive validity- this is done by comparing the actual results with the predicted results.

Web links

• https://journals.openedition.org/bms/348#tocfrom3n2
• https://writing.colostate.edu/guides/page.cfm?pageid=1415&guideid=68
• https://onlinelibrary.wiley.com/doi/abs/10.1002/9781444316568.wiem02051
• https://journals.openedition.org/bms/348
• https://surveymethods.com/blog/how-to-pre-test-a-survey-questionnaire/
• https://www.researchgate.net/publication/282796273_The_importance_of_pretesting_questio nnaires_a_field_research_example_of_cognitive_pretesting_the_Exercise_referral_Quality_o f_Life_Scale_ER-QLS