12 Field testing of material

Ms. Nishat Farooq

epgp books

Content Outline

  •  Objectives of the module
  •  Understand the nature of community.
  •  Understand community as a system
  •  Understand  community in the Indian context.
  •  Communities and Social Change
  •  Frame effective measures of intervention for human betterment.
  •  Let us sum up
  •  Multiple choice questions
  •  Suggested readings.

 

Introduction

 

Different types of motivational and teaching learning material are produced for adult education. Although a producer may develop and produce these materials to her/his satisfaction but it may not necessarily be to the satisfaction of our readers and user. Therefore, the material should be tried out in the field with the potential users before finalizing and printing to get an idea whether the material is being understood and enjoyed by the learners. The producers, who are concerned about readability and acceptability of their material by the learners and want the learners to become self-reliant, always field test their material. In the present module, we will learn about need, importance and objectives of pre-testing. We will also learn about methods of pre-testing of different type of adult education material.

Objectives

 

After completing the module, the learner will be able to:

  1. State need, importance and objective of pre-testing of adult education material.
  2. Explain method of pre-testing of different types of material.
  3. Cite examples of some sample questionnaires for pre-testing.

Need and importance of pre-testing of material

 

Pre-testing or field testing is an indispensable step in material development. It is a process of evaluation for finding out the short comings and to make further improvement in draft material before finalizing for mass production. Adult education material should be need based. Therefore, it is necessary to find out if content and presentation of content are relevant and useful to the learners’ context or not. It is also necessary to find out if the language used in material is easily understood by the learner in terms of vocabulary, difficulty in reading and is not creating any other problems. Visuals are abundantly used in teaching learning materials and play vital role in adult education. Therefore, visuals are pre-tested to find out if used visuals are attractive, accurate and understood by learners and are conveying messages and how these could be made more attractive. Format of material is as important as content and language etc. Format are pre-tested to find out if it is appropriate for learners. Wrong format of booklets may create psychological problem in reading. Format is also pre-tested to find out if it is convenient to be used by the teachers/facilitators. In addition to these other elements like size of book, quality of paper, title of material is also field-tested for their acceptance and catchiness. In short pre-testing help to find out whether material interest, impress and attract the potential readers/users or not.

 

Objectives of pre-testing

 

Pre-testing is conducted to assess the quality of material and quality of material depends on acceptance of material by the readers and users of material. Objectives of pre-testing of material are as follows –

  • To find out the impact of material
  • To find out if the objective set in the curricular unit are achieved after use of material.
  • To find out the shortcomings of draft material, which may restrain learners from accepting the material e.g. difficulties in reading and comprehension due to language, unfamiliar and irrelevant content and the format which may create psychological problems in reading.

Method of pre-testing of material

 

Field-testing is a very important step of material production. Therefore, it is done very carefully. Pre-testing may be structured or unstructured. Structured pre-testing is questionnaire based. Various forms of pre-testing techniques may be utilized for unstructured pre-testing, like observation, seminars, discussion, etc.

 

Before going to field it is necessary to get information about local conditions of the target group where material will be tested. Testing should be conducted by facilitator/teacher in the presence of writer, artist and material developer. Target group should be genuine because pre-testing is done on sample basis to understand and factify the language difficulty, comprehension problem and acceptability of material.

 

Method of pre-testing of material depends on the type of material being pre-tested. In case of adult education teaching learning materials could be broadly divided in following categories-

  1. Print material – books, flash cards, posters, charts, newsletters, etc.
  2. Non-print material – audio-visual aids like cassettes, radio-programmes, puppets, street corner play, etc.

For pre-testing of materials different arrangements have to be made according to format and category of these materials.

 

Field-testing of print material

 

In case of print material model material is prepared using paper size and number of pages, etc. corresponding to the final product. Material is field-tested in such a form that it gives image and effect of complete material. For pre-testing of books as many model copies are made as potential readers are there. In case of poster, flash card, etc. single copy is enough. In model copies some space is kept to add and make changes in accordance with the suggestions of experts and field test results. Information sheet having information about target group, objectives of material, method of application, anticipated effects is prepared for pre-testers.

 

Field testing of non-print material

 

For field-testing of non-print material in addition to information sheet, a simple instructor’s manual with simple explanation on how to handle the material (slide projector, tape recorder, filmstrip, etc.) is prepared. If needed simple outline, pamphlet and other supplementary material are also prepared. For example, selected important slides may be shown on pamphlet or theme of street corner play may be distributed during the show.

 

Place for field testing is arranged in advance. Arrangements for equipment required are made and it is ensured that all of these equipment and electricity points are in order. In case of projected materials, it is also ensured that arrangements are there to make the room dark.

 

Steps in pre-testing

 

Following step are followed in field-testing of both type of materials:

  1. Preparation of model draft material – Sufficient number of copies of materials to be tested are made in advance. If enough copies could not be arranged due to certain reasons, learners/users are asked to share the materials.
  2. Preparation of questionnaire – Questionnaire is prepared and sufficient copies are made for potential users and pre-testers.
  3. Preparation of pre-tester’s manual/information sheet – A manual/information sheet is prepared for pre-testers who conduct the field testing. The manual/information sheet has information about target group, objective of field-testing, method of application, anticipated effects, etc.
  4. Preparation of pre-test agenda – For getting maximum feedback preparation of pre-test agenda is very important. It has been observed that several problems arise in field if testing is conducted without preparation. It may be complete failure also. Therefore, before proceeding to the field following arrangements are made:
    • Date and time of pre-testing is set with the mutual consent of teacher, pre-testers and learners/user.
    • Place of pre-testing is decided in advance.
    • Information about field-testing is given to potential users of materials in advance.
  5. Arrangement of equipment – Equipment required are arranged in advance, e.g. hanging tools for a poster, audiocassette player, projector, pen and pencil required for filling out questionnaire and taking notes.
  6. Training of pre-testers – Contacts are made with perspective pre-testers and they are trained in advance about their roles as facilitators, performers and note-taker. During training, they are informed about profile of target group including their specific dialects and traditional beliefs. Facilitators practice the presentation of material in advance for many times until they feel confident. They also get familiarised with the questionnaire and learn how to fill it.
  7. Field testing – Material are usually field-tested with at least 50 to 100 learners of different centres. while pre-testing reactions of facilitators and learners are observed. After use of material is complete and it is ensured that learners have read the book /used the material, the questionnaire are distributed among learners/users. The questionnaires are filled by learners with the help of pre-testers. General impressions of the material are discussed with the learners and facilitators and noted down. It is necessary to give enough time for pre-testing.
  8. Compilation of data – After field testing is over at all the centres, questionnaires filled by learners are read and report is compiled. Comments received are also analysed summery is made.
  9. Analysis of pre-testing findings – The result and observation of pre-testing are analysed and scrutinised about advantages and disadvantages of its incorporation in the material. If the material was developed in a group, findings are discussed with group members and points for revision are finalized.
  10. Revision and finalization – Changes and improvements are made in the material with the help of writer, illustrators etc. before publication.

The minor differences in pre-testing of different types of materials are as follows:

 

Pre-testing of booklets

 

Printed books and booklets have certain distinctive qualities which give them a position of pre-eminence among basic literacy and neo-literates materials. Books can actively help in self education whereas most of the other materials need help of other aids. Learners use books at their own time, place and convenience. Two methods are adopted for pre-testing of booklets-

  • All the learners assemble at one place and each person reads the booklet individually then and there. They are given sufficient time to read it. When they finish reading then pre-testers investigate whether they have actually read it.
  • Learners are allowed to take the booklets home, then next day the results are investigated.

During pre-testing of book main focus is on readability of book. Questionnaire for pre-testing of booklet is based on the objective of the booklet. Components of evaluation are attractiveness of title of booklet, relevance, usefulness and correctness of (authenticity from the point of view of learners and facilitators) content, use of local vocabulary, difficulty in reading and comprehension, attractive and correct use of illustration and size of book, etc. Following sample questionnaire is developed based on these components.

Pre-testing of posters

The basic function of poster is to clearly present visually and directly a message to many people at a time. Poster is an effective tool to convey a lasting impression in a short time. Posters usually consist of four basic parts – slogan, visual contents or pictures, secondary message and logo.

Poster may be used for campaign, for educational/instructional use or for initiating discussion. For pre-testing purpose, a copy of poster is prepared corresponding to the colour and size of actual poster. Pre-testing may be done in structured as well as unstructured way, depending on use of poster –

  • Unstructured method – Hang the poster in a place passed by many people and observe their reactions.
  • Structured method – Poster is hanged in designated place. People assemble there. Pre-tester explains the content and purpose of poster.

In case of campaign type poster, it is ascertained if the poster has immediate public appeal.

Instruction oriented posters are explained by pre-tester and discussion is generated. Observation is made to find out to which extent motivation and learning opportunities develop.

Questionnaire is developed based on size of poster, appropriateness of slogan, theme of poster, readability and comprehension of slogan and messages written on poster, attractiveness and correctness of illustration. Overall impact of poster is also evaluated.

Pre-testing of Radio broadcast

 

Radio is the most suitable medium to reach vast majority of rural people. Even they do not have to assemble in one place in order to receive the information. A radio programme can be used to motivate people to participate in literacy programme or stimulating them to get more information on certain subject. Different types of Radio programmes are developed for learners and facilitators, like talk or Radio lessons for functional literacy or equivalency programmes, dialogue, documentary, drama, interview, discussion, question answer sessions, news/market/weather reports and Spots

 

While using Radio for literacy purposes care is taken to write script in spoken language as programme is used for ears and not for eyes. It is also ensured that words used should be suited to the medium of sound. Therefore, during pre-testing focus is on attractiveness of title of programme, relevance of theme, relevance and usefulness of content, clarity and simplicity of language, presentation style of the programme, crispness of dialogues, clarity of voice and sound, etc.

 

For pre-testing of radio programme learners and facilitators are assembled at a place. In case of pre-testing of radio programme development of a guide for pre-tester is a must. All the equipment required for presentation of programme are arranged and checked in advance. The pre-tester must listen to the programme along with the members of the group and be alert on noting the points raised that initiate discussion, may need clarification or explanation after the programme. Aspects of programme that need repetition are also noted down. The pre-tester must carefully observe the reactions of the learners/users. If the first broad caste yields insufficient response the programme is repeat second time also. After filling all the questionnaires, the result is compiled and music, narration, dialogues, etc. are revised as necessary.

Pretesting of song

 

Song is mainly an audio teaching material. Songs are effective tool to convey messages in simple, interesting and effective manner. Songs are used for folk as well as electronic media. The song used for educational purposes should be composed in simple language, easy to understand and remember and have catchy tune.

 

Following is a sample questionnaire for pre-testing of song.

 

Pre-testing of games

 

Educational games provide opportunity for learning unconsciously in a participatory manner. Some conventional games used for literacy are very interesting as these create excitement and competition among learners in pursuit of a goal. Difficult topics may also become enjoyable if tackled in form of carefully designed games. Some conventional games are card games, jigsaw puzzles, snake and ladder, monopoly, Pacheesi, etc.

 

Pacheesi is a game played on a board with moves decided by the throw of a dice. The game can be played by five or more people at a time. It is an interesting ethnic game in which excitement is created to reach the goal and achieve the first position without facing any hurdles. For preparation of the game a topic is selected which is developed step by step. A goal is decided and that goal can be achieved after travelling through a rout having about 20 or 30 blocks. The reasons for progression and regression are clearly written in the blocks. Every block is beautifully illustrated.

 

While pre-testing of the Pacheesi game focus is on following elements

  • Relevance of topic.
  • Clarity and attractiveness of the illustrations
  • Colour of blocks representing penalty/progress
  • Complexity of messages/instructions
  • Vocabulary and language of statements given in blocks

Now you yourself develop questionnaire for field testing of Pachcheesi game.

 

Conclusion

 

Pre-testing or field testing is an indispensable step in material development. It is a process of evaluation of draft material for further improvement before finalization and mass production. Since the material is developed following certain outlines set in the curricular unit, pre-testing points are mainly based on – objectives set in the curricular unit, relevancy and usefulness of content, the comprehensibility of language used in terms of vocabulary and difficulty level, presentation of content, accuracy and attractiveness of illustration, appeal and appropriateness of format. Method of pre-testing of material depends on the type of material. Adult education material may be printed or non-printed. Different arrangements have to be made according to format and category of these materials. Material is usually pre-tested with 50-100 learners/users. Steps in pre-testing are as follows – preparation of pre-test agenda, preparation of sufficient number of copies of draft material, development of questionnaire for learners, facilitators, pre-tester, etc. training of pre-tester and facilitators, arrangement, supply and checking of required material, actual testing in the field, and compilation of result of pre-testing. Revisions are made in the material based on the pre-testing result before mass production.

you can view video on Field testing of material