Redundant versus consistent stems in multiple-choice vocabulary tests and their effects on the pre-university students' performance
Bahman Gorjian
TEFL Department, Islamic Azad University, Abadan Branch, Iran
Alireza Jalilifar
TEFL Department, Islamic Azad University, Science and Research Center, Ahvaz Branch, Iran
Halimeh Mousavi
MA, TEFL Department, Islamic Azad University, Science and Research Center, Ahvaz Branch, Iran
PP: 011 - 024
Abstract
This research attempted to determine the effect of redundant and consistent stems in multiple-choice vocabulary tests on pre-university students' performance. To this purpose, a sample English language proficiency tests was administered to a total of 130 pre-university students.
Sixty homogeneous students were randomly selected and divided into two groups, one consistent group that took the test consisting of consistent stems and the other redundant group that took the test containing redundant stems. Next, three versions of the same test were provided in which the items had the same alternatives but different stems. The first and second versions of the consistent test were assigned to the consistent group, and the first two versions of the redundant tests to the redundant group. In the last versions of the two tests, the groups were reversed. This time, the consistent group took the test containing redundant stems while the redundant group took the test consisting of consistent stems.
All the tests were teacher-made summative achievement tests and their reliability coefficients, the statistical independent t-test, paired samples t-test, and Pearson correlation formula were calculated. The results revealed that there was no significant difference between the students' performance on two kinds of tests (p<0.05). The empirical findings of the present study suggest that pre-university students' competence in vocabulary can be measured through both tests with redundant and tests with consistent stems.
Keywords
multiple-choice tests, redundant stems, consistent stems
Article Text
The goal of testing vocabulary is to assess the subjects' knowledge of lexical items. In preparing vocabulary tests, the first task concerns the selection of the vocabulary to be included in the test. In case of achievement testing, the test constructor chooses the vocabulary items from the materials covered in the course (Farhady, Jafarpur and Birjandi, 2004, p. 179). Ballantyne (2004, p. 1) argues that objective tests are made up of short questions, termed items. The most common test regarding vocabulary is the multiple-choice test. Since the construction of a stem in multiple-choice tests of vocabulary has a predominant function on the students' choice of correct alternative and gives a clear portrait of students' achievement during the course, this study will investigate using redundant stems and consistent stems in multiple-choice tests of vocabulary in order to see the effect of context provided in the stem. This may reveal the fact that contextualized items provide the test takers with more information about the situation, increase their cognitive extension, and also have beneficial backwash effect on their learning process.
... continues ...
References
Ascolon ME, Meyers LS, Davies BW and Smits N (2007) Distractor similarity and item-stem structure: Effects on item difficulty. Applied Measurement in Education 20(2): 153-170.
Ballantyne C (2004) Multiple choice tests. Retrieved June 24 2005 from http://www.tlc.murdoch.edu.Ic/eddev/evaluation/mcq/mctests.html.
Brown JD (1996) Testing in language programs. New Jersey: Prentice-Hall Regents.
Buck G and Tatsuoka K (1998) Application of the rule-space procedure to language testing: Examining attributes of a free response listening test. Language Testing 15(2): 119-157.
Chiang CS and Dunkel P (1992) The effect of speech modification, prior knowledge and listening proficiency on EFL lectures learning. TESOL Quarterly 26(2): 345-374.
Collins J (2006). Writing multiple-choice questions for continuing medical Education activities and self-assessment modules. RadioGraphics 26(2): 543-551. Retrieved June 01 2007 from www.radiographics.rsnajnls.org/cgi/content/full/26/2/5430.
Cooper G (2002) Practical suggestions for writing multiple-choice questions. General suggestions. Retrieved May 14 2004, from http://www.tep.uoregon.edu/resources/assessment/multiplechoicequestions/practicalsuggestions.html.15k-.
Farhady H, Jafarpur A and Birjandi P (2004) Testing language skills from theory to practice. Tehran: The center for studying and compiling University Books in Humanities (SAMT).
Fowler WS and Coe N (1976) Nelson English Language Tests. Kenya: Thomson Nelson and Sons Ltd.
Frary R (2006) Testing memos 10: Some multiple- choice item writing do's and don'ts. Retrieved November 04 2007 from http://www.vt.edu/.
Grounlund NE and Linn RL (1990) Measurement and evaluation in teaching. New York: Macmillan.
Haladyna TM and Downing SM (1989) Multiple- choice item writing. Retrieved July 10 2000 from http://testing.byu/info/handboos/m.
Heaton JB (1988) Writing English language test. New York: Longman Inc.
Hughes A (2003) Teaching for language teachers 2nd edn. Cambridge: Cambridge University Press.
Jackson K (2003) Guidelines for writing multiple choice questions. Retrieved June 11 2005 from http://www.ncbi.nlm.nih.gov/entrez/query.fcgiz.cmd=rwtrieveddb=pumcd&list_uids=37.81991&dopt=Abstract.
Jacob CL (2004) How to write better tests: A handbook for improving test construction. Retrieved April 20 2005 from http://www.indiana.edu/~best/write_better_tests.shtml.
Khodadady E (1999) Multiple choice stems in testing: Practice and theory. Tehran: Rahnama Publications.
Lorusso GD (2004) A style guide for effective and consistent formatting of multiple-choice questions.Pathology education 27(2): 25-32.
Mueller J (2006) Constructing good items. Retrieved April 14 2007 from http://www.mailto.jfmueller@noctri.edu.
Osterlind SJ (2005) Creating quality multiple choice questions. Retrieved June 19 2006 from http://www.ecdledg.com/news/Edvoece-arc-05.10-06.learn-34k-.
Painter J (2004) Writing and reviewing assessment items: Guidelines and tips. Retrieved July 08 2005 from www.unc.edu/~painter/docs/Testpreparation.pdf.
Pappas P (2007). Developing assessments: A guide to multiple choice, constructed response, thematic essays, and DBQ item. Retrieved December 12 2007 from http://www.Peterpappas.com/.
Parker K and Chaudron C (1987) The effects of linguistic simplifications and elaborative modifications on L2 comprehension. UHWPESL 6(2): 107-133.
Passmore C, Dobbie AE, Parchman M and Tysinger J (2002) Guidelines for constructing a survey. Research series 34(4): 281-286. Retrieved June 17 2006 from www.stfm.org/fmhub/fm2002/apro2/rs1.pdf_.
Read J (2000) Assessing vocabulary. Cambridge: Cambridge University Press.
Violato C and Marini AE (1989) Effects of stem orientation and completeness of multiple-choice items on item difficulty and discrimination. SAGE Journal: Educational and Psychological Measurement 49(11): 287-296.
Ying-hui H (2007) An investigation into the task features affecting EFL listening comprehension test performance. Asian EFL Journal 8(2): 1-15. Retrieved December 19 2007 from http://www.asian-edf-journal.com/june_06_hyh.php_54k-.
Zimmaro DM (2004) Writing good multiple-choice exams. Retrieved June 16 2006 from http://www.utexas.edu/academic/mec/research/pdfwritingmcexamshandout.pdf-.
eContent Home
