Teacher and student washback on test preparation evidenced from Taiwan’s English certification exit requirements
Yiching Pan
National Pingtung Institute of Commerce, Pingtung, Taiwan
Timothy Newfields
Toyo University, Faculty of Economics, Japan
PP: 260 - 272
Abstract
This study explores the extent that a policy requiring university students in Taiwan to attain a minimal score from an approved list of EFL proficiency tests has resulted in test-driven instruction and changes in student study patterns. Data from 160 teacher and 1,415 student surveys as well as interviews with 25 teachers and 38 students indicate that test-related instruction and test-driven student study patterns were more prevalent at schools with English proficiency exit requirements than their counterparts. However, this policy appears to have had minimal impact on student learning because it seems clear that many students did not devote substantial time to preparing for the exit exams after class. Moreover, the majority of teachers did not turn their regular English classes into certification-test coaching sessions. Although many students seemed to want more test-preparation instruction and practice in class, teachers in general spent only a small amount of class time on such activities.
Keywords
EFL proficiency tests; GEPT test; washback and test-related instruction; EFL university entry certification
Article Text
Given the importance of English in the globalized workplace, the Taiwanese government- like many other Asian countries such as Thailand, the Philippines, and Japan-has recently expressed "grave anxiety about its national proficiency in English" (Graddol, 2006, p. 95). In order to enhance students' English proficiency, Taiwan's Ministry of Education has encouraged institutes of tertiary education to establish exit requirements for English because policy makers and school administrators generally adhere to the belief that tests exert a significant influence on educational policy (Shohamy, 2001a, 2001b). Because of the exit requirement policy, an increasing number of institutes of tertiary education are attempting to integrate test preparation within their standard curricula (Pan, 2010). Although test preparation is often regarded to be just one component of the wider issue of washback (Lumley & Stoneman, 2000), there is no significant body of empirical evidence investigating these alleged concerns about test preparation courses (Green, 2007). In view of this, the current study will explore whether the exit requirements have increased the amount of "teaching to the test" and "studying for the test" to tertiary institutions. Furthermore, it will determine whether teachers and students tend to have different responses to test mandates. Exploring the relationship between teachers' test preparation practices and students' perceived needs might help us elicit more positive washback for both stakeholders.
Literature review
One way of defining washback is as the influence that testing has on classroom practices (Alderson & Wall, 1993; Cheng, 1999; Hughes, 2003). A number of studies exist on the various effects that tests reputedly have on lesson content, pedagogy, and learning behaviors (Green, 2007; Gu, 2007; Qi, 2005; Stoneman, 2006; Watanabe, 1996, 2004; Wall, 2000, 2005). In order to better understand the relationship between teaching and learning for specific tests, let us consider the reported test effects on teaching, then learning, and finally mention some hybrid studies considering both aspects.
Studies on washback
Teaching and learning
Three types of teaching washback claims have been reported that are pertinent to this study. They comprise: a) tests that affect both how and what teachers teach, but show how not all teachers react in the same way to tests (Alderson & Hamp-Lyons, 1996; Gu, 2007; Watanabe, 1996; Falvey & Cheng, 2010; Green, 2007; Hayes & Read, 2003, 2004; Qi, 2005; Wall & Horak, 2006), b) tests that affect what is taught, but not how teachers teach (Cheng, 2004; Wall & Alderson, 1993; Wall, 2005), and c) tests that do not appear to significantly affect teaching (Li, 2008). This study provides evidence supporting the latter position. Two major student washback claims have been made. These realte to a) how tests influence the amount of study - but not the means, (Stoneman, 2006), and b) how tests have little - if any - impact on study habits (Shih, 2007; Tsai and Tsou, 2009). This study provides evidence supporting the second position.
Hybrid washback studies
The hybrid studies we have examined suggest that tests can influence teaching and learning if the stakes are sufficiently high. Shohamy et al. (1996) demonstrated how a high-stakes test EFL exam appeared to promote teaching and learning changes while a low-stakes Arabic exam had minimal impact. That study suggests that test stakes, format, purpose, the status of the language being tested, and skills tested are all intervening variables that influence teaching and learning washback. In the same vein, Ferman (2004) outlined how teaching and learning reputedly changed as a consequence of a new university entrance test. 94% of the teachers (n=17) and 80% of the students (n=96) in her study claimed to focus on the specific skills measured by the test. Moreover, Gu (2007) explored how an English exit test appeared to influence the learning styles of 2,605 university students and the teaching styles of 1,080 teachers in China. Her results suggest that the exit test led to a greater emphasis on reading skills, and the teacher/student questionnaires and interviews suggest that many students' reading ability improved markedly.
Washback and potential conflicts between teaching and learning
Cheng (1997, 1999, 2004, and 2005) found that although there was a match between most teacher and policymaker perceptions of tests as a means to promote integrated, task-based approaches to high school EFL education in Hong Kong, many teachers had fossilized teaching behaviors that were inure to change. Cheng asserts that without sufficient professional training about new recommended instructional methods, merely superficial changes in teacher behaviors were apt to occur. In Cheng's words, although the recently introduced test "showed teachers something new", it did not show them, ". . . how to teach something new" (2005, p. 247). She commented that this discrepancy diminished the test's washback and further suggested teacher training and collaboration was needed to produce substantial changes in teaching practices.
Chen (2002, 2006) examined the relationship between washback effects and curriculum innovation in a Taiwanese junior high school EFL context. She found a lack of the alignment between the junior high school curriculum and the test she studied. This test had a twofold, and somewhat contradictory purpose: to screen students, yet also to promote "communicative" curricular innovations. However, the test was solely multiple-choice, focusing on reading competences rather than holistic language skills. Hence, the mismatch between curricular objectives and test format and content diluted its impact. The author suggests that "positive washback effects are more likely to occur when a curriculum and a test are highly matched" (2006, p. 5).
Qi (2004, 2005, 2007) further revealed how teachers and test-developers can have conflicting perceptions of test use. Test-developers created the National Matriculation English Test in China to reputedly promote communicatively-oriented approach high school English teaching. However, since the primary function of this test was gate-keeping, most teachers ignored the communicative features and simply focused on drilling factoids into students. As these studies suggest, more research establishing the connection between teaching and learning for specific tests is needed. This might help us understand what prevents some examinations from providing positive effects for teachers and students.
English certification exit requirements in Taiwan
Since 2003, Taiwan's Ministry of Education has encouraged universities and colleges of technology to set English thresholds for graduates to generate a level of English proficiency sufficient to meet the anticipated needs of both domestic and international job markets. The number of four-year technical universities/colleges in Taiwan implementing an English certification exit requirement policy for non-English majors increased from 5% in 2003 to 31% in 2010 (Pan, 2010).
The Ministry established a list of recommended tests to fulfill this requirement. The list includes the TOEIC®, TOEFL®, ITELTSTM and two local tests that some readers might not be familiar with: the General English Proficiency Test (GEPT) and the College Student English Proficiency Test (CSEPT). The GEPT is a 5-level, four-skill general English proficiency examination commissioned by Taiwan's Ministry of Education in 1999. The CSEPT is 2-level, listening-reading-grammar test for university-level students in Taiwan. The GEPT listening and reading sections and entire CEPT is machine-scored and in a multiple-choice format. The GEPT speaking and writing sections are analytically scored by human raters (Pan, 2009). By 2008, nearly one-third of the universities and colleges of technology in Taiwan had adopted the GEPT as one way to fulfill the English proficiency exit exam requirements for non-English majors (Roever & Pan, 2008).
Research questions
This study seeks to determine the impact of a specific testing policy on teaching and learning in a tertiary EFL context in Taiwan by addressing the following research questions:
1) How does the policy of requiring all students to pass an approved EFL proficiency test prior to graduation appear to affect teaching according to teacher self-reports?
2) How do the aforementioned requirements appear to affect student learning according to student self-reports?
3) How do teachers' and students' notions regarding optimal test preparation appear to differ?
References
Alderson, C., & Hamp-Lyons, L. (1996). TOEFL preparation courses: A study of washback. Language Testing, 13(3), 280-297. doi: 10.1177/026553229601300304
Alderson, C., & Wall, D. (1993). Does washback exist? Applied Linguistics, 14, 115-129. doi: 10.1093/applin/14.2.115
Bailey, K. M. (1996). Working for washback: A review of the washback concept in language testing. Language Testing, 13(3), 257-279. doi: 10.1177/026553229601300303
Berry, V., & Lewkowicz, J. (2000). Exit-tests: Is there an alternative? Hong Kong Journal of Applied Linguistics, 5(1), 19-49.
Brooks, F. B. (1993). Some problems and caveats in "communicative" discourse: Toward a conceptualization of the foreign language classroom. Foreign Language Annals, 26 (2) 233-242.
Burrows, C. (1998). Searching for washback: An investigation into the impact on teachers of the implementation into the adult migrant English program of the certificate in spoken and written English. Unpublished doctoral dissertation, Macquarie University, Sydney, Australia.
Burrows, C. (2001). Searching for washback: The impact of assessment in the certificate in spoken and written English. In G. Brindley & C. Burrows (Eds.), Studies in immigrant English language assessment. (Vol. 2, pp. 95-187). Sydney, Australia: National Center for English Language Teaching and Research.
Burrows, C. (2004). Washback in classroom-based assessment: A study of the washback effect in the Australian adult migrant English program. In L. Cheng, Y. Watanabe & A.Curtis (Eds.), Washback in language testing: Research contexts and methods (pp. 113-128). Mahwah, New Jersey: Lawrence Erlbaum.
Chen, C. (2006). College students' perception of the impact of graduation English proficiency benchmarks, GEPT and further English learning. Selected Paper from the Fifteenth International Symposium on English Teaching. Crane Publishing & ETA-R. O. C., 239-249.
Chen, L. (2002). Taiwanese junior high school English teachers' perceptions of the washback effect of the basic competence test in English. Unpublished doctoral dissertation, Ohio State University, Ohio, USA.
Cheng, L. (1997). How does washback influence teaching? Implications for Hong Kong. Language and Education, 11(1), 38-54. doi: 10.1080/09500789708666717
Cheng, L. (1999). Changing assessment: Washback on teacher perceptions and actions. Teaching and Teacher Education, 15, 253-271.
Cheng, L. (2004). The washback effect of a public examination change on teachers' perceptions toward their classroom teaching. In L. Cheng, Y. Watanabe & A. Curtis (Eds.), Washback in language testing: Research contexts and methods (pp. 147-170). Mahwah, NJ: Lawrence Erlbaum.
Cheng, L. (2005). Changing language teaching through language testing: A washback study. Cambridge University Press.
Cheng, L. (2008). Washback, impact and consequences. In E. Shohamy & N. H. Hornberger (Eds.), Encyclopaedia of Language and Education (2nd ed., Vol. 7, pp. 349-364). New York; London: Springer.
Chern, C. (2002). English language teaching in Taiwan. Asia-Pacific Journal of Education, 22(2), 97-105.
Chu, H. Y. (2009). Stakes, needs and washback: An investigation of the English benchmark policy for graduation and EFL education at two technological universities in Taiwan. Unpublished doctoral dissertation, National Taiwan Normal University, Taiwan.
Falvey, P., & Cheng, L. (2010). Paradigm shift: Teachers' classroom assessment practices in Hong Kong and the PRC. Melbourne Papers in Language Testing, 10 (3), 29-51.
Ferman, I. (2004). The washback of an EFL national oral matriculation test to teaching and learning. In L. Cheng, Y. Watanabe & A. Curtis (Eds.), Washback in language testing: Research contexts and methods (pp. 191-210). Mahwah, NJ: Lawrence Erlbaum.
Graddol, D. (2006). English next: Why global English may mean the end of "English as a foreign language". London: British Council.
Green, A. (2007). IELTS washback in context: Preparation for academic writing in higher education. Cambridge University Press.
Guo, X. (2007). Positive or negative - An empirical study of CET washback. Chongqing, China: Chongqing University Press.
Haladyna, T. M., Nolen, S. B., Haas, N. S. (1991). Raising standardized achievement test scores and the origins of test score pollution. Education Researchers, 20 (5), 2-7. doi: 10.3102/0013189X020005002
Hamp-Lyons, L. (1997). Washback, impact and validity: Ethical concerns. Language Testing, 14(3), 295-303. doi: 10.1177/026553229701400306
Hamp-Lyons, L. (2007). The impact of testing practices on teaching: Ideologies and alternatives. In J. Cummins & C. Davison (Eds), The international handbook of English language teaching, (Vol1, pp. 487-504). Norwell, MA: Springer.
Hayes, B., & Read, J. (2003). IELTS Test preparation in New Zealand: Preparing students for the IELTS academic Module. In R. Tolloh (Ed.), IELTS Research Report 4 (pp. 153-206). Canberra: IELTS Australia.
Hayes, B., & Read, J. (2004). IELTS test preparation in New Zealand: Preparing students for the IELTS academic module. In L. W. Cheng, Y. Watanabe & A. Curtis (Eds.), Washback in language testing: Research contexts and methods (pp. 97-111). Mahwah, NJ: Lawrence Erlbaum.
Henderson-Montero, D., Julian, M. W., & Yen, W. M. (2003). Multiple measures: Alternative design and analysis models. Educational Measurement: Issues and Practice, 22(2), 7-12. doi: 10.1111/j.1745-3992.2003.tb00122.x
Huang, C. (2001). Using authentic materials in Taiwan's English classroom: The best way to enhance students' English proficiencies and intercultural competence. Journal of National Taipei University of Technology, 34(1), 225-234.
Hughes, A. (2003). Testing for language teachers (2nd ed.). Cambridge, UK: Cambridge University Press.
Lai, H. (2003). Learning styles and personality types: Identification and comparison of hospitality students in Taiwan and United States. Unpublished doctoral dissertation, Texas Tech University.
Li, Hong. (2008, June). Are teacher teaching to the test? A case study of the College English Test (CET) in China. Poster presented at 30th Annual Language testing Research Colloquium, Hang Zhou, China.
Lumley, T., & Stoneman, B. (2000). Conflicting perspectives on the role of test. Hong Kong Journal of Applied Linguistics, 5(1), 50-80.
Messick, S. (1996). Validity and washback in language testing. Language Testing, 13(3), 241-256. doi: 10.1177/026553229601300302
Munoz, A. P. & Alvarez, M. E. (2010). Washback of an oral assessment system in the EFL classroom. Language testing, 27(1), 33-49. doi: 10.1177/0265532209347148
Nkosana, L. B. (2008). Importance of education officers' attitudes and perceptions in curriculum and assessment reform. Electronic Journal of Foreign Language Teaching, 5(1), 41-67. Retrieved from http://e-flt.nus.edu.sg/v5n12008/nkosana.pdf
Pan, Y. (2009). Voices in the field: An interview with Jessica Wu. SHIKEN: The Japan Association of Language Teaching, Testing & Assessment SIG Newsletter, 13, 9-14. Retrieved from http://jalt.org/test/PDF/Wu.pdf
Pan, Y. (2010). Consequences of test use. Educational and societal effects of English certification exit requirements in Taiwan. Unpublished doctoral thesis, the University of Melbourne.
Qi, L. (2004). Has a high-stakes test produced the intended changes? In L. Ceng, Y. Watanabe & A. Curtis (Eds.), Washback in language testing: Research contexts and methods (pp. 147-170). Mahwah, NJ: Lawrence Erlbaum.
Qi, L. (2005). Stakeholders' conflicting aims undermine the washback function of a high-stakes test. Language Testing, 22(2), 142-173. doi: 10.1191/0265532205lt300oa
Qi, L. (2007). Is testing an efficient agent for pedagogical change? Examining the intended washback of the writing task in a high-stakes English test in China. Assessment in Education, 14(1), 51-74. doi: 10.1080/09695940701272856
Roever, C., & Pan, Y. (2008). General English Proficiency Test. Language Testing, 25(3), 403-408. doi: 10.1177/0265532208090159
Saif, S. (1999). Theoretical and empirical considerations in investigating washback: A study of ESL/EFL learners. Unpublished doctoral dissertation, University of Victoria, Canada.
Saif, S. (2006). Aiming for positive washback: A Case study of international teaching assistants. Language Testing, 23(1), 1-34. doi: 10.1191/0265532206lt322oa
Schmidt, C. (2004). The analysis of semi-structured interviews. In U. Flick, E.V. Kardoff & I. Steinke (Eds.), A companion to qualitative research (pp. 253-258). London: Sage.
Shih, C. (2007). A new washback model of students' learning. The Canadian Modern Language Review, 64(1), 135-162.
Shohamy, E., Donitsa-Schmidt, S., & Ferman, I. (1996). Test impact revisited: washback effect over time. Language Testing, 13(3), 298-317. doi:10.1177/026553229601300305
Shohamy, E. (2001a). The power of tests. Harlow, UK: Pearson Education Limited.
Shohamy, E. (2001b). Democratic assessment as an alternative. Language Testing, 18 (4), 373-391. doi: 10.1177/026553220101800404
Stoneman, B. W. H. (2006). The impact of an exit English test on Hong Kong undergraduates: A study investigation the effects of test status on students' test preparation behaviours. Unpublished doctoral dissertation, The Hong Kong Polytechnic University, Hong Kong, China.
Tsai, Y., & Tsou, C. (2009). A standardised English language proficiency test as the graduation benchmark: Student perspectives on its application in higher education. Assessment in Education: Principles, Policy & Practice, 16(3), 319-330.
Vermon, L. J., Baytops, J. L., McMahon, P. J., Padden, D. H., Walther, T. C. (2003). The reality of increased graduation requirements: A nationwide view of diploma options. National Association of Secondary School Principals. NASSP Bulletin, 87, 635, 2-14.
Wall, D. (2000). The impact of high-stakes testing on teaching and learning: Can this be predicted or controlled? System, 28, 499-509. doi: 10.1016/S0346-251X(00)00035-X
Wall, D. (2005). The impact of high-stakes examinations on classroom teaching. Cambridge University Press.
Wall, D., & Alderson, J. C. (1993). Examining washback: The Sri Lankan impact study. Language Testing, 10(1), 41-69. doi: 10.1177/026553229301000103
Wall, D., & Horak, T. (2006). The impact of changes in the TOEFL examination on teaching and learning in central and eastern Europe: Phrase 1, the baseline study. Princeton, NJ: Educational Testing Service.
Watanabe, Y. (1996). Does grammar translation come from the entrance examination? Preliminary findings from classroom-based research. Language Testing, 13(3), 318-333. doi: 10.1177/026553229601300306
Watanabe, Y. (2004). Teacher factors mediating washback. In L. Cheng, Y. Watanabe & A. Curtis (Eds.), Washback in language testing: Research contexts and methods (pp. 129-146). Mahwah, NJ: Lawrence Erlbaum Associates.
eContent Home
