SEARCH PAPERS   

AWA: Academic Writing at Auckland

About this paper

Title: Journal article evaluation: Warren Matsuoka & David Hirsh

Evaluation: 

An Evaluation (or Critique) evaluates an object of study. This requires understanding of the object and the criteria by which to evaluate it.

Copyright: Karen Dombroski

Level: 

Honours year (postgraduate)

Description: Choose one of the vocabulary research articles in the issue and write an evaluation of it, following the guidelines below: Why is the article interesting to you? How does it relate to your experience as a language learner or language teacher? What are strong features and weak features (or limitations) of the research reported in the article? Are there some aspects of the research that you don't really understand? What are they?

Warning: This paper cannot be copied and used in your own assignment; this is plagiarism. Copied sections will be identified by Turnitin and penalties will apply. Please refer to the University's Academic Integrity resource and policies on Academic Integrity and Copyright.

Writing features

Journal article evaluation: Warren Matsuoka & David Hirsh

  1. Introduction

Matsuoka and Hirsh’s journal article (2010) reports on a study of one commercial English Language Teaching (ELT) coursebook for upper-intermediate level learners – New Headway Upper Intermediate Student’s Book (Soars & Soars, 2005).  The investigation analyzes vocabulary occurrence in the coursebook text as an indicator of opportunities for incidental vocabulary learning through reading. Learning is considered in terms of both breadth and depth of word knowledge through

1) examination of vocabulary size required to read the book, and

2) identification of the aspects of word knowledge included in the text
.

Attention is given to three features of word occurrence which promote favourable conditions for incidental vocabulary learning – word repetition, spacing of repetitions and the type of word repetitions (Nation, 2001). Previous related research is cited in support of the study’s approach, including earlier studies using word frequency lists to investigate vocabulary load and learning possibilities in texts. The researchers claim theirs is the first published study to examine vocabulary occurrence in a commercial ELT coursebook and state that the overall intention of the study is ‘to provide a methodology and initial findings to inform future studies of this type’ (2010:59).

The aim of this assignment is to analyze and evaluate the article and its research results, in light of its strengths, weaknesses and pedagogical implications.

  1. Strengths

The strength of the article lies in the attention given to the compilation of computer ‘baseword lists’ (2010:60), i.e. word frequency lists against which the vocabulary in the coursebook is compared. Seven lists are used, including West’s (1953) General Service List (GSL) of the first and the second 1000 most frequent words in English, Coxhead’s (2000) Academic Word List, and four created by the researchers. The Range computer program (Heatley, Nation & Coxhead, 2002) is employed to show coverage of these lists in the text, the frequency of word families, and the range or spacing of words in coursebook chapters.

To evaluate the vocabulary coverage needed to read a coursebook with adequate comprehension, the measure based on word tokens is chosen, rather than number of word families. This approach is deemed ‘safest’ by Nation (2001:146) because word family totals can vary depending on the type, length and similarity of texts. The selected figure of 95% or one unknown word in twenty appears to be a reasonable level of coverage for understanding an instructional text designed to be read in the classroom with teacher assistance. Nation’s (2001:148) comment supports this choice, saying, ‘learners would need to have 95% coverage for learning vocabulary from meaning-focused input, and 98-100% coverage for fluency development’.

In the examination of word frequency and patterns of repetition in the text, the word family is the unit of classification, i.e. a base word and its inflected and derived forms represent one unit. The current study addresses the issue of consistency of word family membership by using a standard of classification presented in research by Bauer & Nation (1993). Four new baseword lists were created to include proper nouns, metalinguistic terms, references to text structure, and common technological words.  The items are assumed to be understood by upper-intermediate learners who know the most frequent 2000 words in English. The treatment of proper nouns as known words has precedents in previous research, based on the argument that their meanings are revealed in the reading context (Hirsh & Nation, 1992). Learners’ familiarity with words in the other new lists is backed by researcher experience and the fact that new technology words have entered the language since the second 1000 list was first compiled. Words assigned to these lists appear suitable but a caveat is in order. As no one besides the authors provided input into their selection, the question remains as to whether other words should have been included.

The authors refer to a ‘principled approach’ and ‘consistency’ (2010: 61-62) as important considerations when undertaking text/list modifications. Their strategy for handling hyphenated compound words is an illustration of this. Again a caution is warranted in regard to one particular issue. The inability of the Range program to distinguish homographs is discussed in a previous study (Nation & Wang, 1999) but is not addressed here.

  1. Weaknesses    

Three aspects of the article are of concern: 1) the limited scale of the study and the effect this could have on the perceived value of the research, 2) the lack of rationale behind the selection of text for analysis, 3) the selective focus of the study.

3.1   Scale of study

The study examines the New Headway students’ book in depth. As this is limited to one text, however, the study findings cannot be generalized to other coursebooks. No discussion of this limitation is undertaken in the research.

The size of the case study used to investigate vocabulary learning opportunities in the coursebook is also extremely small – only six words were chosen, representing 3.2% of second 1000 word families repeated five or six times, or 1.5% of those repeated from five to over ten times. Although the authors do admit that random sampling may have limited the range of features of word knowledge found, no comment is made regarding the effect of such a narrow sample on the research results.

3.2  Rationale for selection of text for analysis

No reasons are given for the choice of coursebook title or level. Unlike the previous study by Nation and Wang (1999), which questions the representative nature of the texts under investigation, Matsuoka and Hirsh do not ask, ‘how typical is the coursebook examined here?’ or consider how this might impact on study outcomes.

The authors state that ‘all words appearing in the twelve chapters of the text were analyzed’. Within each chapter unit, however, learners are directed to related material, which is located at the back of the book. This includes the entire writing syllabus, a grammar reference section, scripts for listening exercises which are sometimes used as read-while-listening activities, and an extra material section of communication/practice activities. The latter comprises information which would cause a unit to overrun the standard 10 pages, or which needs to be kept separate (e.g. jigsaw reading). It is unclear from the statement above whether words from these sections are part of the analysis. A personal examination of the coursebook, focusing on several of the words in the case study, reveals this material has not been included. For example, the word ‘invite’, which is listed  in the study  (2010:65) as occurring seven times, with only one derivation ‘invitation’, also appears as ‘invitations’  in the Unit 6 grammar section (Soars & Soars, 2005:146). Similarly, there are more occurrences than the ten recorded for the word ‘island’ – including ‘Islands’ in the Unit 1 writing section, ‘Islands’ in tapescript 2.8, and ‘island’ in Unit 2 extra material (Soars & Soars, 2005: 111,126 &155).

If the intention of the study is to cover all the words students would encounter while working through the book’s syllabus, this coverage obviously falls short. Unexamined reading material amounts to 49 extra pages over the 104 pages comprising the twelve units, representing 32% of the total coursebook. Even if listening scripts are omitted, this still amounts to almost a quarter of the entire book.

3.3  Selective focus of study

The authors draw attention to three text-based studies – Hwang and Nation (1989), Hirsh and Nation (1992), and Nation and Wang (1999) – as providing the foundation for their research. Differences exist, however, in the type of reading text under investigation.

The previous studies examine newspapers, fictional texts and graded readers – genres which focus mainly on reading for information, pleasurable or extensive reading.  A multi-syllabus ELT coursebook like New Headway is an instructional document, compiled with a number of learning goals in mind, one of which is vocabulary learning through reading. In their article on graded readers, Nation and Wang (1999) acknowledge that examining the text with a narrow focus, i.e. viewing it only in terms of how reading contributes to vocabulary acquisition, limits research findings:

‘This is not the main goal that graded readers were designed for and is clearly not their only goal. This must be borne in mind when interpreting results of this study.’ (Nation & Wang, 1999:357)

No such caveat is included in Matsuoka and Hirsh’s study of an ELT coursebook.

Reading is only one way ELT coursebooks approach vocabulary acquisition. Thornbury (2002) outlines three main methods: 1) special vocabulary sections, 2) integration into text-based activities (reading/listening), and 3) incidental exposure, through grammar explanations, practice activities and instructions.  As he points out, how a particular coursebook approaches vocabulary can vary markedly.

In the matter of incidental vocabulary learning, word repetition is an essential factor (Nation, 2001). The results of the above-mentioned studies indicate that this is an incremental process and the types of texts providing the most favourable learning conditions are either long, continuous texts or shorter passages related by topic. This would suggest that a coursebook divided into themed units, each incorporating reading texts from a range of sources, along with tasks dealing with grammar and other unrelated areas of instruction, is likely to be a poor candidate for incremental acquisition. This aspect of coursebook design, although not raised in the article, is commented on in the previous study by Hwang and Nation (1989).

With regard to direct methods of vocabulary teaching, the authors of New Headway claim their book has ‘a strong lexical syllabus’ (Soars & Soars, 1999:5). In addition to using reading passages as a context for language-focused learning, the coursebook includes at least one section per unit devoted to vocabulary study and practice. Focus is given to word form, meaning and use – all important aspects in knowing a word (Nation, 2001). This involves work on words with variable stress, homonyms/homophones, affixes, compound words, synonyms/antonyms and collocations, including those with high-frequency verbs. O’Dell (1997:267) comments on the varied approach that has been adopted by the book over time, saying it offers ‘more breadth and greater depth’ in developing vocabulary knowledge. Although breadth and depth are discussed in the current study, this is only from the point of view of incidental learning of vocabulary through reading. Analysis of the direct teaching sections/activities in the coursebook may be beyond the scope of this article, but no acknowledgement is given to its potential contribution towards acquisition.

  1. Findings and Implications

Study outcomes suggest the coursebook presents good opportunities for improving depth of knowledge of the second 1000 most frequent words through incidental reading. This finding is based on the high rate of repetition of these known words across the twelve units, evidence of spaced repetitions and a variety of collocations discovered in the small case study. Conversely, the low rate of repetition of academic and low frequency words indicates limited opportunities exist for incidental learning of these unknown lexical items. Pre-teaching of academic words in the text would be needed in an effort to supplement known words and reach the 95% proposed level for adequate text coverage.

The key recommendation for English as a Foreign Language (EFL) classrooms, made on the basis of these results, is that in order to learn vocabulary outside the high frequency lists, ELT coursebooks should be supplemented by extensive reading programmes, which preferably would include some form of direct vocabulary teaching.

Several presuppositions underlie comments made in the concluding remarks:

  • The EFL context is the same in classrooms everywhere, with no need for definition or qualification.
  • The role of the ELT coursebook in vocabulary development is to support learning through reading. Materials for ‘direct vocabulary instruction and instructionally-enhanced reading’ (2010:67) exist in a sphere outside the coursebook.
  • Study outcomes relate to classroom teachers, learners and other researchers. Publishers and coursebook writers are not part of the equation.

Let us consider each of these in turn. Firstly, an important consideration in any investigation is context. EFL is normally associated with ELT in countries where English is not the primary language. The term also applies, however, to classroom situations in English-speaking countries where the learners are adult non-native speakers of English who have come as visitors from overseas to study English (Tomlinson, 2008). The commercial ELT coursebook is used in both scenarios, but there are significant differences in constraints, particularly in terms of range of choice of materials, time available, class size, and access to learning opportunities outside the classroom.

Matsuoka and Hirsh’s recommendations for setting up extensive reading programmes in classrooms as a supplement to coursebook reading also fails to take into account the practicalities of the local context. Horst (2005) points to several caveats in her extensive reading study, one of which is that ‘volume is clearly crucial’. According to Laufer (2003:573), the goal suggested by Nation & Wang (1999) of reading one or two books a week in order to provide frequent repeated encounters with new vocabulary, is ‘unrealistic’ in light of tight classroom timetables. Added to this is the potential problem of finding enough suitable material for the size of class and at the desired level. Nation and Wang’s (1999) study of graded readers shows that for higher level learners there is a huge gap between the books’ top level of 3000 words and the vocabulary size needed for the transition to comfortable reading of authentic texts.

The second point centres on the treatment of the coursebook as a tool for vocabulary learning. Laufer’s (2003, 2009) research is cited, which highlights the effectiveness of word-focused activities as an adjunct to incidental acquisition through reading, and emphasizes the need for both in a well-balanced second language course. Nevertheless, the current study views this as a strategy to support an extensive reading programme that supplements the use of an ELT coursebook. No comment is provided on how the coursebook itself contributes to direct vocabulary learning or to suggest that it might be used more effectively.

Finally, the third consideration illustrates how the text developers, i.e. the publishers or coursebook writers, are viewed. Their notable absence from the discussion is surprising, as Nation and Wang’s (1999) study of graded readers, which the authors claim to be a key study on which their research is based, includes as specific section with recommendations for publishers. Tomlinson (1998) goes further with his proposal that researchers, teachers, publishers and writers pool expertise and resources in order to generate more effective materials for learners.

  1. Conclusion

Matsuoka and Hirsh’s article investigates an ELT student’s coursebook as a means of incidental vocabulary learning through reading, focusing on the role of repetition. In this selective approach, direct teaching of the coursebook’s vocabulary syllabus is not considered.

In general, the authors are successful in their attempt to establish a methodology that evaluates word occurrence in such a coursebook. Compilation of word lists against which the text is compared is based on a principled and consistent approach. Unanswered questions remain, however, over the issue of homographs and the selection of a limited portion of the text for analysis.

Pedagogical implications are limited by the scale and narrow focus of the study. The examination of one coursebook makes it uncertain as to whether findings would be similar in other titles. Also, research into the type of word repetitions in the text is based on an extremely small case study. Findings, showing vocabulary learning through reading a coursebook text is not sufficient and needs to be supplemented, are not unexpected, as this is not the main goal of the text. Nevertheless, how learning is to be effectively achieved remains unresolved. Recommendations to practitioners for setting up extensive reading programmes fail to take into account practical considerations, such as time and resources in different classroom contexts. Certainly, direct teaching through form-focused activities is needed for a balanced programme, but the role the coursebook plays in this regard is unclear and requires further investigation. So too is the contribution that coursebook publishers/writers can make to improving vocabulary acquisition.

From a personal standpoint, the study reinforces the practices I have observed in the EFL context where I teach, at a private language school in New Zealand. In addition to coursebook-based General English lessons, learners are given opportunities for exposure to longer texts through special reading/writing skills classes. Access to a substantial number of graded readers, audio books, magazines, newspapers and textbooks is also provided through a Student Learning Centre.  For those wanting extra form-focused learning, vocabulary and academic options classes are offered. In my role as a classroom teacher in this supportive environment, I am able to provide word-focused instruction for my learners from a range of coursebooks and other materials, and also promote regular extensive reading in an effort to help them improve their breadth and depth of vocabulary knowledge.

Word Count:  2740

 

 

References

Bauer, L., & Nation, P. (1993). Word families. International Journal of Lexicography, 6 (4), 253-279.

Coxhead, A. (2000). A new academic word list. TESOL Quarterly, 34 (2), 213-238.

Heatley, A., Nation, I. S. P., & Coxhead, A. (2002). Range computer program. http://www.vuw.ac.nz/lals/staff/Paul_Nation.

Hirsh, D., & Nation, P. (1992). What vocabulary size is needed to read unsimplified texts for pleasure? Reading in a Foreign Language, 8 (2), 689-696.

Horst, M. (2005). Learning L2 vocabulary through extensive reading: a measurement study. The Canadian Modern Language Review, 61 (3), 355-382.

Hwang, K., & Nation, P. (1989). Reducing the vocabulary load and encouraging vocabulary learning through reading newspapers. Reading in a Foreign Language, 6 (1), 323-335.

Laufer, B. (2003). Vocabulary acquisition in a second language: do learners really acquire most vocabulary by reading? Some empirical evidence. The Canadian Modern Language Review, 59 (4), 567-587.

Laufer, B. (2009). Second language vocabulary acquisition from language input and from form-focused activities. Language Teaching, 42 (3), 341-354.

Matsuoka, W., & Hirsh, D. (2010). Vocabulary learning through reading: does an ELT course book provide good opportunities? Reading in a Foreign Language, 22 (1), 56-70.

Nation, I. S. P. (2001). Learning vocabulary in another language. Cambridge: Cambridge University Press.

Nation, P., & Wang, K. (1999). Graded readers and vocabulary. Reading in a Foreign Language, 12 (2), 355-380.

O’Dell, F. (1997). Incorporating vocabulary into the syllabus. In N. Schmitt & M. McCarthy (Eds.), Vocabulary: Description, acquisition and pedagogy (pp. 6- 19). Cambridge: Cambridge University Press.

Soars, L., & Soars, J. (1999). New headway upper intermediates teacher’s book. Oxford: Oxford University Press.

Soars, L., & Soars, J. (2005). New headway upper intermediates student’s book – the new edition. Oxford: Oxford University Press.

Thornbury, S. (2002). How to teach vocabulary. Harlow, Essex: Pearson Education.

Tomlinson, B. (1998). Introduction. In B. Tomlinson (Ed.), Materials development in language teaching (pp. 1- 24). Cambridge: Cambridge University Press.

Tomlinson, B. (2008). Language acquisition and language learning materials. In B. Tomlinson (Ed.), English language learning materials (pp. 3-13). London: Continuum.

West, M. (1953). A general service list of English words. London: Longman, Green & Co.