This is the third entry on the topic of educational research. Here I will look at one particular example which is the title of this post. The title quite clearly states that we can expect “a qualitative study” which means that the collected data will be predominantly words, as opposed to a quantitative study which heavily relies on numbers and statistics. We also can assume that there will be less reliance on a literature review since there is probably inadequate pre-existing information. (This is a question to ask though. Is it that there is no information on the area, or was it neatly kept for the findings section?)
Although there are many studies into teaching grammar, finding anything that is only qualitative in the literature was surprisingly rare. This is the first reason this article may seem dated. However, this qualitative study has been cited over 600 times from one source online. It also obviously holds an important place considering Rod Ellis, one of the heavyweights of grammar teaching, has quoted it too. Finally, the “TESOL quarterly” journal is rated as Q1 and would not publish this study if it weren’t peer-reviewed. This study then must hold significant value to the field of second language acquisition and TESOL.
The author or researcher, Borg (1998, p.9) states the problem is that there is not enough research into the cognitive roots of teachers’ choices in regards to grammar instruction in second language classrooms. This is stated in the abstract, making it quite clear that this is going to be an exploratory piece as qualitative studies usually are and since there is no literal problem already identified that requires study. Just the lack of studying the field seems to be enough to justify it, which from the abstract at least, is not as compelling or clear as to why the study is important as it could be. (Or is this just the nature of this style of research?)
Borg then defines the terms in the title: “pedagogical system” as the accumulation of what the teacher knows, believes, thinks, assumes etc. which forms those personal grammar teaching decisions. He acknowledges that it is an “interpretative study” here too. Interpretation in qualitative research means a number of things. According to Creswell and Guetterman (2019, p.259, 485) the researcher “steps back” to consider the overall inferred meanings and draw out the bigger concepts for possible conclusions to be made either through their own personal positions, comparison to what already exists in the literature or both.
To someone new approaching the evaluation of research, the first impression may be: where is the title for the “REVIEW OF THE LITERATURE” with this article? Indeed it would help make it clear that this is what we start to read into after the abstract. However, we learn again from Creswell and Guetterman (2019, p.79-80) that it is common practice for quantitative research to include the title since the review plays a much greater role and therefore warrants a section to itself, but not so important for qualitative research, since here the review is minimal and only serves the purpose of “justifying or documenting” the need for the study.
That is why the introduction (which also lacks a title) is in essence also the literature review. The introduction takes a broad to narrow approach on the phenomena. Starting with the general field of education on the most similar topic, then shifting into TESOL, and finally establishes the importance of grammar specifically in TESOL and finally not so much a research question, but a need to study this area.
This article explores the nature of the maps L2 teachers utilise in determining the role and nature of grammar teaching in their classroom practice.
(Borg, 1998, p.10)
The expectation for the reader of the title and this quoted statement is on the plural noun “teacherS.” There may be generalisations made from a number of teachers’ experiences and therefore quite a great deal more a defensible argument may finally be formed. Still, this could be ambiguously interpreted that exploration will take place in the literature after the data has been collected. A reading which is weaker, but weakens further when noting “this article explores the nature of the mapS.” Here “maps” is ambiguous because they could be read as the “guidelines” for the choices one teacher makes.
The first heading and opening sentence in the paper is:
PURPOSE AND CONTEXT
The initial aim of the study was to provide an emic perspective on the manner in which an L2 teacher’s personal pedagogical system informed his approach to grammar.
(p.10)
The expectations of the reader are now met with a singular case only and in addition, a feminist lens would also query the pronoun choice. This clearly raises the question: why the intention of selecting only one male teacher? There is at no time an answer given to this.
In this quote, there is a key concept from anthropology: “emic” as an insider’s perspective (“etic” is the opposite). Also, since we are still on the first page of the study, we can raise the question: what kind of qualitative study is this? It has not been spelt out for us. If this were traditional ethnography in the field of TESOL, we would expect it would typically be about a group of teachers and or their students. Or look at one classroom and talk to all involved in it. We would expect a triangulation or multiple perspectives to get that thick description. (Richards, 2003, p.15)
In the same section, we discover a little about both why he was chosen and which students would be in the presence of the data collection:
He was one of the most highly qualified and experienced teachers in his institute and was chosen for this study on the basis of his reputation as a professionally committed L2 teacher. The fieldwork for this study was conducted with a group of intermediate-level 18- to 35-year-old EFL students from Germany, Poland, Switzerland, and Italy.
The choice of Malta is only discovered at the end of the paper where there is a note where Borg is working.
There is no mention before this that the study was interested in what successful teachers do with adult students nor has there been a mention why this age group was chosen, or more importantly whether ethical considerations were considered. How will these adults be affected by the researcher’s presence? How did the researcher gain permission to sit these classes?
There is in the footnote (p.11) reference at least to the protection of the anonymity of the teacher:
“… in order to protect the teacher’s anonymity I am unable to provide any additional specific information about his back”
(p.11)
We can now confirm that the design is a case study, a type of ethnography, of one teacher only. It may further be categorised, following Creswell & Guetterman’s (2019, p.479) explanations, as an intrinsic case because he is a uniquely experienced teacher, and possibly be an instrumental case on best practise of grammar instruction. The benefit of a singular case only is that it will allow for greater depth than if there were multiple cases.
When moving onto the next section, there is a legitimising of the framework which is elaborated on in its own section:
RESEARCH FRAMEWORK
The research I present here was conceived within an exploratory-interpretive paradigm (Grotjahn, 1987). Within this framework, the goal of research is to understand the inner perspectives on the meanings of the actions of those being studied.(p.11)
Terms such as exploration, interpretation, and inner perspectives had already been discussed but are now connected to the literature by one of the specialists in the fields of research. Ellis (2010, p. 125-6) considers Grotjan’s 1987 article as one of the most informative articles written about research methodology in the field of TESOL. Ellis also positions research which is “exploratory-interpretive” purely as qualitative through a reading of Grotjan and also refers to another study on only one teacher’s development. The validity of this type of inquiry now begins to stand its own ground. (Although one could have reservations with the connections between Borg and Ellis both being in New Zealand at the same time, and not declaring this. They do refer to each others’ work quite conveniently.) Also there is a noticeable shift from the third person formality in the abstract to the first person “I” for the researcher. Could this make the reader question how vigorous a study this will be?
The following sentence makes it also clear that Borg is explaining alienating concepts for a wider audience to his research and research terminology in general., making it more accessible for the layman to the field. In simple terms, it becomes more readable.
It is characterised by an idiographic conceptual framework (i.e., which focuses on the meaning of particular events), by its aim to generate rather than to verify theory (i.e., it does not set out to test a priori hypotheses), and by naturalistic rather than experimental research designs. (Borg, 1998, p.11)
The reference to “naturalistic” is not explained, however. It suggests that the teacher will be studied in his everyday working environment in ways most humans inquire about things around them, as opposed to carrying out experiments and surveys which are unnatural. He will be observed how he teaches and interacts with the students, and the researcher will listen to what he has to say about it, think about what the teacher achieves and does, comprehend what his narrative means and then report it back to the teacher. (Beuving & De Vries, 2015, p.15)
Questions continue to be unanswered: Why idiographic? Does that refer to the individual, and non-general characteristics of this teacher? Will Borg be able to defend more general claims about the gaps in research based on idiosyncrasies of this one successful teacher? Generalizability is a very common problem when it comes to case study research. (Duff 2008, p.44) Indeed, why did Borg not identify this earlier as an individual case study? It may have been easier to follow and be believable to signal the reason for an individual case study on the grounds of being able to go into greater depth of the causes of what makes a good teacher, like his background education, when it is only one individual. For Duff (2008, p.43), researching a unique case is beneficial since it has everything necessary for an easy to read profound examination which might produce novel challenging suppositions, representations and comprehension of systems that take place in a language classroom.
Borg continues to explain awareness of the self-reflexivity of the research ahead using the literature to justify his stance on his reasoning, that what we know is not objective, but subjectively created.
This approach to research views knowledge not as an objective reality that the researcher describes scientifically; rather it acknowledges the personally constructed nature of all knowledge (Bassey, 1991). A consequence of this epistemology is that, from an exploratory-interpretive perspective, research is conceived as a task of interpreting human action by understanding why people behave in the way they do. (Borg, 1998, p.11)
Borg’s ontology, that is his philosophy of being, is more relativist in that it suggests that various individuals, such as our teacher in this study, has his own belief system for his particular teaching for his particular class of students. It assumes a more subjectivist position in that what he knows about teaching grammar is constructed locally over time. The question for this critique, however; is whether Borg’s own research of the teacher reflects this in the paper under critique.
The overall effect on the reader may be that the study seemed to be transparantly revealing ways of working in the classroom that did not exist documented before. For example, why was literature on the identified key aspect of “error analysis” not connected to a wealth of existing literature over the last 50 years? For example Richards (1980). Second language acquisition: Error analysis. Annual Review of Applied Linguistics, 1, 91-107 or Celce‐Murcia (1991). Grammar pedagogy in second and foreign language teaching. TESOL quarterly, 25(3), 459-480 in which error analysis has been shown to have been a large part of and recommended way of improving students’ grammar? Since the method for collecting was cyclical in that what the researcher observed was then reflected back to the teacher, was it not possible to have introduced a connection to the history of this?
Although the case study here of only one teacher, it should be seen as it is positioned against other possibilities, a survey of hundred’s of teacher’s beliefs on similar topics, that Borg (2008) headed into with a quantitative angle almost a decade later.
The data collection and analysis section also contains references to the appendices with more details which helps answer questions that arise. The researcher states that the pre-observation interview was semi-structured. This means that he had already chosen questions that may frame the ideas, although he left it open to other directions. One must ask though that for theories to truly arise from the data, would it be better not to start with any questions or interview prior. If one is already set to ask a series of questions, are they really open to just observing?
In the following section
PRESENTATION OF DATA
… Thus the presentation of the data is organised around teaching behaviours that characterised the teacher’s approach to grammar work; I discuss these behaviours in turn with reference to the teacher’s own analytic commentary on them, and it is through an analysis of this commentary that the key features of the teacher’s pedagogical system emerge. (6 SEE FOOTNOTE) This form of presentation mirrors and makes transparent to readers the inductive processes of data analysis that were central to this study; it also ensures that all assertions in the account are clearly grounded in the data from which they emerged.
footnote: 6 See Appendix C for a list of areas of classroom practice that informed the selection of the teaching behaviours I illustrate in this account. The features of the teacher’s pedagogical system I discuss here are taken from Appendix D. A full discussion of all the categories listed in Appendixes C and D is not possible in the space available; hence, this article focuses on recurrent teaching behaviours that provide insight into the predominant features of the teacher’s pedagogical systems.
(p.14)
we are introduced to how the categories were chosen. It is claimed that the inductive process is made “transparent” to the reader. Alhtough not stated earlier, it seems assumed that this is grounded theory. The footnotes continue to include insights on limitations not expressed elsewhere. Where it is stated that there is not space to go into all the categories of the data.
How can we judge a piece of research without considering what may be better or worse?
We could consider the title for a start of Watson’s (2015). “The problem of grammar teaching: a case study of the relationship between a teacher’s beliefs and pedagogical practice” being much clearer than Borg’s. Watson also provides a much more powerful reason for the research and it is quite clearly set up throughout. The topic is quite similar and actually refers to Borg a number of times. It also outlines the reason for a case study more transparently. It is clearer what kind of research it will be and the literature review prior is more convincing.
Indeed, Watson shows much more reflexivity with the observation, remarking that the teacher stated she would do something she doesn’t usually because she was going to be observed. This is something that Borg did not do.
There are also other pieces of research such as by Johnston and Goettsch (2000) that spell out the research question much more clearly:
INTRODUCTION
This study looks at aspects of the knowledge base that experienced ESL teachers draw on in their work. We will analyze interviews with teachers and examine classroom data comprising explanations given by these teachers in the course of their day-to-day teaching in classes labelled ‘Grammar.’ Our principal research question is, ‘What kinds of knowledge do practising teachers actually possess and utilize in their teaching?’ … (p.438)
There is no real return to take the findings from the teacher back to the literature either. For example, what was already known about error analysis. Did what the teacher say about this reflect what has already been theorised?
Finally, Borg (1998) offers a detailed case study of a single teacher in a Maltese EFL classroom. Though the class itself is not exclusively grammar-focused, Borg concentrates on grammar teaching; like us, he is interested in looking at ‘practicing teachers’ understandings of
(p.442) Johnston and Goettsch
grammar teaching’ (p. 32). His approach is to analyze the ‘pedagogical system’ of the teacher he studied. This pedagogical system includes a belief in the importance of work on grarrunar both in terms of awareness-raising and in terms of applying grarrmiatical rules; the need to relate grarrunar work to the state of knowledge and the needs of the actual students in the class; and the need to engage the students actively in their own learning. These beliefs emerge not only from interviews with the teacher but also from an analysis of his classroom practice. Finally, Borg comments on the importance of training and experience in the development of the teacher’s pedagogical system.
REFERENCES
Beuving, J., & De Vries, G. (2015). Introduction: The arc of naturalistic inquiry. In Doing Qualitative Research: The Craft of Naturalistic Inquiry (pp. 15-26). Amsterdam: Amsterdam University Press. doi:10.2307/j.ctt130h8g7.6
Borg, S. (1998). Teachers’ pedagogical systems and grammar teaching: A qualitative study. TESOL Quarterly, 32(1), 9-38. http://tesol.aua.am/tqd_2000/tqd_2000/tq_d2000/Vol_32_1.pdf#page=10
Borg, S., & Burns, A. (2008). Integrating grammar in adult TESOL classrooms. Applied linguistics, 29(3), 456-482.
BURNS, A., FREEMAN, D. and EDWARDS, E. (2015), Theorizing and Studying the Language‐Teaching Mind: Mapping Research on Language Teacher Cognition. The Modern Language Journal, 99: 585-601. doi:10.1111/modl.12245
Celce‐Murcia, M. (1991). Grammar pedagogy in second and foreign language teaching. TESOL quarterly, 25(3), 459-480.
Creswell, J., & Guetterman, T. (2019). Educational research : planning, conducting, and evaluating quantitative and qualitative research (Sixth edition.). New York, NY: Pearson. ProQuest Ebook Central, https://ebookcentral.proquest.com/lib/qut/detail.action?docID=5812713
Duff, P. (2008). Case study research in applied linguistics . Boca Raton, FL: Lawrence Erlbaum Associates.
Dufon, M. (1993). Ethics in TESOL Research. TESOL Quarterly, 27(1), 157–160. https://doi.org/10.2307/3586970
Ellis, R. (2010). Editorial. Language Teaching Research, 14(2), 125–127. https://doi.org/10.1177/1362168809353863
Johnston, B., & Goettsch, K. (2000). In search of the knowledge base of language teaching: Explanations by experienced teachers. Canadian Modern Language Review, 56(3), 437-468.
Mahboob, A., Paltridge, B., Phakiti, A., Wagner, E., Starfield, S., Burns, A., … De Costa, P. (2016). TESOL Quarterly Research Guidelines. TESOL Quarterly, 50(1), 42–65. https://doi.org/10.1002/tesq.288
Richards, J. C. (1980). Second language acquisition: Error analysis. Annual Review of Applied Linguistics, 1, 91-107.
Richards, K. (2003). Qualitative inquiry in TESOL . New York: Palgrave Macmillan.
Watson, A. (2015). The problem of grammar teaching: a case study of the relationship between a teacher’s beliefs and pedagogical practice. Language and Education, 29(4), 332–346. https://doi.org/10.1080/09500782.2015.1016955