A new research method for SLA researchers: Scoping Review!
The field of SLA and applied linguistics has witnessed a rapid growth in the number of studies employing research synthesis and meta-analysis over the years—but most of the time, these different methods have been used interchangeably. One reason for this confusion might be the availability of many relevant yet different terminologies (e.g., systematic review, research synthesis, meta-analysis, methodological synthesis; just to name a few). A new research method slowly finding its way in L2 research is ‘scoping review’. A scoping review shares so many similarities with research synthesis but obviously there are certain differences. Mays, Roberts and Popay (2001) argue that scoping review is useful for “mapping rapidly the key concepts underpinning a research area and the main sources and types of evidence available, and can be undertaken as stand-alone projects in their own right, especially where an area is complex or has not been reviewed comprehensively before (p. 194). Similarly, Arksey and O’Malley (2005) list four common reasons to employ scoping review: (1) to examine the extent, range and nature of research activity; (2) to determine the value of undertaking a full systematic review; (3) to summarize and disseminate research findings; (4)to identify research gaps in the existing literature.
To read up on an example scoping review study in L2 research, please see Gurzynski-Weiss, and Plonsky’s (2017) chapter entitled “Look who’s interacting: A scoping review of research involving non-teacher/non-peer interlocutors.”
Probably, a few, if not many, of you have heard of the name Paul Erdős. He was a Hungarian-born, phenomenal mathematician who loved only numbers. His research interests centered mostly around number theory and combinatorics. He was also one of the most prolific mathematicians of the 20th century. He has more than 1,500 mathematical papers! It is simply amazing! What is even more interesting is that he did a lot of joint research, collaborating more than 500 authors, in a field where collaboration is said to be very scarce. His immense joint research has given rise to the Erdős number project. Basically, an Erdős number indicates the collaborative distance between an author and Erdős. So, those who directly co-authored with Erdős have Erdős number 1. And those who co-authored a paper with someone with Erdős number 1 (but not with Erdős himself) have Erdős number 2, and so forth. Erdős’s Erdős number is 0. I think this is a great way to honor Erdős and to boost the collaborative work in the field at the same time. Given that, I think it would be interesting to have a similar project in the field of second language acquisition and applied linguistics. For instance, how does Gass number sound? (Professor Susan Gass is one of the leading L2 scholars, with a lot of collaborative work). My Gass number is 2 for now. What is your Gass number?
Missing data occur in almost any research we conduct, but a more important point is whether and how we should deal with missing data. Most commonly, researchers simply ignore the missingness or use the classic methods of listwise deletion or mean substitution. However, in many cases neither of these produces good results. The two important steps in missing data management are to determine the type and amount of missing data. In particular, the type of the missingness allows us to determine how to handle the missing data. However, there are limited discipline-specific sources on handling missing data in the fields of SLA and AL. Hopefully, with a couple of SLAtisticians, I am going to provide a picture of the current practices for missing data management in L2 research. Stay tuned for the details of this study!
In the field of SLA, L2 researchers have begun to employ more advanced and sophisticated statistical methods over the years. Factor analysis (FA) is a such method. FA is a series of complex structure-analyzing procedures . Although it is mostly used with survey data, it can also be used for hypothesis testing, instrument development, or summarizing patterns of interrelationships.There are two broad types of FA: confirmatory factor analysis (CFA) and exploratory factor analysis (EFA). CFA is generally preferred when researchers have a specific expectation regarding the number and nature of factors whereas EFA is, as its names suggests, exploratory in nature. Along with the differences between CFA and EFA, there is some ambiguity in the terminology used within EFA. EFA is often used as umbrella term covering both principal components analysis (PCA) and EFA, although there are two schools of thought on the differences between PCA and EFA. In addition to the terminological ambiguity, there are several decision points that researchers need to consider when conducting FA such as the decisions on the appropriateness of FA (factorability of data), the factor extraction model used, the factor retention criteria, the factor rotation methods, and the interpretation of the factor solution. Because of its complexity and availability of many options in each step, the state of the art of FA in SLA is less than optimal. Fortunately, a number of discipline-specific advanced stats sources have been introduced in recent years. Let’s factor analyze!
Loewen, S., & Gonulal, T. (2015). Exploratory factor analysis and principal components analysis. In Plonsky, L. (Ed), Advancing quantitative methods in second language research. New York: Routledge. [pdf]
Plonsky, L., & Gonulal, T. (2015). Methodological synthesis in quantitative L2 research: A review of reviews and a case study of exploratory factor analysis. Language Learning, 65, (S1), 9-36. [pdf]
Bandalos D. L., & Boehm-Kaufman M. R. (2009). Four common misconceptions in exploratory factor analysis. In C. E. Lance & R. J. Vandenberg (Ed.), Statistical and methodological myths and urban legends: Doctrine, verity and fable in the organizational and social sciences. New York: Routledge.
The field of second language acquisition (SLA) doesn’t have a long history as a discipline but the use of quantitative research methods and statistics in the field has increased considerably since the field’s inception. Although qualitative-oriented research is slowly increasing, most SLA research today relies on stats in some form or another. Recently, the status quo of quantitative research methods has received some scholarly and editorial attention. Now, more researchers are interested in the current state of statistical literacy in the field. I am so happy that the Second Language Studies program here at Michigan Stats(!) University is playing a pioneering role in this area of research. My advisor, Dr. Shawn Loewen, I and some other stats aficionados in the program are investigating the development and assessment of statistical literacy among SLA researchers. In a few years from now, we might see a couple of studies addressing this research topic in SLA journals.
Rasch hour time, dear fellow slatisticians! Yes, we need to Rasch! Rasch analysis is a statistical method used for estimating the probability of success of particular persons on particular items. These probabilities are expressed in logits. The Rasch models can be applied to a wide range of data types. The Rasch family consists of the simple Rasch model (can only be used to analyze dichotomous data such as right or wrong), the rating scale model (handles data with rating scales such as likert-scale items), the partial credit model (in a writing or speaking test) and the many-facet Rasch model (suitable for analyzing multiple aspects of testing such as the test location, the experience of raters and so on). Some of the modern software packages used for Rasch models are Winsteps, Facets, RUMM, and Conquest. Probably one of the most useful output from a Rasch analysis is item/person map (aka Wright Map). This map (see the figure above) provides a vertical histogram describing the representation of persons and items on the same measurement scale. The Rasch models are slowly finding their way into L2 research. I am planning to use a Rasch model for my dissertation. I will update this post as I learn something new about the Rasch family…stay tuned!
Knoch, U., & McNamara, T. (2015). Rasch analysis. In L. Plonsky (Ed.), Advancing quantitative methods in second language research (pp. 275-304). New York: Routledge.
Hi my secret readers—hiding in dark rooms behind the glow of their computers or tablets. When reading an article from the Economist or a top-tier journal in your field, have you ever encountered sentences beginning with “statistics have shown that” or “the results are statistically significant”? I am quite sure you did! Well…because statistics are everywhere! But the question is whether to trust statistics or not! One funny yet true definition of statistics that I recently came across on the internet perfectly highlights this question: “statistics is the only science wherein two recognized experts, using exactly the same set of data, may come to completely opposite conclusions”. And also, what is important in statistics is not what statistics tell but what statistics do not tell. As the famous saying goes, “statistics are like bikinis. What they reveal is is suggestive, but what they conceal is vital”. To blindly accept statistical results depends on your knowledge of statistics. So, how statistically literate are you?