by Radomir Ray Mitic, University of North Dakota, United States;Takeshi Yanagiura, University of Tsukuba, Japan; and Yukikazu Hidaka, Independent Researcher, United States
As critical researchers of the internationalization of higher education, we often face epistemological and methodological challenges when attempting to explain large-scale phenomena and challenge entrenched systems of power. The recent trend towards quantitative methods with a critical lens and a rejection of positivist paradigms with a purpose of transforming higher education practice has opened the door to large-scale empirical studies with a focus on uprooting the status quo (Tabron & Thomas, 2023). In particular, we argue that a quantitative approach to addressing issues of racism, sexism, classism, and heteronormativity, among other socially constructed systems of oppression, is expanding its available tools to conduct empirical research.
The emergence of artificial intelligence (AI) on the world scene has been foretold through works of science fiction for decades. Whether it is the benevolent android Data in Star Trek: The Next Generation or the evil supercomputer Hal 9000 in 2001: A Space Odyssey, these examples demonstrate the ability of artificial intelligence to reflect the best and worst of humanity. Because the algorithms programmed into these devices originate in the human mind with our own biases, we should be prudent to be careful when applying them in real world or empirical research situations. One need not look far to see the dangers of facial recognition in policing to see how systems of oppressions can be reinforced through AI (Buolamwini et al., 2020). In this Critical Voices piece, we lay out the current usage of AI in social science research and suggest how best to conceptualize and execute critical internationalization research with AI tools.
Using AI in Social Science Research
Because of the focus of critical research on deconstructing the dominant relationships in human affairs, we consider critical quantitative work that utilizes AI within a similar framework proposed by Kincheloe and McLaren (1994). We argue that an AI-based critical approach also must employ a theoretical or conceptual framework that pushes the research towards a liberatory purpose. In addition, criticality must be infused throughout selecting the topic and phenomenon of interest, the empirical approach, and the interpretation. In essence, AI is a tool, and its value hinges on the ability of researchers to give it a clear and meaningful purpose.
Rather than simply describe an object of interest, AI can help us to interrogate the barriers to a more equitable educational system rather than be neutral arbiters. As the title of Howard Zinn’s autobiography (1994) You Can’t be Neutral on a Moving Train suggests, chasing objectivity is neither achievable nor preferred. Similarly, critical, transformative, and liberatory frameworks in social science research aim to change structures that oppress communities. As we move into a new era of research using AI tools, we heed the call of Rios-Aguilar (2014) to engage in methodological self-reflection.
Conceptualizing AI Use in Critical Internationalization Research
Much like critical research in general, critical internationalization has taken a more qualitative approach. But as we ask research questions about topics such as racial hierarchy and whiteness in different higher education systems and Westernized notions of knowledge production, we find opportunities to use large datasets to address these issues. Once a study is grounded in a critical framework and engages communities of practice, we offer the following methodological and ethical considerations and recommendations surrounding trust, validation, and teamwork to help prepare researchers who wish to conduct research using AI.
Trust and Validation
Trust remains a hallmark of not only research, but human-AI interactions. For example, artificial intelligence in the form of a photo recognition program acts as a tool and as a contributor to the work. While we do not engage in the debate of whether AI merits attribution in scholarly work, we must recognize that the line between passive tool and active part of the research process is blurred. But how do we trust what the AI finds? Much like current attempts to establish trustworthiness in qualitative research or validity and reliability in quantitative research, research using AI must consider trust of self and the tool.
Thus, validating AI results is important. In particular, AI algorithms are susceptible to bias and sometimes make discriminatory decisions for individuals who belong to a certain demographic group (Mehrabi et al., 2021). As AI technology continues to advance, the learning cost of AI-based methods is anticipated to decline, making them more accessible to a broader range of individuals with varying skills and expertise, just as the past development of statistical software has lowered the barrier to conduct quantitative research for many social scientists. While more innovative research could possibly come out in education, it is not hard to imagine that careless or irresponsible application of AI will also increase. It is the responsibility of researchers utilizing AI to not only report the results generated by AI, but also to ascertain that the results are not biased. Numerous validation methods have been established, and researchers utilizing AI should possess a basic knowledge of how to validate AI’s findings. Additionally, since validating AI outputs often requires both time and resources, researchers should opt for AI models that are transparent about known biases and take measures to address them too (e.g., Kärkkäinen & Joo, 2019; Ding et al., 2022).
But such steps at validation should not end with technical approaches. Much like our colleagues who engage in qualitative research, practicing reflexivity through critical self-reflection builds on the validity of any study (Kleinsasser, 2000). Examining our own backgrounds, biases, and how we contribute to the systems we are studying is essential to conceptualizing a study as well as interpreting results generated by AI. Common approaches include analytic memoing and peer debriefing to bridge the gap between our ethical obligations to our participants and the rigors of empirical research.
The importance of having a diverse team cannot be overstated, as collaboration is key to successful research projects. No single person is likely to possess all the necessary skills to carry out all tasks alone. For instance, an AI programmer may excel at running a facial recognition algorithm but may lack familiarity with international education contexts to generate pertinent questions. On the other hand, international education researchers might know which questions are crucial but not possess the technical expertise to utilize AI to address those questions. Moreover, these two individuals may struggle to communicate effectively due to their different views associated with their disciplines, necessitating a third person who understands both domains enough to facilitate the conversation and bridge the gap between their perspectives. A well-rounded team is vital for conducting innovative research that leverages AI in international education.
Use of Findings to Inform Practice and Policy
Ultimately, critical internationalization research must have an emphasis in praxis where the findings are used to transform our educational institutions and systems (Freire, 1970). Having a critically-oriented team that utilizes advanced AI methods to address uneven power structures in education around the world is one way we can leverage this emerging technology. AI-informed research is one tool among many in the methodological satchel and should be complementary rather than in competition with other critical approaches that generally use qualitative approaches. Policymakers at institutional and national levels can be influenced by quantitative data that are contextualized and relevant to issues facing their constituents.
Buolamwini, J., Ordóñez, V., Morgenstern, J., & Learned-Miller, E. (2020, May 29). Facial recognition technologies: A primer. https://people.cs.umass.edu/~elm/papers/FRTprimer.pdf
Ding, L., Yu, D., Xie, J., Guo, W., Hu, S., Liu, M., Kong, L., Dai, H., Bao, Y., & Jiang, B. (2022). Word embeddings via causal inference: Gender bias reducing and semantic information preserving. Proceedings of the AAAI Conference on Artificial Intelligence, 36(11), 11864-11872. https://doi.org/10.1609/aaai.v36i11.21443
Friere, P. (1970). Pedagogy of the oppressed. Continuum.
Kärkkäinen, K., & Joo, J. (2021). Fairface: Face attribute dataset for balanced race, gender, and age for bias measurement and mitigation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (pp. 1548-1558). https://doi.org/10.1109/WACV48630.2021.00159
Kincheloe, J. L., & McLaren, P. L. (1994). Rethinking critical theory and qualitative research. In
N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research (pp. 138–157). Sage Publications, Inc.
Kleinsasser, A.M. (2000). Researchers, reflexivity, and good data: Writing to unlearn. Theory into Practice, 39(3), 155-162. https://www.jstor.org/stable/1477547
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2021). A survey on bias and fairness in machine learning. ACM Computing Surveys (CSUR), 54(6), 1-35. https://doi.org/10.1145/3457607
Rios-Aguilar, C. (2014). The changing context of critical quantitative inquiry. New Directions for Institutional Research, 158, 95–107. https://doi.org/10.1002/ir.20048
Tabron, L.A., & Thomas, A.K. (2023). Deeper than wordplay: A systematic review of critical quantitative approaches in education research (2007-2021). Review of Educational Research. https://doi.org/10.3102/00346543221130017
Zinn, H. (1994). You can’t be neutral on a moving train. Beacon Press.
About the Authors
Radomir Ray Mitic is an Assistant Professor of Higher Education at the University of North Dakota. Dr. Mitic’s program of research centers on the equitable outcomes of higher education at the local, national, and international levels. He holds a Ph.D. in Higher and Postsecondary Education from New York University.
Takeshi Yanagiura is an Associate Professor at the University of Tsukuba, Japan, specializing in the application of AI, machine learning, and causal inference methods to address challenges in higher education. He holds a Ph.D. in Economics and Education from Teachers College, Columbia University.
Yukikazu Hidaka has a Ph.D.in Computer Science from the University of Southern California. He has been working as a Machine Learning Engineer in tech companies. His research interests include fair and robust applications to education, healthcare, trust, and safety domains.