Deductive qualitative analysis at scale using text embeddings

Markus Fleten Kreutzer, Jonas Timmann Mjaaland and Halvor Tyseng: 

Qualitative analysis is a cornerstone of social science research, used in fields as diverse as education research, sociology, history, and business. Traditional qualitative analysis, however, is time consuming, labor intensive, and difficult to both scale and replicate. To tackle these issues, we have devised a fast, replicable, and scalable technique for deductive, qualitative research on text-data, by utilizing novel advances in machine learning and artificial intelligence. 

Picture made with Chat GPT

To prove the concept, we thematically analyze 23 years of the Physics Education Research Conference proceedings literature. Using text embedding models, we create vector-based representations of texts in a high-dimensional meaning space. Then we define topics, measure and transform distances in the meaning space, and arrive at results showing trends in the field of physics education.

We will present our method, critically discuss its features, uses, and limitations, and argue that it holds promise for flexible deductive qualitative analysis of a wide variety of text-based data that avoids many of the drawbacks inherent to prior NLP methods.

 

We will serve refreshments, coffee and tea. Welcome! 

The bi-weekly ODD seminar series at CCSE

The Open Discussions on Didactics (ODD) is a seminar series on Mondays at 13:00-14:00 every other week (odd week numbers) at CCSE.

The seminar will be maximum one hour, often closer to half an hour. It is an informal arena to present and discuss learning theory, educational research and teaching experiences within computational science. To cater to the highly heterogeneous backgrounds and interests of students, teachers and researchers in our environment, we aim for seminars that introduce listeners to new ideas within a broad spectrum of aspects, and that invites reflection and discussion.

Presentations need not be mature and polished - to the contrary we hope that as many as possible wants to share undigested observations and reflections in short presentations of varied form and topics. We hope to have enough contributions to frequently have the meetings as lightning talk sessions, where three different speakers will each give a 5-10-minute presentation followed by discussion.

Published Apr. 3, 2024 3:04 PM - Last modified Apr. 3, 2024 3:19 PM