SRA conference keynote 2024

The Good, the Bad and the Ugly of AI in Qualitative Analysis

Dr Christina Silver

Director, QDAS | Qualitative Data Analysis Services

Associate Professor, CAQDAS Networking Project, University of Surrey

Dr Steve Wright

Senior Lecturer in Medical Education, University of Lancashire

Christina Silver

Dr Christina Silver is a researcher and facilitator of qualitative data analysis methods and digital tools.

She is Founder and Director of QDAS | Qualitative Data Analysis Services and Associate Professor (Teaching) in the Department of Sociology at the University of Surrey, and Director of the university's CAQDAS Networking Project which since 1994 has raised awareness and built capacity in the use of qualitative software.

She is also Co-Director of the department’s Short Courses in Social Research programme.

Christina has trained many thousands of researchers and students in qualitative methods/software since 1998.

Steve Wright

Dr Steven Wright is an independent CAQDAS consultant and accredited trainer for a range of software.

He is the author of a series of datasets for SAGE Research Methods exploring new developments in CAQDAS packages including working with auto-transcription, machine-learning content analysis and systematic reviews, as well as developing guidance for working with phenomenological approaches.

He worked as a qualitative research consultant and learning technologist at Lancaster University for 18 years, and is now Senior Lecturer in Medical Education at the University of Central Lancashire.

Abstract

Building on the Symposium on AI in Qualitative Analysis that Christina organised in collaboration with the SRA late 2023, this talk considers where we are now at in terms of the role and use of AI in the qualitative research process.

Christina and Steve discuss some of the key methodological and ethical considerations that face the qualitative community of practice in this new AI-infiltrated world.

Whether you work with in academic, government, industry, voluntary, or other sectors, there are issues to critically evaluate before using Generative-AI which Christina and Steve urge every qualitative and mixed methods researcher to consider

Download Slides

References mentioned

Paulus TM & Marone V (2024), "In Minutes Instead of Weeks": Discursive Constructions of Generative AI and Qualitative Data Analysis. Qualitative Inquiry. Published Online May 8th 2024

Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. (2021), On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT '21). Association for Computing Machinery, New York, NY, USA, 610–623.

Kristi Jackson, Queri, Trena Paulus, University of Georgia & Nicholas H. Woolf, Woolf Consulting (2018), The Walking Dead Genealogy: Unsubstantiated Criticisms of Qualitative Data Analysis Software (QDAS) and the Failure to Put Them to Rest

Relevant podcast and youtube videos

Mystery AI Hype Theatre 3000

Episode 28: LLMs Are Not Human Subjects. March 4, 2024

CAQDAS Chat with Christina

Episode 10: Christina Chats with Janet Salmons. May 31, 2024

From 30 mins we begin talking about research integrity and the ethics of using generative AI

Symposium on AI in Qualitative Analysis

Organised by the Social Research Association in partnership with the CAQDAS Networking Project

Part 1 AI in Qualitative Analysis: Developments and Methodological implications. November 25, 2023. Full recording available on YouTube

Part 2 AI in Qualitative Analysis: Research Applications, Opportunities and Challenges, December 2, 2023. Full recording available on YouTube

Blogposts

Artificial Intelligence and Machine Learning in Qualitative Analysis, Steve Wright, Independent CAQDAS trainer. Published on Steve's personal blog. June 5th 2024.

Important Issues for Automated Transcription, David Woods, Lead Developer of Transana. Published on the Transana Blog

Automated transcription and some risks of machine interpretation, Daniel Turner, Director of Quirkos Software. Published on the Quirkos Blog April 23rd 2023.

AI in Qualitative Research, blog series,Christina Silver, Director of CAQDAS Networking Project, University of Surrey & Director of Qualitative Data Analysis Services (QDAS)

Software mentioned in the talk

(NB: these are not all the available options, just those mentioned in this talk).

For reviews of these and other CAQDAS packages see the CAQDAS Networking Project website and the Qualitative AI blog series.

AILYZE - founded 2023 - Generative-AI for Qualitative Analysis (proprietary models not OpenAI)
ATLAS.ti - founded 1989 - Established CAQDAS package incorporating Generative-AI tools since 2023 (using OpenAI models)
CoLoop - founded 2023 - Generative-AI for Qualitative Analysis (using OpenAI models)
DiscoverText - founded 2009 - Established CAQDAS combining human interpretation and machine-learning to perform text classification
MAXQDA - founded 1989 - Established CAQDAS package incorporating Generative-AI tools since 2023 (using OpenAI models)
NVivo - founded 1980s - Established CAQDAS package incorporating sentiment analysis, pattern-based auto-coding and automated identification of noun-phrases as well as automated transcription (no generative-ai capabilities, yet)
Quirkos - founded 2013 - Established CAQDAS package incorporating automated transcription since 2023
Reveal - founded 2024 - Generative-AI for Qualitative Analysis (using OpenAI models)
Transana - founded 2002 - Established CAQDAS package incorporating automated transcription since 2023