The Good, the Bad and the Ugly of AI in Qualitative Analysis

Christina Silver, CAQDAS Networking Project, University of Surrey

Steve Wright, University of Central Lancashire

SRA Annual Conference, 6 June 2024.

www.qdas.co.uk

Christina Silver

Dr Christina Silver is a researcher and facilitator of qualitative data analysis methods and digital tools.

She is Associate Professor (Teaching) in the Department of Sociology at the University of Surrey, where she is also Director of the CAQDAS Networking Project which since 1994 has raised awareness and built capacity in the use of qualitative software.

She is also Co-Director of the department’s Short Courses in Social Research programme.

Christina has trained many thousands of researchers and students in qualitative methods/software since 1998.

www.caqdas.co.uk

Steven Wright

Dr Steven Wright is an independent CAQDAS consultant and accredited trainer for ATLAS.ti, NVivo, MAXQDA and Quirkos.

He is the author of a series of datasets for SAGE Research Methods exploring new developments in CAQDAS packages including auto-transcription, machine-learning content analysis and systematic reviews.

He worked as a qualitative research consultant and learning technologist at Lancaster University for 18 years.

Hs is now Senior Lecturer in Medical Education at the University of Central Lancashire.

Abstract

Building on the Symposium on AI in Qualitative Analysis that Christina organised in collaboration with the SRA late 2023, this talk considers where we are now at in terms of the role and use of AI in the qualitative research process.

Christina and Steve discuss some of the key methodological and ethical considerations that face the qualitative community of practice in this new AI-infiltrated world.

Whether you work with in academic, government, industry, voluntary, or other sectors, there are issues to critically evaluate before using Generative-AI which Christina and Steve urge every qualitative and mixed methods researcher to consider

Download slides

References mentioned

Paulus TM & Marone V (2024)

"In Minutes Instead of Weeks": Discursive Constructions of Generative AI and Qualitative Data Analysis. Qualitative Inquiry. Published Online May 8th 2024

Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. (2021)

On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT '21). Association for Computing Machinery, New York, NY, USA, 610–623.

Kristi Jackson, Queri, Trena Paulus, University of Georgia &
Nicholas H. Woolf, Woolf Consulting (2018)

The Walking Dead Genealogy: Unsubstantiated Criticisms of Qualitative Data Analysis Software (QDAS) and the Failure to Put Them to Rest

Relevant podcast and youtube videos

Mystery AI Hype Theatre 3000

Episode 28: LLMs Are Not Human Subjects. March 4, 2024

CAQDAS Chat with Christina

Episode 10: Christina Chats with Janet Salmons. May 31, 2024

From 30 mins we begin talking about research integrity and the ethics of using generative AI

Symposium on AI in Qualitative Analysis

Organised by the Social Research Association in partnership with the CAQDAS Networking Project

Part 1 AI in Qualitative Analysis: Developments and Methodological implications. November 25, 2023. Full recording available on YouTube

Part 2 AI in Qualitative Analysis: Research Applications, Opportunities and Challenges, December 2, 2023. Full recording available on YouTube

Blogposts

Artifical Intelligence and Machine Learning in Qualitative Analysis

Steve Wright

Important Issues for Automated Transcription,

David Woods, Lead Developer of Transana.

Automated transcription and some risks of machine interpretation

Daniel Turner, Director of Quirkos Software

AI in Qualitative Research, blog series

Christina Silver, Director of CAQDAS Networking Project, University of Surrey

& Director of Qualitative Data Analysis Services (QDAS)

SOLD OUT

June 17

AI-Assisted Qualitative Data Analysis

Join us in this interactive workshop considering the appropriate and ethical use of generative-AI tools to analyse qualitative data

Mon 17th June

09:30-16:30 BST / 10:30-17.30 CET

July 4 & 5

Using AI for Qualitative Analysis througout the Research Cycle

Explore the potential using AI to do analysis this workshop covering the principles, practices and ethics of using AI throughout the analytic workflow.

Thu 4th & Fri 5th July

09:30-16:30 BST / 10:30-17:30 CET

Sept 20

AI-Assisted Qualitative Data Analysis

Join us in this interactive workshop considering the appropriate and ethical use of generative-AI tools to analyse qualitative data

Fri 20th Sept

09:30-16:30 BST / 10:30-17.30 CET

Software mentioned in the talk

(NB: these are not all the available options, just those mentioned in this talk).

For reviews of these and other CAQDAS packages see the CAQDAS Networking Project website and the Qualitative AI blog series.

AILYZE - founded 2023 - Generative-AI for Qualitative Analysis (proprietary models not OpenAI)
ATLAS.ti - founded 1989 - Established CAQDAS package incorporating Generative-AI tools since 2023 (using OpenAI models)
CoLoop - founded 2023 - Generative-AI for Qualitative Analysis (using OpenAI models)
DiscoverText - founded 2009 - Established CAQDAS combining human interpretation and machine-learning to perform text classification
MAXQDA - founded 1989 - Established CAQDAS package incorporating Generative-AI tools since 2023 (using OpenAI models)
NVivo - founded 1980s - Established CAQDAS package incorporating sentiment analysis, pattern-based auto-coding and automated identification of noun-phrases as well as automated transcription (no generative-ai capabilities, yet)
Quirkos - founded 2013 - Established CAQDAS package incorporating automated transcription since 2023
Reveal - founded 2024 - Generative-AI for Qualitative Analysis (using OpenAI models)
Transana - founded 2002 - Established CAQDAS package incorporating automated transcription since 2023

The Good, the Bad and the Ugly of AI in Qualitative Analysis

Abstract

References mentioned

Relevant podcast and youtube videos

Blogposts

Future workshops