A content analysis process for qualitative software engineering research

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

A review of the qualitative research methods discussed in papers that study software engineering teams showed most of those papers did not follow a systematic process during the qualitative analysis. This finding is concerning as this deficiency in research analysis procedure may reduce the validity and/or completeness of the qualitative results. Such a lack of rigor may be a result of qualitative research not being as firmly established in software engineering as quantitative research methodologies. In engineering research, quantitative methods are typically more prevalent and qualitative analysis is part of a mixed-method analysis process. However, when researching teams, where human activity is abundant, qualitative analysis may need to take precedence. In this paper, we focus on the qualitative analysis method called content analysis with the goal of presenting a rigorous process for content analysis in the context of software engineering. We then present and demonstrate the use of that content analysis process for software engineering researchers using two examples. An analysis of 215 articles that were a result of a mapping study on software engineering team research is presented. Those papers were analyzed to determine which utilized a qualitative data analysis method in their research in addition to the rigor and type of qualitative analysis performed. We ultimately included 23 papers in this study. We present a mapping study and a content analysis process that include a straightforward way to select, code, and present data in both inductive and deductive studies. We demonstrated the process using the keywords from the papers included in this study as well as on a second data set that utilized responses from structured interview transcripts from practicing software engineers. The first dataset also resulted in a taxonomy to categorize software engineering team research. We presented and demonstrated a content analysis process in terms of software engineering in order to improve future qualitative software engineering research that would benefit from systematic content analysis.

Original languageEnglish (US)
Pages (from-to)129-141
Number of pages13
JournalInnovations in Systems and Software Engineering
Volume13
Issue number2-3
DOIs
StatePublished - Sep 1 2017

Fingerprint

Engineering research
Software engineering
Taxonomies
Engineers

All Science Journal Classification (ASJC) codes

  • Software

Cite this

@article{babc877ede4d45819fef59819f8b51fc,
title = "A content analysis process for qualitative software engineering research",
abstract = "A review of the qualitative research methods discussed in papers that study software engineering teams showed most of those papers did not follow a systematic process during the qualitative analysis. This finding is concerning as this deficiency in research analysis procedure may reduce the validity and/or completeness of the qualitative results. Such a lack of rigor may be a result of qualitative research not being as firmly established in software engineering as quantitative research methodologies. In engineering research, quantitative methods are typically more prevalent and qualitative analysis is part of a mixed-method analysis process. However, when researching teams, where human activity is abundant, qualitative analysis may need to take precedence. In this paper, we focus on the qualitative analysis method called content analysis with the goal of presenting a rigorous process for content analysis in the context of software engineering. We then present and demonstrate the use of that content analysis process for software engineering researchers using two examples. An analysis of 215 articles that were a result of a mapping study on software engineering team research is presented. Those papers were analyzed to determine which utilized a qualitative data analysis method in their research in addition to the rigor and type of qualitative analysis performed. We ultimately included 23 papers in this study. We present a mapping study and a content analysis process that include a straightforward way to select, code, and present data in both inductive and deductive studies. We demonstrated the process using the keywords from the papers included in this study as well as on a second data set that utilized responses from structured interview transcripts from practicing software engineers. The first dataset also resulted in a taxonomy to categorize software engineering team research. We presented and demonstrated a content analysis process in terms of software engineering in order to improve future qualitative software engineering research that would benefit from systematic content analysis.",
author = "Defranco-Tommarello, {Joanna F.} and Laplante, {Phillip A.}",
year = "2017",
month = "9",
day = "1",
doi = "10.1007/s11334-017-0287-0",
language = "English (US)",
volume = "13",
pages = "129--141",
journal = "Innovations in Systems and Software Engineering",
issn = "1614-5046",
publisher = "Springer London",
number = "2-3",

}

TY - JOUR

T1 - A content analysis process for qualitative software engineering research

AU - Defranco-Tommarello, Joanna F.

AU - Laplante, Phillip A.

PY - 2017/9/1

Y1 - 2017/9/1

N2 - A review of the qualitative research methods discussed in papers that study software engineering teams showed most of those papers did not follow a systematic process during the qualitative analysis. This finding is concerning as this deficiency in research analysis procedure may reduce the validity and/or completeness of the qualitative results. Such a lack of rigor may be a result of qualitative research not being as firmly established in software engineering as quantitative research methodologies. In engineering research, quantitative methods are typically more prevalent and qualitative analysis is part of a mixed-method analysis process. However, when researching teams, where human activity is abundant, qualitative analysis may need to take precedence. In this paper, we focus on the qualitative analysis method called content analysis with the goal of presenting a rigorous process for content analysis in the context of software engineering. We then present and demonstrate the use of that content analysis process for software engineering researchers using two examples. An analysis of 215 articles that were a result of a mapping study on software engineering team research is presented. Those papers were analyzed to determine which utilized a qualitative data analysis method in their research in addition to the rigor and type of qualitative analysis performed. We ultimately included 23 papers in this study. We present a mapping study and a content analysis process that include a straightforward way to select, code, and present data in both inductive and deductive studies. We demonstrated the process using the keywords from the papers included in this study as well as on a second data set that utilized responses from structured interview transcripts from practicing software engineers. The first dataset also resulted in a taxonomy to categorize software engineering team research. We presented and demonstrated a content analysis process in terms of software engineering in order to improve future qualitative software engineering research that would benefit from systematic content analysis.

AB - A review of the qualitative research methods discussed in papers that study software engineering teams showed most of those papers did not follow a systematic process during the qualitative analysis. This finding is concerning as this deficiency in research analysis procedure may reduce the validity and/or completeness of the qualitative results. Such a lack of rigor may be a result of qualitative research not being as firmly established in software engineering as quantitative research methodologies. In engineering research, quantitative methods are typically more prevalent and qualitative analysis is part of a mixed-method analysis process. However, when researching teams, where human activity is abundant, qualitative analysis may need to take precedence. In this paper, we focus on the qualitative analysis method called content analysis with the goal of presenting a rigorous process for content analysis in the context of software engineering. We then present and demonstrate the use of that content analysis process for software engineering researchers using two examples. An analysis of 215 articles that were a result of a mapping study on software engineering team research is presented. Those papers were analyzed to determine which utilized a qualitative data analysis method in their research in addition to the rigor and type of qualitative analysis performed. We ultimately included 23 papers in this study. We present a mapping study and a content analysis process that include a straightforward way to select, code, and present data in both inductive and deductive studies. We demonstrated the process using the keywords from the papers included in this study as well as on a second data set that utilized responses from structured interview transcripts from practicing software engineers. The first dataset also resulted in a taxonomy to categorize software engineering team research. We presented and demonstrated a content analysis process in terms of software engineering in order to improve future qualitative software engineering research that would benefit from systematic content analysis.

UR - http://www.scopus.com/inward/record.url?scp=85019019227&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85019019227&partnerID=8YFLogxK

U2 - 10.1007/s11334-017-0287-0

DO - 10.1007/s11334-017-0287-0

M3 - Article

VL - 13

SP - 129

EP - 141

JO - Innovations in Systems and Software Engineering

JF - Innovations in Systems and Software Engineering

SN - 1614-5046

IS - 2-3

ER -