Abstract:
In this exploratory pilot study, we trace students’ experiences in designing, developing and deploying unit and integration tests, as well as setting up and running continuous integration, during a semesterlong, industry-partnered software engineering course project. We conducted a qualitative study using semi-structured interviews, and combined them with researcher memos, and reflective researcher journals. In so doing, we identified various difficulties that our novice testers were plagued by during the testing and DevOps process. Some of those difficulties include: communication within the team and other stakeholders, prioritization of features to be tested, entry and exit criteria for tests, difficulties with learning tools associated with testing, the time commitment involved in designing, writing and implementing meaningful tests, not knowing what kind of questions to ask and of whom, and how to look for test completeness beyond code coverage. We investigate and discuss these difficulties in detail. We also explore the cognitive biases identified as impediments to these testers in accomplishing their testing goals.
PPIG 2021 - 32nd Annual Workshop
Student difficulties in Unit Testing, Integration Testing and Continuous Integration: An exploratory pilot qualitative study