Abstract:
Research on mental representations formed by programmers during program comprehension has not yet been applied to parallel programming. The goals of the pilot study were to validate a stimulus set, consisting of 80 programs written in C using OpenMP 4.0 directives, that will be used in subsequent studies on mental representations formed by expert parallel programmers and to serve as a resource for researchers who want to replicate or expand the research on program comprehension to include the parallel programming paradigm. The task used to stimulate the comprehension process was determining the presence of data races. Responses to the data race question were analyzed to determine the validity of the stimuli.
The results of the pilot study indicate that the level of difficulty of the stimuli (accuracy rate of .65) and the time limit for exposure to the stimuli are both appropriate and do not need to be adjusted for the main study. Participants’ self-perceived level of expertise correlated with their accuracy indicating this is a reasonable measure of expertise. Given the disparity of responses when asking participants what cues or program components they used to determine whether or not there was a data race, the main study will also include specific questions about components of the code to determine the type of information that is included in their mental representations.
The results of the pilot study indicate that the level of difficulty of the stimuli (accuracy rate of .65) and the time limit for exposure to the stimuli are both appropriate and do not need to be adjusted for the main study. Participants’ self-perceived level of expertise correlated with their accuracy indicating this is a reasonable measure of expertise. Given the disparity of responses when asking participants what cues or program components they used to determine whether or not there was a data race, the main study will also include specific questions about components of the code to determine the type of information that is included in their mental representations.