Jared Freeman
Cognitive Technologies, Inc.

Knowledge of software bugs among professional and student programmers was examined in two experiments. In the first study, six professional C language programmers, ten undergraduate programmers, and nine subjects with no programming experience performed a free recall task. Quantity, quality, and speed of recall generally correlated with level of expertise, though differences between professional and student programmers were not significant. Professionals recalled items in significantly larger groups, or chunks, than did students or naives, however. The recall strategies and protocols of novices correlated with surface features of the stimuli; those of experts did not.

In the second study, 50 professional C language programmers and 54 student C programmers sorted descriptions of thirty bugs into piles and labelled the piles. They performed this task three times, once to organize materials in a generally “useful” manner (the unprimed condition) and twice in preparation to debug different applications (the primed conditions). Agreement on sorting schemes was slightly higher among experts than novices. The effect of application priming was marginally smaller for experts, as was reliance on surface features during sorting.

The quality of sorts differed between groups. Only professional programmers formulated a coherent group of interface bugs, while only students differentiated between pointer and non-pointer data type errors. Students with four or fewer years of general plus C programming experience differed most and most consistently from experts in this regard. More senior students sorted much like professionals. These patterns and differences in subjects’ descriptions of sorted piles reflected the influence of the social and technical context of professional programming and student programming, respectively. A definition of expertise that incorporated environmental variables differentiated most strongly between groups, while a measure of simple length of experience was the next most powerful definition. A psychologically realistic classification of bugs is proposed. Implications for education, training, and expert/novice research are discussed.