Spotlight on PPIGers

Tags:

Sebastian Jekutsch

Sebastian's research topic is entitled: Investigating programmer's work to analyse defect insertions

With more extensible and more integrated programming environments at hand (e.g. [1]), and with growing insights into the dynamics of programming (e.g. [2], [3]) we started a project with the aim of automatically observing a programmer's behaviour to predict defective code.

We assume that code defects are actively being built into software, because

  • programmers - being humans - make errors
  • making errors is not bad luck and therefore defects do not occur by chance and should not be treated by statistical means only
  • defect prevention, i.e. prevention of making errors, is to a great extent a task of process observation and process adjustment.

We are focussing on what we call 'the micro-process of software development'. The micro-process is a stream of events occurring during development. Restricted to coding, this can be:

  • extending code parts bottom-up
  • browsing through code
  • executing the program
  • pausing for a while, etc.

In our study, we try to find significant correlations between some suspect coding behaviour (i.e. episodes of coding events at some code location) and later changes of this code, especially defect removals. We want to assist the programmer in answering questions like:

  • What happened when I constructed this defective code?
  • Which part of the code was constructed in the same manner?
  • What seems to be my typical coding behaviour which results in coding defects?

Typical defect-prone micro-process patterns or episodes are:

  • being interrupted and not resuming work where left off
  • changing a small part of code very often
  • not changing copied code fragments consistently, etc.

We will build our hypotheses on the observations published in the area of programming psychology, especially in programming seen as a problem solving activity and its known anomalies.

The optimal result of our research would be a tool which automatically issues a warning to the programmer as soon as suspect episodes are detected (via just-in-time analysis of the event stream). It remains to be seen whether this will result in some really useful hints or if the amount of false positives will render it useless. A captured and stored micro-process may alternatively be replayed as a means for the programmer to learn from mistakes.

The research steps are:

  1. collecting micro-process events (using a general framework [4])
  2. doing qualitative research to build hypotheses on defect-prone coding episodes
  3. trying to automatically detect those episodes
  4. doing experiments on the applicability of the recognizer and its predictive power concerning later defect removals

Other work in our working group [5] includes studies to examine API documentation usage and programming task influence of the usefulness of pair programming.

We would be very pleased to get any comment about known issues in human error in programming and its observable characteristics, about related projects and research questions, and about any experimental work already done for programmer's coding behaviour, especially concerning coding defects.

[1] http://www.eclipse.org

[2] Willemien Visser, Jean-Michel Hoc. Expert Software Design Strategies. In: J.-M. Hoc, T.R.G. Green, R. Samurcay, D.J. Gilmore: Psychology of Programming, Academic Press 1990

[3] Simon P. Davis. Models and theories of programming strategy. Int. Journal Man-Machine Studies (1993) 39, 237-267

[4] http://csdl.ics.hawaii.edu/Tools/Hackystat/

[5] http://www.inf.fu-berlin.de/inst/ag-se/

Sebastian is a student at the Freie Universitat Berlin, Working Group on Software Engineering. You can contact Sebastian by e-mailing him using jekutsch (at) inf.fu-berlin.de.

Derek Jones

Derek has recently carried out a series of experiments that explore memory for short sequences of assignment statements.

The papers describing these experiments and their associated results can be found by following the below link:

ACCU 2004 Memory Experiment

Derek Jones is an ex-compiler writer who is trying to find something else to do.

Gerold Keefer

Gerold has recently published an article entitled Pair Programming : An Alternative to Reviews and Inspections.

This article takes an in-depth look at the empirical research results available on pair programming and inspections. It reveals that a prominent experiment in that area has a range of flaws that were detected by several researchers and practioners independently.

A copy of this paper can be found by following the below link:

Pair Programming : An Alternative to Reviews and Inspections

Michael P. O'Brien

Micheal is a student at the Department of Computer Science and Information Systems at the University of Limerick, Ireland. He introduces his research topic: Characterizing Information Behaviour During Software Maintenance - An Empirical Approach. A paper describing Micheals' earlier work has been recently published in the Journal of Software Maintenance and Evolution.

Much research has been carried out to date suggests that both comprehension and information seeking are major parts of software maintenance. However, it remains the view of many computer scientists that the standard of empirical software engineering in this area leaves scope for improvement.

There is, however, an increasing awareness in the software engineering community that empirical studies are a vital aspect in the process of improving methods and tools, for software development and maintenance.

It is interesting that up to 70% of expenditure on software systems, occurs after delivery - during maintenance and evolution. Of this 70%, researchers report that over half the effort expended is due to 'understanding' the system. Unfortunately, as a research community, we can provide little guidance to professional software engineers as they carry out their maintenance/evolution tasks.

Preliminary studies already carried out as part of this research, suggest that understanding large commercial software systems seems not guided by the cliched comprehension strategies, said to be employed during maintenance activities. Instead, the process is driven by an information requirement.

Essentially this research aims to analyse high ecologically valid talk-aloud data from several empirical studies of programmers carrying out 'real' maintenance tasks in situ. The data will be examined (in a data-centric grounded theory fashion) from several perspectives (already hypothesised upon / studied in the community) in the information-seeking paradigm - and will aim to answer questions; for example, at a low level, how do programmers use lexical meaning in the sessions?

At a higher level, what do they reason about in terms of structure (architectural, file, etc) or do they neglect it altogether. The data will be further analysed to identify the information sources programmers use during software maintenance along with the interleaving of these information sources.

Limited research has been carried out (mainly by the use of questionnaires), as to the information sources programmers rely on - however, to date, this research has not been specifically empirically validated in-situ.

Olga Smyslova

Alexander E. Voiskounsky and Olga V. Smyslova, M.I.N.D. Lab, Moscow State University, Russia, have been working on a study of computer hackers' motivation for a few years. We developed a model of hackers' motivational development, based on the flow experience paradigm. Hackers seem to have a strong intrinsic (i.e. with no external rewards, such as money or peer-recognition) motivation for their preferred activity.

A well-developed theory of flow experience, introduced by M. Csikszentmihalyi within the school of positive psychology, is intimately related to intrinsic motivation. As a practical tool, the flow experience is being intensely measured, taken various types of human-computer interaction.

Taken as granted that hackers experience flow, it was hypothesized that flow increases with the increase of hackers' competence in the IT use. Self-selected subjects were recruited on specialized web sources; 457 hackers filled out a web questionnaire. Competence in the IT use, specific flow experience and demographic data were questioned.

An on-line research was administered within the Russian-speaking community (though one third of subjects are non-residents of Russian Federation). Two differing strategies of task choice were self-reported by subjects: a step-by-step increase of the difficulty of choices leads to a match of challenges and skills (and to preserving the flow experience); putting choices irrespective of the likelihood of solution leads to a flow crisis.

The findings gave productive hints on processes of hackers' motivational development. The flow-based model of computer hackers' motivation was developed. It combines both empirically confirmed and theoretically possible ways of hackers' 'professional' growth.

The results of the study can be found in an article in Cyber-Psychology and Behavior ( Flow-Based Model of Computer Hackers' Motivation. Alexander E. Voiskounsky, Olga V. Smyslova. CyberPsychology & Behavior. Volume 6, Page 171-180, April 2003). You can reach authors at vae-msu (at) mail.ru (Alexander Voiskounsky) or korobka (at) mail.ru (Olga Smyslova) to get a copy of the paper.

Future plans include cross-cultural study of software engineers in Russia, States and, probably, other cultures; study of the ethics in the software development. Other scientific interests of the Lab members include: Usability, Accessibility, Virtual Environments, Flow Experience of MUD-players, Online Communities.

Olga is planning a study of programming languages influence on thinking, comparing Object-Oriented Programming, Functional programming, and Procedural Programming.

If you have any suggestions or thoughts, or literature on the topic, please, share!

Mads Søgaard

I have just finished a report called Object Orientation Redefined: From abstract to direct objects and toward a more understandable approach to understanding.

The report investigates the connection between object-oriented formalisationand human thought processes. The report redefines/rethinks Object Orientation by problematising the distinction between Model (underlyingmeaning) and Interface, first from a philosophical and epistemological perspective and thereupon from a more practically oriented HCI, Computer Science, and Interface Design perspective.

In broad terms, the report orients Object Orientation in the direction of HCI and Interaction Design and conversely explains why Object Orientation may be regarded as a part of the field of HCI.

I will be most grateful for every remark, as well as advice on where to publish key points of the report. Since the report redefines objectorientation as a 'cross section' of the areas/theories/subject matters/etc that I introduce, I have trouble finding the right 'disciplinary pigeonhole'for my work.

The report is available here: Object Orientation Redefined

Please feel free to e-mail Mads using mads (at) acm.org