|Psychology of Programming
|about newsletters workshops resources contents|
Editor: Chris Douce
Welcome to the Summer 2011 edition of the PPIG newsletter.
Quite a lot of time has passed since the last issue and quite a lot has happened too. It's not possible to write about all the events and publications that have occurred and emerged between 2009 and 2011, but I do hope that what you find within this newsletter is both interesting and informative.
This issue begins with some useful information for the forthcoming 2011 workshop, which is to be held in the University of York, UK, in September. This is followed with a report from both the most recent work in progress (WIP), and main PPIG workshop. Jennifer Ferreira from the Open University has written a trip report for the 8th International Advanced School in Empirical Software Engineering.
This issue contains two short articles, one by Fabian Fagerholm. The articles are followed by a short resource section and then to two familiar columns: the spotlight on PPIG members section, and a suitably random collection of assorted links.
I hope you enjoy this somewhat belated issue of the PPIG newsletter. As always, if you have any ideas or suggestions, please don't hesitate to get in touch. I'll always be happy to hear from you.
PPIG 2011 is due to take place at the University of York between 6 and 8 September. More information about both the event and the city can be found by visiting the below links.
PPIG 2011 website
University of York
City of York
[ top ]
by Mark Zarb
The seventh PPIG WIP was hosted by Dr Chris Roast, from the Culture, Communications and Computing Research Institute at Sheffield Hallam University, and Jag Gill from the GIST Foundation between 18 and 19 April 2011.
The PPIG work-in-progress workshop is a forum in which researchers at all levels can present and discuss their current work, results, findings and developments. It is an informal forum which provides constructive criticism and suggestions to allow researchers to take their work forward. Despite the name, PPIG is not only interested in the psychology of programming, but also in psychological aspects of related activities, as can be seen in some of the presentations and workshops that took place during this conference.
The presentations were divided into four sessions spanning across the two days: Constraints, Novice Learning, Continuing Learning Constraints, and Solution Design.
The novice learning session started off with a presentation on how dancing robots could be used to teach programming concepts to students at any level by Chris Martin, followed by a presentation on flow (being in "the zone"), and its future applications of programming by Mark Zarb. Charles Boisvert ended the session by talking about the development process for novice programmers, and showcased a tutoring system he had spent years devising.
After a coffee break, the delegates had the option of attending one of two workshops – an introduction to video prototyping (by Liliana Rodriguez), or a discussion about coaching (by Marc Johnson). Following this, a collection of pizzas were taken to the GIST Lab, allowing people to mingle whilst the next activities were set up.
An excellent workshop was devised by Chris Martin with his Arduino (Wikipedia) robots, where the aim was to introduce programming concepts with Arduino by getting the delegates to make these robots perform in a dance-off competition. Following this, Alex McLean demonstrated his Acid House Coding system, which provided music to end the night.
The second day started with Chris Bates talking about the importance of facework in pair-programming sessions. Following this, John Rooksby gave a very visual presentation on coding dojos; forums which have pair programmers working in front of an audience. Rebecca Yates spoke about how experts understand unfamiliar code and about her case study on 'onboarding' (when one developer shows another around the code).
Finally, Marc Johnson gave a brief recap of the previous day's discussion on coaching, and proceeded to explain the idea of coaching in detail, discussing advantages and experiences he had had with being both a coach, and being coached.
The final session was on Solution Design. Liliana Rodriguez returned to talk about her workshop and video-prototyping, as well as give examples of how video prototypes can help define a product.
Chris Martin returned to talk about a new, unique module at the University of Dundee called 'Physical Computing', where first year undergraduates had to use Arduino components as part of their assessed coursework. Finally, Tim Duckett spoke about iterative development and user feedback, pointing out that end users never behave in a formally specified way.
PPIG WIP 2011 was closed with a keynote speech by Dr Alan Blackwell, who spoke about the history of computing, including development of the GUI in programming, and therefore, interest in metaphors, which leads to the psychology of programming.
The following is a list of prizes that were awarded at the closing of PPIG 2011 by Maria Kutar:
Luke Church has taken some fabulous photographs during the Work in Progress workshop. These can be viewed by visiting his photography website
[ top ]
By Rebecca Yates and Fabian Fagerholm
Universidad Carlos III de Madrid, Leganes, Spain.
For the students, the conference began with the Doctoral Consortium held in the nearby Tryp Hotel. Chaired by Maria Kutar, with Thomas Green, Mary Anne Rosson and Rachel Bellamy on the panel, the consortium provided advice, opinions and constructive criticism on PhDs at various stages of completion.
Rebecca Yates asked for advice on collecting data on software developer mentoring, and Sami Pietinen presented his work on studying collaborative programming through eye-tracking. Fabian Fagerholm had the unusual situation of too much data, collected from his lab's Software Factory. Aleksandra Pawlik discussed her work on the relationship of computational science developers to software engineering, and Scheila Martins asked for advice on completing her PhD on students' motivation in learning to program.
Margaret Burnett opened the conference with a keynote on Gender HCI and Programming. She reported on a series of investigations conducted by her and her students. They found that purportedly gender-neutral software tools do interact with gender differences, resulting in lower problem-solving effectiveness for female users. In particular, males were more prone to explore and attempt problem-solving by trial and error, while females did not explore as much and stayed with familiar functions. Female end-user effectiveness in programming environments like Excel could be improved by taking gender differences into account. This would not necessarily mean the tool would be less usable by males; in fact, many groups of people could benefit from the improvements.
In the session on usability issues, Luke Church showed how the concept of 'liveness' could be applied to describe feedback cycles in music and programming environments, and Catherine Letondal compared computing-oriented and interaction-oriented programming. Chris Parnin gave a cognitive neuroscience perspective on programming, describing the capture of subvocalisations to provide clues to programmers' thought processes, and John Daughtry described a method to analyse API usability through measurement of 'perceived self-efficacy'.
In the first session on teaching and learning programming, Leonard Mselle discussed the use of Random Access Memory to improve comprehension when teaching programming, and asked attendees for any examples of such diagrams in educational books. David Moffat reported on an experiment in using Scratch (a visual programming language aimed at children) for ITC lessons in a school. The results were encouraging, with pupils performing well and enjoying their introduction to programming.
David continued with a survey of students' changing attitudes to programming as they worked through a college-level introductory programming course. The unusually practical course improved students' self-efficacy, but was not able to remove the fear of programming in all cases. Finally, Zhen Li described students' difficulties with concurrent programming. A hierarchy of misconceptions at various levels, such as the definition of "lock" and "block" terminology, left them struggling to answer questions about a concurrent system.
After coffee, the teaching and learning programming theme continued with Sylvia da Rosa's application of genetic epistemology to our field. Interviewing students about a real-world search procedure (finding a word in a dictionary) revealed their growing understanding of the underlying binary search mechanism and its requirements. Next, Randy Kaplan presented the development of a collection of 'programming wisdom' intended to help novices acquire the meta-skills required for effective programming. The session concluded with a series of five-minute presentations by the students who attended the doctoral consortium. This provided and opportunity for the wider conference audience to comment and advise on ongoing PhD work.
The conference banquet, held that evening at the Simona Restaurant, featured course after course of tasty Spanish food, and finished with a round of traditional liqueurs of unidentified flavours.
The second day of PPIG was dedicated to the topics Software Engineers and Practise and End-User Programming. In the first session, Edna Rosen presented insights in the areas of distributed pair programming, Rien Sach spoke about "programmer personality" as depicted by the Myers-Briggs personality type indicator in several studies, and Gul Calikli presented results on a study of confirmation bias in software development and testing.
After the first coffee break, the program continued with two discussions. Alan Blackwell started the first discussion on the topic of teaching psychology of programming to masters-level students. Could an explicit PPIG syllabus be feasible within a "hard" computer science context? There was general agreement that teaching PPIG would be useful, but the audience saw many challenges that needed to be overcome. However, there was hope that students could be interested despite the difference between PPIG and traditional computer science.
The next topic for discussion was titled "Help Luke and Thomas write a book", alternatively (and depending on which version of the PPIG 2010 program you happened to look at) "Help Thomas and Luke write a book". Thomas Green and Luke Church led the discussion around a book-idea about usability techniques for practitioners, particularly focusing on cognitive dimensions. The audience compared existing works on information visualisation and design patterns and pondered whether the book should be in paper form, electronic, or both. An initial prototype of a web version was shown. The discussion did not reach an ultimate conclusion, but many interesting ideas were voiced.
The following section on end-user programming delved into the mind of professional end-users via a case description by Alan Blackwell. The differences in reasoning between a professional end-user and a programmer were exposed. Matthew Dinmore showed thought-provoking statistics about end-user programming behaviour in the Yahoo!Pipes system. Patterns of use and reuse provided insight into how end-users behaved in the system. Finally, PPIG attendants dived into the creative and artistic world of bricolage programming. Alex McLean showed how programming is constructed as an art form: as an iterative navigation of a conceptual space where the artist manipulates the programming system and observes -- and is sometimes surprised by -- how the system changes its behaviour.
The remainder of the day was devoted to a panel on information foraging. After having reached final agreement on who was to chair and who was to give the introductory talk, Joseph Lawrance and Scott Fleming showed how the theory of information foraging could be applied to help programmers in their search for bugs in source code. They showed an algorithm that implemented the theoretical ideas, and that had given promising results. The discussion was diverse and included questions on both theory and practise. Could similar results be achieved by applying simple principles of locality, as the end-user is quite likely to search nearby the current source code location? What kinds of results could be achieved by selecting other basic need than foraging for food, and deriving a theory from that?
Finally, Thomas Green conducted the closing ceremony of PPIG 2010, which featured awards for several categories of announced and unannounced competitions, as well as an improvised foraging exercise to find the thank-you presents for co-chairs Rachel Bellamy and Joseph Lawrance. The presents were found behind the speaker's podium, which was the most natural place to put them to be found, but the least natural place in which to search for them for the particular end-user in question. The fact that there was no certainty about what the presents looked like may have influenced the search.
Biggest snout: Alex McLean
Information foraging award for discovering hidden products: Luke Church
Most innovative design: Edna Rosen ("Benefits remain undiscovered")
Unfortunately defective: Margaret Burnett ("You have to lie in your own bed")
Thomas Green ("Throw the baby out of the bath")
Most unexpected concept: Randy Kaplan (pre-natal programming)
Most results in any one presentation: Zhen Li
First report of animal studies in PPIG: Chris Parnin
Self-referentiality prize: Scheila Martins
[ top ]
by Jennifer Ferreira
This is a trip report on the 8th International Advanced School in Empirical Software Engineering that took place on 15 September 2010, as part of the Empirical Software Engineering week held in Bolzano, Italy.
The event lasted a full day and the aim was to introduce attendees to the use of ethnographic methods for studying the complexities of software practice. The topics for the day focused on conducting, analysing, presenting and evaluating ethnographic studies. The chairs for this event were Helen Sharp, Yvonne Dittrich, Cleidson de Souza and myself. While Helen and I were physically present, Yvonne and Cleidson were connected throughout the day via Skype.
The seven attendees came from various countries around the world, including Italy, Denmark, the US and Brazil. Some were already doing qualitative and/or empirical research, while others were about to begin. So experience ranged from those who were interested but not necessarily doing ethnographic research, through those who had just begun, to those who had conducted ethnographic studies before. Attendees were a combination of PhD students, post-doctoral researchers and lecturers.
Next, is my very condensed account of what was an engaging and informative day ...
The day kicked off with introductions and we made a list of the burning issues that attendees wanted to address during the day. Most were interested in finding out more about qualitative studies and ethnography in particular.
Helen's presentation started with asking "What is Ethnography?" She emphasised that the ethnographer sets out to understand the informant's point of view by being present at the research site and participating (to various degrees) in what happens there. We also considered when NOT to use ethnography. In response to this, Yvonne spoke about agendas and, as a researcher, being aware of the agendas of management especially. If you realise that you don't agree with the ethics of management's agenda (for example, if their reason for allowing you to carry out research with them is to reduce staff) then continuing to work them might not be a good idea.
After the morning coffee break, we continued with "Myths and expectations" and "Doing ethnography." As part of the discussion of the myths we discussed the unexpected aspects to doing ethnography. For example, the unexpected amount of explaining you have to do at the research site. You may have negotiated access to a research site with one person, who may or may not be part of the team that you want to observe.
Inevitably, when you arrive on site, you run into many other people who have no idea why you're there and who may not even like that you're there. This led to a discussion about the importance of trust and how that is linked with the kind of access the researcher can expect. We agreed that building trust is an ongoing process between the researcher and participants, meaning that aside from non-disclosure agreements and other confidentiality arrangements, the levels of access to the research site can vary throughout the study depending on whether the researcher is trusted. In the few minutes before lunch the attendees were given a practical exercise to have a go at during the lunch break: to find a group of people to observe and spend approximately 10 minutes collecting ethnographic data.
After lunch, attendees were invited to report back and reflect on their experiences. The waiters at the sit-down lunch turned out to be popular "targets" for observations. Participants had collected various types of data – some had taken photographs, conducted short interviews while others covertly made field notes. Consensus at the end was that next time, they would like to inform their participants of what they are doing before they launch into observations. However, the point the exercise conveyed was that the challenges of ethnographic studies are the (often social) challenges that come with working with real people in real settings. Next, we considered ways of analysing and presenting qualitative findings, using the data collected for the practical exercise.
One of the issues that came up was how to deal with documenting the researcher's reflections while in the field and how to deal with that during the analysis. Yvonne recommended a strategy that can be applied to written field notes. Each page in your notebook can be divided into three sections: one for noting down the observations (what you see happening), one for your reflections on what you're observing and a third for noting emotions or mood. Some attendees made the observation that mood could affect how we perceive what is going on in the field. Therefore, along with issues such as bias, the key is to be as self-aware as possible during the collection as well as analysis stages - a self-awareness that prevents us from asking leading questions during interviews, or carrying out emotionally-charged analyses.
After the final coffee break we moved on to dealing with the "so what?" question. We agreed that our findings could have implications for tools, processes and company strategy; that our findings can bring to light the problems, conflicts and successes of software engineering in practice; and that our ultimate aim is to improve software engineering.
As the final bit of excitement for the day, we projected Yvonne on to the big screen so that she could present to the group via Skype. Her presentation dealt with applying an action research approach in combination with ethnography. The action researcher intervenes and changes practice to solve real world problems. As an example, an action researcher may identify a lack of communication between designers and developers at a research site, introduce a new method to aid the problem and then evaluate whether their communication improved. Clearly, this approach aligns well with our overall intention of improving software engineering practice.
By the end of the day everyone felt that their burning issues and questions had been addressed and they had gained plenty of food for thought. Thanks to the enthusiasm of the attendees, their interesting questions and lively discussions this year's advanced school was a great success. The chairs also had excellent support from the conference organisers Barbara Russo, Bruno Rossi and Andrej Gavrilov.
[ top ]
fabian.fagerholm (at) helsinki.fi
Department of Computer Science
University of Helsinki
Software Factory is a platform for research and education. It maintains close contact to industry and fosters entrepreneurship by collaborating with small companies and students who are considering a future as software entrepreneurs. At the end of 2009, I joined a project to design and build the first factory at the Department of Computer Science, University of Helsinki. In this article, I relate my experiences with managing this complex project.
During the doctoral consortium of PPIG 2010 in Madrid, several fellow doctoral students expressed their woes over lack of good data for their research. In general, availability of good data sources is limited. Any kind of PPIG-related research that requires dealing with human subjects has this same problem – and it is hard to imagine how most PPIG-related research could avoid human subjects. Part of the difficulties arise because interesting projects often happen where you cannot study them in detail. For example, companies may not want to open their doors for researchers who have not yet proven themselves – but to prove themselves they would need access to high-quality, valid data. Open Source project participants may be reluctant to answer yet another online questionnaire – where is the benefit for them?
Also, if you do get an opportunity to study real-life projects, the degree of control over experimental variables can be very low. Even as a project intern, you must often do with what you get, which is to accept the fact that you cannot intervene or manipulate independent variables to the degree that you would like. When the most interesting moment comes – for example, the realisation that a project may have to reduce its scope because of schedule concerns – you may be excluded from the discussion because it will occur at a higher managerial level. The basis for the final decision may be political, and your ability to draw conclusions relevant to your research will be limited. At best, you will get good data for an interesting case study. At worst, your experiment will become invalid.
In 2009, while discussing funding of my PhD work, my former MSc supervisor hinted at the possibility of a new project starting at the Department of Computer Science, University of Helsinki (CS department). I started to get involved in discussions concerning this project, and in the autumn of 2009 I was offered a position in the project. Pekka Abrahamsson, the visionary professor behind the project, needed someone to lead and coordinate planning, construction and initial operation of what was then known as a global software research lab.
The vision was of an environment to support research, education, and entrepreneurship. In professor Abrahamsson's words, software engineering research often lacks industrial impact, current studies are poorly generalised, and many results are never commercialised or understood by industry. In education, universities may expect industry to educate students about real software development. Academic educators are rarely exposed to real development complexities like restraints on cost and schedule. [Abrahamsson 2009]
The vision also painted a picture of what this environment would be in the physical world: a Software Factory with 12-15 developers working regular business hours, doing real-life software development with business goals, deadlines, delivery dates, real stakeholders, end-users and other complexities that are common in software design and development. It would be an actual facility equipped with all the necessary tools for modern software development. It would also be possible to record entire projects in high-definition video with multiple cameras to allow researchers to study software development in detail as it unfolds. The vision also had a global dimension: several similar factories would be set up around the world to simulate distributed software projects. [Abrahamsson 2009; personal communication; Abrahamsson 2010]
As we explored this vision, new ideas emerged. The educational aspect grew, and we realised that this was an opportunity to support masters- and doctoral-level education, and to exercise the so-called third mission of the university: to bridge the gap between university and society by allowing budding entrepreneurs to reach out, and existing businesses to reach in. This would also ensure that we would have a set of interesting project ideas with backing from motivated domain experts. [Kurhila 2010; Kettunen 2010a; Kettunen 2010b]
As most visions, this one was optimistic about schedules, and omitted some restrictions imposed by reality. Our funding was secured until the end of the year, which was only three months away. If we could not deliver something tangible by then, the vision would probably never take form. My challenge was to make the vision concrete and decide what needed to be implemented first. With less than three months at my disposal, I soon began to doubt whether it was possible to reach any goal at all. But I had already begun developed a vision of my own: I knew what the end result should be like. I had the choice of trying to implement it, or to let it remain a dream. I chose the former.
With so little time on my hands, delegation was the only possibility. I started involving people, letting them know about the vision, and letting them go to work, planning and implementing in very short cycles, sometimes just hours. In line with good project management, I made sure everyone's intermediate goals were as clear as possible, and in Lean fashion, I made sure that there was as little waiting as possible – making efforts run in parallel wherever feasible.
We already had secured the facility – a project room at the department – and basic construction work could start without further elaboration. We cooperated with an interior design firm to make a modular floor plan with the major activity areas mapped out: tables for working, a sofa for resting and socialising, and "hot spots" with accompanying group work equipment to facilitate information sharing and visualisation. The major structures along with electricity and network were already in place. I contacted the building architect, who connected me with key persons in the Premises and Property Services at the university. We added additional electrical outlets, lighting, and found a solution for bringing cabling to work tables in a discreet and safe way.
In parallel, I worked with the university's audiovisual services to find suitable solutions for the video and audio recording equipment. My background in media helped formulate our requirements in sufficient detail, including what types of camera angles we were looking for, what kind of lighting conditions we could expect, what the technical specifications for image quality would be, and so on. The audiovisual services staff also had experience with several kinds of group work equipment, and we chose a SmartBoard to enable collaborative and interactive planning. We also wanted to provide our software development teams with the ability to visualise their work, and settled on a three-by-three screen information radiator that the team could program with whatever visualisation was suitable for them. Finally, we also installed traditional whiteboards, which are reliable and can be used for multiple purposes.
Also in parallel, I worked with the CS department's computing facilities and Helsinki Institute for Information Technology HIIT staff to get development workstations, monitors, and the building blocks for the information radiator. Also, they provided IP networks for the factory and video equipment, hosting for the network video recorder, disk space in their SAN, and an internal wiki and version control system.
Recording entire projects is challenging not only technically, but also from a legal and ethical perspective. Ethical issues are highly relevant for software research [Singer 2002]. I realised early on that this aspect was largely neglected in the visionary ideation. It is easy to forget that computer science is not only a mathematically or technically oriented field, there are humans involved that can become research subjects even if the research is primarily concerned with software and development processes. They can also be the primary focus of the research in some cases. Not being a lawyer, all I could do was gather the needed information, present it to my group, and make sure I had provisions for handling all data in the correct manner. Some specific details had to be in place: there had to be a separate, secure network for the video data, and it had to be stored securely and be accessible only to persons who needed it.
There was also the question of copyrights: who owns the works produced in the factory? I discovered there was little knowledge in my group about copyrights, patents, and other intellectual property concepts. They mostly seem to be considered a nuisance that is put off as long as possible. Another course at the CS department had had similar needs in the past when working with industry partners. In that course, participants either all licensed their work under an open source license, or they transferred their copyright to the industry partner. In this case, however, support for a more diverse range of scenarios was needed. In addition to the open source and complete transfer alternatives, there was also the possibility of several academic and industry partners in a single project, continuation projects where previous rights holders would not necessarily be available to approve further use of their works, and projects targeting a spinoff company. The university's legal department helped draft a contract that would enable the first step to support these scenarios.
Software Factory became operational in January 2010. The first project was launched on the 18th of January, and was concluded seven weeks later, on the 15th of March. From the first day of operation, Software Factory has been a platform that solves the problems of data acquisition and access to projects. For researchers, it provides a testbed for research methods and for doctoral training. For students, it provides a valuable learning experience to prepare them for project work in software development organisations. For entrepreneurs, both external to the university and to students wishing to start their own company, it provides an environment where ideas for software products and services can be taken to a prototype stage.
Software Factory has now been operating for a year and a half at the CS department. We have been able to try several kinds of studies, ranging from qualitative, observation- and participation-based research, to quantitative analysis of project metrics. Our master's students have appreciated the opportunity to study real projects for their masters theses, and PhD students have been able to train research skills needed in complex projects.
Participation in Software Factory currently is in the form of a course. Each course corresponds to a project – however in some cases, several projects might be run in the same course, if they have common elements or are small enough. The course is far from easy. Students participating as development team members have the responsibility to carry out the project from start to finish. There is no "pause button" that can be used to get more time for the project; the deadline is known in advance and students must balance time, resources, and requirements as in a real project. Our experiences so far have shown that our students differ greatly in technical skills, motivation, and ability to participate actively and take initiative in the projects. Personally, I think of this as a positive challenge: how can we develop the course and the rest of the curriculum to support the skills needed in complex software development projects?
Until now, Software Factory has predominantly been a single-site environment. However, modern software development often occurs in a distributed setting, with several sites collaborating on the same project. The ultimate goal of Software Factory is to have a number of factories around the world which can be connected on demand to collaborate on common projects. This would allow research and education in Software Factory to be on the same level as state-of-the-art software projects. The first steps towards this have already been taken: the University of Eastern Finland, Joensuu campus, have applied the Software Factory concept in their own setting. The first video conference has been held, and in 2011, we have carried out two joint projects. We are also expecting sites in Spain [Martín Ruiz et al. 2010] and Italy to join our collaboration. Meanwhile, we are moving our own Software Factory to a bigger room, so that we have the space necessary for bigger projects and communication equipment for distributed development.
Education in Software Factory bears little resemblance to traditional university lectures. It is also different from other lab projects at the CS department because such a large responsibility is placed on the students. Had I been offered such a course during my master's studies, I would not have hesitated to take it. Having worked in the software industry, I also know that the skills that can potentially be learned during a Software Factory project are invaluable in the workplace. However, it is not clear exactly what these skills are, how to ensure that they are being learned, or how to grade students in this kind of a project. I have taken some initial steps, but there is much to be learned about learning in this kind of an environment.
Finally, a valuable contribution to the field of software research would be the release of a rich software project data set. Many other fields have standard data sets that are used in education and method development, but software research lacks a set of complete project data. The legal and ethical challenges are great, since we must balance between the confidentiality and anonymity of research subjects, and open access to enable meaningful research. This will be one of the great challenges for Software Factory in the coming years.
Software Factory extends an open invitation to join our network of factories working to advance research, education, and entrepreneurship in the field of software. Setting up a Software Factory at your university requires a lot of work and some resources, but we have made the process easier and can give helpful advice and blueprints. With some dedication, you can build it in less than three months. (It has been proven possible...)
[Abrahamsson 2009] Abrahamsson, P.: Global Software Laboratory
Vision. Unpublished slide set. September 2009.
[Abrahamsson 2010] Abrahamsson, P., Kettunen, P., Fagerholm, F.: The Set-Up of a Valuable Software Engineering Research Infrastructure of the 2010s. The 11th International Conference on Product Focused Software Development and Process Improvement (PROFES 2010) / Workshop on Valuable Software Products (VASOP 2010). June 2010.
[Kettunen 2010a] Kettunen, P.: Strategic outlook: Software Factory fosters next-generation software production and business. Software Software Factory Magazine. Volume 1, January 2010, p. 5.
[Kettunen 2010b] Kettunen, P.: Software Factory research helps in factoring high-performance software enterprises – even at large scale. Software Software Factory Magazine. Volume 1, January 2010, p. 7.
[Kurhila 2010] Kurhila, J.: Educational perspectives in Software Factory. Software Software Factory Magazine. Volume 1, January 2010, p. 6.
[Martín Ruiz et al. 2010] Martín Ruiz, Juan Luis and Yagüe, Agustín and Gonzales Ortega, Eloy and Garbajosa, Juan: Making Software Factory truly global: the Smart Software Factory Project. Software Software Factory Magazine. Volume 1, January 2010, p. 19.
[Singer 2002] Singer, J. and Vinson, N.: Ethical Issues in Empirical Studies of Software. IEEE Transactions on Software Engineering Engineering, Vol. 28, Issue 12, December 2002, pp. 1171-1180.
[ top ]
by Chris Douce
I haven't been working in the field of software development for a number of years now. During this time, all the coding I've done has been some PHP development on a research project and migrating five or six small C++ programs to Java. When I say small, I mean programs ranging between one hundred and five hundred lines long, nothing substantial. The programs didn't do much either: they were designed to exercise different OO features and might be used as experimental materials for a small project I'm involved with.
The thing that struck me about doing this small programming task was how much time it took me. I struggled a bit for the first couple of hours. I quickly realised that I was 'rusty'. I could barely remember the Java syntax (which reminded me of the Shneiderman/Mayer model, which was one of the first psychology of programming papers I read) and had to constantly refer to documentation and websites for help. Programming began to take on the form of a process of re-learning how to do stuff.
Here's another dimension to this: just before starting to compile this article I downloaded a web development IDE (so as to not blind myself with too many angle brackets). I visited the vendors website (which I hadn't visited for a couple of years) and was surprised to see that everything had changed. The original product I was once familiar with had morphed into something else with loads of new unfamiliar functions.
These two points made me think of things from the side of technology recruiters (a subject that is related to one of the 'connected links' that features within a later part of this newsletter). How might my skills be interpreted by a recruiter. Although I might be able to say, 'yeah, I've used C++, I've four or five years experience of using it...', I am likely to be asked (as I have been asked before) 'but when was the last time you have used it?'
One of my long standing interests in this area of the psychology of programming is that of memory. We use different types of memory whilst we code: different types of short term memory (spatial and articulatory), long term memory where we hold facts about how things work, and autobiographical, which is facts about stuff what we did.
I guess my own long-term memory for programming can be divided into two main areas. The first is memory that relates to skills, such as knowing how to do stuff in a particular language or a product. The second relates to knowledge of systems that I have worked on. What I mean is that if I'm sat in front of the code for a project that I worked on five years ago, I'll probably remember quite a lot of what goes where, and also remember why I coded things in particular way. But might this be something unique to me? I doubt it. Perhaps it might be useful to understand more about the notion of developer skill degradation (and if there is such a phenomenon).
I guess technical (and software development) skills exist in hierarchies. At the top there's operating systems, followed by how to use integrated development environments (where menu options and compiler options can move around), followed by programming languages. Another dimension is the notion of software development frameworks or libraries. Knowing such products, of course, enables developers to gain huge productivity boosts.
Early in my studies, I discovered a whole vein of research where cognitive psychologists were trying to understand the nature of expertise through specialist domains, such as chess. Computer programming was one of those domains too. Not so long ago a colleague mentioned in passing that expertise is something that takes ten thousand hours of study or work to acquire (don't ask me for a reference for this, since I don't have one!)
I've lost count of the number of times bloggers and writers have compared the activity of software development and design as a craft skill. Perhaps computer programming expertise ought to be conceptualised less in terms of individual products (such as languages), but instead in terms of sets of tools?
I guess this short collection of (quite random) paragraphs is ultimately asking a number of questions. These might be: how is technical expertise perceived by others? Do developers tire of developing? Is there such a thing as a career ladder for engineers who don't wish to move into management? My final questions are, 'what does it actually mean when you say that you're rusty?' and 'how long does it take for an experienced developer to get moving again?'
[ top ]
A copy of The Psychology of Programming (1990), as edited by Hoc, Green, Samurcay and Gilmore can be found on the information pages which relate to Alan Blackwell's Usability of Programming Languages course which Alan writes about later within this newsletter. Many thanks to Alan for making this resource available. It should be noted that this resource is only for educational and research use, since copyright remains, of course, with the publishers.
Not so long ago I was directed to a paper entitled Scratch: programming for all by Mitchel Resnick et. al., published in the Communications of the ACM
Scratch forms an important part of a new Open University course that is entitled TU100 My Digital Life. Students can make use of Scratch to interface with some physical hardware, enabling students to gain an understanding of both interfacing and how to combine different programming constructs together.
[ top ]
PPIG members who attended the 2010 conference will remember the discussion of a practical course oriented toward research skills in Psychology of Programming. I can now report on the experience of a first year teaching that syllabus.
The course 'Usability of Programming Languages', was offered as a module in the Cambridge MPhil in Advanced Computer Science for 2010/11. Seven students out of a total cohort of 30 elected to take this module, and all successfully completed it. I expect that some of these will be submitting results of their research projects to PPIG 2011, and that PPIG members will have a chance to meet them there.
The main textbook for the course was Hoc, Green, Samurçay and Gilmore's 1990 book 'Psychology of Programming', supplemented by Carroll's collection of models, theories and frameworks in HCI research, and the recent overview by Cairns and Cox of research methods for HCI. The observant will note that there is one author who has contributed to all of these course texts - thank you Thomas!
As a practical course, formal presentation time was minimised, with only four one-hour lectures. The first of these gave an overview of the most common theoretical approaches to PoP research, and of those groups around the world with the most active programmes of research.
The second outlined the research methods that are most often applied in the field, and the third the particular classes of user that receive most attention: students, 'end-user' programmers, and in our group at Cambridge, domestic and arts programming. The final lecture led the student through a process of research design, after which they started work on the design and execution of individual experimental studies. In retrospect, an additional lecture on cognitive dimensions would have been appreciated. Cambridge undergraduates become familiar with CDs during their degree, but none of the students on this module had been at Cambridge previously.
The bulk of the course work consisted of the completion of an experiment, exploring questions that ranged from evaluation of novel programming languages created by the students themselves, to assessment of manufacturer's claims for systems such as Lego Mindstorms and Google AppInventor. Some interesting studies also applied PPIG perspectives to more conventional HCI problems, such as the specification of email filtering rules in different mail systems, or the usability consequences of creating spreadsheet formulae when working with extremely large data sets.
Over a period of 5 weeks, experimental design, testing and data analysis was punctuated by seminars, at which the students discussed their work in progress, including the familiar challenges of recruiting participants with specific experience, running pilot studies, and balancing internal and external validity in the search for controlled experiments that demonstrate statistically significant effects. All students submitted final reports in the format of the PPIG proceedings, having read extensively in the online PPIG archive.
I am pleased to say that despite the rather intensive workload, student feedback was overwhelmingly positive, and that this module has been a very welcome addition to the Masters syllabus. It will certainly be repeated. Several of the students who took this module are now planning future research careers in which psychology of programming will form an important element, including two who will be starting PhDs in Cambridge next year. In marking their final assignments, I encouraged several to submit their research to PPIG - with luck, newsletter readers will meet them soon!
Full details of the course, including lecture notes and slides, and links to other teaching material, can be found on the course website
Cairns, P. and Cox, A.L. (2008) Research Methods for Human-Computer Interaction. Cambridge University Press.
Carroll, J.M. (Ed) (2003). HCI Models, Theories and Frameworks: Toward a multidisciplinary science. Morgan Kaufmann.
Hoc, J.M, Green , T.R.G, Samurcay, R and Gilmore, D.J (Eds.) (1990) Psychology of Programming. Academic Press.
I am a current PhD research student working in the School of Computing and Mathematics at Keele University, UK. My research involves investigating innovative methods of teaching introductory programming with the aim of promoting a greater understanding of the subject and increasing student motivation. I am working under the supervision of Dr Theocharis Kyriacou and Professor Pearl Brereton.
My initial research, after performing a Systematic Literature Review (SLR), has found how the use of robots as teaching tools may help to promote a greater understanding of programming and can overcome some of the difficulties that impact upon the successful teaching of it. A paper reporting the results of the SLR has been presented at the 15th International Conference on Evaluation and Assessment in Software Engineering - EASE 2011 (Durham University, UK, 11 - 12 April 2011).
The SLR highlighted how robots can be a powerful and effective tool when used in an introductory programming course but that the potential remains to further investigate methods for their implementation. In particular it was found how there remains scope to evaluate the effectiveness of simulated robots as tools to teach programming.
The results of the SLR will provide the platform upon which future research activities can be built. The development and testing of simulated software, that utilises the concept of robotic agents, will be undertaken in order to further assess the effectiveness of such a method. It is also being considered how such a tool may be implemented in a high school environment and a preliminary session has already been held with trainee high school teachers to determine their opinions of such a tool.
There is a wealth of literature related to Systematic Literature Reviews available. In a nutshell a SLR is a form of secondary study that aims to provide an objective and unbiased approach for finding relevant primary studies, and for extracting and aggregating the data from these. The Evidence-Based Software Engineering website (http://www.dur.ac.uk/ebse/) is a great place to start if you want to learn more about SLR's and other similar tools. I have also provided the reference of my SLR below .
I would be grateful of any feedback from the PPIG community in regards to the research which I am performing. Likewise I would be happy to answer any questions which you may have. I look forward to meeting some of you in September at the annual PPIG workshop! My email address is: email@example.com
 Major, L., Kyriacou, T. and Brereton, O. P. Systematic Literature Review: Teaching Novices Programming Using Robots. In 15th International Conference on Evaluation and Assessment in Software Engineering (EASE 2011), Durham University, UK, 11 - 12 April 2011. pp. 21-30.
My paper for PPIG 2010 was about bricolage programmers, in particular artists who write software without any clear plan, but just reacting to the results of each edit. From feedback it is clear that the paper could have done with a decent case study, so I thought I'd contribute the following example to this newsletter. This is not meant to illustrate great art, or indeed great programming, but just to act as a talking point when discussing alternative approaches of programming. Full versions of the examples are available on sketchpatch.
Imagine a visual artist, programming their work using the Processing environment, a language based on Java. They begin with an urge to draw superimposed curved lines, and come up with the following program, shown with its output:
On seeing the output, they are struck first by how hairy it looks, but then by the suggestion of a scribble. They decide that they are interested in the latter, and change their program to join the curves together, removing the hairiness and accentuating the scribble:
The artist reflects upon the letter-like quality of the scribble forms, and decides to try writing letters across the page, grouped into word-like forms:
The output has a handwritten quality, almost appearing to be readable, a quality of 'automatic writing' used by mystics to supposedly channel the spirit world. This may bring further conceptual development in our artist's mind, but at this point we leave will them pondering.
Alex is from Goldsmiths, University of London
It's your first day at your new job at Acme Software. Or you've just moved to a new team. Or you need to resurrect ten-year-old software. You browse through the source code with a sinking feeling: where do you start? If only there was someone you could ask... but the original author is stuck in a meeting. Or on holiday. Or left the company years ago.
Knowing how to program is only part of software development. Developers need to understand at least part of a codebase before they can make effective modifications, and accumulating this knowledge is time-consuming especially when no mentor is available.
My research begins to address this problem by investigating how experts explain their code to new team members. Through video recordings, screen capture and code analysis, I'm seeking to understand what information about a codebase is most useful in this situation.
I'm always looking for opportunities to collect data, so if you'll be bringing someone up to speed on your code (or you know someone who is) I'd be very grateful to hear from you. Contact information can be found below, or through my homepage.
Contact: rebecca.yates (at) lero.ie
Would you like to tell other PPIGers how you are and what you are doing through the newsletter? If so, please e-mail c.douce (at) open.ac.uk
[ top ]
Some time ago, I heard of a new programming language called Go (which has apparently been developed by Google, as described in an article on the same subject). I have to confess I don't know much about it, but it appears to be dynamically typed and have a C like syntax.
Some comments on commenting: I recently discovered an article that discusses whether the elegance of comments may reflect the elegance of code. I have to confess I agree with this view. I seem to remember rolling my eyes on too many occasions when I've discovered one too many HACK comments and the sentence, 'this seems to work but I'm not quite sure why' (I'm sure we've all see these!)
Some bugs are just plain weird, like this one about a more bugs are written in software between midnight and 4am than at other times of the day.
The domain of visual programming (or whatever that means) remains and interest, but developers at Microsoft confess their attachment to the text editor in an article entitled: Microsofts top developers prefer old school coding methods (this begs the question, 'so, what are the new school coding methods?')
You've heard of carbon offsetting, right? Well, there is this crazy (or innovative?) idea of 'code offsetting', which allows you to pay for your past software development sins (go on, admit it, we all have them… they come from a time when we were just learning the craft and didn't understand the consequences of our choice of constructs and abstractions). More information can be found through The Alliance for Code Excellence if you have enough time.
On a more serious (but related note), I really enjoyed a slashdot discussion about how to tackle a large codebase.
I have also made a note of an article about 12 programming mistakes to avoid (I'm now wondering whether there is a shorter list of just seven).
There is also a link to another blog post about perennial discussions (or perhaps 'old chestnut' discussions?) about how important mathematics is for computer science education (and software development jobs in general). I guess it comes down to what kind of job you would like to do.
Frank Wales circulated an interesting opinion piece that compares hypnosis and programming I think the key point is that are both difficult to understand from an outsiders perspective, and both can be considered to be a craft of some form or another.
Have you ever debugged using sound? Here's an inspired use of sound that demonstrates the operation of sorting algorithms. I could have sworn I heard the bubblesort algorithm played in a Manchester nightclub I once went to...
The sound clips do have a slight retro feel to them, which leads me onto a TV clip entitled BBC Micros used in retro programming classes. It was also interesting to see the use of kinaesthetic learning (which can be briefly witnessed at the start of the clip). I love the quote, 'I managed to turn the plane from white to black.'. Ah, those were the days...
Here's a related link to an article entitled tools for teaching kids to code.
It contains a number of embedded YouTube videos which give a flavour of the different environments (the first video relates to the Scratch paper that was mentioned earlier in the newsletter). Roman Bednarik also points us to the Finch Robot, by CMU, which can be used for computer science education. I do like the idea of its design simplicity, and the fact that you can simply connect it to a USB port.
When a programmer turns professional there is a lot of advice available, including this handy list of strategies for becoming a better programmer. (I'm not sure about the idea of unlearning stuff – some stuff is just hard to forget, like the confusion I faced when trying to figure out Prolog for the first time).
Even if you've memorized all these tips, it may take time to find your feel at a programming shop. Tech recruiters are always faced with the difficulty of trying to figure out what people know. This issue is reflected in an article why the new guy can't code (I'm assuming we can forgive the author's gender bias through his footnote).
And finally, the idea that social media might be able to tell us a little more about the world of software development and coding. On an article about programming language popularity it appears that Twitter users make more references to PHP than any other language.
[ top ]
If you would like to become the new PPIG newsletter editor, why not send a message to c.douce (at) open.ac.uk with the subject heading of, 'I want to become the new newsletter editor'.
Personal qualities should include unbridled enthusiasm, a willingness to send out numerous e-mail messages, and sufficient patience to code pages of HTML. I look forward to hearing from you!
[ top ]
Many thanks are extended to all contributors and the efforts of the reviewers of this edition of the newsletter are appreciated. Thanks goes to Roman Bednarik for providing some of the connected links for this issue and for co-ordinating the collection of some of the articles that feature within this issue.