Research on Agile technical practices are few and far between. The research that exists often uses students as the test subjects so doesn’t necessarily reflect professional practice, particularly not by senior professionals. In 2006 I got my company of the time involved in some research on pair programming. What was different about this study was that the subjects were professional developers. The results were interesting as, in general, they didn’t bear out claims that pair programming improved quality or speeded up development. But pairing does take a lot more effort. This is why I am selective about encouraging pair programming, restricting my active encouragement to discovery mode.
Arisholm et al. (2007) studied professional Java developers from 29 companies in Norway, Sweden, and the UK. The report talks about “consultancies” and “consultants” so I presume all of the companies were software houses like the one I worked for. In other words the participants were professional developers working in a context where the company specialised in software development.
The 295 participants were hired for one day to participate in a controlled experiment on pair programming. The participants used professional Java tools to perform several change tasks on both a simple Java systems and a more complex one. Participants were divided into junior, intermediate and senior levels of ability, and pairs were formed from people of the same ability.
The study showed that “pair programming in general does not reduce the time required to solve the tasks correctly nor increases the proportion of correct solutions”. But it also showed that pair programming takes 84% more effort – not too surprising given it takes two people to form a pair.
The “in general” bit in the report summary is significant. They did find benefits to pair programming but only for certain combinations of system complexity and developer expertise. Intermediate and senior level developers completed tasks faster as a pair than as individuals on the simple system, but not the complex system. Juniors pairs were more likely to produce correct solutions on the complex system but not the simple system.
I’ve summarised the results here:
|Effect on||Simple system||Complex system|
|Effort to perform the tasks correctly||+84%|
|Time taken to solve the tasks correctly||-20% mainly for intermediates and seniors||0%|
|Proportion of correct solutions||0%||+48% mainly for juniors|
These results are rather underwhelming given the hype around pairing. This is why I am selective about encouraging pair programming. In general I encourage Seniors to work alone – effectively they get twice as much done as they are working in parallel. I restrict my active encouragement for pairing to those times when either developer is in discovery mode. Juniors are in discovery mode pretty much all the time, at least when they join a team. Seniors are also in discovery mode when they are exploring a new problem space.
Please drop me a line if you know of other Lean-Agile research on professionals.
Arisholm, Gallis, Dybå, & Sjøberg. (Feb., 2007). Evaluating Pair Programming with Respect to System Complexity and Programmer Expertise. IEEE Transactions on Software Engineering, 33(2), 65-86.