Research on Agile technical practices are few and far between. The research that exists often uses students as the test subjects so doesn’t necessarily reflect professional practice, particularly not by senior professionals. In 2006 I got my company of the time involved in some research on pair programming. What was different about this study was that the subjects were professional developers. The results were interesting as, in general, they didn’t bear out claims that pair programming improved quality or speeded up development. But pairing does take a lot more effort. This is why I am selective about encouraging pair programming, restricting my active encouragement to discovery mode.
Arisholm et al. (2007) studied professional Java developers from 29 companies in Norway, Sweden, and the UK. The report talks about “consultancies” and “consultants” so I presume all of the companies were software houses like the one I worked for. In other words the participants were professional developers working in a context where the company specialised in software development.
The 295 participants were hired for one day to participate in a controlled experiment on pair programming. The participants used professional Java tools to perform several change tasks on both a simple Java systems and a more complex one. Participants were divided into junior, intermediate and senior levels of ability, and pairs were formed from people of the same ability.
The study showed that “pair programming in general does not reduce the time required to solve the tasks correctly nor increases the proportion of correct solutions”. But it also showed that pair programming takes 84% more effort – not too surprising given it takes two people to form a pair.
The “in general” bit in the report summary is significant. They did find benefits to pair programming but only for certain combinations of system complexity and developer expertise. Intermediate and senior level developers completed tasks faster as a pair than as individuals on the simple system, but not the complex system. Juniors pairs were more likely to produce correct solutions on the complex system but not the simple system.
I’ve summarised the results here:
| Effect on | Simple system | Complex system |
|---|---|---|
| Effort to perform the tasks correctly | +84% | |
| Time taken to solve the tasks correctly | -20% mainly for intermediates and seniors | 0% |
| Proportion of correct solutions | 0% | +48% mainly for juniors |
These results are rather underwhelming given the hype around pairing. This is why I am selective about encouraging pair programming. In general I encourage Seniors to work alone – effectively they get twice as much done as they are working in parallel. I restrict my active encouragement for pairing to those times when either developer is in discovery mode. Juniors are in discovery mode pretty much all the time, at least when they join a team. Seniors are also in discovery mode when they are exploring a new problem space.
Please drop me a line if you know of other Lean-Agile research on professionals.
References
Arisholm, Gallis, Dybå, & Sjøberg. (Feb., 2007). Evaluating Pair Programming with Respect to System Complexity and Programmer Expertise. IEEE Transactions on Software Engineering, 33(2), 65-86.
Pair programming requires training. It goes far beyond sitting two random programmers in front of the same computer.
The rearch should be done with people that know how to pair. Maybe even with pairs that already know each other.
Finally the goal of pairing is not doing things faster, but with more quality and ensuring transmission of knowledge and standards compliance
Txau, I agree.
In fact the research project addressed most of your concerns:
– The participants who paired already knew each other and had worked together. At least that was the case in my company.
– They were instructed in how to pair although this was not formal training.
– The outcomes considered more than just elapsed time (going faster); it also considered effort and quality.
But a research project cannot cover everything. This project didn’t address “transmission of knowledge and standards compliance”. I agree these are worthy outcomes of pairing. However research on this would probably have to be long term. Finding professionals willing to engage in a long term study could be a challenge . . . or be expensive. Which might explain why there is a gap in the research.
I admit a short introduction to pairing is not the same as “training”. However from my experience people learn pairing by pairing. Now days there might be enough professionals with experience of pairing that a new study restricted to experienced pairs might be possible. That wasn’t possible a few years ago.
Your argument about Seniors working faster is perhaps true in the short term … Yes they can pumped out more, but over time sharing knowledge and developing skills in aggregate over a team will benefit development velocity and avoid siloed knowledge. Teams that are happier working together also end up building more consistent well thought out products.
Watched a great talk “Pairing is caring” about the benefits of remote pair programming recently:
http://youtu.be/zCzc5W7vHQg
Amongst other points, pairing with an external expert can ramp up team members on new technologies and methodologies like TDD.
My company has a service that let’s companies book experts on demand to assimilate knowledge quickly and fill knowledge gaps within teams:
http://www.airpair.com/pair-programming
Jonathan
I think we largely agree. I’m just using different language. From my perspective seniors work faster. Juniors are more often in discovery mode. Seniors pairing with juniors help the juniors through discovery mode (https://itsadeliverything.com/pair-programming-is-best-in-discovery-mode). That can help a junior become a senior. At which point they are in discovery mode less often and the need for pairing, at the personal level, declines.
At a departmental or organisational level then having less silos is a good idea and pairing can also address that. There are, of course, other ways to do this.
I don’t see “pairing” being correlated to “happier working together”. There are many teams that are happy working together that don’t pair. I have also seen pairing leading to unhappy coworkers.
Thanks for the link to Avdi Grimm’s talk.
Cheers
Steven
I propose that it is also beneficial for anything that is blocked at any level of expertise, especially for debugging. A second person naturally challenges your assumptions and provides their own.
This “research” is highly suspicious. Taking 300 programmers who aren’t accustomed to pairing and requiring them to pair for a day is nothing remotely like the conditions of a disciplined pairing team. Pairing is a skill that has to be learned; different people acclimate to it at different rates, and senior programmers (i.e. those who are most ingrained in their existing practices) tend to take the longest to pick it up.
While it is true that many programmers chose the profession because it tended to be a solitary one, that does not give them the right to insist that it remain so at the cost of productivity and quality. And while, for a small number, pairing may truly not be the best option, in my experience developers who claim that to be true do so long before they have actually tried it long enough to reach an informed conclusion. Unfortunately, blog posts like this one – propagating “studies” and “findings” that bear no resemblance to reality – give them the excuse they are looking for to remain close minded and stuck in the past.
So what’s your alternative? Just assume that it is better despite no evidence? Do you have any empirical evidence?
The research compares developers working in pairs to developers working individually. And, by definition, it comes with empirical evidence.
This study (http://hakanerdogmus.net/weblog/uploads/papers/EE2003-ErdogmusWilliams.pdf) concludes that pair programming generally creates superior economic value
Interesting paper but I’d be wary of drawing universal conclusions from it. Like much research on pair programming the subjects are students. So, to paraphrase the abstract, “our analysis demonstrates a distinct economic advantage of pair [novice] programmers over solo [novice] programmers”. That result is aligned with the research I was involved in, and my own observations, that juniors benefit from pair programming more than seniors.
There is a further caveat that, for these conclusions to be valid, all of these students must also be using a variation on the PSP [Personal Software Process]. And I’m not aware of any commercial organisations that insist on PSP, not even in 1999 when the original research used as the basis for this paper was conducted.
I’m surprised no one noticed this but the flaw in the experiment is this:
“pairs were formed from people of the same ability.”
The goal of pair programming is to pair senior & junior together. This helps with knowledge transfer and is the best way to bring improve the junior programmer’s skills.
Pairing two seniors or two juniors is not very interesting in general, although I’ve often paired with someone for a particularly difficult task.
Like everything else, it’s not one size fits all. One must try it and adapt.
I don’t see why it is a flaw. Although is common “to pair senior & junior together” but that isn’t the “goal” of pair programming and many teams have other combinations of developers in a pair. As it happens the research provides empirical evidence why the senior-junior pairing “helps with knowledge transfer and is the best way to bring improve the junior programmer’s skills”. I agree that juniors benefit from pairing more than seniors (the research also supports this), except, as you say when the seniors are working on “a particularly difficult task”.
It’s good to see some formal research for a change. Pair programming is a very expensive practice and it has to prove itself. Now, determining the R of the ROI is always hard with software methodologies – a lot of the benefits attributed to pairing are long-term and ‘soft’ (knowledge sharing, communication, morale, etc.) – but given how many resources insist quite explicitly that pairing will substantially improve correctness, it’s very important to evaluate those claims.
One thing I would observe is that the benefits of pairing seem to be quite woolily defined. Some pairing advocates reject negative accounts of pairing where developers have equal skill levels, insisting that it’s all about learning. Others insist that inequality leads to passivity and disengagement by the less skilled programmer. Others say it’s just a matter of having the right personality. Yet I’ve seen other studies online that suggest personality is irrelevant. This makes it hard to push back against pairing as a practice, because it doesn’t seem clear what we’re actually expecting pairing to achieve.
Pairing is definitely a personality thing, especially for senior developers. I don’t understand how can any study suggest otherwise, It’s like asking is partying good for you, sure it helps you socialise, but introverts simply cannot stand partying every night.
I have seen quality went down in pair programming, because 1. people don’t take code ownership like they use to. 2. People’s thought are constantly distracted by each other.
Pair programming is beneficial in discovery mode and knowledge transfer, but that’s about it. Never force developers to pair, developers will pair when they see the need. Pair programming long exist before the term “pair programming” was invented.