The Power of Pairs

In my current contract we have a policy of full-time pair programming. It has been working incredibly well. I thought I'd share some of my observations as to why this might be the case.

Previous experience of pairing

I've used and worked with pairs in previous roles with various degrees of success. However, none of those previous roles adopted the full-time pairing model. I've always seen some benefit in pairing, especially on complex or critical pieces of code: two minds working together on these sorts of problems is invaluable. My previous roles have, however, always gone back to singletons for the more mundane and less critical coding tasks.

The one thing I have never seen previously is any of the purported productivity boosts from pair working. This could be because I've only used pairs on for complex tasks, which would take a long time anyway. But surely I should have seen some benefit in the past? Why didn't I?

Our pairing model

The team that I am working with at the moment is made up of five developers. Each of us comes with experience of many successfully completed projects. We all have different language and domain backgrounds, but there seems to be a common thirst from every member of the team for learning and improving. I think this is perhaps one of the key reasons why our pairing model works so well.

Each day we swap the pairs. Sometimes if we are deep in a task we might keep the same pair for two days, but never longer. The dynamic of the team is therefore always changing. It also encourages up to keep tasks small so that we can generally complete them in a single pairing session.

Hold on, don't we have an odd number of people? Yes, and this I think works very well. Each day we pair up and one person gets to work on their own for the day. We try to ensure that each person gets to be the singleton regularly. This single person picks up simple technical debt tasks, carries out investigative spike or just refactors and tidies up the code base. They are requested not to pick up any complex development. Working in pairs is incredibly demanding of concentration and being the singleton for a day also helps keep people fresh.

The benefits

So, why does this pairing model work so well, and why have we created a team that appears to be incredibly productive? Here's my thoughts…

The right team members - All of the developers on the team are good developers and more importantly they are open and keen to learn and improve. We have a number of different language and platform backgrounds on the team, but rather than being a problem it is just more useful information to share between us. The team members have created an environment of sharing and constant learning.

Willingness to question and compromise - Every member of the team has the ability and confidence to question the approach being taken by the other developer in their pair. While I think experience helps in this, you really want even junior team members to have this confidence to question. Linked to this is the fact that each team member is open to new ideas and is either prepared change their thinking or, even if they disagree, come to some form of compromise to allow the pair to continue moving forward. Developers who argue, fail to question or who are not prepared to learn and adapt will kill a pair's productivity.

Shared knowledge - By rotating the pairs on a daily basis, knowledge of the application, language and development approaches is spread throughout the team. This just wouldn't happen if pairs were fixed or people working alone.

The right tools and environment - The team have created a development and build environment that they are comfortable with and that is productive. The team are empowered to make any changes that allow them to work better. These could be changes to the build process or introduction of a new library the cuts the amount of or simplifies code.

A focus on quality - All the members of the team have the same view of the importance of quality and testing. There us a mutual agreement of what constitutes good quality and all team members strive for this goal. Teams where there are differing notions of quality between individual members just don't work as well - even in a non-pairing environment.

The ability to technically invest - By having one team member constantly assigned to cleaning up technical debt, refactoring and improving the code base it allows the pairs to spend all of their time investing in new features. The code base is kept clean and this makes it much easier to introduce new approaches or libraries that further improve the code.

Conclusion

So, would I adopt full-time pair programming again in the future? Definitely! However, to get the real benefit you need the right team of people and you need to create the right environment and culture for them to work in and give them the right set of tools. I'd also try to always have an odd number of developers so that pairs could be rotated and one person be free to clear down any technical debt so that the pairs can invest.

Scala eXchange London 2011

It's been an information packed two days of learning at the Scala eXchange 2011, hosted by Skills Matter. Twenty-one fantastic sessions covering a wide range of subjects. It was great to have so many in-depth sessions. Previous Scala conferences I have been to have tended to be full of introduction level sessions. This conference was different, all of the sessions went into a great deal of depth, which perhaps shows how mature the Scala community is becoming.

The highlight of the two days for me had to be James Strachan's entertaining keynote on the Scalate templating engine. It's certainly going to right to the top of my toolbox when it comes to needing a templating solution. The laughs were long and loud when a small table crashed to the ground in the corner and James commented "...that's the JSP developer falling over!". Scalate is certainly many steps beyond JSP.

Victor Klang's imaginatively titled session "The Promising Future of Akka" was my second highlight. Not only an inspired name of the presentation but some great in depth details on how Akka is implementing the Future/Promise model and how these complement the existing Actor model.

The overall impression I got from the two days was that it was a gathering of very smart people who "get it". The Scala community seems to be made up of the best and the brightest, who are investing in finding better ways to write software. The depth of the talks and the many conversations had over coffee show a great maturity of thought. These are people thinking beyond simple POJOs, DI Frameworks and ORM tools. I think we are on the verge of another major shift in the way software is designed, constructed and deployed.

The other thing that I drew from the two days is that Scala is no longer an experimental plaything, it's now a mature technology. The core language is now stable, the tools are mature to the point where they are almost as functionality complete as their Java counterparts and there is a wide understanding of how to use and apply Scala. The effort has now shifted onto solving more advanced problems like huge parallelism, complex enterprise integrations and complex event driven challenges. From being trailblazers, the Scala community are instead becoming ambassadors: showing how Scala leads to cleaner code, more elegant solutions and how Scala fits so well with the agile process.

Invariance, Covariance and Contravariance

While learning Scala I spent a fair amount of time getting my head around covariant and contravariant concepts. Just recently I found myself tryying to explain these to a collegue. I've since refined my explanation and decided to write it down for all those people who want a clear, maths free explanation.

Invariance

Let's consider a simple domain model. We have a base class 'Employee' and a sub-class 'Manager':

scala> class Employee(val number: String, val name: String)
defined class Employee

scala< class Manager(number: String, name: String, val department: String) extends Employee(number, name)
defined class Manager

Next, we have an award class that we implement using a parameterised recipient type so that awards can be used for more than just our Employee/Manager model:

scala> class Award[T](val recipient: T)
defined class Award

So, lets create an award that is for one of our managers:

scala> val mike = new Manager("1", "Mike", "Sales")
mike: Manager = Manager@712c20d9

scala> val managerAward = new Award[Manager](mike)
managerAward: Award[Manager] = Award@411a9435

All well and good. We have a manager award instance. However, say we have a variable that is of type Award[Employee]. It would seem natural to be able to assign our manager award to this variable as a manager is a type of employee, so their award should also be part of the general set of awards given to employees, right? Well...

scala> val employeeAward: Award[Employee] = managerAward
<console>:12: error: type mismatch;
 found   : Award[Manager]
 required: Award[Employee]
Note: Manager <: Employee, but class Award is invariant in type T.
You may wish to define T as +T instead. (SLS 4.5)
       val employeeAward: Award[Employee] = managerAward

So, we see that this is not possible. We also can't add our manager award to a list of awards given to employees, even though it seems natural that we should be able to do so:

scala> val awards = List[Award[Employee]](managerAward)
<console>:12: error: type mismatch;
 found   : Award[Manager]
 required: Award[Employee]
Note: Manager <: Employee, but class Award is invariant in type T.
You may wish to define T as +T instead. (SLS 4.5)
       val awards = List[Award[Employee]](managerAward)

This is an example of an invariant type. Even though the parameterised type is a subclass of the more general one, we can't assign it to an instance that has been parameterised with the more general type. This is the standard generics model of Scala and is also the way that the Java language handles generics.

Covariance

Fortunately, Scala's type system allows us to overcome this problem by allowing covariant types. We can redefine the awards class using a plus (+) character on the type to indicate that it is covariant:

scala> class Award[+T](val recipient: T)
defined class Award

Now, we can create a new manager award and can assign it to a variable of type employee award or add it to a collection of employee awards:

scala> val managerAward = new Award[Manager](mike)
managerAward: Award[Manager] = Award@7796649

scala> val employeeAward: Award[Employee] = managerAward
employeeAward: Award[Employee] = Award@7796649

scala> val awards = List[Award[Employee]](managerAward)
awards: List[Award[Employee]] = List(Award@7796649)

By using covariant types we are in effect saying that for any parameterised class its super classes include those parameterised with any of the superclasses of the type used as the parameter. Java provides a very basic mechanism for supporting covariance with the extends keyword in generic declarations:

Award<? extends Employee> employeeAward = managerAward;

However, the Java solution is less powerful because it is the variable using the instance that defines whether it allows covariance rather than allowing this to be encapsulated inside the class itself.

Contravariance

It is slightly more difficult to explain contravariance, and we first need to expand our domain model slightly. We now want to create different classes of awards and restrict who they can be presented to. Firstly we start by adding an additional generic parameter to the Award class to control the class of the award. We also create a couple of example awards restricted on this parameter:

class Award[+T, V](val recipient: T)

class AttendanceAward[T](recipient: T) extends Award[T, Employee](recipient)
class TeamLeadershipAward[T](recipient: T) extends Award[T, Manager](recipient)

The above effectively states that Attendance Awards are available to employees, while a Team Leadership Award can only be presented to Managers. Let's add a couple of functions that deal with presenting the awards:

def presentManagementAward(award: Award[Manager, Manager]) = { /* lots of pomp */ }
def presentEmployeeAward(award: Award[Employee, Employee]) = { /* a simple affair */ }

These state that a management award (with all the pomp) must be an award for managers that is presented to a manager, while an simple affair is available to an employee receiving and employee award. Wait, what happens when we want to present an employee class award to a manager with all the pomp?

scala> presentManagementAward(new AttendanceAward(mike))
<console>:14: error: type mismatch;
 found   : AttendanceAward[Manager]
 required: Award[Manager,Manager]
Note: Employee >: Manager (and AttendanceAward[Manager] <: Award[Manager,Employee]), but class Award is invariant in type V.
You may wish to define V as -V instead. (SLS 4.5)
       presentManagementAward(new AttendanceAward(mike))

We can't do this as the presentManagementAward function is expecting an award of class manager given to a manager but instead we are passing an award of class employee. However, in this case it seems wrong that this doesn't work. Why shouldn't the manager also be able to receive more general awards? It turns out that we can achieve this my modifying the award classifier to be contravariant. This makes it possible to substitute a more general parameter. If we can present a management class award then we can also present a more general employee award:

class Award[+T, -V](val recipient: T)

scala> presentManagementAward(new AttendanceAward(mike))

So there we have it, invariance, covariance and contravariance.

Alternative Votes, Politics and Software Projects

Aside for my non-UK readers: On May 5th 2011, the UK is holding a referendum on switching our electoral voting system from 'First Past The Post' (voters select a single candidate and the one with the most votes wins) to 'Alternative Vote (AV)' (voters rank candidates in order of preference, multiple analysis rounds take place where the candidate with the least votes is eliminated and their votes are redistributed to the next preferences until one candidate has 50% of the vote).

Yes, this is a software blog and yes, this is a political posting. Have I gone mad? Well, not quite. On May 5th 2011 I will be voting YES to change the UK electoral voting system from First Past The Post to Alternative Vote (AV). I have felt for a long time that the current political system is fundamentally broken. Switching to AV is the first tiny step in what will hopefully become a gradual move to a better way of doing politics.

So why do I think this is the case, and what has it got to do with software? Read on to find out....

A Badly Run Project

Consider a badly run software project. Typically the project starts out full of hope. The business defines its requirements and a development team is established to build the solution. An Architect is appointed, technologies are selected and a high-level architecture is defined that will make delivering all the requirements a doddle. Development commences. After many months the project fails to deliver, moral among the development team is very low and the business is disillusioned with the project, the Architect and development team.

Why has this happened? I'm sure any engineer who has worked on such a project can produce a myriad of possible reasons! However, there are some broad areas that tend to be common across most of these failed projects:

Over Ambitious Requirements and Scope: At the start of the project the business defines a set of requirements that encompass everything they want to see done. They expect the new software project to be able to change the world and the developers don't give them any reason to doubt this is possible. As the project starts to struggle, the business is reluctant to let go of or revise these requirements so the development team is forced to press on even though they now know it will lead to failure.

Overly Complex Architecture and Poor Technology Choice: The project's Architect has some great new technologies and patterns they have read about and want to apply. The technical requirements for the project are defined in a way to justify needing these new approaches and an architecture is built around them. As the project develops it is found that some of the choices were not good but it's either too late to go back and undo them or the Architect is unwilling to admit that he/she got it wrong. Extra architecture is added to work round the limitations of the original choices and complexity grows. Productivity is significantly reduced and quality suffers.

Poor Day-to-Day Project Management The management team for the project fails to adequately manage the scope of the project. They allow features to creep in from the requirements that are not truly essential to the completed project. They allow technical debt to accumulate in order to deliver agreed upon feature sets on prematurely early fixed dates. They fail to manage the relationship between the developers and the business. Deadlines are missed, problems covered up and an air of mistrust develops between the project team and the business.

So what happens next? Well, the project fails to deliver and the business gets upset. The project is stopped and the senior people are removed (CTO, Architect, Programme Lead and so on). The business still needs the software so it looks for a new team to take over the project. The new team reviews what was done before and decides that the only way forward is to rip up most of what was done and start over. The project is full of hope. The business reviews the requirements and comes up with a new over ambitious set. The new Architect selects his set of preferred new technologies and patterns. The team are excited again and agree to deliver the new requirements on the new architecture. The project managers vow to not repeat the poor management of the old project. Every thing starts again and the same happens all over again. Sound familiar?

A Bad Political System

Now, let's take the above and replace 'business' with 'electorate', 'development team' and 'project management' with 'government' and 'Architect' with 'Prime Minister / President'. See where I'm coming from?

It's election time and the electorate have become disillusioned with the current politicians' ability to deliver. They create a whole lot of expectations of what the government should be doing (requirements). The various parties and candidates promise they have a set of policies (architecture) that can deliver these expectations. A new party (development team) with its great leader (Architect) is voted into power. They start by throwing out the policies of the previous government and putting their new approaches into place. They make sacrifices to the long term (technical debt) to allow demonstration that they can deliver. After a while they find their their policies are not making the changes that they had hoped, but it's too late to turn back. They push on, adding extra policies to cover up the failed short-termist ones that came before. Finally their failure to deliver becomes entrenched, the electorate becomes disillusioned, a new party is voted into power and the whole process starts over again. See the similarity?

A Well Run Project

Let's consider the alternative of a well run software project. Yes, they do exist!

In this project the business has a set of requirements that they want to achieve. A good working dialogue is established between the development team and the business. The requirements are discussed. The dev team identifies those requirements that will be difficult or complex to implement. They add their own input about additional (non-functional) requirements that will be needed plus work that will be required to ensure the long-term health of the project and avoid the build up of technical debt. The business and dev team then prioritise the work, ensuring that there is a good balance between delivering essential requirements early while also ensuring the long-term objectives are taken into account.

The Architect doesn't throw away everything that came before just to try something new. Instead he selects tried and tested solutions and the stuff that is working. In those areas where there are problems or specific challenges then new approaches are tried, perhaps using a new technology that seems to be the correct fit for the problem. Planning is more about long-term adaptation and evolution of the architecture through refactoring rather than sudden and seismic shifts in architectural approach.

Given this more evolutionary approach to the architecture, the business is more willing to commit to investing some percentage of development effort into the architecture and ensuring technical debt is not built up. At the same time the development team sees an ongoing commitment to the programme and is thus less likely to take short-termist decisions to meet immediate goals that they know will hurt them later.

Communication between the business, dev team and project management remains high, open and honest. When things go wrong (and they always will) then this can be brought into the open, discussed and priorities changed, without recrimination, to resolve the issue. If requirements change then a good dialogue is already present that allows a different direction to be taken with minimal upheaval.

This is a perfect example of a collaborative, agile project that is not stuck in a particular dogma of process, technology or mindset.

What About Politics?

So, if a well run software project can work in this way, why not government? My belief is that the current adversarial, party-political system is fundamentally in conflict with this collaborative approach. The fact is that that generally one party is in power. They are trying to deliver an over ambitious set of promises built around a particular philosophy quickly enough to ensure that immediate results are visible to ensure that they get elected again. This is identical to the model of our failing software project.

What I ultimately want to see is a model much closer to our well run project: one where there are no party politics. All politicians should be elected on their individual merit. They should then come together to form a government of collaboration. There is honesty and openness with the electorate over what can be achieved and when. Policies that are working are allowed to continue (as opposed to being abandoned because they are too left/right-wing). Those that are failing can be addressed with new approaches. Policies that will need a longer period of time to deliver results are allowed the time to do so. There should be continuity rather than fundamental flips in philosophy every ten years or so.

And the AV Referendum?

My true hope is that by voting for a switch to the AV system that we will end up with a series of coalition governments. This will start a process of continued government through collaboration. There should also be a level of continuity as, in the three party system we have in the UK, it will mean that one party always remains in joint power across elections. I would then hope that over time the UK electorate will grow to expect a collaborative coalition government and that this will foster the parties to work together rather than against each other. From a small seed of AV, hopefully a large (and fundamentally different) tree will grow.

We all want to live in a successful, well governed country don't we? I know that I certainly want to be on the well run software project.

Thought For The Day: Programmers

A BAD programmer...

...writes loads of code that usually doesn't work. When they do get it to work they have no idea why it does so [programming by coincidence].

An AVERAGE programmer...

...writes loads of code that works. They are happy that it is working and they broadly understand why it does.

A GOOD programmer...

...implements the same working functionality as the average programmer, but they use far less (and more simple) code to do so. As their code is smaller and simpler, they fully understand why it works.

An OUTSTANDING programmer...

...writes the same code as the good programmer but then refactors until most of it gone. What remains is clean an elegant. Their code is so good that even the bad programmer can understand it.