Concise, Elegant and Readable Unit Tests

Just recently I was pair programming with another developer and we were writing some unit tests for the code we were about to write (yes, we were doing real TDD!). During this exercise we came to a point where I didn't feel that the changes being made to the code met my measure of what makes a good test. As in all the best teams we had a quick discussion and decided to leave the code as it was (I save my real objections for times when it is more valuable). However, this got me thinking about what actually I was uncomfortable with, and hence the content of this post.

As a rule I like my code to have three major facets. I like it to be concise: there should be no more code than necessary and that code should be clear and understandable with minimum contextual information required to read it. It should be elegant: the code should solve the problem at hand in the cleanest and most idiomatic way. Also, it must be readable: clearly laid out, using appropriate names, following the style and conventions of the language it is written in and avoiding excessive complexity.

Now, the test in question was written in Groovy, but it could just as well be in any language as it was testing a fairly simple class that maps error information into a map that will be converted to Json. The test is:

@Test
void shouldGenerateAMapOfErrorInformation() {
    String item = "TestField"
    String message = "There was an error with the test field"
    Exception cause = new IllegalStateException()

    ErrorInfo info = new ErrorInfo()
    info.addError(item, message, cause)

    Map content = info.toJsonMap()
    assert content.errors.size() == 1
    assert content.errors[0].item == item
    assert content.errors[0].message == message
    assert content.errors[0].cause == cause.toString()
}

Now, to my mind this test is at least three lines too long - the three lines declaring variables that are then used to seed the test and in the asserts. These break my measure of conciseness by being unnecessary and adding to the amount of context needed in order to follow the test case. Also, the assertions of the map contents are less than elegant (and also not very concise), which further reduces the readability of the code. I'd much rather see this test case look more like this:

@Test
void shouldGenerateAMapOfErrorInformation() {
    ErrorInfo = new ErrorInfo()
    info.add("TestField", "There was an error with the test field", 
                 new IllegalStateException())

    Map content = info.toJsonMap()
    assert content.errors.size() == 1
    assert hasMatchingElements(contents.errors[0], 
          [ item : "TestField",
             message : "There was an error with the test field",
             cause : "class java.lang.IllegalStateException" ])
}

This test implementation is more concise. There are less lines of code and less context to keep track of. To my eye I find this much more elegant and far more readable.

"But, you've added duplication of the values used in the test", I hear you cry! That is true and it was totally intentional. Let me explain why…

Firstly, as we have already mentioned, duplicating some test values actually makes the code more concise and readable. You can look at either the test setup/execution or the assertions in isolation without needing any other context. Generally, I would agree that avoiding duplication in code is a good idea, but in cases such at this, the clarity and readability far out way the duplication of a couple of values!

Secondly, I like my tests to be very explicit about what they are expecting the result to be. For example, in the original test, we asserted that the cause string returned was the same as the result of calling toString() on the cause exception object. However, what would happen if the underlying implementation of that toString() method changed. Suddenly my code would be returning different Json and my tests would know nothing about it. This might event break my clients - not a good situation.

Thirdly, I like the intention of my code to be very clear, especially so in my tests. For example, I might create some specific, but strange, test data value to exercise a certain edge case. If I just assigned it to a variable, then it would be very easy for someone to come along, think it was an error and correct it. However, if the value was explicitly used to both configure/run the code and in the assertion then I would hope another developer might think more carefully as to my intent before making a correction in multiple places.

My final, and possibly most important reason for not using variables to both run the test and assert against is that this can mask errors. Consider the following test case:

@Test
void shouldCalculateBasketTotals() {
    Basket basket = new Basket()
    LineItem item1 = new LineItem("Some item", Price(23.88))
    LineItem item2 = new LineItem("Another item", Price(46.78))
 
    basket.add(item1)
    basket.add(item2)
 
    assert basket.vatExclusivePrice == 
                       item1.vatExclusivePrice + item2.vatExclusivePrice
    assert basket.vat == item1.vat + item2.vat
    assert basket.totalPrice == item1.price + item2.price
}

In this test, what would happen if the LineItem class had an unnoticed rounding error in its VAT calculation that was then replicated into the basket code? As we are asserting the basket values based on the fixture test data we may never notice as both sides of the assertion would report the same incorrectly rounded value. By being more explicit in our test code we not only create code that is more concise, elegant and readable but we also find more errors:

@Test
void shouldCalculateBasketTotals() {
    Basket basket = new Basket()
    LineItem item1 = new LineItem("Some item", Price(23.88))
    LineItem item2 = new LineItem("Another item", Price(46.78))
 
    basket.add(item1)
    basket.add(item2)
 
    assert basket.vatExclusivePrice == 58.88
    assert basket.vat == 11.78
    assert basket.totalPrice == 70.66
}

Finally, a couple more points that people might raise and my answers to them (just for completeness!):

Q: Can't we move the constant variables up to the class level and initialise them in a setup to keep the tests methods concise?
A: This makes an individual method contain less lines of code but fails my consiseness test as there is context outside of the test method that is required to understand what it does. You actually have to go and look elsewhere to find out what values your test is executing against. This is also less readable!

Q: Isn't is better to share test fixtures across test methods in order to keep the test class as a whole more concise?
A: This might be appropriate for some facets, such as any mocks that are used in the test or if there is a particularly large data set required for the test which is not part of the direct subject of the test. However, I'd rather see slightly longer individual test methods that are explicit in their test data and assertions, even if this means some duplication or a slightly longer overall test class.

Type Classes and Bounds in Scala

I've been doing some initial experiments with the Scalaz library (https://github.com/scalaz/). This is a great extension to the Scala standard library that adds a wide range of functional programming concepts to the language. Many of its features are based around the concept of Type Classes.

The type class concept originally comes from Haskell, and provides a way to define that a particular type supports a certain behavioural concept without the type having to explicitly know about it. For example, a type class may define a generic 'Ordered' contract and implementations can be defined for different types separate from the definition of those types. Any type that has an implementation of the 'Ordered' type class can then be used in any functions that work with ordering.

The Scala language provides a way to support type classes via its implicit mechanism. In order to get my head around this fully I created some examples to experiment with how this works. Now I fully grasp the mechanisms, the Scalaz library makes much more sense. I therefore thought I'd share my experiment in case it proves useful to anyone and as an aide-mémoire to myself for the future.

In addition, my examples also make use of the different type bounds mechanisms provided by Scala, so I will demonstrate these in my examples as well.

The full source code can be found in the following gist: https://gist.github.com/1247752.

Without Type Classes

In the first example, I built a simple solution that doesn't make any use of the type class concept. This solution is much like you would implement in Java, with an interface defining the behaviour and classes that implement this interface:

trait Publishable {  
  def asWebMarkup: String
}

case class BlogPost(title: String, text: String) extends Publishable {
  def asWebMarkup =
    """|
%s
       |
%s
""".stripMargin format(title, text)
}

Here we have a trait Publishable that indicates that something can represent itself as a web markup string. Then we have a case class that implements the trait and provides the method to return the markup. Next, we declare a class that can do the publishing:

class WebPublisher[T <: Publishable] {
  def publish(p: T) = println(p.asWebMarkup)
}

Note that we have defined this as a templates class and that we have used the <: bounds notation to require that our type T is only valid if it extends the Publishable trait. All we need to do now is use it:

val post = new BlogPost("Test", "This is a test post")
val web = new WebPublisher[BlogPost]()
web publish(post)

Ok, so that works fine. However, what happens when we either can't or don't want BlogPost to implement the Publishable trait? There can be many reasons for this: perhaps we don't have the BlogPost domain object source; perhaps the object is already quite complex and we don't want to pollute it with publishing knowledge; perhaps it's shared by multiple teams or projects and only ours needs publishing knowledge. So, what do we do?

Without type classes there are some options: we might extends the domain class to implement the trait; we might create a wrapper class or we might create a helper utility. However, all of these result in a level of indirection and complication in our code. Let me explain...

If we internalise the knowledge of the wrapper/helper/sub-class in our publishing code we end up with some horrific type matching:

class WebPublisher {
  def publish(p: AnyRef) = p match {
    case blogPost: BlogPost => BlogPostPublishHelper.publish(blogPost)
    case _ => …
  }
}

However, if we externalise the knowledge of publishing then our client code has to do the conversion:

class WebPublisher {
  def publish(p: Publishable) = ...
}

web publish(new PublishableBlogPostWrapper(blogPost))

Clearly, both solutions lead to fragile boilerplate code that pollutes our main application logic. Type classes provide a mechanism for isolating and reducing this boilerplate so that it is largely invisible to both sides of the contract.

Type Classes with Implicit Views

Fortunately, Scala provides a mechanism whereby we can declare an implicit conversion between our BlogPost and a Publishable version of our instance. So, let's start with the simple stuff:

trait Publishable {
  def asWebMarkup: String
}

case class BlogPost(title: String, text: String) 

So, we now have a blog post that doesn't implement the Publishable trait. Let's now define the conversion that can implicitly turn our blog post into something that supports publishing (we would typically add this into our publishing code rather than the domain object in order to isolate all knowledge of Publishable to just the area that needs it):

implicit def BlogPostToPublishable(blogPost: BlogPost) = new Publishable {
  def asWebMarkup =
    """|
%s
       |
%s
""".stripMargin format(blogPost title, blogPost text)
}

Our conversion just creates a new Publishable instance that wraps our blog post and implements the required methods. But, how do we make use of this? We have to change our web publisher very slightly:

class WebPublisher[T <% Publishable] {  
  def publish(p: T) = println(p.asWebMarkup)
}

All we have in fact changed is the <: bounds to a <% bounds. This new one is called a view bounds and defines that we can instantiate a WebPublisher with type T only if there is a view (in this case as implicit conversion) in scope from T to Publishable.

Our code to call this remains the same:

val post = new BlogPost("Test", "This is a test post")
val web = new WebPublisher[BlogPost]()
web publish(post)

Fantastic, our publish code gets objects that it knows are publishable, while our client code can just pass domain objects. We have all the boilerplate for conversion separated out from the main code.

However, while this all seems good, this approach does have its downsides. As you can see we are using the wrapper approach: creating a new instance of a Publishable class that wraps the original object. In a high volume system, the additional object allocations of new Publishable wrapper instances on each call to the publish method may have some less than ideal memory and garbage collection impacts.

The other problem with this approach is that is reduces our ability to compose methods in interesting ways. The reason for this is that once the publishing code has actually invoked the implicit conversion it now has the Publishable wrapper rather than the original object. If it passes this Publishable to other methods or classes than these are not aware of the original wrapped type or instance.

We can overcome this problem by modifying the Publishable trait to have a generic parameter type and support a get method to extract the wrapped value - but this then should really be called PublishableWrapper and some of the simplicity starts to break down.

I think the root of the problem here is that type classes are a very functional concept and implicit views tries to coerce these into a hybrid object/functional world. Fortunately, as of Scala 2.8 there is an additional way to implement the type class approach that leads to a more functional style of coding...

Type Classes with Implicit Contexts

An alternative to implicitly converting one class to a wrapper version of that class that adds additional behaviour is to follow more of a helper like approach. In this model we provide an implicit evidence parameter that implements the type class specific behaviour. This is a much more functional approach in that we don't alter the type we are working on. So, on with the code...

trait Publishable[T] {     
  def asWebMarkup(p: T): String
}

case class BlogPost(title: String, text: String)

This is our new Publishable trait and BlogPost class. Note that our Publishable is no longer intended to be implemented or used as a wrapper. Instead, it is now a contract definition of functional behaviour that takes a parameter of type T and transforms it into a String. A much cleaner abstraction. In fact, we can even create a single object instance that implements this behaviour for blog posts:

object BlogPostPublisher extends Publishable[BlogPost] {     
  def asWebMarkup(p: BlogPost) =
     """|
%s
        |
%s
""".stripMargin format(p title, p text)
}

We are also going to need our implicit. This time however, rather than being a conversion it becomes an evidence that we have something that implements Publishable for the type BlogPost:

implicit def Publishable[BlogPost] = BlogPostPublisher 

We also need to update our WebPublisher a bit:

class WebPublisher[T: Publishable] {  
  def publish(p: T) = println(implicitly[Publishable[T]] asWebMarkup(p))
}

There are two interesting things about this class. First, the bounds has now switched from view (<%) to context (:). The context bounds effectively modifies the class declaration to be:

class WebPublisher[T](implicit evidence$1: Publishable[T]) {  ...
}

The second change is the use of implicitly[Publishable[T]] which is a convenience for getting the implicit evidence of the correct type so that you can call methods on it.

Our code to call this remains exactly the same:

val post = new BlogPost("Test", "This is a test post")
val web = new WebPublisher[BlogPost]()
web publish(post)

One advantage that should be immediately obvious is that we are no longer creating new instances for each implicit use. Instead, we are using the implicit evident parameter (which is in this case an object) and passing our instance to it. This is more efficient in terms of allocations.

Also, we explicitly show where we make use of the implicit evidence, which is clearer. This also means that we are never creating a new wrapped type of T that we pass on to other code. We always pass on instances of type T that have implicit evidence parameters that allow T to behave as a particular type class instance.

Conclusion

We have looked at two different approaches to solving the problem of adding behaviour to an existing class without requiring it to explicitly implement a particular contract or extend a specific base class. Both of these make use of Scala implicits and type classes.

Implicit conversions with view bounds provides an approach where a type T can be converted into the type required by the type class. This works, but there are issues associated with the wrapping or transformation aspects of the conversion.

Implicit evidence parameters within context bounds overcome these problems and provide a far more functional approach to solving the same problem.

Learnings From A Scala Project

I’ve recently been working as an associate for Equal Experts. I’ve been part of a successful project to build a digital marketing back-end solution. The project started out life as a JVM project with no particular language choice. We started out with Java and some Groovy. However, the nature of the domain was one that leant itself to functional transformations of data so we started pulling in some Scala code. In the end we were fully implementing in Scala. This post describes the learning gained on this project.

The Project

The project team consisted of five experienced, full-time developers and one part-time developer/scrum master. One of the developers (me) was proficient in Scala development (18 months, part-time) while the others were all new to the language.

The initial project configuration (sprint zero) was based on a previous Java/Groovy project undertaken by Equal Experts. The initial tooling, build and library stack was taken directly from this project with some examples moved from Groovy to Scala. The initial project stack included:

• Gradle (groovy based build system) with the Scala plugin bolted in
• IntelliJ IDEA with the Scala plugin
• Jersey for RESTful web service support
• Jackson for Json/Domain Object mapping
• Spring Framework
• MongoDB with the Java driver

Tooling

The biggest hurdle faced by the project was tooling, specifically tool support for the Scala language. While Scala tooling has come on a huge amount over the last year, it is still far from perfect. The project faced two specific challenges: build tool support and IDE support.

Build Tool Support

The Gradle build tool is primarily a build tool for Java and Groovy. It supports Scala via a plugin. Similar plugins exist for other build tools like Maven and Buildr.

What we experienced on this project was that this was not the ideal scenario for building Scala projects. Build times for loading the Scala compiler were rather slow and support for incremental compilation was not always as good as we would have liked. More often than not we just had to undertake a clean and build each time. This really slowed down test-driven development.

Currently (in my opinion) the only really workable build solution for Scala is the Simple Build Tool, also known as SBT (https://github.com/harrah/xsbt/wiki). This is a Scala specific build tool and has great support for fast and incremental Scala compilation plus incremental testing.

With hindsight it would probably have been better to switch over from Gradle to SBT as soon as the decision was made to use Scala for the primary implementation language.

IDE Support

The Scala plugin for IntellJ IDEA is one of the best editors for Scala. In particular its code analysis and highlighting has come on in leaps and bounds over the last few months. However, the support for compilation and running tests within the IDE is still less than ideal. Even on core i7 laptops we were experiencing long build times due to slow compiler startup. Using the Fast Scala Compiler (FSC) made a difference, but we found it somewhat unstable.

What was more annoying was that after failed compilations in IDEA, we often found that incremental compilation would no longer work and that a full rebuild of the project was required. This was VERY slow and really hampered development.

One proven way of doing Scala development within IDEA is to use the SBT plugin and run the compilation and tests within SBT inside an IDEA window. This has proven very successful on other Scala projects I have been involved with and again with hindsight we should have investigated this further.

Some developers on the team also found the lack of refactoring options, in IDEA, when working with Scala somewhat limiting. This is a major challenge for the IDE vendors given the nature of the language and its ability to nest function and method definitions and support implicits. Personally this doesn’t bother me quite as much as my early days were spent writing C and C++ code in vi!

Libraries

Scala has excellent integration with the JVM and can make good use of the extensive range of Java libraries and frameworks. However, what we experienced in a number of cases was that integrating with Java libraries required us to implement more code than we would have liked to bridge between the Java and Scala world. Also, much of this code was less than idiomatic Scala.

What we found as the project progressed was that it became increasingly useful to switch out the Java libraries and replace them with Scala written alternatives. For example, very early we stopped using the MongoDB Java driver and introduced the Scala driver, called Casbah. Then we added the Salat library for mapping Scala case classes to and from MongoDB. This greatly simplified our persistence layer and allowed us to use much less and more idiomatic Scala code.

Aside: If you are working with MongoDB then I strongly recommend Scala. The excellent work by Brendan McAdams of 10gen and the team at Novus has created a set of Scala libraries for MongoDB that I think are unrivaled in any other language. The power of these libraries, while maintaining simplicity and ease of use, is amazing.

We never got a chance to swap out some of the other major libraries that the original project structure was built on. However with more hindsight we should perhaps have made the effort to make these swaps as they would have resulted in much less and much cleaner code. Some specific swaps that we could have done:

• Jersey for Unfiltered or Bowler
• Jackson for sjson or lift-json
• Spring for just plain Scala code

Scala projects work well with Java libraries. However when a project is being fully implemented in Scala then it’s much better to swap these Java libraries for their Scala implemented alternatives. This results in much less and more idiomatic Scala code being required.

Team Members

Having highly experienced team members makes a big difference on any project and this one was no exception. The other big advantage on this project was that all of the developers were skilled in multiple languages, even though most had not known Scala before. One thing I have found over many years is that developers who know multiple languages can learn new ones much more easily than those who only know one. I’d certainly recommend recruiting polygot developers for new Scala projects.

It also helped having at least one experienced Scala developer on the team. The knowledge of the Scala eco-system was helpful as was the ability to more quickly solve any challenging Scala type errors. However, the main benefit was knowledge transfer. All of the team members learned the Scala language, libraries and techniques much more quickly by pairing with someone experienced in the language.

Additionally, as the experienced Scala developer I found it very valuable working with good developers who were still learning the language. I found that explaining concepts really clarified my understanding of the language and how to pass this on to others. This will be of great benefit for future projects.

Any new project that is planning on using Scala should always bring on at least one developer who already has experience in the language. It creates a better product and helps the rest of the team become proficient in the language much more quickly.

Object-Oriented/Functional Mix

We found Scala’s ability to mix both object-oriented and functional approaches very valuable. All team members came from an object-oriented background and some also had experience with functional approaches. Initially development started mainly using object-oriented techniques, but leaning towards immutable data wherever possible.

Functional approaches were then introduced gradually where this was really obvious, such as implementing transformations across diverse data sets using map and filter functions. As the team got more familiar with the functional programming techniques more functionality was refactored into functional style code. We found this made the code smaller, easier to test and more reliable.

Perhaps the only real disadvantage of moving to more functional leaning Scala code is that some code became less readable for external developer with less exposure to the functional style. It became necessary to explain some of the approaches taken to external developers: something that would probably have not been necessary with more procedural style code. This is worth bearing in mind for projects were there will be a handover to a team not skilled in Scala and functional programming techniques.

Conclusions

The project was very successful. We delivered on time and more features than the customer originally expected. I don’t think we would have achieved this if the implementation was in Java, so from that point the smaller, more functional code base produced by developing in Scala was a real winner. Using Scala with MongoDB was a major benefit. Tools was the real let-down on the project. Some problems could be alleviated by moving from Java libraries and solutions to their Scala equivalents. However, there is still some way to go before Scala tools (expecially IDEs) match the speed an power of their Java equivalents.

Technical Investment vs Technical Debt

A subject that has been bothering me for a while is the concept of Technical Debt. It’s an interesting analogy, but I think it has become increasingly misused and misunderstood in the agile world. In this post I want to focus on the real meaning of Technical Debt and describe an alternative approach that worked on a current project.

What is Technical Debt?

In its simplest form, Technical Debt is work that you really should be doing but which you put off until later in order to meet some short-term delivery goal - it’s a compromise! For example, I might know that my JSON to Domain Object mapping approach is sub-optimal bit I decide to live with that for a while in order to complete a set of urgent features required for an important customer demo.

The key part of Technical Debt is that you MUST pay it back at some point in the future; and the sooner you do the better. To continue my example: as soon as the customer demo is complete, the fixing of the mapping approach should be prioritised as an urgent task.

How is it Misused?

One of the main problems I have encountered on many projects is that they misuse Technical Debt. The two most common misuses are:

• Being far too willing to accept Technical Debt; and
• Allowing Technical Debt to exist for too long (i.e. not paying it back quick enough)

Both of the above burden a project and, in my experience, they are allowed to happen because people don’t understand the true implications of Technical Debt correctly. There seems to be a general miscomprehension that Technical Debt can be captured as a story in the backlog that can be played at some time in the future much like any other story.

What I believe that most projects miss is the fact that having a Technical Debt story in the backlog not only requires that the debt is paid back in the future, but that its mere presence incurs an ongoing interest payment on the project. This interest payment comes in many different forms, some of which include:

• increased difficulty understanding the code base
• unnecessary complexity in the code
• more difficult and time consuming refactoring
• needing to write more code than necessary to add new features
• slower build and test times

All of the above (and other forms) result in one major impact: reduced team velocity. So, any Technical Debt, besides being a story on the backlog that has to be played in the future, also reduces velocity. Too much of it and velocity grinds to a halt.

Cumulative Impact

Projects that tend to allow Technical Debt to be readily created and to persist also tend to allow multiple pieces of Technical Debt to be present at the same time. This has a cumulative effect in that features are built on one piece of debt and then new debt grows on these features and so on. After a while the ongoing interest payments increase exponentially and the effort required to repay each piece of Technical Debt increases significantly.

At this point a project tends to come to a halt completely because the interest payments of having the Technical Debts stop the project adding any new features, while the effort required to pay back the debt has become too huge. The project either fails or needs a reboot.

Is There An Alternative?

From time to time, any project must accept some compromise in order to meet some critical goal. I see no problem with this. However, what projects need to do is to ensure that these compromises are kept as small as possible and have the smallest possible payback cost. So, how do we ensure this? The solution I would propose is what I have coined as ‘Technical Investment’.

Using the debt/investment analogy for personal finances, we can see two approaches: I can borrow a load of money to buy what I need now and then have to deal with interest payments until I can pay the debt back; or, I can put some money away into an investment so that when I need to buy something I can do so without having to go into debt. I temporarily reduce the size of my investment to make the purchase but then I quickly replenish it and grow it again ready for another future purchase. Hopefully my investment also grows over time so I can buy more things or reduce my investment portfolio by less each time.

So, how does this Technical Investment work in agile software development? What we do is we dedicate a small portion of the team during each iteration to refactoring, cleaning, rewriting, upgrading libraries and so on. This may be in any area of the code, not just that directly touched by the current stories in play. We invest in the code base a small amount each iteration. Perhaps we might deliver one small story or feature less each sprint, but as the project progresses we avoid the large reductions of velocity that are inherent in a Technical Debt driven model.

So, Does It Work?

On a recent project we adopted the above approach (although somewhat informally, which is why I wanted to document it more formally in this post). We had a team of five developers working on weekly sprints. Each day we created two pairs to work on features, while the fifth team member was left free to invest in the code base. The lone person changed each day or so. What did we find?

Firstly we found that the need to incur Technical Debt (or draw down from our investment, to follow the analogy) was significantly reduced. Quite often the areas where we would have had to make a compromise to meet a sprint goal were already addressed and improved before making the compromise became necessary. We were able to preempt many areas of improvement before they became an issue.

Secondly, we found that in cases where a compromise was made (where we drew down from our investment) that we were able to quickly fix the compromise and get back to adding to our investment much sooner. We also avoided the problem of any cumulative compromises, because payback was smaller, which made correcting individual cases much simpler when we decided to accept them.

By working in this way we were able to have a continually improving code base, while still maintaining a high velocity of new features. The code base continually improved in quality and clarity throughout the project even though features were continually being added.

But Shouldn’t This Happen Anyway?

This is an interesting question. Yes, a good agile team should be continually refactoring and improving code as part of every story. However in practice, the pressure to complete features often reduces the amount of improvement that is done. It’s also often difficult to justify changes to code in an area that no-one is currently working on just because it needs some love: which may benefit some unknown work in the future.

Having a concept called ‘Technical Investment’ also gives a concrete thing that can be talked about with Product Owners. We can discuss the investments made during reviews and present them in a positive way, something that I have found much more difficult to do with debt.

I suspect that it may also be possible to measure and visualise the investment in a similar way to the measuring of velocity. This is certainly something I’d like to explore in the future as being able to show how investing in the code reduces future reductions in velocity is a great way to justify the practice.

Conclusion

Keeping the code base clean and building up good quality through continual Technical Investment on each iteration has many advantages over the use of Technical Debt to capture and deal with compromises. When applied, the need to compromise becomes less and the effort involved in fixing each compromise is significantly reduced. The chances of building up cumulative, velocity zapping compromises also lessens significantly. If you have a longer-running or larger project then continual Technical Investment is the only sure way to make sure the project doesn’t grind to a halt due to debt interest as some time in the future.

The Power of Pairs

In my current contract we have a policy of full-time pair programming. It has been working incredibly well. I thought I'd share some of my observations as to why this might be the case.

Previous experience of pairing

I've used and worked with pairs in previous roles with various degrees of success. However, none of those previous roles adopted the full-time pairing model. I've always seen some benefit in pairing, especially on complex or critical pieces of code: two minds working together on these sorts of problems is invaluable. My previous roles have, however, always gone back to singletons for the more mundane and less critical coding tasks.

The one thing I have never seen previously is any of the purported productivity boosts from pair working. This could be because I've only used pairs on for complex tasks, which would take a long time anyway. But surely I should have seen some benefit in the past? Why didn't I?

Our pairing model

The team that I am working with at the moment is made up of five developers. Each of us comes with experience of many successfully completed projects. We all have different language and domain backgrounds, but there seems to be a common thirst from every member of the team for learning and improving. I think this is perhaps one of the key reasons why our pairing model works so well.

Each day we swap the pairs. Sometimes if we are deep in a task we might keep the same pair for two days, but never longer. The dynamic of the team is therefore always changing. It also encourages up to keep tasks small so that we can generally complete them in a single pairing session.

Hold on, don't we have an odd number of people? Yes, and this I think works very well. Each day we pair up and one person gets to work on their own for the day. We try to ensure that each person gets to be the singleton regularly. This single person picks up simple technical debt tasks, carries out investigative spike or just refactors and tidies up the code base. They are requested not to pick up any complex development. Working in pairs is incredibly demanding of concentration and being the singleton for a day also helps keep people fresh.

The benefits

So, why does this pairing model work so well, and why have we created a team that appears to be incredibly productive? Here's my thoughts…

The right team members - All of the developers on the team are good developers and more importantly they are open and keen to learn and improve. We have a number of different language and platform backgrounds on the team, but rather than being a problem it is just more useful information to share between us. The team members have created an environment of sharing and constant learning.

Willingness to question and compromise - Every member of the team has the ability and confidence to question the approach being taken by the other developer in their pair. While I think experience helps in this, you really want even junior team members to have this confidence to question. Linked to this is the fact that each team member is open to new ideas and is either prepared change their thinking or, even if they disagree, come to some form of compromise to allow the pair to continue moving forward. Developers who argue, fail to question or who are not prepared to learn and adapt will kill a pair's productivity.

Shared knowledge - By rotating the pairs on a daily basis, knowledge of the application, language and development approaches is spread throughout the team. This just wouldn't happen if pairs were fixed or people working alone.

The right tools and environment - The team have created a development and build environment that they are comfortable with and that is productive. The team are empowered to make any changes that allow them to work better. These could be changes to the build process or introduction of a new library the cuts the amount of or simplifies code.

A focus on quality - All the members of the team have the same view of the importance of quality and testing. There us a mutual agreement of what constitutes good quality and all team members strive for this goal. Teams where there are differing notions of quality between individual members just don't work as well - even in a non-pairing environment.

The ability to technically invest - By having one team member constantly assigned to cleaning up technical debt, refactoring and improving the code base it allows the pairs to spend all of their time investing in new features. The code base is kept clean and this makes it much easier to introduce new approaches or libraries that further improve the code.

Conclusion

So, would I adopt full-time pair programming again in the future? Definitely! However, to get the real benefit you need the right team of people and you need to create the right environment and culture for them to work in and give them the right set of tools. I'd also try to always have an odd number of developers so that pairs could be rotated and one person be free to clear down any technical debt so that the pairs can invest.