Technical Investment vs Technical Debt

A subject that has been bothering me for a while is the concept of Technical Debt. It’s an interesting analogy, but I think it has become increasingly misused and misunderstood in the agile world. In this post I want to focus on the real meaning of Technical Debt and describe an alternative approach that worked on a current project.

What is Technical Debt?

In its simplest form, Technical Debt is work that you really should be doing but which you put off until later in order to meet some short-term delivery goal - it’s a compromise! For example, I might know that my JSON to Domain Object mapping approach is sub-optimal bit I decide to live with that for a while in order to complete a set of urgent features required for an important customer demo.

The key part of Technical Debt is that you MUST pay it back at some point in the future; and the sooner you do the better. To continue my example: as soon as the customer demo is complete, the fixing of the mapping approach should be prioritised as an urgent task.

How is it Misused?

One of the main problems I have encountered on many projects is that they misuse Technical Debt. The two most common misuses are:

• Being far too willing to accept Technical Debt; and
• Allowing Technical Debt to exist for too long (i.e. not paying it back quick enough)

Both of the above burden a project and, in my experience, they are allowed to happen because people don’t understand the true implications of Technical Debt correctly. There seems to be a general miscomprehension that Technical Debt can be captured as a story in the backlog that can be played at some time in the future much like any other story.

What I believe that most projects miss is the fact that having a Technical Debt story in the backlog not only requires that the debt is paid back in the future, but that its mere presence incurs an ongoing interest payment on the project. This interest payment comes in many different forms, some of which include:

• increased difficulty understanding the code base
• unnecessary complexity in the code
• more difficult and time consuming refactoring
• needing to write more code than necessary to add new features
• slower build and test times

All of the above (and other forms) result in one major impact: reduced team velocity. So, any Technical Debt, besides being a story on the backlog that has to be played in the future, also reduces velocity. Too much of it and velocity grinds to a halt.

Cumulative Impact

Projects that tend to allow Technical Debt to be readily created and to persist also tend to allow multiple pieces of Technical Debt to be present at the same time. This has a cumulative effect in that features are built on one piece of debt and then new debt grows on these features and so on. After a while the ongoing interest payments increase exponentially and the effort required to repay each piece of Technical Debt increases significantly.

At this point a project tends to come to a halt completely because the interest payments of having the Technical Debts stop the project adding any new features, while the effort required to pay back the debt has become too huge. The project either fails or needs a reboot.

Is There An Alternative?

From time to time, any project must accept some compromise in order to meet some critical goal. I see no problem with this. However, what projects need to do is to ensure that these compromises are kept as small as possible and have the smallest possible payback cost. So, how do we ensure this? The solution I would propose is what I have coined as ‘Technical Investment’.

Using the debt/investment analogy for personal finances, we can see two approaches: I can borrow a load of money to buy what I need now and then have to deal with interest payments until I can pay the debt back; or, I can put some money away into an investment so that when I need to buy something I can do so without having to go into debt. I temporarily reduce the size of my investment to make the purchase but then I quickly replenish it and grow it again ready for another future purchase. Hopefully my investment also grows over time so I can buy more things or reduce my investment portfolio by less each time.

So, how does this Technical Investment work in agile software development? What we do is we dedicate a small portion of the team during each iteration to refactoring, cleaning, rewriting, upgrading libraries and so on. This may be in any area of the code, not just that directly touched by the current stories in play. We invest in the code base a small amount each iteration. Perhaps we might deliver one small story or feature less each sprint, but as the project progresses we avoid the large reductions of velocity that are inherent in a Technical Debt driven model.

So, Does It Work?

On a recent project we adopted the above approach (although somewhat informally, which is why I wanted to document it more formally in this post). We had a team of five developers working on weekly sprints. Each day we created two pairs to work on features, while the fifth team member was left free to invest in the code base. The lone person changed each day or so. What did we find?

Firstly we found that the need to incur Technical Debt (or draw down from our investment, to follow the analogy) was significantly reduced. Quite often the areas where we would have had to make a compromise to meet a sprint goal were already addressed and improved before making the compromise became necessary. We were able to preempt many areas of improvement before they became an issue.

Secondly, we found that in cases where a compromise was made (where we drew down from our investment) that we were able to quickly fix the compromise and get back to adding to our investment much sooner. We also avoided the problem of any cumulative compromises, because payback was smaller, which made correcting individual cases much simpler when we decided to accept them.

By working in this way we were able to have a continually improving code base, while still maintaining a high velocity of new features. The code base continually improved in quality and clarity throughout the project even though features were continually being added.

But Shouldn’t This Happen Anyway?

This is an interesting question. Yes, a good agile team should be continually refactoring and improving code as part of every story. However in practice, the pressure to complete features often reduces the amount of improvement that is done. It’s also often difficult to justify changes to code in an area that no-one is currently working on just because it needs some love: which may benefit some unknown work in the future.

Having a concept called ‘Technical Investment’ also gives a concrete thing that can be talked about with Product Owners. We can discuss the investments made during reviews and present them in a positive way, something that I have found much more difficult to do with debt.

I suspect that it may also be possible to measure and visualise the investment in a similar way to the measuring of velocity. This is certainly something I’d like to explore in the future as being able to show how investing in the code reduces future reductions in velocity is a great way to justify the practice.

Conclusion

Keeping the code base clean and building up good quality through continual Technical Investment on each iteration has many advantages over the use of Technical Debt to capture and deal with compromises. When applied, the need to compromise becomes less and the effort involved in fixing each compromise is significantly reduced. The chances of building up cumulative, velocity zapping compromises also lessens significantly. If you have a longer-running or larger project then continual Technical Investment is the only sure way to make sure the project doesn’t grind to a halt due to debt interest as some time in the future.

The Power of Pairs

In my current contract we have a policy of full-time pair programming. It has been working incredibly well. I thought I'd share some of my observations as to why this might be the case.

Previous experience of pairing

I've used and worked with pairs in previous roles with various degrees of success. However, none of those previous roles adopted the full-time pairing model. I've always seen some benefit in pairing, especially on complex or critical pieces of code: two minds working together on these sorts of problems is invaluable. My previous roles have, however, always gone back to singletons for the more mundane and less critical coding tasks.

The one thing I have never seen previously is any of the purported productivity boosts from pair working. This could be because I've only used pairs on for complex tasks, which would take a long time anyway. But surely I should have seen some benefit in the past? Why didn't I?

Our pairing model

The team that I am working with at the moment is made up of five developers. Each of us comes with experience of many successfully completed projects. We all have different language and domain backgrounds, but there seems to be a common thirst from every member of the team for learning and improving. I think this is perhaps one of the key reasons why our pairing model works so well.

Each day we swap the pairs. Sometimes if we are deep in a task we might keep the same pair for two days, but never longer. The dynamic of the team is therefore always changing. It also encourages up to keep tasks small so that we can generally complete them in a single pairing session.

Hold on, don't we have an odd number of people? Yes, and this I think works very well. Each day we pair up and one person gets to work on their own for the day. We try to ensure that each person gets to be the singleton regularly. This single person picks up simple technical debt tasks, carries out investigative spike or just refactors and tidies up the code base. They are requested not to pick up any complex development. Working in pairs is incredibly demanding of concentration and being the singleton for a day also helps keep people fresh.

The benefits

So, why does this pairing model work so well, and why have we created a team that appears to be incredibly productive? Here's my thoughts…

The right team members - All of the developers on the team are good developers and more importantly they are open and keen to learn and improve. We have a number of different language and platform backgrounds on the team, but rather than being a problem it is just more useful information to share between us. The team members have created an environment of sharing and constant learning.

Willingness to question and compromise - Every member of the team has the ability and confidence to question the approach being taken by the other developer in their pair. While I think experience helps in this, you really want even junior team members to have this confidence to question. Linked to this is the fact that each team member is open to new ideas and is either prepared change their thinking or, even if they disagree, come to some form of compromise to allow the pair to continue moving forward. Developers who argue, fail to question or who are not prepared to learn and adapt will kill a pair's productivity.

Shared knowledge - By rotating the pairs on a daily basis, knowledge of the application, language and development approaches is spread throughout the team. This just wouldn't happen if pairs were fixed or people working alone.

The right tools and environment - The team have created a development and build environment that they are comfortable with and that is productive. The team are empowered to make any changes that allow them to work better. These could be changes to the build process or introduction of a new library the cuts the amount of or simplifies code.

A focus on quality - All the members of the team have the same view of the importance of quality and testing. There us a mutual agreement of what constitutes good quality and all team members strive for this goal. Teams where there are differing notions of quality between individual members just don't work as well - even in a non-pairing environment.

The ability to technically invest - By having one team member constantly assigned to cleaning up technical debt, refactoring and improving the code base it allows the pairs to spend all of their time investing in new features. The code base is kept clean and this makes it much easier to introduce new approaches or libraries that further improve the code.

Conclusion

So, would I adopt full-time pair programming again in the future? Definitely! However, to get the real benefit you need the right team of people and you need to create the right environment and culture for them to work in and give them the right set of tools. I'd also try to always have an odd number of developers so that pairs could be rotated and one person be free to clear down any technical debt so that the pairs can invest.

Scala eXchange London 2011

It's been an information packed two days of learning at the Scala eXchange 2011, hosted by Skills Matter. Twenty-one fantastic sessions covering a wide range of subjects. It was great to have so many in-depth sessions. Previous Scala conferences I have been to have tended to be full of introduction level sessions. This conference was different, all of the sessions went into a great deal of depth, which perhaps shows how mature the Scala community is becoming.

The highlight of the two days for me had to be James Strachan's entertaining keynote on the Scalate templating engine. It's certainly going to right to the top of my toolbox when it comes to needing a templating solution. The laughs were long and loud when a small table crashed to the ground in the corner and James commented "...that's the JSP developer falling over!". Scalate is certainly many steps beyond JSP.

Victor Klang's imaginatively titled session "The Promising Future of Akka" was my second highlight. Not only an inspired name of the presentation but some great in depth details on how Akka is implementing the Future/Promise model and how these complement the existing Actor model.

The overall impression I got from the two days was that it was a gathering of very smart people who "get it". The Scala community seems to be made up of the best and the brightest, who are investing in finding better ways to write software. The depth of the talks and the many conversations had over coffee show a great maturity of thought. These are people thinking beyond simple POJOs, DI Frameworks and ORM tools. I think we are on the verge of another major shift in the way software is designed, constructed and deployed.

The other thing that I drew from the two days is that Scala is no longer an experimental plaything, it's now a mature technology. The core language is now stable, the tools are mature to the point where they are almost as functionality complete as their Java counterparts and there is a wide understanding of how to use and apply Scala. The effort has now shifted onto solving more advanced problems like huge parallelism, complex enterprise integrations and complex event driven challenges. From being trailblazers, the Scala community are instead becoming ambassadors: showing how Scala leads to cleaner code, more elegant solutions and how Scala fits so well with the agile process.

Invariance, Covariance and Contravariance

While learning Scala I spent a fair amount of time getting my head around covariant and contravariant concepts. Just recently I found myself tryying to explain these to a collegue. I've since refined my explanation and decided to write it down for all those people who want a clear, maths free explanation.

Invariance

Let's consider a simple domain model. We have a base class 'Employee' and a sub-class 'Manager':

scala> class Employee(val number: String, val name: String)
defined class Employee

scala< class Manager(number: String, name: String, val department: String) extends Employee(number, name)
defined class Manager

Next, we have an award class that we implement using a parameterised recipient type so that awards can be used for more than just our Employee/Manager model:

scala> class Award[T](val recipient: T)
defined class Award

So, lets create an award that is for one of our managers:

scala> val mike = new Manager("1", "Mike", "Sales")
mike: Manager = Manager@712c20d9

scala> val managerAward = new Award[Manager](mike)
managerAward: Award[Manager] = Award@411a9435

All well and good. We have a manager award instance. However, say we have a variable that is of type Award[Employee]. It would seem natural to be able to assign our manager award to this variable as a manager is a type of employee, so their award should also be part of the general set of awards given to employees, right? Well...

scala> val employeeAward: Award[Employee] = managerAward
<console>:12: error: type mismatch;
 found   : Award[Manager]
 required: Award[Employee]
Note: Manager <: Employee, but class Award is invariant in type T.
You may wish to define T as +T instead. (SLS 4.5)
       val employeeAward: Award[Employee] = managerAward

So, we see that this is not possible. We also can't add our manager award to a list of awards given to employees, even though it seems natural that we should be able to do so:

scala> val awards = List[Award[Employee]](managerAward)
<console>:12: error: type mismatch;
 found   : Award[Manager]
 required: Award[Employee]
Note: Manager <: Employee, but class Award is invariant in type T.
You may wish to define T as +T instead. (SLS 4.5)
       val awards = List[Award[Employee]](managerAward)

This is an example of an invariant type. Even though the parameterised type is a subclass of the more general one, we can't assign it to an instance that has been parameterised with the more general type. This is the standard generics model of Scala and is also the way that the Java language handles generics.

Covariance

Fortunately, Scala's type system allows us to overcome this problem by allowing covariant types. We can redefine the awards class using a plus (+) character on the type to indicate that it is covariant:

scala> class Award[+T](val recipient: T)
defined class Award

Now, we can create a new manager award and can assign it to a variable of type employee award or add it to a collection of employee awards:

scala> val managerAward = new Award[Manager](mike)
managerAward: Award[Manager] = Award@7796649

scala> val employeeAward: Award[Employee] = managerAward
employeeAward: Award[Employee] = Award@7796649

scala> val awards = List[Award[Employee]](managerAward)
awards: List[Award[Employee]] = List(Award@7796649)

By using covariant types we are in effect saying that for any parameterised class its super classes include those parameterised with any of the superclasses of the type used as the parameter. Java provides a very basic mechanism for supporting covariance with the extends keyword in generic declarations:

Award<? extends Employee> employeeAward = managerAward;

However, the Java solution is less powerful because it is the variable using the instance that defines whether it allows covariance rather than allowing this to be encapsulated inside the class itself.

Contravariance

It is slightly more difficult to explain contravariance, and we first need to expand our domain model slightly. We now want to create different classes of awards and restrict who they can be presented to. Firstly we start by adding an additional generic parameter to the Award class to control the class of the award. We also create a couple of example awards restricted on this parameter:

class Award[+T, V](val recipient: T)

class AttendanceAward[T](recipient: T) extends Award[T, Employee](recipient)
class TeamLeadershipAward[T](recipient: T) extends Award[T, Manager](recipient)

The above effectively states that Attendance Awards are available to employees, while a Team Leadership Award can only be presented to Managers. Let's add a couple of functions that deal with presenting the awards:

def presentManagementAward(award: Award[Manager, Manager]) = { /* lots of pomp */ }
def presentEmployeeAward(award: Award[Employee, Employee]) = { /* a simple affair */ }

These state that a management award (with all the pomp) must be an award for managers that is presented to a manager, while an simple affair is available to an employee receiving and employee award. Wait, what happens when we want to present an employee class award to a manager with all the pomp?

scala> presentManagementAward(new AttendanceAward(mike))
<console>:14: error: type mismatch;
 found   : AttendanceAward[Manager]
 required: Award[Manager,Manager]
Note: Employee >: Manager (and AttendanceAward[Manager] <: Award[Manager,Employee]), but class Award is invariant in type V.
You may wish to define V as -V instead. (SLS 4.5)
       presentManagementAward(new AttendanceAward(mike))

We can't do this as the presentManagementAward function is expecting an award of class manager given to a manager but instead we are passing an award of class employee. However, in this case it seems wrong that this doesn't work. Why shouldn't the manager also be able to receive more general awards? It turns out that we can achieve this my modifying the award classifier to be contravariant. This makes it possible to substitute a more general parameter. If we can present a management class award then we can also present a more general employee award:

class Award[+T, -V](val recipient: T)

scala> presentManagementAward(new AttendanceAward(mike))

So there we have it, invariance, covariance and contravariance.

Alternative Votes, Politics and Software Projects

Aside for my non-UK readers: On May 5th 2011, the UK is holding a referendum on switching our electoral voting system from 'First Past The Post' (voters select a single candidate and the one with the most votes wins) to 'Alternative Vote (AV)' (voters rank candidates in order of preference, multiple analysis rounds take place where the candidate with the least votes is eliminated and their votes are redistributed to the next preferences until one candidate has 50% of the vote).

Yes, this is a software blog and yes, this is a political posting. Have I gone mad? Well, not quite. On May 5th 2011 I will be voting YES to change the UK electoral voting system from First Past The Post to Alternative Vote (AV). I have felt for a long time that the current political system is fundamentally broken. Switching to AV is the first tiny step in what will hopefully become a gradual move to a better way of doing politics.

So why do I think this is the case, and what has it got to do with software? Read on to find out....

A Badly Run Project

Consider a badly run software project. Typically the project starts out full of hope. The business defines its requirements and a development team is established to build the solution. An Architect is appointed, technologies are selected and a high-level architecture is defined that will make delivering all the requirements a doddle. Development commences. After many months the project fails to deliver, moral among the development team is very low and the business is disillusioned with the project, the Architect and development team.

Why has this happened? I'm sure any engineer who has worked on such a project can produce a myriad of possible reasons! However, there are some broad areas that tend to be common across most of these failed projects:

Over Ambitious Requirements and Scope: At the start of the project the business defines a set of requirements that encompass everything they want to see done. They expect the new software project to be able to change the world and the developers don't give them any reason to doubt this is possible. As the project starts to struggle, the business is reluctant to let go of or revise these requirements so the development team is forced to press on even though they now know it will lead to failure.

Overly Complex Architecture and Poor Technology Choice: The project's Architect has some great new technologies and patterns they have read about and want to apply. The technical requirements for the project are defined in a way to justify needing these new approaches and an architecture is built around them. As the project develops it is found that some of the choices were not good but it's either too late to go back and undo them or the Architect is unwilling to admit that he/she got it wrong. Extra architecture is added to work round the limitations of the original choices and complexity grows. Productivity is significantly reduced and quality suffers.

Poor Day-to-Day Project Management The management team for the project fails to adequately manage the scope of the project. They allow features to creep in from the requirements that are not truly essential to the completed project. They allow technical debt to accumulate in order to deliver agreed upon feature sets on prematurely early fixed dates. They fail to manage the relationship between the developers and the business. Deadlines are missed, problems covered up and an air of mistrust develops between the project team and the business.

So what happens next? Well, the project fails to deliver and the business gets upset. The project is stopped and the senior people are removed (CTO, Architect, Programme Lead and so on). The business still needs the software so it looks for a new team to take over the project. The new team reviews what was done before and decides that the only way forward is to rip up most of what was done and start over. The project is full of hope. The business reviews the requirements and comes up with a new over ambitious set. The new Architect selects his set of preferred new technologies and patterns. The team are excited again and agree to deliver the new requirements on the new architecture. The project managers vow to not repeat the poor management of the old project. Every thing starts again and the same happens all over again. Sound familiar?

A Bad Political System

Now, let's take the above and replace 'business' with 'electorate', 'development team' and 'project management' with 'government' and 'Architect' with 'Prime Minister / President'. See where I'm coming from?

It's election time and the electorate have become disillusioned with the current politicians' ability to deliver. They create a whole lot of expectations of what the government should be doing (requirements). The various parties and candidates promise they have a set of policies (architecture) that can deliver these expectations. A new party (development team) with its great leader (Architect) is voted into power. They start by throwing out the policies of the previous government and putting their new approaches into place. They make sacrifices to the long term (technical debt) to allow demonstration that they can deliver. After a while they find their their policies are not making the changes that they had hoped, but it's too late to turn back. They push on, adding extra policies to cover up the failed short-termist ones that came before. Finally their failure to deliver becomes entrenched, the electorate becomes disillusioned, a new party is voted into power and the whole process starts over again. See the similarity?

A Well Run Project

Let's consider the alternative of a well run software project. Yes, they do exist!

In this project the business has a set of requirements that they want to achieve. A good working dialogue is established between the development team and the business. The requirements are discussed. The dev team identifies those requirements that will be difficult or complex to implement. They add their own input about additional (non-functional) requirements that will be needed plus work that will be required to ensure the long-term health of the project and avoid the build up of technical debt. The business and dev team then prioritise the work, ensuring that there is a good balance between delivering essential requirements early while also ensuring the long-term objectives are taken into account.

The Architect doesn't throw away everything that came before just to try something new. Instead he selects tried and tested solutions and the stuff that is working. In those areas where there are problems or specific challenges then new approaches are tried, perhaps using a new technology that seems to be the correct fit for the problem. Planning is more about long-term adaptation and evolution of the architecture through refactoring rather than sudden and seismic shifts in architectural approach.

Given this more evolutionary approach to the architecture, the business is more willing to commit to investing some percentage of development effort into the architecture and ensuring technical debt is not built up. At the same time the development team sees an ongoing commitment to the programme and is thus less likely to take short-termist decisions to meet immediate goals that they know will hurt them later.

Communication between the business, dev team and project management remains high, open and honest. When things go wrong (and they always will) then this can be brought into the open, discussed and priorities changed, without recrimination, to resolve the issue. If requirements change then a good dialogue is already present that allows a different direction to be taken with minimal upheaval.

This is a perfect example of a collaborative, agile project that is not stuck in a particular dogma of process, technology or mindset.

What About Politics?

So, if a well run software project can work in this way, why not government? My belief is that the current adversarial, party-political system is fundamentally in conflict with this collaborative approach. The fact is that that generally one party is in power. They are trying to deliver an over ambitious set of promises built around a particular philosophy quickly enough to ensure that immediate results are visible to ensure that they get elected again. This is identical to the model of our failing software project.

What I ultimately want to see is a model much closer to our well run project: one where there are no party politics. All politicians should be elected on their individual merit. They should then come together to form a government of collaboration. There is honesty and openness with the electorate over what can be achieved and when. Policies that are working are allowed to continue (as opposed to being abandoned because they are too left/right-wing). Those that are failing can be addressed with new approaches. Policies that will need a longer period of time to deliver results are allowed the time to do so. There should be continuity rather than fundamental flips in philosophy every ten years or so.

And the AV Referendum?

My true hope is that by voting for a switch to the AV system that we will end up with a series of coalition governments. This will start a process of continued government through collaboration. There should also be a level of continuity as, in the three party system we have in the UK, it will mean that one party always remains in joint power across elections. I would then hope that over time the UK electorate will grow to expect a collaborative coalition government and that this will foster the parties to work together rather than against each other. From a small seed of AV, hopefully a large (and fundamentally different) tree will grow.

We all want to live in a successful, well governed country don't we? I know that I certainly want to be on the well run software project.