Life Without Objects

Over the last few years I have been learning and experimenting with a wider range of programming languages. In particular I have started using Scala as my primary development language, adopting a more and more functional style. I have also become very interested in Haskell (a pure functional language) and Clojure (a modern lisp dialect).

I have therefore been moving away from the object-oriented development principles that have made up the bulk of my 17 year career to date. More and more I am beginning to feel that objects have been a diversion away from building concise, well structured and reusable software.

As I pondered on this topic, I realised that this isn’t a sudden switch in my thinking. The benefits of objects have been gradually declining over a long period of time. The way I use objects today is very different to how I used them when they were new and shiny. In this post I explore this change in my thinking about object-oriented development.

The Promise of Objects

Back in the 1990s, objects were new and exciting. The promise of being able to create reusable classes built around common design patterns was seductive. The ability to then combine these classes into reusable and configurable business components seemed like the Mecca of software development. New languages like C++ and then, slightly later, Java held the promise of a new way of building great software.

Business Components Aren’t Reusable
It didn’t take us long to discover that the ability to create reusable business components was just a giant fallacy. Each business is significantly different from another, even in the same industry. Each similar project has very different business rules.

The only way to build reusable business components at this level is to make them hyper-configurable by adding such things as rules engines and embedded scripting languages. Hardly a component model and more like a bloatware model. This promise gone, people either buy into the big bloatware systems (suckers) or build their custom business objects on a project by project basis.

Patterns Don’t Build Well Structured Software
The next thing we learnt was that excessive reliance on design patterns doesn’t lead to the good software structure. Instead it leads to software that is overly complex, hard to understand and difficult to maintain. Some patterns even turned out to be anti-patterns (the singleton pattern makes software almost impossible to unit test, for example).

We soon learnt to use patterns judiciously. More often than not it’s just cleaner to code the software as you understand the model rather than try to abstract it into a more generalised pattern.

Frameworks for Class and Component Reuse Give Few Benefits
Another early promise of objects was rich, tightly coupled frameworks of classes which when used together would make building applications from reusable component a breeze by hiding all the technical complexity and implementation plumbing. Think EJB and similar. Experience soon showed that these just did not work. They were just too restrictive and cumbersome for what people were trying to achieve.

These heavy-weight frameworks soon died out to be replaced with more lightweight libraries and toolkit type approaches. Collections of more loosely coupled classes that you can pull together as needed are now the preferred way to build software. Take just what you need and nothing more.

Inheritance Creates Brittle Software
The ability to support interface and implementation inheritance was one of the key tenets of object oriented software development. We could spot common code and behaviour and push this down into a shared base class so that future abstractions could benefit from having this shared code available to build on.

Sadly, this just didn’t work out well. Each sub-class turns out to be subtly different from its peers, resulting in lots of overrides of base class behaviour or making the base classes even more generic. The net result was super-fragile software, where any small changes to a common base class would break most, if not all, of the sub-class implementations.

These days we don’t use inheritance much, and especially not for creating technical base classes. Its use is pretty much restricted to interface inheritance to indicate an object supports a certain behaviour or to occasional domain models where there is a true inheritance relationship. Other than that we tend to extract commonality in to separate ‘mixin’ type classes and compose them together through an aggregation approach.

Violation of Encapsulation
Another key feature of the object-oriented model was the ability to encapsulate state and then expose behaviours (via methods) that access and update this state. Unfortunately it turns out that there are a large number of cases, where we are actually interested in the vales of the state rather than the behaviour.

For example, asking an object to render itself as HTML turns out to be a pretty poor approach. Knowledge of HTML rendering gets spread across the code base and a small change in approach causes us to change many, many classes. Instead we tend to pass the object to a dedicated HTML rendering/template component, which pulls the data values from the object.

Anti-patterns have even emerged around this to allow us to have light-weight objects that just encapsulate state without behaviour (Java Beans, Data Transfer Objects and so on). If we are doing this, then why not just work directly with first-class structured data as opposed to objects?

Mutable State Causes Pain

Another perceived benefit of encapsulation was the ability to mutate the state of an object instance without impacting on the clients that use that object. However, anyone who has built a significant sized object-oriented system can tell you stories of trawling through many files of code to find the location that mutated the state of an object to an unexpected value that happened to make your software blow up in a completely different place (usually where you output or store the state of that object).

More and more we now favour immutable state and stateless services so that these problems do not occur. There’s also the additional benefit that immutable state is a much better model for building highly concurrent systems and for getting the most out of modern multi-core hardware. It’s also far easier and less error prone than trying to work with threads, locks and concurrency safe data structures.

Behavioural Interfaces Cause Code Complexity and Bloat
One of the things we do frequently in object-oriented languages is create small marker interfaces that have just a single method. Any class wanting to support this behaviour extends the interface and implements the method. We can also declare anonymous implementations for ad-hoc use.

However, we have found that neither of these approaches are particularly good. Implementing the marker interfaces often pollutes classes with implementations that are not their direct concern (thus violating the single responsibility principle). Anonymous classes are just unnecessary bolierplate that makes our code more difficult to understand and maintain.

Life Without Objects

So, is it possible to go back on 17 years of experience and contemplate a life without objects? I’m not sure that I’m 100% there just yet, but using a multi-paradigm language like Scala is allowing me to overcome many of the limitations of the object-oriented approach.

For example, Scala’s support for mixin traits makes it almost unnecessary to ever use implementation inheritance. It’s rich collections framework plus the ability to use case classes to create data structure like concepts obviates working around encapsulation issues. A recommendation to use immutable data and collections makes code easier to debug and reason about. The ability to use functions as general abstractions and type classes to extend behaviour while maintaining single responsibilities makes it much easier to build well structured, reusable software.

In fact, what I find I am doing more and more is using simple objects in the form of case classes to represent data structures, with a few behavioural methods to simplify working with this data. Then I’m just using mixin traits as a modular approach for grouping related functions together. Then I’m combining these together to form components in which I compose together various functions that transform data from one for into another.

Perhaps I’m further away from the pure object-oriented approach than I’d thought. I’m certainly building smaller, cleaner and better structured software than I ever was before.

Option

In my previous post I introduced Dave and we looked at map, flatten and flatMap. In this new episode we take a look at the Scala Option type:

Here's some code...

case class Person(name: String, partner: Option[Person])


sealed trait Product
case object XBox extends Product
case class WeekendAway(person1: Person, person2: Person) extends Product

type Cash = Int
case class Purchase(product: Product, change: Cash)


def buyXBox(cash: Cash): Purchase = Purchase(XBox, cash - 20000)

def bookWeekendAway(p1: Person, p2: Person, cash: Cash): Purchase = Purchase(WeekendAway(p1, p2), cash - 15000)


val lucy = Person("lucy", None)
val dave = Person("dave", Some(lucy))
//val dave = Person("dave", None)
val wages = 50000

val purchase = dave.partner map (bookWeekendAway(dave, _, wages)) getOrElse buyXBox(wages)
println(purchase)

Map, Flatten and FlatMap

I've recently been working with some developers who are new to the Scala world. They've picked up the language really easily and in only a few week have become competent Scala developers. Who says it's too complex and difficult to learn?

Anyway, one of these developers is a visual thinker such as myself. We prefer communication through diagrams rather than words. While explaining Scala I've been drawing some pictures to illustrate some of the more interesting concepts. She suggested that I should publish these and we had the idea that an animation would be fun.

Here's my first animated post showing the concepts for Map, Flatten and FlatMap. It's a pretty amiturish stab at animation, but hopefully it explains the concepts while also adding a bit of humour along the way.

If you are interested here's the Scala code for the concepts covered in the animation:

case class World[+A](revolvesAround: A) {

  def map[B](f: A => B): World[B] = World(f(revolvesAround)) 

  def flatten[B](implicit ev: A <:< World[B]): World[B] = World(revolvesAround.revolvesAround)

  def flatMap[B](f: A => World[B]) = World(f(revolvesAround).revolvesAround)

  def get: A = revolvesAround
}

def lift[A](entity: A): World[A] = World(entity)

case class Person(name: String) {
  def smite = DeadBody(name)
}
case class DeadBody(usedToBe: String)
case class Insect(species: String)

def kill(p: Person): DeadBody = p.smite
def reincarnate(b: DeadBody): World[Insect] = World(Insect("ant"))

val dave: Person = Person("Dave")
val antDave: Insect = lift(dave) map kill flatMap reincarnate get 

println(antDave)

Done and Dusted

In a recent retrospective our team dug out our Definition of Done and gave it a good review. We ended up removing a number of unnecessary items. This got me thinking about what is Done?

In previous places that I have worked the Definition of Done was a long check-list of things that had to have been completed and verified. Is this the best that a philosophy that claims to value "individuals and interactions over processes and tools" can come up with? This got me thinking...

Isn’t Done actually just the state when the team agree that the story is complete? Why should it need to be any more than that?

Done: The state when all team members agree that the story is complete

We could expand this slightly to define the roles that the different team members play in this definition:

• Developers agree that the code is complete, tested and refactored to an acceptable standard
• QAs agree that the acceptance criteria for the story has been met, that the functionality works and that they have not found any defects introduced by the new feature
• BAs and POs agree that the feature meets the business objectives for the story
• (we could also add points for DBAs, Ops etc, as needed)

So, if we accept that the team can decide for themselves when something is Done, why do we end up with long checklists, often including a whole range of metrics? It could be because 'management' aren’t able to release their need to monitor everything and produce nice reports. However, it’s more likely a matter of trust.

Teams are just not trusted to self-organise and make good decisions. If the team say that a story is Done then for many managers there is an immediate suspicion that the team is not telling the truth and is trying to cut corners. They then require that there is concrete proof that this is not the case. This is despite the fact that a group of professionals all happen to agree that it is Done. It just doesn’t make sense.

If the team has never released a story as Done when it clearly wasn’t then why doubt they will suddenly stop doing this in the future? Show trust in the team that is delivering completed work.

If the team are marking stories as Done when they clearly aren’t then there is a much more serious problem with the team and they probably either need replacing, mentoring or a firm guiding hand. Adding a requirement to satisfy a long checklist and provide associated metrics will never improve a bad team.

As with many things agile, it’s not about the process: it’s about changing attitudes; it’s about trusting teams to deliver the best solutions possible; it’s about not bogging the teams down with unnecessary controls.

Sometimes you just have to be lazy...

...or, solving problems with lazy evaluation.

I’ve recently been working on a simple template system, which also has the potential to grow into a full-blown document generation system. The basic principle is that a document is composed of a number of fragments. Each fragment is called in turn to generate its piece of the document and these pieces are concatenated together to product the final result.

My first implementation was a super simple one. We just fold over the list of fragments, accumulating the output from each into a string builder. Each fragment may alternatively generate an error, which we collect along the way as well. At the end of the fold we return either the string document or a list of errors.

  def render(fragments: List[Fragment]): Either[List[String], String] = {
    fragments.foldLeft(RenderResult()) { (acc, fragment) =>
      fragment.generate fold (
        { err => acc.copy(errors = err :: acc.errors) },
        { text => acc.doc.append(text); acc }     
      )
    }.toEither
  }

  case class RenderResult(doc: StringBuilder = new StringBuilder(), errors: List[String] = List()) {
    def toEither: Either[List[String], String] = ...
  }

Now, this works great for documents of only a few kilobytes in size. However, as the documents grow to multi-megabyte sizes this approach becomes infeasible. The amount of memory required to hold the document being generated becomes huge. Multiple concurrent generations become impossible. We need a better solution.

The typical suggestion at this point is to stream output from each fragment straight into an output writer of some form rather than collect and build a string.

  def render(fragments: List[Fragment], writer: Writer): Option[List[String]] = {
    fragments.foldLeft(List[String]()) { (acc, fragment) =>
      fragment.generate fold (
        { err => err :: acc.errors },
        { text => writer.write(text); acc }     
      )
    }.getErrors
  }

Unfortunately this doesn’t work here because of the possibility that fragments may generate errors. We don’t want to stream out partially complete documents. We could output to a temporary file and then copy this to the ultimate output on success, but this seems like a less than ideal solution.

A common approach in an imperative style would be to first run a check phase to see if there are any errors and then run a second render phase to produce the output.

  def check(fragments: List[Fragment]): List[String] = 
    fragmens.foldLeft(List[String]()) { (acc, fragment) =>
      fragment.checkForErrors map (_ :: acc) getOrElse acc
    }

  def render(fragments: List[Fragment], writer: Writer): Unit = 
    fragments foreach { fragment => writer.write(fragment.render) }

However, in this case the building of fragments might be quite costly and we don’t want to have to process them once to see if there is an error and then again to render.

However, as we are using Scala and programming in a functional style, we have an alternative approach. Instead of returning a string or writing direct to a writer we can return a lazily evaluated function. This function can encapsulate all the side-effect generating logic of doing the actual write - which is also nice from the point of view of being able to reason about our rendering code.

  def render(fragments: List[Fragment]): Either[List[String], (Writer) => Unit] = {
    fragments.foldLeft(RenderResult()) { (acc, element) =>
      fragment.generate fold (
        { err => acc.copy(errors = err :: acc.errors) },
        { func => acc.copy(fragmentFuncs = func :: fragmentFuncs }     
      )
    }.toEither
  }

  case class RenderResult(fragmentFuncs: List[() => String] = List(), errors: List[String] = List()) {
    def toEither: Either[List[String], List[(Writer) => Unit] = {
      ...
      (writer: Writer) => fragmentFuncs foreach { f => writer.write(f()) }
      ...
    }
}

The way this works is that the render method folds over each fragment, asking the fragment to do enough work to be certain that no errors will occur (e.g. validate data, take a temporary copy of any volatile values and so on). However, it doesn’t need to do the actual string generation. Instead, each fragment returns a function that will be called at some time in the future (or an error). The render code then accumulates all these small functions into one larger function and returns this.

The returned function, when called, takes a writer and iterates over all the small string generating functions, writing the output of each straight into the writer. Job done and in a mere fraction of the memory required by the original solution.

In cases where you need to generate large amounts of data or where you want to delay side effecting execution to a well-defined location in your code then lazy evaluation is a great approach to follow.

NOTE: All above code example are significantly simplified from the real code in order to demonstrate the approach.