Fast Track to Haskell Course

I've spent the last two days on a Fast Track to Haskell course. It was a very interesting two days and I learnt a huge amount. The course was taught by Andres Löh from Well-Typed and hosted by the excellent people at Skillsmatter.

I went into the course having read the superb Learn You A Haskell book and having built a couple of experimental Haskell programs. From that starting point there was not a huge amount of content in the course that I was not aware of. However, it was great to have my learning verified and my understanding in many areas significantly improved. There were also a few things that changed my thinking on certain topics. All in all, a well spent two days.

The first day started with a whistle stop tour of the Haskell language and how to build things in it. There were plenty of excellent exercises to help cement learning and understanding. Then we looked in more detail at types and how to reason about software using types. The day concluded with looking at some more advanced type concepts including higher-order functions and type classes. The second day followed on with some more about types and then went on to look at how Haskell deals with I/O (which is very different to most other languages I have encountered). The rest of the day was then spent looking at common patterns that can be found in Haskell code and how these can be generalised into concepts such as Functors and Monads. Plenty more exercises followed.

I will now be going away and working on some private projects in Haskell, with the hope of attending the advanced course some time later this year. In the mean time, this blog post looks at some of the key things that I came away from the course with and how they relate back to the Scala and Java that I tend to do for my day job.

Focus on the Types, they are the API

When using an OO language, such as Java (or even Scala), we tend to think about classes, their interfaces and what behaviour they have, and then use this as a starting point to develop from. In Haskell you tend to think first about the types that are involved and then work forward from there. This is a very interesting approach as it makes it possible to think about a problem in a very general and quite abstract way without getting too bogged down in detail too quickly. The types become an API to your system (rather than the interfaces, methods and domain model in an OO solution).

My experience from the course was that this tends to lead to many more types and type aliases in Haskell than I would typically define in Java or Scala. However, this soon proves to be a huge advantage as it then makes it much easier to reason about what each function does/should do and makes its behaviour very clear via the type signature. I will certainly be looking at following this practice in any Haskell programs I write and also trying to introduce more types into my Scala code.

It's Pattern Matching and Recursion All The Way Down

In a language like Java there is nothing like pattern matching and recursion has to be used sparingly if you want to avoid blowing up the stack and writing poorly performing code. In Scala there is much more scope for both, although writing tail recursive functions is still very important to ensure that you get the best optimisations from the compiler. In Haskell, pattern matching and recursion are the bread and butter of the language. Almost every single function tends to be a pattern match on arguments and the language supports a naturally recursive style across the board. Given the lazy semantics of Haskell and the optimisation in the runtime for recursive algorithms this is the best approach for building programs in Haskell.

One area for improvement that I can see in my Scala code is to make more use of pattern matching, especially partial functions.

Thinking Curried Helps a Lot

In the Scala world we typically only curry functions when there are very obvious places to partially apply them. More often than not we just partially apply a function using the underscore for one of its parameters. This is needed to ensure compatibility with Java and the object-oriented model. However, in Haskell every function is completely curried. There are no multi-argument functions. Everything is a single argument function that returns either a result or another function.

Initially I found that this made sense in the model that I'm used to: partially applying functions. However, once it came to defining new types as actually being functions as well I found that this started to fry my mind slightly. The conclusion that I came to is that it's best to just think of everything from a curried mindset and throughout the course things gradually became clearer. This thinking is essential to understand things like the State monad (which I'm still not 100% confident about to be totally honest).

Parametric Polymorphism is Very Useful

One of my biggest take-aways from the course was how useful and important parametric polymorphism is. As an (overly) simple example, what does a function from Int -> Int do? It could return the same number, add something to it, multiply it, factor it, modulus it, ignore the input and return a constant - the possibilities are nearly endless. However, what does a function from a -> a do? Given no other constraints it can only do one thing - return itself (this is called the identity function - id in Haskell).

The ability to define functions in this polymorphic way makes it much easier to reason about what a function might do and also to restrict what a function actually can do. This second point greatly reduces the possibility of introducing unexpected defects or creating functions that have multiple interweaved concerns. As another example, consider the function: Ord a => [a] -> [a]. What does this do? Well, it could just return the same input list, but because it takes the Ord type class as a constraint it's quite likely to be some form of sorting or filtering by order function. As the writer of this function I'm restricted to working with lists and just using equality and ordering functionality so the possibility of doing anything unexpected is greatly reduced.

Parametric Polymorphism is possible in Scala, but not quite as powerful as there aren't type classes for may common concepts (e.g. equality, ordering, numbers, functor, applicative, monoid, monad etc.). Introducing Scalaz fixes this for quite a lot of cases, but that's not to the taste of many projects and mixed Scala/Java teams may find this a step too far. However, there's certainly more scope for using parametric polymorphism in everyday Scala code and this is something I'm going to work on ove the coming weeks (expect some more blog posts).

Don't Try to Over Generalise

Something that I've seen in every language is that once developers discover the power of generalising concepts they tend to go overboard with this. Remember the time when every piece of code was a combination of seven billion design patterns? The power of Haskell's type classes and parametric polymorphism makes it very easy to see patterns that may not actually hold. I can see that it would be very easy to waste a lot of time and effort trying to make a data type fit into various different type classes or trying to spot type class patterns in your code and pull them out into generic concepts.

As with any kind of design pattern, generalisation or similar, the best approach (in any language) is to just build what you need to solve a problem and then when it becomes apparent that it either fits an existing pattern or represents a general pattern then this change can be made as part of a refactoring process.

Try to Keep Pure and I/O Code Separate

Haskell's I/O model at first seems very different, unusual and restrictive. Once you get your head around it, it makes so much sense. Separating pure code from code that talks about doing an I/O action from code that actually does the I/O makes for software that is much more easy to reason about, test and reuse. Very powerful and something I'm looking forward to exploring more during my journey into Haskell.

In the Scala world, I'm not sure about the need to go to such lengths as using an I/O Monad or similar. However, I can really see the advantage of separating pure code from code that performs I/O. Having pure functions that transform data and then wiring them together with the I/O performing functions at a higher level seems a natural way to create software that is better structured and easier to test. I'll certainly be playing with some approaches in Scala to experiment with this concept further.

Functors, Applicatives, Monoids and Monads are Nothing Special

A final observation from the course is that many people learning and talking about functional programming get a bit obsessed with concepts like Functors, Applicatives, Monoids and Monads (especially Monads). What I learnt is that most of these are just simple patterns that can be represented by one or two basic functions and a few simple rules. Nothing more to them really. Certainly nothing to get all worked up about!

Yes, there's a great deal of skill involved in knowing when to use these patterns, when to make your own code follow a pattern and so on, but that's true of any programming language or design pattern. It's called experience.

Conclusion

If you are a Scala developer who is enjoying the functional programming aspects of the language I would certainly recommend taking a look at Haskell. I found that it clarified lots of things that I thought I understood from Scala but still had some doubts about. Haskell is, in my opinion, a more elegant language than Scala and once you start using it you realise just how many hoops the Scala team had to jump through to maintain compatibility with the JVM, Java code and object-oriented features. Over the two days I learnt a lot about Haskell, a lot about functional programming and a lot about how to get even more value from Scala.

Life Without Objects

Over the last few years I have been learning and experimenting with a wider range of programming languages. In particular I have started using Scala as my primary development language, adopting a more and more functional style. I have also become very interested in Haskell (a pure functional language) and Clojure (a modern lisp dialect).

I have therefore been moving away from the object-oriented development principles that have made up the bulk of my 17 year career to date. More and more I am beginning to feel that objects have been a diversion away from building concise, well structured and reusable software.

As I pondered on this topic, I realised that this isn’t a sudden switch in my thinking. The benefits of objects have been gradually declining over a long period of time. The way I use objects today is very different to how I used them when they were new and shiny. In this post I explore this change in my thinking about object-oriented development.

The Promise of Objects

Back in the 1990s, objects were new and exciting. The promise of being able to create reusable classes built around common design patterns was seductive. The ability to then combine these classes into reusable and configurable business components seemed like the Mecca of software development. New languages like C++ and then, slightly later, Java held the promise of a new way of building great software.

Business Components Aren’t Reusable
It didn’t take us long to discover that the ability to create reusable business components was just a giant fallacy. Each business is significantly different from another, even in the same industry. Each similar project has very different business rules.

The only way to build reusable business components at this level is to make them hyper-configurable by adding such things as rules engines and embedded scripting languages. Hardly a component model and more like a bloatware model. This promise gone, people either buy into the big bloatware systems (suckers) or build their custom business objects on a project by project basis.

Patterns Don’t Build Well Structured Software
The next thing we learnt was that excessive reliance on design patterns doesn’t lead to the good software structure. Instead it leads to software that is overly complex, hard to understand and difficult to maintain. Some patterns even turned out to be anti-patterns (the singleton pattern makes software almost impossible to unit test, for example).

We soon learnt to use patterns judiciously. More often than not it’s just cleaner to code the software as you understand the model rather than try to abstract it into a more generalised pattern.

Frameworks for Class and Component Reuse Give Few Benefits
Another early promise of objects was rich, tightly coupled frameworks of classes which when used together would make building applications from reusable component a breeze by hiding all the technical complexity and implementation plumbing. Think EJB and similar. Experience soon showed that these just did not work. They were just too restrictive and cumbersome for what people were trying to achieve.

These heavy-weight frameworks soon died out to be replaced with more lightweight libraries and toolkit type approaches. Collections of more loosely coupled classes that you can pull together as needed are now the preferred way to build software. Take just what you need and nothing more.

Inheritance Creates Brittle Software
The ability to support interface and implementation inheritance was one of the key tenets of object oriented software development. We could spot common code and behaviour and push this down into a shared base class so that future abstractions could benefit from having this shared code available to build on.

Sadly, this just didn’t work out well. Each sub-class turns out to be subtly different from its peers, resulting in lots of overrides of base class behaviour or making the base classes even more generic. The net result was super-fragile software, where any small changes to a common base class would break most, if not all, of the sub-class implementations.

These days we don’t use inheritance much, and especially not for creating technical base classes. Its use is pretty much restricted to interface inheritance to indicate an object supports a certain behaviour or to occasional domain models where there is a true inheritance relationship. Other than that we tend to extract commonality in to separate ‘mixin’ type classes and compose them together through an aggregation approach.

Violation of Encapsulation
Another key feature of the object-oriented model was the ability to encapsulate state and then expose behaviours (via methods) that access and update this state. Unfortunately it turns out that there are a large number of cases, where we are actually interested in the vales of the state rather than the behaviour.

For example, asking an object to render itself as HTML turns out to be a pretty poor approach. Knowledge of HTML rendering gets spread across the code base and a small change in approach causes us to change many, many classes. Instead we tend to pass the object to a dedicated HTML rendering/template component, which pulls the data values from the object.

Anti-patterns have even emerged around this to allow us to have light-weight objects that just encapsulate state without behaviour (Java Beans, Data Transfer Objects and so on). If we are doing this, then why not just work directly with first-class structured data as opposed to objects?

Mutable State Causes Pain

Another perceived benefit of encapsulation was the ability to mutate the state of an object instance without impacting on the clients that use that object. However, anyone who has built a significant sized object-oriented system can tell you stories of trawling through many files of code to find the location that mutated the state of an object to an unexpected value that happened to make your software blow up in a completely different place (usually where you output or store the state of that object).

More and more we now favour immutable state and stateless services so that these problems do not occur. There’s also the additional benefit that immutable state is a much better model for building highly concurrent systems and for getting the most out of modern multi-core hardware. It’s also far easier and less error prone than trying to work with threads, locks and concurrency safe data structures.

Behavioural Interfaces Cause Code Complexity and Bloat
One of the things we do frequently in object-oriented languages is create small marker interfaces that have just a single method. Any class wanting to support this behaviour extends the interface and implements the method. We can also declare anonymous implementations for ad-hoc use.

However, we have found that neither of these approaches are particularly good. Implementing the marker interfaces often pollutes classes with implementations that are not their direct concern (thus violating the single responsibility principle). Anonymous classes are just unnecessary bolierplate that makes our code more difficult to understand and maintain.

Life Without Objects

So, is it possible to go back on 17 years of experience and contemplate a life without objects? I’m not sure that I’m 100% there just yet, but using a multi-paradigm language like Scala is allowing me to overcome many of the limitations of the object-oriented approach.

For example, Scala’s support for mixin traits makes it almost unnecessary to ever use implementation inheritance. It’s rich collections framework plus the ability to use case classes to create data structure like concepts obviates working around encapsulation issues. A recommendation to use immutable data and collections makes code easier to debug and reason about. The ability to use functions as general abstractions and type classes to extend behaviour while maintaining single responsibilities makes it much easier to build well structured, reusable software.

In fact, what I find I am doing more and more is using simple objects in the form of case classes to represent data structures, with a few behavioural methods to simplify working with this data. Then I’m just using mixin traits as a modular approach for grouping related functions together. Then I’m combining these together to form components in which I compose together various functions that transform data from one for into another.

Perhaps I’m further away from the pure object-oriented approach than I’d thought. I’m certainly building smaller, cleaner and better structured software than I ever was before.

Is Scala Like EJB2?

In this post and then in this followup, Stephen Colebourne (@jodastephen) states his criticisms of the Scala language and how it feels to him similar to the same feeling of when he was working with EJB2. A colleague of mine also tweeted the following:

"This is why I will never use Scala: def ++ [B >: A, That] (that: TraversableOnce[B])(implicit bf: CanBuildFrom[List[A], B, That]) : That"

Here's my thoughts and experience with the above.

I started out as a C and C++ developer, but joined the Java set right at the very beginning. I've been fortunate enough to have been working in Java since 1998. Or unfortunate, if you include the early days of the Servlet API & JSP, EJB1 and then EJB2! About 2007 I really started to feel the need for a different language, one that would allow me to express my ideas more concisely and without writing so much boilerplate code. I looked at Groovy, but found it too easy to keep dropping back and writing Java. I then learned Ruby. This taught me how expressive a good language can be and how powerful closures can be. I dabbled with a few more languages before finally finding Scala at the beginning of 2009. These days I mostly code a mix of Scala, Java and Groovy. Next challenge is Clojure.

Changing Mindset

Now, I'm an engineer at heart and my typical approach is to not only understand how to use something but also (and usually at the same time) understand how and why it works. This approach always served me well when learning languages like Java and Ruby. These languages have a fairly small surface area and a relatively simple type system. It's easy to read up on them, understand how to use them and understand exactly how they work. It's easy to look at the standard and third-party libraries and see how these work as well. For me, Scala was different and involved me making a fundamental shift in mindset about how I learned the language.

When I started out I approached Scala in the same way I did other languages: I read a book and tried to build some test applications while at the same time delving into the library and language to see how they worked. This was daunting! I can fully appreciate Stephen's position that it feels a bit like EJB2. Let's put that in context. EJB2 was a horrid mess of unnecessary complexity and complication. It aimed to simplify building enterprise apps involving persistent entities and business services, but it just added way more complexity and mind effort than it saved.

Now, back to Scala. It's a big language with a very extensive and powerful type system. If you attack trying to understand it as a whole I can see how it could be mistaken as another EJB2: there seems to be loads of complexity, a lot of which you can't see the reason for and a lot of stuff just requires so much mind effort to understand. Much of it initially seems unecessary.

It was after my initial failure to understand how the Scala language and libraries actually were built and worked that I took a step back and reconsidered my learning style. I could already see some value to the language and I wanted to explore more, but how? I then made a fundamental change in my approach. I decided that it was okay for me to start using the language without necessarily understanding exactly how all the details worked. As I used the language more I would gradually try to understand the how and why, one small step at a time.

I started learning to write Scala code. When I wanted to achieve something I researched how to do it until I was familiar with the use of a particular language feature or method. If there was some syntax, implementation detail or concept that I didn't get at that point I just noted it down as something to come back to. Now after a year or so writing Scala I look back at what I didn't initially understand and it mostly makes sense. I still occasionally encounter things that don't make total sense initially, so I just learn enough to use them and then come back later to learn the why or how. It's a system that seems to work incredibly well.

When you attack looking at and learning Scala in this more gradual way, first understanding how to use it then gradually, and incrementally, delving more deeply into the type system, you begin to realised that it's not so bad. Yes, there is complexity there, but it's not the unnecessary complexity of EJB2, it's moderate complexity that helps you be expressive, write safe code and get things done. As my knowledge of Scala has grown I've yet to find any features that I haven't been able to see the value of. It's a very well thought out language, even if it doesn't quite seems so at first.

Switching from a mindset where I have to have a full understanding or how something works before I use it to one where I'm comfortable with how to use it even if I don't know why it works was a very disconcerting change in my thinking. I'm not going to say it was an easy change to make, because it wasn't. However the effort was worth it for me because it got me using a language that I now find to be many more times more productive than Java. I think I also understand the language more deeply because I have put it to use and then come back later to understand more deeply exactly how and why it worked. I would never have got to this depth of understanding with my approach to learning other languages.

An Example

Consider the method:

  def ++ [B >: A, That] (that: TraversableOnce[B])
                        (implicit bf: CanBuildFrom[List[A], B, That]) : That

There's a lot of implementation detail in there. However, in order to use this method all I really need to understand is that it allows me to concatenate two immutable collections, returning a third immutable collection containing the elements of the other two. That's why the Scala api docs also include a use case signature for this method:

  def ++ [B](that: GenTraversableOnce[B]): List[B]

From this I understand that on a List I can concatenate anything that is GenTraversibleOnce (Scala's equivalent of a Java Iterable) and that I will get a new list returned. Initially I need understand nothing more in order to use this method. I can go away and write some pretty decent Scala code.

Some time later I learn that the >: symbol means 'same or super type of' and understand this concept. I can then refine my understanding of the method so that I now know that the collection that I want to concatenate must have elements that are the same (or a supertype) as the collection I am concatenating onto.

As I later progress and learn type classes I can see that CanBuildFrom is a type class that has a default implementation for concatenating lists. I can then take this further and create my own CanBuildFrom implementations to vary the behaviour and allow different types to be returned.

I'm first learning how to use the code, then gradually refining my understanding as my knowledge of the language grows. Trying to start out by learning exactly what the first signature meant would have been futile and frustrating.

Conclusion

If you start using Scala and you take the approach of trying to see it all, understand it all and know how and why it all works then it's a scary starting place. The size and complexity of the language (particularly its type system) can seem daunting. You could easily be confused in feeling it similar to EJB2, where reams of needless complexity just made it plain unusable.

However, by switching mindset, learning to use the language and then growing understanding of how it works over time it becomes much more manageable. As you do this you gradually realise that the perceived initial complexity is not as bad as it first seemed and that it all has great value. That initial hurdle from wanting to understand it all to being happy to understand just enough is very hard to get over, but if you do then a very powerful, concise and productive language awaits you.

Invariance, Covariance and Contravariance

While learning Scala I spent a fair amount of time getting my head around covariant and contravariant concepts. Just recently I found myself tryying to explain these to a collegue. I've since refined my explanation and decided to write it down for all those people who want a clear, maths free explanation.

Invariance

Let's consider a simple domain model. We have a base class 'Employee' and a sub-class 'Manager':

scala> class Employee(val number: String, val name: String)
defined class Employee

scala< class Manager(number: String, name: String, val department: String) extends Employee(number, name)
defined class Manager

Next, we have an award class that we implement using a parameterised recipient type so that awards can be used for more than just our Employee/Manager model:

scala> class Award[T](val recipient: T)
defined class Award

So, lets create an award that is for one of our managers:

scala> val mike = new Manager("1", "Mike", "Sales")
mike: Manager = Manager@712c20d9

scala> val managerAward = new Award[Manager](mike)
managerAward: Award[Manager] = Award@411a9435

All well and good. We have a manager award instance. However, say we have a variable that is of type Award[Employee]. It would seem natural to be able to assign our manager award to this variable as a manager is a type of employee, so their award should also be part of the general set of awards given to employees, right? Well...

scala> val employeeAward: Award[Employee] = managerAward
<console>:12: error: type mismatch;
 found   : Award[Manager]
 required: Award[Employee]
Note: Manager <: Employee, but class Award is invariant in type T.
You may wish to define T as +T instead. (SLS 4.5)
       val employeeAward: Award[Employee] = managerAward

So, we see that this is not possible. We also can't add our manager award to a list of awards given to employees, even though it seems natural that we should be able to do so:

scala> val awards = List[Award[Employee]](managerAward)
<console>:12: error: type mismatch;
 found   : Award[Manager]
 required: Award[Employee]
Note: Manager <: Employee, but class Award is invariant in type T.
You may wish to define T as +T instead. (SLS 4.5)
       val awards = List[Award[Employee]](managerAward)

This is an example of an invariant type. Even though the parameterised type is a subclass of the more general one, we can't assign it to an instance that has been parameterised with the more general type. This is the standard generics model of Scala and is also the way that the Java language handles generics.

Covariance

Fortunately, Scala's type system allows us to overcome this problem by allowing covariant types. We can redefine the awards class using a plus (+) character on the type to indicate that it is covariant:

scala> class Award[+T](val recipient: T)
defined class Award

Now, we can create a new manager award and can assign it to a variable of type employee award or add it to a collection of employee awards:

scala> val managerAward = new Award[Manager](mike)
managerAward: Award[Manager] = Award@7796649

scala> val employeeAward: Award[Employee] = managerAward
employeeAward: Award[Employee] = Award@7796649

scala> val awards = List[Award[Employee]](managerAward)
awards: List[Award[Employee]] = List(Award@7796649)

By using covariant types we are in effect saying that for any parameterised class its super classes include those parameterised with any of the superclasses of the type used as the parameter. Java provides a very basic mechanism for supporting covariance with the extends keyword in generic declarations:

Award<? extends Employee> employeeAward = managerAward;

However, the Java solution is less powerful because it is the variable using the instance that defines whether it allows covariance rather than allowing this to be encapsulated inside the class itself.

Contravariance

It is slightly more difficult to explain contravariance, and we first need to expand our domain model slightly. We now want to create different classes of awards and restrict who they can be presented to. Firstly we start by adding an additional generic parameter to the Award class to control the class of the award. We also create a couple of example awards restricted on this parameter:

class Award[+T, V](val recipient: T)

class AttendanceAward[T](recipient: T) extends Award[T, Employee](recipient)
class TeamLeadershipAward[T](recipient: T) extends Award[T, Manager](recipient)

The above effectively states that Attendance Awards are available to employees, while a Team Leadership Award can only be presented to Managers. Let's add a couple of functions that deal with presenting the awards:

def presentManagementAward(award: Award[Manager, Manager]) = { /* lots of pomp */ }
def presentEmployeeAward(award: Award[Employee, Employee]) = { /* a simple affair */ }

These state that a management award (with all the pomp) must be an award for managers that is presented to a manager, while an simple affair is available to an employee receiving and employee award. Wait, what happens when we want to present an employee class award to a manager with all the pomp?

scala> presentManagementAward(new AttendanceAward(mike))
<console>:14: error: type mismatch;
 found   : AttendanceAward[Manager]
 required: Award[Manager,Manager]
Note: Employee >: Manager (and AttendanceAward[Manager] <: Award[Manager,Employee]), but class Award is invariant in type V.
You may wish to define V as -V instead. (SLS 4.5)
       presentManagementAward(new AttendanceAward(mike))

We can't do this as the presentManagementAward function is expecting an award of class manager given to a manager but instead we are passing an award of class employee. However, in this case it seems wrong that this doesn't work. Why shouldn't the manager also be able to receive more general awards? It turns out that we can achieve this my modifying the award classifier to be contravariant. This makes it possible to substitute a more general parameter. If we can present a management class award then we can also present a more general employee award:

class Award[+T, -V](val recipient: T)

scala> presentManagementAward(new AttendanceAward(mike))

So there we have it, invariance, covariance and contravariance.

Cleaner Code with the Scala Type System

Last week I happened to be sitting with a colleague looking at some prototype Java code. We came across a method that neither of us was particularly happy with, but which represented the typical messy code that occurs when wanting to do anything involving types and classes in Java. My initial comment was that it "would be so much cleaner to do in Scala". I therefore thought I'd better have play with the code to prove (if only to myself) that I was right. This post discusses the result of my effort and my conclusions from having done this work.

The Problem

In order to understand the code, let's first look at the problem that it is tying to solve:

The domain is one related to content that can appear on a website. We have a persistent domain model representing this content. However, a number of the content items are sourced from and can be updated in external systems (e.g. a CMS system or a remote content provider). We therefore need to synchronise the state of these external systems into our domain model from time to time. The java code that we were looking at was inside this synchronisation component.

The content is retrieved from the external systems, wrapped in a subtype of ForeignObject. The subtype is based on where the content was obtained from. The ForeignObject implementations can examine the content they contain and return information about its type and its unique business identifier. The synchronisation code then uses this information to lookup the associated object in the persisted domain (which may or may not exist). The ForeignObject and domain object are then passed to a mapper that updates the domain object from data that it extracts from the ForeignObject instance.

The full source code and tests described below can be obtained from my Github account: http://github.com/skipoleschris/type-safe-sync.

The Java Code

Let's first take a look at the Java code that we were less than happy about. Firstly, the ForeignObject implementation that returns the detail about the type of content that it contains:

public class CMSForeignObject implements ForeignObject {
     // Other code removed

    @Override
    public Class getLocalClass() {
        if ("Story".equals(type)) return Story.class;
        else if ("Image".equals(type)) return Image.class;
        else if ("BinaryObject".equals(type)) return BinaryObject.class;
        else if ("RelatedLinks".equals(type)) return RelatedLinks.class;
        else throw new IllegalStateException("Unknown type: " + type);
    }
}

As we can see it looks at some string based type field that it extracts from the wrapped content and returns the associated domain class that represents that content. Perhaps not the nicest code and returning class instances always feels a bit wrong. However, I don't have too many problems with this approach. Where the messy code comes in is in the synchronisation logic that uses this class:

public class SynchronisationService {
     // Setup code remove for clarity

    @SuppressWarnings( { "unchecked" })
    public Content sync(final ForeignObject foreignObject) {
        return findById(foreignObject.getLocalClass(), foreignObject);
    }

    @SuppressWarnings("unchecked")
    private  T findById(final Class clazz, final ForeignObject foreignObject) {
        if(Story.class.equals(clazz)) {
            CMSForeignObject cmsForeignObject = (CMSForeignObject)foreignObject;
            return (T)repository.findStoryById(cmsForeignObject.getId());
        }
        else if(Image.class.equals(clazz)) {
            CMSForeignObject cmsForeignObject = (CMSForeignObject)foreignObject;
            return (T)repository.findImageById(cmsForeignObject.getId());
        }
        else if(BinaryObject.class.equals(clazz)) {
            CMSForeignObject cmsForeignObject = (CMSForeignObject)foreignObject;
            return (T)repository.findBinaryObjectById(cmsForeignObject.getId());
        }
        else {
            throw new IllegalStateException(
                "We have managed to try and synchronise an unsupported class. This is a bug.");
        }
    }
}

How's that for a nice bit of logic? Imagine when we have many different types! I know this is only prototype code and there's certainly plenty we can do to improve it. However, it's quite typical to see this sort of stuff in production code whenever you have to go down into type based logic in Java.

The First Scala Version

To come up with a better solution, the first thing I did was to just convert the Java code directly into Scala. Nothing special here, other than using Scala syntax and modifying to return Option[Story] rather than null. So, first the ForeignObject implementation:

class CMSForeignObject(typeName: String, val id: String) extends ForeignObject {
  def localClass[C <: Content]() = typeName match {
    case "Story" => classOf[Story]
    case "Image" => classOf[Image]
    case "BinaryObject" => classOf[BinaryObject]
    case "RelatedLinks" => classOf[RelatedLinks]
    case _ => throw new IllegalStateException("Unknown type: " + typeName)
  }
}

And the synchronisation code:

class SynchronisationService(repository: ContentRepository) {

  def sync(foreignObject: ForeignObject): Option[Content] = findById(foreignObject.localClass, foreignObject)

  private def findById[T <: Content](clazz: Class[_], foreignObject: ForeignObject) = {
    if ( clazz == classOf[Story] )
      repository.findStoryById(foreignObject.asInstanceOf[CMSForeignObject].id).asInstanceOf[Option[T]]
    else if ( clazz == classOf[Image] )
      repository.findImageById(foreignObject.asInstanceOf[CMSForeignObject].id).asInstanceOf[Option[T]]
    else if ( clazz == classOf[BinaryObject] )
      repository.findBinaryObjectById(foreignObject.asInstanceOf[CMSForeignObject].id).asInstanceOf[Option[T]]
    else throw new IllegalStateException("We have managed to try and synchronise an unsupported class. This is a bug.");
  }
}

Works fine, but not really much better than the Java code.

Adding Scala Type Goodness

Once I had an initial Scala version I then spent a couple of hours playing around with the code and some Scala type goodness. The resulting synchronisation code is:

class SynchronisationService(repository: ContentRepository) extends ContentFinder {

  def sync(foreignObject: ForeignObject): Option[Content] = findById(foreignObject)

  private def findById[C <: Content, FO <: ForeignObject](foreignObject: FO) =
    Option(finder(findStory) orElse finder(findImage) orElse 
           finder(findBinaryObject) apply foreignObject)

  private def findStory(foreignObject: CMSForeignObject) = repository findStoryById (foreignObject id)
  private def findImage(foreignObject: CMSForeignObject) = repository findImageById (foreignObject id)
  private def findBinaryObject(foreignObject: CMSForeignObject) = repository findBinaryObjectById (foreignObject id)

No nasty if/else block. No need to deal with Class instances. No nasty type casting. Much more readable. Hopefully this is much more friendly for developers and will be far easier to maintain in the future as we come to add more types of content to be synchronised. So, how did I manage to achieve this much more readable code?

The first piece of the puzzle was to eliminate the returning of Class instances from the ForeignObject. Instead we now return a new ContentType instance:

class CMSForeignObject(typeName: String, val id: String) extends ForeignObject {

  def contentType[C <: Content, FO <: ForeignObject] = {
    val contentType = typeName match {
      case "Story" => new ContentType[Story, CMSForeignObject]
      case "Image" => new ContentType[Image, CMSForeignObject]
      case "BinaryObject" => new ContentType[BinaryObject, CMSForeignObject]
      case "RelatedLinks" => new ContentType[RelatedLinks, CMSForeignObject]
      case _ => throw new IllegalStateException("Unknown type: " + typeName)
    }
    contentType.asInstanceOf[ContentType[C, FO]]
  }
}

The ContentType instance supports generics on both the type of content and the type of foreign object. It captures the actual erased types used via implicit manifests on its constructor:

class ContentType[C <: Content, FO <: ForeignObject]
        (implicit val mC: Manifest[C], val mFO: Manifest[FO]) {

  def compatibleWith(cClass: Class[_], foClass: Class[_]) = (cClass == mC.erasure) && (foClass == mFO.erasure)
}

It also provides a compatibleWith method that is used internally within the finder code that we will look at below.

This final piece of the puzzle is the implementation of theContentFinder trait that is mixed in to the synchronisation code. This is where most of the type magic occurs in order to simplify the actual synchronisation code:

trait ContentFinder {
  implicit protected def finderToPartialFunction[FO <: ForeignObject, C]
        (handler: LookupHandler[FO, C])(implicit mFO: Manifest[FO], mC: Manifest[C]) =
    new PartialFunction[AnyRef, C] {
      def apply(foreignObject: AnyRef) =
        if ( isDefinedAt(foreignObject) ) handler(foreignObject.asInstanceOf[FO])
        else throw new IllegalStateException("We have managed to try and synchronise an unsupported class. This is a bug.")

      def isDefinedAt(foreignObject: AnyRef) =
        foreignObject.asInstanceOf[FO].contentType.compatibleWith(mC.erasure, mFO.erasure)
    }

  protected def finder[FO <: ForeignObject, C](lookup: FO => Option[C]) = new LookupHandler(lookup)

  protected class LookupHandler[FO <: ForeignObject, C](lookup: FO => Option[C]) {
    def apply(foreignObject: FO) = lookup (foreignObject).getOrElse(null).asInstanceOf[C]
  }
}

The first insight that I had when working with this code is that the if/else condition across classes is actually a partial function across content types. We have a ForeignObject that can return a ContentType instance that contains the type information about the type of content and the type of foreign object. We therefore just have to see if there is a partial function defined at that particular combination and if there is then we can apply it.

The resulting implementation therefore defines a finder method that takes a function containing the actual code that should be applied to the ForeignObject/Content pair. It returns this wrapped in a LookupHandler instance. The synchronisation code can then just define find functions for each type that it can deal with and pass them to the finder method. Finally we define an implicit conversion for the LookupHandler instances that convert them to partial functions so that we can compose them with the orElse method. The partial function captures the erased types from the handler using implicit manifests and compares these with those captured by the content type to see if the function can be applied.

Conclusion

We can clearly see that the resulting synchronisation code is significantly cleaner, more easily understood and more maintainable than the original Java code. However, this comes at the expense of slightly more lines of Scala code and some fancy logic around finders and implicit partial functions. Is the benefit really worth the extra?

I can see both sides of the argument. I can certainly understand that for the Scala novice that the ContentFinder is somewhat impenetrable. However, its the synchronisation code that is likely to be updated a number of times as more types of content are bought into the domain. The code to deal with all of the type goodness is unlikely to change that much and can be easily maintained by more experienced Scala developer. Therefore the readability and maintainability of the business code wins out in my mind over some extra, well abstracted complexity.

This tends to be the typical approach to the Scala language and one of the reasons that it can be so incredibly powerful. Use all the power of the language and its incredible type system in the framework and support code so that the actual application logic can be as minimal and simple as possible.

The Architecture Doesn't Work

I've been building web and other large-scale, enterprise type applications for many years, following the standard accepted approaches to application architecture. I've been increasingly feeling that these approaches are somewhat flawed and that we need something better. Over this series of three blog posts I'm going to talk about some new technologies and architectures that I have been having some great success with. In this first of the three posts I want to look at why...

The Architecture Doesn't Work

If like me you've been building enterprise style applications in Java (or pretty much most other similar languages) then you will be intimately familiar with this layered architectural pattern:

For those, who need a quick refresher, here's a quick summary of each layer:

• Presentation Layer - Deals with rendering the current state of the application to the user/client. Could be a web page, an GUI application or even some structured representation (e.g. XML, JSON). Usually renders the contents of the Domain Objects, often by some form of template system.
• Controllers - Handles events generated by the user/client and orchestrates the processing and Presentation. Typical approach is to accept an incoming request, extract any supplied request values, call a Service, pass the resulting Domain Objects to the appropriate Presentation component and finally return a result.
• Service Layer - Executes business logic and processes against the Domain Objects and returns the resulting objects. Typically stateless and transactional. Processing usually involves using the Persistence Layer to load Domain Objects from a Persistent Store, working with the Domain Objects to mutate state, using the Persistence Layer to save the changes and then returning the resulting Domain Objects for presentation.
• Domain Objects - The main objects representing entities in the business, their relationships and the business rules associated with each. Entities are usually persisted to a Persistent Store. Some people prefer anaemic Domain Objects, but I have argued that rich models are significantly better.
• Persistence Layer - Provides an interface between the Service Layer and the Persistent Store that talks the language of Domain Objects. Allows the Service Layer to undertake CRUD (Create, Read, Update Delete) and Find operations against the Persistent Store.
• Persistent Store - The storage for the data in the application. Usually a Relational Database Management System (RDBMS). Domain Objects are typically mapped to tables. Supports transactional semantics.
• Object-Relational Mapping (ORM) - A library that assists the mapping between Domain Objects and the Persistent Store. Tries to solve the impedance miss-match between object and relational structures. Used by the Persistence Layer. Modern solutions use annotations on the Domain Objects (e.g. JPA & Hibernate)
• Container Services - General services provided by a container. Supports features including: dependency injection; transactions; configuration; concurrency; service location; aop etc.

So, why doesn't this work?

It has become apparent to me over many projects that the above architectural approach suffers from a number of technical weaknesses. Often these have come about through the aim of simplifying the programming model for developers, but as we will see later the result is often the opposite. First, let's look at what I believe to be the main general technical problems that we find in this standard architecture. Later we will look at the impacts of these problems on the application and developers.

Thread-per-request model

Typically, in the above architecture, each request is handled in its own thread. Threads are usually managed in a pool and are then assigned to a request when it arrives. The thread and request are then linked until a response is returned, at which time the thread is returned back to the pool. This approach aims to simplify the development model by reducing the need for developers to have to think about concurrency issues - effectively giving them a single-threaded application path for handling each request.

Unfortunately, assigning a separate thread to each request is a very inefficient way of utilising system resources. Threads are fairly heavy-weight entities, having to maintain their own stacks, address space and so forth. The work involved in scheduling them is also quite large and given today's i/o intensive applications, many of them will be in a blocked state at any point in time. Additionally, most systems can only support a finite number of threads before suffering degradation in performance. (Some languages have tried to overcome the thread problem by introducing lighter weight constructs, such as fibers, but these still share many of the same problems for concurrent programming.)

The technical limitation of this thread-per-request approach is to actually stifle system scalability and impact application performance. Additionally, when the need to share state occurs, then developers must step outside their comfort zone into the world of concurrent programming: something the thread-per-request model has isolated them from.

Constant reloading from the persistent storage

Given a thread-per-request model, there is a natural approach to load all objects required to service a request from the persistent store on each request. Often this is completely unnecessary as a large percentage of the required data is unlikely to have changed from one request to another, or to be different between two separate running requests. However, with no convenient way to store state safely between requests or share infrequently changing state across threads we are forced down the load everything each time direction. The alternative is to drop down into shared state and concurrent programming - and we've already explored why that's not the best option! Techniques such as lazy loading try to minimise the overhead, but we have to accept that much of the data an application uses is being reloaded unnecessarily on each request.

Using database transactions for optimistic locking & concurrency control

Given each request is working with its own copy of application data kept in its own thread, how does the architecture handle multiple threads trying to update the same data at the same time? The general approach is to use database transactions to implement optimistic locking. Two requests can each load a separate copy of some data in their own threads and update it as they wish. When a thread's current transaction is committed, the state is written back to the persistent store. The database's transaction support ensures that each thread's changes are committed in an atomic and correctly isolated manner. Optimistic concurrency can follow a last-commit-wins model, or many ORM solutions offer a row versioning approach so the second transaction to commit changes to an object will fail with a stale data exception.

However, relying on the lowest level in the application stack to prevent concurrent updates isn't always the best solution. For a start, it allows both transactions to pretty much complete all their processing before one discovers that all its work is no longer valid. Secondly it causes problems when concurrency in the application layers above is required as some state would be shared across threads, and thus transactions - not nice to deal with. Finally, it doesn't actually simplify the problem because instead of having to deal with concurrency, the developer must now deal with the possibility of transaction rollback due to stale data, and the need for replaying of transactions. You'd be surprised how many applications I've come across that either don't address the problem (just raise a 500 error) or try to do something in this area and fail miserably!

Tight coupling between domain and persistent store through ORM

Another problem inherent in this architectural model is the tight coupling between the domain objects and the persistent store. This is caused by the need for the domain objects to be closely coupled to the object-relational mapping (ORM) solution. Why would we want to couple our domain objects this tightly? Well, it turns out that quite often we have to do this for performance reasons related to needing to reload from the persistent store for each request. Given that we are loading data so frequently we find that we have to tune our domain model not to make it optimal to our business model but to allow us to work most efficiently with the persistent store. And, don't even get me started on annotation based configuration (aka JPA) which forces us to consider and include all the details of our persistence strategy inside our representation of the business domain!

Scaling through adding servers

Once we have built our application using our architecture, we then have to deploy and scale it. Given that our thread-per-request approach doesn't make the best use of resources, the typical approach to scaling is to add additional servers, each running a separate copy of the application. Unfortunately this is not a particularly good way to achieve scale because we are still dependent on a shared database - and we are making heavy use of the database by loading data on each request and also using it for concurrency control.

Additionally, this approach to scalability also causes session based data sharing problems. With a couple of servers we can perhaps use session clustering with some success. As the number of servers grows we end up either having to use sticky sessions (i.e. tie each session to a single server - which means sessions are lost if that server crashes) or use some other way of persisting session data. This usually involves either adding session state into our already stressed database or introducing a whole new technology (distributed cache) to handle this data.

Static configuration

While not as significant as many of the above items, our standard architecture, and the libraries it is built upon, tends to push us towards a model of static configuration held in XML/YAML/Properties or other similar files. This increases the difficulty in managing our application at scale and in trying to make dynamic changes to values to tune the running system. When we want to support dynamic configuration we are often forced down the route of exposing properties to a management console (i.e. JMX) or pushing configuration down into the database or distributed cache, which we already saw above is not the ideal direction.

Poor approach to fault tolerance

Finally, we have the problem of fault tolerance. Typically, the approach to exceptions at any stage of processing a request is one of: catch the exception at the top level, log it and then display a generic "Oops, something went wrong!" message to the user. Hardly a great user experience. Unfortunately our architecture makes doing anything more than this prohibitively hard. For example, we might want to put some context information in the message. However, when the exception was thrown the developer might not have added this context information to the exception or, more likely, the exception was thrown by a library (such as the ORM) which is unaware of the context in the first place. We might also try to implement a re-try strategy, but implementing retries in a thread-per-request model is actually very difficult - you have to remember to throw everything you currently have away (because memory state in the thread may not be consistent with the database), clear all caches and so on. Try it, it's pretty hard!

And what is the impact?

So, given all of the above technical weaknesses with our current architectural approach, what effects do these have on our application?

Achieving performant applications is more difficult than it should be

The first major impact is that the architecture makes it hard to achieve application performance: the thread-per-request model makes poor use of resources and, loading unchanged data from the database for each request is inefficient and overloads the database.

Given these problems we generally find that we have to consider performance far earlier and in much greater depth than we really should. I'm not saying that we should never develop without considering performance (that would be negligent), but that the need to optimise our well written code and architecture for performance should be something we do rarely and in response to specific issues identified during performance testing. Designing a domain model so that it has the most efficient mapping to a database in order to achieve basic application performance is neither desirable nor the best way of capturing an accurate representation of the business domain. Compromising simple code for more complex but performant alternatives just to meet basic performance criteria is never desirable.

Another impact of the poor performance characteristics of our architecture is that we often end up having to circumvent the simplified thread-per-requets model in order to get a performant and usable application. What do we mean by this? We have to add some form of shared state - exactly what the simplified request model was put in place to avoid. Typically, we do this in one of two ways: adding caching or doing concurrent programming.

When an application proves to be less performant than required, most developers will initially suggest adding caches for data and results that don't change regularly. There are plenty of places in an application where caching can be applied. However, implementing caching is SIGNIFICANTLY more complex than most developers like to assume. Let's consider some of the things that need to be addressed: how do we keep the cache in sync with the database? what's the expiry strategy? what's the update strategy? do we pre-warm? do we block reads during an update or just block concurrent updates? is a cache per server fine or do we need a distributed solution? how will we ensure we don't get cache deadlock? As you can see, implementing a cache is actually a very complex option which requires great skill to get right. Caches should be reserved for special cases and should not be a catch-all for badly performing code and architecture.

The alternative approach to a cache is to introduce some form of shared state with concurrent access. Rather than loading all objects on each request, we keep common objects in memory and share them across and between requests. However, this comes with many of the same problems as using caching, plus the requirement of transaction management for updating this shared state. If done right, shared state with concurrent access can be simpler, more elegant and more flexible than caching, but it's the doing it right which is the challenge.

Now, this might be an over-generalisation, but my experience has shown that most 'average' developers can't write concurrent code using shared mutable state, multiple threads and synchronisation blocks. They usually end up with deadlocks, inconsistent state and difficult to track down bugs. Why? Because it is HARD and also because the single-threaded thread-per-requet model that they are used to programming to has isolated them from any considerations of concurrency. It's true that modern concurrency libraries (e.g. java.util.concurrent) have helped the problem, but most developers are using these without understanding exactly the reasons why. Just ask an average java programmer to explain the 'volatile' keyword and you will see what I mean!

Additionally, our dominant L-mode brain activity is optimised for linear, single-threaded thought. This in turn makes thinking about concurrency and all its implications a very unnatural process. I don't have any proof, but I have some very strong suspicions that a large proportion of the developer community is just unable to comprehend the real impacts of concurrent threads reading and writing in a shared state model.

Scaling applications is difficult

The other major impact of our current architectural approach is that it doesn't scale very well as application load increases. Firstly, the thread-per-request model just doesn't make good use of resources. Application servers have a finite set of threads that they are able to handle at any one time. Going beyond this limit significantly slows down the server - even though at any time many of the threads will be blocked on i/o. Once a server has reached thread saturation, the only solutions are to speed up performance (which we have seen is very hard) to free threads up more quickly, or add more application servers.

However, adding more application servers doesn't actually solve the problem. All it does is enable more concurrent threads to run, which in turn puts more load on the underlying database. Given the approach of re-retrieving data on each request we just end up overloading the database. We then need to scale the database to handle the extra load - and scaling databases is a VERY specialist job and VERY expensive as well. There's no way we can follow a path of continued scalability following this model. Clearly having an architecture based around cheap commodity hardware running app servers doesn't work if you need to massively scale databases to get any performance - so we are back to caching and shared state again, and we already explored what a nightmare these can be.

The final difficulty of scaling the application comes in the way configuration and shared session state is managed. At low scale, configuration by static config files is fine. Sharing session state through session clustering or sticky sessions is also acceptable. However, as the number of servers grows and infrastructure becomes more complex you soon end up in configuration hell - different versions of the application with slightly different configurations, massive challenges tuning parameters on running servers and so forth. Additionally, sticky sessions become less feasible and any form of session state clustering becomes impossible. Session state therefore needs to be pushed down into the database, increasing its overload and scalability issues or we start having to introducing distributed shared state solutions, with all the problems this entails.

So, it really doesn't work. Where do we go now?

We've seen that the current standard layered architecture that we are using to build applications causes a number of performance and scalability problems. Mostly these have come about from the desire to shield 'average' developers from the complexity of concurrent programming with shared mutable state. Unfortunately this simplicity causes performance problems and impedes scalability, requiring that developers must work around the restrictions in order to build something that works. There must be a better approach that solves these problems without needing developers to become shared mutable state concurrent programming gurus!

In the next post in this series I will look at a solution to these problems using lightweight asynchronous 'actors' and immutable messages. The final post will then look at some specific technology solutions that I have employed to implement this new approach.

Anti-Patterns and Rich Domains

At one of my customers we have been doing a lot of work on rich domain models, based on the principles of Domain Driven Design by Eric Evans. During this work we identified a particular anti-pattern in the existing code. I was trying to explain this to one of the developers on my team and it proved quite a subtle distinction. After thinking about it a bit more I decided to write this blog entry with some code examples to explain it in more detail.

Background

In the particular domain in question we have a (very simplified!) notion of Content and Metadata. Each piece of Content has some Metadata attached. However, the Content is sourced from one location (for example a CMS system) and Metadata is sourced from another (such as a remote web service). The challenge is combining the objects from these two different sources into a model that can be used by the higher layers of the application.

Anti-Pattern Solutions

The current solutions to this problem are based around the concept of an aggregator that pulls content from the different sources and merges them together into the returned objects. In all cases the returned objects follow the anaemic domain pattern of just fields with setter and getter methods.

Anti-Pattern 1: Transient Relationship

In this pattern, the aggregator pulls one object (the Content) from its source, uses an id field in this object to obtain the other object (the Metadata) from its source, and then sets the Metadata on the Content as a transient field:

public class Content {
    private String id;
    private Long metadataId;
    // Other fields

    private transient Metadata metadata;

    // Constructors, setters & getters
}

public class Metadata {
    private Long id;
    // Other fields

    // Constructors, setters and getters
}

public class Aggregator {
    // Fields, constructors and setters

    public Content getContent(String id) {
        Content content = cms.getContent(id);
        Metadata metadata = metadataService.getMetadata(content.getMetadataId());
        content.setMetadata(metadata);
        return content;
    }
}

The above code has a number of major problems:

• Clients of the Content class can use both the metadataId and metadata properties, possibly interchangeably. This makes for fragile client code that is not receptive to change.
• Any serialisation will result in the transient field being lost. How do we get it back?
• When we want to update the metadata relationship do we update the metadataId, metadata or both? How do we keep them in sync?
• What happens if we can't get the metadata, but successfully get the content? Do we throw an exception and prevent the content being used (even if it can)? or do we return content with a null transient field - requiring the client code to do null checking?

So, clearly having and id and a transient field representing the same relationship is not a good approach, can we do it without the transient?

Anti-Pattern 2: Decoupled Objects

In this pattern we avoid the transient object by keeping the two objects (Content and Metadata) separate and using the aggregator to retrieve each:

public class Content {
    private String id;
    private Long metadataId;
    // Other fields

    // Constructors, setters & getters
}

public class Metadata {
    private Long id;
    // Other fields

    // Constructors, setters and getters
}

public class Aggregator {
    // Fields, constructors and setters

    public Content getContent(String id) {
        return cms.getContent(id);
    }

    public Metadata getMetadata(Long id) {
        return metadataService.getMetadata(id);
    }
}

This code solves SOME of the problems of the first solution, but introduces problems all of its own:

• It is the total responsibility of the client code to manage the relationship between Content and Metadata - making the whole codebase more fragile
• It is very easy for client code to bypass the aggregator entirely and make direct calls to the metadata service, increasing complexity and coupling

So, what's the better way to do this?

The Rich Domain Solution

Our solution to the problem is to correctly model the relationship between Content and Metadata. We can introduce a Repository for Content that correctly sets up and manages the relationship, hiding the underlying management of ids and using real objects instead:

public class Content {
    private String id;
    private Metadata metadata;
    // Other fields

    // Constructors, setters & getters
}

public class Metadata {
    private Long id;
    // Other fields

    // Constructors, setters and getters
}

public class ContentRepository {
    // Fields, constructors and setters

    public Content getContent(String id) {
        Content content = cms.getContent(id);

        // We could proxy the metadata here for lazy loading or to delay exceptions being
        // thrown until the metadata is requested. However, in this simple example we will
        // build the relationship directly
        Metadata metadata = metadataService.getMetadata(content.getPrivateMetadataId());
        content.setMetadata(metadata);
        return content;
    }

    public void save(Content content) {
        // Persist changes and update the relationship if it has changed
    }
}

By hiding the management of the relationship between Content and Metadata in the core structure of the object model we prevent the details of how this is created and managed escaping from the domain model. Changes in the relationship between Content and Metadata can be correctly managed by the domain model. We also have options such as proxying the Metadata in Content to support lazy loading or delayed exception handling - something we can't do when details of the relationship are exposed to or managed by the client code.

Just goes to show how much a bit of good rich domain modelling can improve the structure and maintainability of software, even in very subtle ways.

Addendum

Just been thinking further on the subject. The above adequately addresses the technical aspects of the anti-pattern and the solution. However, there is also a huge cognitive element as well. The aggregator anti-pattern encourages a mental model of the domain as one of disconnected objets that are being forced together. Clients of the domain are therefore much more likely to create code that teats the model in this way: code that will be more complex and fragile. However, following the rich domain approach, clients have a mental model of a single object (Content) and it's relationship to other objects (such as Metadata) without being burdened with any of the separation knowledge. This cleaner mental model of the domain is much more likely to lead to clean, well architected client code that is easier to test, maintain and enhance.

Sprinting with BDD

After some initial experiments into doing BDD using Cucumber our team has decided to adopt a full BDD and TDD approach to our current sprint. It's slightly experimental at the moment as we work out the best ways of working. Once we are done we will retrospect and see if we want to adopt the approach going forward. Initial feelings from our first couple of days are that it is going well. Here's what we have done so far.....

Sprint Planning

After deciding to give BDD and TDD a try for the sprint, the first thing was the sprint planning. Fortunately for this sprint we have a story that is pretty well understood, but sufficiently challenging to be of value in evaluating the approach. The story basically consists of taking some existing data, retrieving additional data from a new remote system and delivering the combination of the two to a remote destination (basic decoration of existing content).

Our sprint planning consisted of two parts:

1. a high level technical discussion of the story and agreement of the general architectural structure and approach
2. the writing of Cucumber 'Feature' files containing all of the acceptance scenarios for the story

These two parts went well and we created a good set of acceptance scenarios, but we then found ourself struggling to get our heads around how to go about implementing them. Our reasoning was that rather than writing out individual technical tasks, we would just create one card for each scenario. These scenario cards would then form our sprint backlog.

However, the first scenario was a 'happy path' one and we were thinking that in order to implement it we would have to build the entire solution; the remaining scenarios would then just be tiny additions. This didn't feel right in that we would then have one scenario stuck as 'in progress' for days and then all of them moving within a few hours right at the end of the sprint. Our next thought was to therefore break down this scenario into a number of technical tasks. As it turns out, this wasn't the correct approach and it soon became clear that by doing this we were missing the benefits of BDD and TDD: we were defining the detailed solution in the planning session, rather than letting it emerge from the implementation of the tests.

We decided to take a break and think over our approach. It was then that it came to us that we were still thinking in a non-TDD mindset. To pass the first scenario, we didn't need to implement the whole solution, just enough to make the BDD test pass. As we then moved on to the next scenario we could implement more of the functionality, add depth and so on. With this in mind we decided to start the sprint and inspect and adapt as we went.

Setup and Configuration

The first step in the sprint was to add Cucumber support to our project. Given that we are using BDD for acceptance tests, we decided to add this support into our functional test suite. Unit level TDD will continue to be done using our existing testing tools (JUnit, Hamcrest and mockito).

Our functional tests are written in Java and are built and executed using Maven. We therefore added a dependency to cuke4duke, which runs Cucumber on top of the JRuby platform. This setup included adding a dependency on the maven-cuke4duke-plugin, which runs the Cucumber 'Features' within a maven test phase.

Aside: Why did we pick Cucumber (which is a Ruby framework) when we are doing a Java project? There were a number of reasons, but the main ones were:

• We liked the HTML report format
• A number of the team are already using Cucumber on external projects (in Ruby and Scala) so there was an existing familiarity
• It seems to keep out of the way and be less invasive than many of the Java based alternatives

With this set up, we added the Features file that we built during spring planning into the appropriate features directory and created step definitions with pending methods for each of the Given/When/Then cases.

The First Scenario

The first scenario was to set up some existing content and some additional content, decorate the existing content with the additional content and then verify that the content delivered at the destination had been correctly decorated. The steps for Cucumber were written accordingly and the assertions added the 'Then' case. Now it was time to implement.

This was where our TDD thinking really kicked in. Previously we would have started building the whole solution. Instead we just implemented the elements of the destination required to allow us to verify the decorated content. Given that we were only testing with one piece of content we just hardcoded to return a fixed set of additional data. Test passed.

Now, we know that this is not the full solution and that we have plenty more work to do, but we have a green on our test report. We have 8 scenarios to complete and the story is not done until they all pass, so what's wrong with this approach? Our next scenario tests with a different set of content and additional data, so this is broken by the hardcoding approach, so we can then work on the next step of delivering different sets of additional data to the destination. In this case we will still probably hardcode the data that is delivered based on the content, but progress will be made. Later scenarios will then add the calls to the new remote system to get actual data. By the end of the scenarios we will have built the entire solution driven solely by tests.

Areas of Uncertainty

Even though things are going well, we still have a number of areas of uncertainty that we need to resolve by then end of the sprint. These include:

• Whether we can successfully complete the story in this way
• Whether this approach impacts our productivity
• Whether the team members are able to adapt to working in this TDD style
• Whether the business will identify with the inherent value of this approach
• Whether we can get the BAs and the POs to start thinking of the stories as BDD scenarios

We hope to resolve these as the sprint progresses. I do think that initially productivity will be reduced due to the time it takes to learn a new approach and to become proficient in it. However, while productivity may be reduced for a short time, I believe that quality will go up due to the test-driven nature of the work and that we will be more certain of building the correct solution. I it my hope that the team members will see the value of this approach and will opt to continue with it for a few more sprints to allow them all time to become proficient in the approach. We will then be able to evaluate its success or failure more objectively.

To help the business understand the value of this approach (assuming of course that it is successful) I will plan to include a quick overview of the BDD approach in the Sprint Review next week. I will give a short presentation on the approach we took, show the scenarios that we created during planning, talk through the approach we took to implement them and show the Cucumber report that is produced. It will be interesting to see the reactions.

I'll post a follow-up at the end of the sprint to let you know how we got on.

From Java to Scala in O's and X's

I've been doing a lot of evangelising on the Scala language, mainly to Java devs that I work with. A number of the have become very interested in the language and have started making an effort to learn more about it. The most common questions that seem to come up relate to how to move from the Java to Scala language and the key differences (such as a functional approach, building apps though composition and mixins, the type system and so on).

With this in mind I decided to build a small application that starts as a typical Java development and over a number of versions becomes a full-blown Scala implementation. All versions of the application are maintained so that it becomes possible to step through and see each new level of progression into the Scala way of doing things. The progressions can be presented quickly in an hour or so, or it is possible to spend much longer on each one discussing design decisions and so forth. Even without the Scala focus, it's also a good application for discussing different design and implementation strategies with Java developers.

To have an application where the domain is easy to understand I picked the age old Noughts and Crosses game (Tic-Tac-Toe for my American readers). The advantages of this game are that it is very simple to learn and understand, it has some interesting data structures and interesting set of rules that can be explored for selecting the best move to make. It can also be implemented quite happily in a handful of classes and a few hundreds of lines of code. The particular approach I selected was to have the game played by two computer players who each take it in turns to pick the best move to make until one either wins or the game is drawn because the grid is full.

Aside: Initially the rules were so optimised that my implementation only ever resulted in a drawn game. The final solutions therefore ensure that the first couple of moves are randomly selected in order to introduce a bit of interest.

All of the source code that I'm going to discuss is available on my github account at: http://github.com/skipoleschris/OandX

Version 1 - Typical Java Code

After spending so much time writing Scala (and the most Scala like Java code possible) it was surprisingly difficult dropping back into the type of Java code that you find on most projects. For this version I tried to adopt the Java coding style of a typical average Java developer. Code is written in basic Java and tests in Java using TestNG. The solution works find but has a number of code and design smells, specifically:

• 'null' is used to represent empty positions on the grid
• The grid is represented in a Java Bean style and so all details of how to work with the contents of the Grid are actually outside the Grid class - leading to a pattern of utility classes that contain large amounts of domain logic
• There is a lot of duplication in the WinScanner and MoveFinder classes
• MoveFinder in particular makes it very difficult to separate the rules being applied from the complex code implementing them
• MoveFinder has the most fantastic if ( ... == null ) multi-repeated check method!

Version 2 - Slightly Improved Java Code with ScalaTest

The next version improves the Java code slightly. (It's still code way below the level I would normally write, but this is an exercise in converting from Java to Scala, not in producing perfect Java code!). In particular, the Grid class now contains more domain logic; the WinScanner and MoveFinder classes have duplication removed; and the MoveFinder is implemented with a chain of filters rather than the nasty if clause. The MoveFinder in particular is still WAY too complex.

The other main feature of this version is that tests have been ported over to ScalaTest. I went for the BDD style of tests using FeatureSpec, GivenWhenThen and MustMatchers as I feel this shows the full power of DSL-like test constructs. Most Java developers I have show this to seem to find it very readable. Typical tests now look like:

feature("The OnX grid") {

  scenario("can have a token added to it") {
    given("a new grid")
    val grid = new Grid

    when("a token is added at a given position")
    grid.addToken(Grid.MIDDLE, Token.nought)

    then("that position contains the token")
    grid.getToken(Grid.MIDDLE) must be (Token.nought)
  }

}

Hopefully much more readable!

Version 3 - Java Without Semicolons

The third version ports the Java code directly to Scala with minimal use of advanced Scala language features. All I did here was convert classes and data structures directly, remove boilerplate code and use Scala for loop syntax. At this point it just looks and reads pretty much like a Java application.

For example, here's the same method from the MoveFinder class implemented in both Java and Scala:

private static class DoubleWinPositionFinder implements PositionFinder {
    @Override
    public Position findPosition(final Grid grid, final Token token) {
        final Set positions = new HashSet();
        for ( final Line line : grid.getLinesWithMatchingTokenAndTwoSpaces(token)) {
            for ( final Position position : line.getEmptyPositions() ) {
                if ( positions.contains(position) ) return position;
                else positions.add(position);
            }
        }
        return null;
    }
}

private class DoubleWinPositionFinder extends PositionFinder {
  def findPosition(grid: Grid, token: Token): Position = {
    var positions = Set[Position]()
    for ( line <- grid.linesWithMatchingTokenAndTwoSpaces(token) ) {
      for ( position <- line.emptyPositions.toList ) {
        if ( positions.contains(position) ) return position
        else positions.add(position)
      }
    }
    null
  }
}

This approach is a great way to get working with Scala. Just bring all your Java knowledge and coding style with you, make use of the Scala tools, test frameworks and so on and gradually move towards advanced Scala when you feel comfortable.

Version 4 - Moving Forwards With Scala

In this version, I moved to a much more Scala like model. In particular:

• The Grid class is now immutable, returning a new Grid on each change
• The Option[T] concept is now used as a replacement for null to allow better representation of empty positions
• The code uses more advanced functional concepts such as map, flatMap, filter and so forth.
• Mixins are used to compose the Grid with traits representing line handling, win scanning and move finding. This gives a much better represented Grid concept and smaller API surface area while still allowing separation of concerns into separate traits and classes

While I think this is a much better implementation than version 3, I am still not happy with the implementation of some of the line building logic and the move finder is still too complex and makes it difficult to separate the rules being applied from how they are implemented. In particular code like this (which finds all the empty positions on a list of lines) is just way too messy:

def emptyPositions(lines: List[Tuple2[List[Option[Token]], Position]]) =
  lines.filter(_._1 contains None).flatMap(empties => empties._1 zip positions(empties._2)).filter(_._1 == None).map(_._2)

This leads me to believe that while I am using some of the more advanced language features that something is wrong with my underlying data structures if the code gets this complex. However, looking at our move finder function we can see some simplification starting to take place:

private case object DoubleFreePositionFinder extends PositionFinder {
  def findPosition(token: Token) = {
    val allEmptyPositions = emptyPositions(linesWithMatchingTokenAndTwoSpaces(token))
    if ( allEmptyPositions.contains(Position(1, 1))) Some(Position(1, 1))
    else if ( allEmptyPositions.isEmpty ) None
    else Some(allEmptyPositions.head)
  }
}

Version 5 - The Final Solution

For version 5 I decided to approach the problem from a much more functional perspective. By switching my thinking from the object approach (i.e. what is my state and how can I encapsulate it) to a functional approach (i.e. what functions do I want to perform) I was able to realise an important concept. Although the noughts and crosses game is represented as a grid, all of the functions we want to apply are either on an individual position on the grid or on one of the eight lines that can be made from the grid. By switching my data structure to one more appropriate for applying these functions we end up with a significantly simplified set of functions for evaluating the game state.

For example, we can take a list of tokens, zip them with a list of positions and then apply functions across either the token part or the position part of the pair:

class Grid(tokens: List[Option[Token]]) extends TokensWithPositions
                                        with Lines
                                        with LineQueryDSL
                                        with WinCheck
                                        with MoveFinder {

  require(tokens.length == 9)

  val positions = for (row <- 0 to 2; column <- 0 to 2) yield Position(row, column)
  private val values = tokens zip Grid.positions

Lines can therefore be represented in exactly the same manner:

type Line = List[Pair[Option[Token], Position]]

This new structure greatly simplifies the rest of the application code when it comes to building and working with lines:

def emptyPositions(lines: List[Line]) = lines.flatMap(_.filter(_._1 == None).map(_._2))

The other main change in this version is separation of the rules for finding the best position for a player from how those rules are implemented. This is achieved through a custom DSL that allows defining the rules in a highly readable way. For example, the rule we saw previously now becomes:

private def doubleFreePosition(token: Token) =
  find linesHaving 2 positions Empty and 1 tokenMatching token select First take EmptyPosition

The DSL is a fairly simple one, implemented as a chain of filtering functions that reduce the lines down and then a selector that pulls the correct result from the matching lines. It actually turns out that the DSL is pretty easy to read and follow through, which also significantly simplifies the code and reduces duplication.

Although version 5 is a much better solution, there is still room for improvement and some functions that could certainly be implemented in a better way. Let me know when you find them and send me your better solution :-)

Version 6 - Adding Some Actors

The final version takes the code from version 5 and wraps actors around the classes. One actor for the game coordinator, one for the grid and one for each player. These actors are loosely coupled through message passing.

Noughts and Crosses is not the best demonstration for actors and concurrency as it's turn based nature makes it a purely sequential game. However, version 6 does allow the concept of actors to be demonstrated in an easily understood way and leads on to more detailed conversations about using actors for concurrent problems.

Conclusions

The Noughts and Crosses example shows how a Java programmer can migrate to Scala in a gradual manner, adding new Scala concepts as they become more comfortable. However, it's important to point out that Scala is certainly not a silver bullet for good software. Like all development it requires careful thought, good design, solid programming practice, good testing and constant refactoring. Any Scala application will only be as good as the developer(s) building it.

Rich vs Anaemic Domain Models

Is your domain model a rich one, or is it one of those ultra anaemic ones? Where should business logic live - encapsulated inside the domain or in a number of higher-level services/controllers? Over and over I seem to have had the same debate with different companies and developers. So, once and for all here is my definitive view on the subject...

Anaemic Domain Models

An anaemic domain model is one typified by the Java BeansTM style of programming: simple Java domain classes containing just fields and setter/getter methods for those fields; logic for manipulating the domain objects is contained in higher level classes (typically a service layer). 

For example, consider the following simple anaemic domain model consisting of a Person and a list of Addresses associated with that person:

1:  public class Person {  
2:    
3:    private Long id;  
4:    private String forename;  
5:    private String surname;  
6:    private Date dob;  
7:    private List<Address> addresses;  
8:    
9:    public Person() {  
10:    }  
11:    
12:    public Long getId() {  
13:      return id;  
14:    }  
15:    
16:    public void setId(Long id) {  
17:      this.id = id;  
18:    }  
19:    
20:    public String getForename() {  
21:      return forename;  
22:    }  
23:    
24:    public void setForename(String forename) {  
25:      this.forename = forename;  
26:    }  
27:    
28:    public String getSurname() {  
29:      return surname;  
30:    }  
31:    
32:    public void setSurname(String surname) {  
33:      this.surname = surname;  
34:    }  
35:    
36:    public Date getDob() {  
37:      return dob;  
38:    }  
39:    
40:    public void setDob(Date dob) {  
41:      this.dob = dob;  
42:    }  
43:    
44:    public List<Address> getAddresses() {  
45:      return addresses;  
46:    }  
47:    
48:    public void setAddresses(List<Address> addresses) {  
49:      this.addresses = addresses;  
50:    }  
51:    
52:    @Override  
53:    public String toString() {  
54:      ...  
55:    }  
56:    
57:    @Override  
58:    public boolean equals(Object o) {  
59:      ...  
60:    }  
61:    
62:    @Override  
63:    public int hashCode() {  
64:      ...  
65:    }  
66:  }  
67:    
68:    
69:    
70:  public class Address {  
71:    
72:    private String line1;  
73:    private String line2;  
74:    private String line3;  
75:    private String town;  
76:    private String county;  
77:    private String postcode;  
78:    private String countryCode;  
79:    
80:    public Address() {  
81:    }  
82:    
83:    public String getLine1() {  
84:      return line1;  
85:    }  
86:    
87:    public void setLine1(String line1) {  
88:      this.line1 = line1;  
89:    }  
90:    
91:    public String getLine2() {  
92:      return line2;  
93:    }  
94:    
95:    public void setLine2(String line2) {  
96:      this.line2 = line2;  
97:    }  
98:    
99:    public String getLine3() {  
100:      return line3;  
101:    }  
102:    
103:    public void setLine3(String line3) {  
104:      this.line3 = line3;  
105:    }  
106:    
107:    public String getTown() {  
108:      return town;  
109:    }  
110:    
111:    public void setTown(String town) {  
112:      this.town = town;  
113:    }  
114:    
115:    public String getCounty() {  
116:      return county;  
117:    }  
118:    
119:    public void setCounty(String county) {  
120:      this.county = county;  
121:    }  
122:    
123:    public String getPostcode() {  
124:      return postcode;  
125:    }  
126:    
127:    public void setPostcode(String postcode) {  
128:      this.postcode = postcode;  
129:    }  
130:    
131:    public String getCountryCode() {  
132:      return countryCode;  
133:    }  
134:    
135:    public void setCountryCode(String countryCode) {  
136:      this.countryCode = countryCode;  
137:    }  
138:    
139:    @Override  
140:    public String toString() {  
141:      ...  
142:    }  
143:    
144:    @Override  
145:    public boolean equals(Object o) {  
146:      ...  
147:    }  
148:    
149:    @Override  
150:    public int hashCode() {  
151:      ...  
152:    }  
153:  }  
154:    

Then, we define some services on top of the domain model that obtain, use and update the domain objects to implement the functionality required by the business:

1:  public class PersonService {  
2:    
3:    private final PersonRepository repository;  
4:    
5:    public PersonService() {  
6:      repository = new PersonRepository();  
7:    }  
8:    
9:    public Long getPersonId(String surname, String forename) {  
10:      return repository.findPerson(surname, forename);  
11:    }  
12:    
13:    public Person getPerson(Long personId) {  
14:      return repository.getPerson(personId);  
15:    }  
16:    
17:    public void addAddress(Long personId, Address newAddress) {  
18:      Person person = repository.getPerson(personId);  
19:    
20:      List<Address> addresses = person.getAddresses();  
21:      if ( addresses == null ) {  
22:        addresses = new ArrayList<Address>();  
23:        person.setAddresses(addresses);  
24:      }  
25:      addresses.add(newAddress);  
26:    }  
27:    
28:    public void makeDefaultAddress(Long personId, Address defaultAddress) {  
29:      Person person = repository.getPerson(personId);  
30:    
31:      List<Address> addresses = person.getAddresses();  
32:      if ( addresses == null || !addresses.contains(defaultAddress) ) {  
33:        throw new IllegalArgumentException();  
34:      }  
35:    
36:      // Default address is always the first address in the list  
37:      addresses.remove(defaultAddress);  
38:      addresses.add(0, defaultAddress);  
39:    }  
40:  }  
41:    
42:    
43:  public class MailShotService {  
44:    
45:    public void sendMailShot(Person person, Long mailShotId) {  
46:      List<Address> addresses = person.getAddresses();  
47:      if ( addresses == null || addresses.isEmpty() ) {  
48:        // No mailshot can be sent  
49:        return;  
50:      }  
51:    
52:      Address sendTo = addresses.get(0);  
53:    
54:      // Code here to locate the mailshot and call the printing routine!  
55:    }  
56:  }  
57:    

The above code is overly simplistic (and somewhat contrived), but it demonstrates a key problem with this approach, namely that the encapsulation of the addresses property is broken. Specifically:

1) The fact that addresses are stored as a list is exposed to the service layer (and beyond). In fact, the PersonService is even responsible for creating the list instance. Changing the way addresses are stored in Person would mandate changing all the services (and perhaps controllers, pages and so on) that work with Person objects.

2) The knowledge that the first address in the list is the default address has escaped the domain model into the service layer. In particular there are two different services that both contain this knowledge. Should we want to change this approach we have to change and test code in two places (or more likely we change it in one place, forget the other and then wonder why our app behaves inconsistently).

Now, many proponents of the anaemic domain approach will tell you that the above problems can be avoided by correctly implementing your service layers. For example, only one service class is ever used to deal with Person. Any other services, controllers or whatever that need to access Person must use this service to do so. For example, the PersonService could have a new method: getDefaultAddress which would be called by the MailShotService. However, in my experience this never works for the following reasons:

1) Unless your developers are INCREDIBLY disciplined then this approach will always be violated. It's right before a deadline and a developer needs to access the default address from some controller in the system. Will they do all the work to inject the PersonService or will they just pull the first element off the address list? Most likely the second, and as soon as it's been done once then you can guarantee that that code will at some time be reused as a template for other code and the problem just proliferates from there. In 15 years I have never seen an anaemic domain pattern where this hasn't happened.

2) You end up with the higher level services and the controllers all having to inject large numbers of other services in order to get anything done. This results in a tighter coupling of the system and significantly increases the complexity of unit and integration testing (unit: need to define many mocks; component: need to pull in almost the whole system to test just one component). In every case I've seen, the anaemic domain pattern done in this way results in a small handful of controllers or services that pull in almost every other service in the system, which makes them really difficult to test and even more difficult to modify.

In my humble opinion, the anaemic domain model should be considered one of the most destructive anti-patterns of our time. It breaks the concept of good object oriented design and encapsulation and leads to service layers (and above) that become difficult to maintain and overly complex.

Rich Domain Models

An alternative is the rich domain model, where we attempt to encapsulate as much information about the domain inside the actual domain classes. We then expose these rich objects to higher levels, which can utilise the domain objects directly with less need for services containing arbitrary domain and business logic.

Looking at our RICH Address and Person objects:

1:  public class Person {  
2:    
3:    private Long id;  
4:    private String forename;  
5:    private String surname;  
6:    private Date dob;  
7:    private final List<Address> addresses;  
8:    
9:    public Person(final String forename, final String surname, final Date dob) {  
10:      this.forename = forename;  
11:      this.surname = surname;  
12:      this.dob = new Date(dob.getTime());  
13:      this.addresses = new ArrayList<Address>();  
14:    }  
15:    
16:    public Long getId() {  
17:      return id;  
18:    }  
19:    
20:    public void setId(final Long id) {  
21:      this.id = id;  
22:    }  
23:    
24:    public String getForename() {  
25:      return forename;  
26:    }  
27:    
28:    public void setForename(String forename) {  
29:      this.forename = forename;  
30:    }  
31:    
32:    public String getSurname() {  
33:      return surname;  
34:    }  
35:    
36:    public void setSurname(String surname) {  
37:      this.surname = surname;  
38:    }  
39:    
40:    public Date getDob() {  
41:      return new Date(dob.getTime());  
42:    }  
43:    
44:    public void setDob(Date dob) {  
45:      this.dob = new Date(dob.getTime());  
46:    }  
47:    
48:    public void addAddress(final Address address) {  
49:      addresses.add(address);  
50:    }  
51:    
52:    public void removeAddress(final Address address) {  
53:      addresses.remove(address);  
54:    }  
55:    
56:    public Collection<Address> getAllAddresses() {  
57:      return Collections.unmodifiableCollection(addresses);  
58:    }  
59:    
60:    public void makeDefaultAddress(final Address address) {  
61:      if ( !addresses.contains(address) ) {  
62:        throw new IllegalArgumentException();  
63:      }  
64:    
65:      addresses.remove(address);  
66:      addresses.add(0, address);  
67:    }  
68:    
69:    public Address getDefaultAddress() {  
70:      if ( addresses.isEmpty() ) throw new IllegalStateException();  
71:      else return addresses.get(0);  
72:    }  
73:    
74:    @Override  
75:    public String toString() {  
76:      ...  
77:    }  
78:    
79:    @Override  
80:    public boolean equals(Object o) {  
81:      ...  
82:    }  
83:    
84:    @Override  
85:    public int hashCode() {  
86:      ...  
87:    }  
88:  }  
89:    
90:    
91:  public class Address {  
92:    
93:    private final String line2;  
94:    private final String line1;  
95:    private final String line3;  
96:    private final String town;  
97:    private final String county;  
98:    private final Postcode postcode;  
99:    private final Country country;  
100:    
101:    public Address(final String line1, final String line2, final String line3,  
102:            final String town, final String county, final Postcode postcode,  
103:            final Country country) {  
104:      this.line1 = line1;  
105:      this.line2 = line2;  
106:      this.line3 = line3;  
107:      this.town = town;  
108:      this.county = county;  
109:      this.postcode = postcode;  
110:      this.country = country;  
111:    }  
112:    
113:    public String getLine1() {  
114:      return line1;  
115:    }  
116:    
117:    public String getLine2() {  
118:      return line2;  
119:    }  
120:    
121:    public String getLine3() {  
122:      return line3;  
123:    }  
124:    
125:    public String getTown() {  
126:      return town;  
127:    }  
128:    
129:    public String getCounty() {  
130:      return county;  
131:    }  
132:    
133:    public Postcode getPostcode() {  
134:      return postcode;  
135:    }  
136:    
137:    public Country getCountry() {  
138:      return country;  
139:    }  
140:    
141:    @Override  
142:    public String toString() {  
143:      ...  
144:    }  
145:    
146:    @Override  
147:    public boolean equals(final Object o) {  
148:      ...  
149:    }  
150:    
151:    @Override  
152:    public int hashCode() {  
153:      ...  
154:    }  
155:  }  
156:    
157:    
158:    

From the above you can see a couple of significant changes. Firstly, I've made as much of the data as possible immutable. In particular I've made addresses immutable and thus to change an address you have to remove the old one and insert a new one.  This stops any users of the domain model from making changes to objects that should always be managed within the domain. Secondly, the addresses field is fully encapsulated. Users of the domain model know none of its implementation detail and they cannot manipulate the contents of the underlying collection as this is never exposed in a mutable form.

Additionally, I've added specific types for Postcode and Country which will encapsulate all the conversion and validation logic for converting between user entered Strings and their actual meaning - logic that would normally be in a controller or service in an anaemic domain model.

This approach greatly simplifies the services layer. The PersonService needs only to provide methods to get the Person and the MailShotService can just call a simple get method on the Person object:

1:  public class PersonService {  
2:    
3:    private final PersonRepository repository;  
4:    
5:    public PersonService() {  
6:      repository = new PersonRepository();  
7:    }  
8:    
9:    public Long getPersonId(final String surname, final String forename) {  
10:      return repository.findPerson(surname, forename);  
11:    }  
12:    
13:    public Person getPerson(final Long personId) {  
14:      return repository.getPerson(personId);  
15:    }  
16:  }  
17:    
18:    
19:  public class MailShotService {  
20:    
21:    public void sendMailShot(final Person person, final Long mailShotId) {  
22:      Address sendTo = person.getDefaultAddress();  
23:    
24:      // Code here to locate the mailshot and call the printing routine!  
25:    }  
26:  }  
27:    

However, this is not the end of the story as there are still improvements to be made. In particular, exposing the ability to change a domain object in a layer above the services still is problematical: needs session in view pattern; may result in multiple places needing modification if a new cross-cutting concern is required (e.g. audit changes). So, it still has some of the weaknesses of the anaemic model.

We can therefore refine the domain objects even further through introduction of an interface to represent the public face of our main domain entities that expose only the accessor functionality:

1:    
2:  public interface Person {  
3:    
4:    Long getId();  
5:    
6:    String getForename();  
7:    
8:    String getSurname();  
9:    
10:    Date getDob();  
11:    
12:    Collection<Address> getAllAddresses();  
13:    
14:    Address getDefaultAddress();  
15:  }  
16:    
17:    
18:  public class BasicPerson implements Person {  
19:    
20:    ...  
21:  }  
22:    

Now we can return a Person instance that clients of the domain object can use to access the domain details, but they cannot mutate them via this interface. Then, I can update the PersonService to contain all the methods for modifying Person instances:

1:  public class PersonService {  
2:    
3:    private final PersonRepository repository;  
4:    
5:    public PersonService() {  
6:      repository = new PersonRepository();  
7:    }  
8:    
9:    public Long getPersonId(final String surname, final String forename) {  
10:      return repository.findPerson(surname, forename);  
11:    }  
12:    
13:    public Person getPerson(final Long personId) {  
14:      return repository.getPerson(personId);  
15:    }  
16:    
17:    public void modifyPerson(final Long personId, final String forename,  
18:                 final String surname, final Date dob) {  
19:      BasicPerson person = repository.getBasicPerson(personId);  
20:      person.setForename(forename);  
21:      person.setSurname(surname);  
22:      person.setDob(dob);  
23:    }  
24:    
25:    public void addAddress(final Long personId, final Address newAddress) {  
26:      BasicPerson person = repository.getBasicPerson(personId);  
27:      person.addAddress(newAddress);  
28:    }  
29:    
30:    public void removeAddress(final Long personId, final Address newAddress) {  
31:      BasicPerson person = repository.getBasicPerson(personId);  
32:      person.removeAddress(newAddress);  
33:    }  
34:    
35:    public void makeDefaultAddress(final Long personId, final Address defaultAddress) {  
36:      BasicPerson person = repository.getBasicPerson(personId);  
37:      person.makeDefaultAddress(defaultAddress);  
38:    }  
39:  }  
40:    

Finally, I make the constructor of the BasicPerson protected and add a factory so that there is now only one place that Person instance can be created (I also did the same for address, but it's pretty simple so I haven't shown it here):

1:  public class PersonFactory {  
2:    
3:    private final PersonRepository repository;  
4:    
5:    public PersonFactory() {  
6:      repository = new PersonRepository();  
7:    }  
8:    
9:    public Person create(final String forename, final String surname, final Date dob) {  
10:      BasicPerson person = new BasicPerson(forename, surname, dob);  
11:      repository.add(person);  
12:      return person;  
13:    }  
14:  }  
15:    

Thus, I have clean, well encapsulated domain objects that don't leak details of their internal implementation to the outside world. Wherever possible data has been made immutable to avoid accidental change. Any clients of the domain model can access its state via the exposed interface, but only the service can modify this state - thus making a single location for adding cross-cutting concerns (such as audit). I can therefore be certain that any code using my domain model will not be dependent on implementation details and will not be impacted by changes (provided I ensure the Person interface contract doesn't change).

You just don't get these benefits from an anaemic model without taking incredibly great care and ensuring that every developer who uses your code in the future also takes the same level of care. With a rich, well encapsulated domain model you protect yourself and those who use your code in the future by preventing the bad usage patterns from ever happening (even accidentally).