The Best Way to Apply BDD

I am currently in the process of rewriting a simple e-commerce application for one of my customers. The new version is implemented using Scala and the excellent Lift framework and is a rewrite of a six year old struts/JSP version. The core application is largely identical but I am adding a wide range of new and improved administration functions.

Typically I build a simple, well understood application such as this using ordinary TDD principles to create unit tests and drive out the code. I'd also usually add some fairly basic UI tests using something like Selenium or even just HtmlUnit to validate the front-end. However, I like to use these sorts of projects to explore better ways of doing things. I've therefore decided to use a Behaviour Driven Development approach using Aslak Hellesøy's excellent Cucumber framework. (I'll post more details on installing and using Cucumber and cuke4duke with sbt once I get it all figured out!)

In this post I want to explore some of my thinking about exactly how and where to use BDD within the project and the places where I want to integrate the Cucumber features and scenarios.

What To Use BDD For

I am a strong advocate of the TDD approach, and I intend to follow this within this development for the low level details (i.e. unit and component tests). I'm therefore looking at using BDD as a way of capturing the higher-level business requirements and acceptance criteria. My aim is to use BDD and Cucumber to ensure that the application I am building actually does what the business expects it to. Lower level tests will still be written using ScalaTest to ensure the code that I am writing does what I expect it to do and also to drive out the architectural and implementation details.

Where To Integrate BDD

Given this positioning for BDD, where I have been struggling is the correct level in my application to target the Cucumber step definitions (step definitions are the code that actually executes the scenario and conditions described by the behaviour). The following sections describe the different options I have considered and advantages and disadvantages of each:

BDD at the User Interface - In this approach the step definitions are implemented to execute against the running web application. This is usually achieved using a technology such as Selenium or HtmlUnit. Each scenario loads web pages, clicks links, fills and submits forms and makes assertions about the HTML documents and associated resources that are returned.

Advanages:

• The behaviour is asserted across the complete application.
• The BDD approach can be used to drive the development of the UI as well as the application logic.

Disadvanages:

• You can only execute the feature steps against a built and deployed application - which may make development slower.
• A web-based UI is usually the least stable part of a web application and thus feature steps may be broken as the UI is tweaked, making it appear that behaviour is being lost. Step definitions may become quite fragile.
• Using BDD at this level may not fit well when user experience or visual designers are being used as this often results in rapid changing of the UI
• Generally testing at this level requires a lot of boilerplate code to be written - which makes wringing BDD step definitions more complex.
• Testing of web-based user interface may be better suited to a suite of UI tests that can focus just on verifying the UI display and interaction.
  

BDD at the Web Protocol - In this approach the step definitions are written to directly exercise the web protocol used to interface to the application. For example, the steps make HTTP calls to the application and assert that the returned results are correct and contain the expected content.

Advanages:

• Does not need to worry about the complexities of web pages, javascript and so on that are perhaps better tested in a dedicated suite of UI tests.
• Very well suited to RESTful or other web services that return structured data instead of HTML documents.

Disadvanages:

• You can still only execute the feature steps against a built and deployed application
• Building complex sequences of requests to simulate thinks such Ajax may be more complex than testing against the UI.
• Asserting against returned HTML content may be more complex than using assertions in a framework like Selenium at the UI layer.
• Building and testing at this layer requires that you also support a suite of UI tests.
• For a web application, the interface at the HTTP layer may change quite frequently, requiring frequent changes to the step definitions. This will be particularly common in an Ajax heavy application.

BDD at the Controllers - In this approach, we ignore the UI and view parts of the application. Instead we wire up the whole application from the controllers/snippets layer down. Step definitions are then written that invoke the controllers/snippets with pre-defined requests and assert that the response action and data that would be used to generate a view is correct.

Advanages:

• Features can be tested as part of the test phase of a build - no need to deploy.
• Testing at this level tends to be more stable and less fragile as the application emerges.
• It is usually much simpler to invoke and assert at a code level rather than a UI level

Disadvanages:

• A good set of testing features is required by the chosen web framework, so that requests and responses can be easily simulated and asserted against.
• Tests at this level tend to spend a significant portion of logic dedicated to the interactions with the UI rather than asserting against business rules.
• A suite of UI tests is required in order to test the UI and these will also generally exercise the same controllers/snippets in order to drive the UI.

BDD at the Services - In this final approach we test behaviour at the level directly below the controller/snippet. This is usually testing against service level code, but may also require writing some step definitions against domain level objects. As this level we are much more interested in testing the business behaviour of the application rather than the behaviour of how the application interacts with the user.

Advanages:

• Features can be tested as part of the test phase of a build - no need to deploy.
• Code at this level will usually be more stable than at the higher levels, thus making the tests less brittle.
• Step definitions at this level will usually be simpler to write and maintain than those written against the UI or controllers.
• We are testing against actual business rules rather than how a user might interact with the application.
• We require less testing specific tools and framework support.

Disadvanages:

• Testing at this level exercises less of the application flow than the tests at higher levels.
• A suite of tests is additionally required to validate the UI and the controllers to ensure that the user interaction with the system calls the services with the correct data in the correct order and correctly displays the results.
• The business is more likely expecting the behaviour of how they interact with the application to be verified.
• Developers must be VERY disciplined to ensure that no business rules, logic or behaviour is implemented in the controllers layer.

So, Where From Here?

After looking at all the possible places to use BDD and Cucumber, I have to draw the conclusion that there is no outright winner. It feels to me that the best mix is a combination of step definitions that test behaviour at the service layer (to verify business rules and behaviour) plus another set that test behaviour of the UI. By combining both of these approaches it should be possible to cover the application sufficiently to have confidence that its behaviour is correct.

This then leads me on to think about the best way to create the feature descriptions of the behaviour. Do we create a single description of the desired behaviour and then write two step definitions - one for the service layer and one for the UI? Or do we write a feature definition describing the required behaviour for the business services and then have an alternative behaviour specific for the user interaction with the system?

I'm still unsure of the correct answers to these questions. Anyone out there got any thoughts or experience? In the mean time I'm going to build my application trying both approaches and see which works out best.

Scala Lift Off London - Day 2

Day 2 of Scala Lift Off London started with some quick-fire presentations on a number of interesting subjects:

• OSGi and Scala
• Lifty
• Minesweeper Problem

Lifty especially looks like a great tool for faster creation of Lift apps using SBT. Certainly something I'm going to be installing.

Scala Test Frameworks and DSLs

The first session I attended was an group discussion about some of the test frameworks for Scala. We looked at both Specs and ScalaTest and compared their approaches and the differences in their DSL offerings. We then looked at ScalaCheck as an alternative solution for testing. Especially useful when you need to generate a sequence of values to test against.

Finally we took DSLs even further by looking at test narratives for making tests clearer. For example you can write direct in the test case:

def test() = {

     Alive cell dies when there are 3 neighbours

}

Is implemented by a DSL such that Scala converts it into the following series of chained method calls:

Alive.cell(dies).when(there).are(3).neighbours

A very interesting approach.

Monds, Functional Concepts, Type Concepts

I decided to have a go at hosting my own session to try to get some clarifications on a number of topics that have been bugging me for a while. I'm coming from a Java background and thus the functional elements of Scala along with its much more complex type system are two areas where I have had the most difficult transformation.

The aim of the session was to get those people who wanted clarifications on topics to post them on a whiteboard and then get members from the audience who were experienced in these areas to explain what they mean and how to use them in 5-10 minutes each. The session started well with some good audience participation. After a while it then dropped down to just Kevin Wright and Jon Pretty doing most of the work (thanks guys!).

During the hour (and the extra bit that we ran over into lunchtime) we covered:

• Monads
• Partial Functions
• Currying and Partially Applied Functions
• Monoids and Type Classes
• Implicit Conversions
• Self Types
• Covariance and Contravariance

I learnt a huge amount from this session and a lot of the stuff I was only partially clear on now makes much more sense. I'm pleased I proposed the session as it was exactly what I was looking for.

Minesweeper

The third session of the day was a great talk by Kevin Wright (@thecoda) about implementing the Minesweeper game in Scala and Lift. An excellent example of how to write really great Scala and provide an elegant solution to the problem. Very interesting to see how this was done with only immutable structures and how ajax and Lift were combined in the front-end implementation.

I'd certainly recommend visiting Kevin's github page to explore this code in more detail.

Type Classes and the Cake Pattern

Another great presentation, this time by Jon Pretty. The first part was a more detail look at Type Classes and the Monoid pattern. This got pretty in-depth, but Jon explained it well. I've since built up an example of using Type Classes to solve a real world problem that I encountered. I'll post a description next week.

The second part of the session was the Cake pattern. This is a great approach to building componentised Scala applications as an alternative to dependency injection solutions  like Spring or Guice. I'm already very familiar with the Cake pattern as I've done a fair bit of experimenting with it on my 'discala' experimental project. I'm going to follow up with Jon to perhaps explore some ideas in more detail.

Overall I'd have to say that Scala Lift Off was a great two days. There's a very vibrant Scala community in London and across Europe. I learnt lots and was very privileged to be in the same room as some very talented people. Also a big mention to SkillsMatter for hosting such a great event and providing some fantastic facilities. I'm looking forward to next year already!

Scala Lift Off - Day 1

I've been at Scala Lift Off London today (#scalalol). Had a great time and attended talks by some really knowledgable people. Also met some people doing some really great suff with Scala. The 'unconference' format took some getting used to at the start, but the group created a great agenda and it all came together nicely. Here's my take on today's talks and some ideas that I might propose for tomorrow.


Session 1 - Moving from Java to Scala
An interesting discussion of how a company moved from building apps in Java to their first Scala deployment. An interesting study on how you can move along incrementally, bringing the whole team along as you go. Their progress can be roughly described as:

1. Start writing unit tests for Java code in Scala
2. Use Scala for main code, but still using Java coding styles (i.e. Java without the semi-colons, getters/setters and boilerplate)
3. Starting to make use of Scala features to improve code (e.g. Option instead of null)
4. Using more of the Scala language features (e.g. closures, immutable types etc.)
5. Adopt the additional functional aspects of the language (e.g. map/flatMap)

What I thought was very interesting of their approach of not moving on to the next level of Scala until all of the team were up to speed with the current level. Also, their approach of getting the whole team on board in terms of style was excellent.


This certainly proves it is possible to move a team of Java programmers to the Scala language. I'd certainly love the opportunity to take one of the teams that I work with on this journey on a project over the space of 6-9 months.


Session 2 - Akka
A fantastic session with Jonas Boner (@jboner) presenting the main elements of the Akka framework. The actors model for building scalable concurrent applications really excites me. I can see many cases where things that I'm working on now could be made much simpler, but at the same time more scalable and performant by adopting this model.


I can certainly see some major architectural changes in the future for building large systems. Applications composed of thousands of lightweight actors, all working concurrently and passing messages around has got to be better than the current model of a single thread handling a single request and trying to deal with all the synchronisation issues whenever shared state is involved.


Also very impressed with the supervisor model within Akka, where a supervising actor can manage a number of other actors and control how they get restarted in failure cases and so forth. Also, the distributed actor model looks great and is almost seamless (although deployment might be more challenging!)


I'm certainly looking forward to building some actor based applications. I'll post details as I do.


Session 3 - Experience of using Scala and Akka
Although this was an excellent session, it was probably the least valuable one of my day. It was great to hear about a successful project that used Scala and the Akka framework - especially how quickly and easily it all came together. However, not much new information.


Session 4 - Software Transactional Memory (STM)
Another great session with Jonas Boner, looking at the STM support within Akka. I've played a bit with STM in the past, but combining this with the actor model looks very powerful. It's all very new at the moment, but I can see some great applications for this approach. The session also turned into a great discussion of STM and I learnt a lot of good stuff.


Other Thoughts
Martin Odersky (@odersky) did what looked like a great talk on advanced Scala and implicit conversions. Sadly I missed this as I was in the STM session. Looking forward to watching the video on this one.


Today for me was definitely an Akka day. However, a lot of the other sessions seemed to be on quite advanced or specialised topics. I'm hoping tomorrow to focus more on Scala and Lift, with perhaps some more intermediate and generic sessions. Some suggestions I might propose:

• WTF is a Monad (and other Functional Goodness): I'm coming from a Java background and while I get the functional approach in Scala I still find it fairly challenging to work with. Some real world examples would help
• The Scala Type System
• Building Lift Applications for developers familiar with Java frameworks (e.g. Wicket, Tapestry, JSP, JSF)

Looking forward to tomorrow.