Automating in Gherkin syntax with Specflow

I’m quite a fan of Gherkin’s Given-When-Then syntax for test cases, so I was pleased to discover Specflow. Specflow is a .NET implementation of Gherkin, with complete integration with Visual Studio.

Steve Sanderson’s blog post describes it most excellently, and is essential reading if you plan on using Specflow. I recommend that you have a little read of it before you read the rest of this post.

The best thing about Specflow is that it works out of the box. This may seem like low standards to some, but anyone who’s had to use any test tool ever will just about fall out of their chair in shock upon reading that. I was bracing myself for a good 4+ hours of pain when I installed it, but I had my first test case up and running within about 20 minutes.

I was almost unreasonably excited when I finished writing this:

Scenario : Create a new client with import access
Given I have created a new client
And my clients have full access to their accounts
When I login as the client
Then I should be able to import a campaign

Why excited? Because when I executed that test case, the first two lines used my product’s API to create a client, and the last two lines used WebAii to verify the result via the browser interface. Specflow, NUnit, .NET, WebAii, MbUnit and Internet Explorer all working in harmony! Which is cool, but that’s not even the most exciting bit. Check out this output:

Given I have created a new client
-> done: ClientStepDefinition.GivenIHaveCreatedANewClient() (0.6s)
And my clients have full access to their accounts
-> done: ClientStepDefinition.GivenMyClientsHaveFullAccessToTheirAccounts() (0.1s)
When I login as the client
-> done: ClientStepDefinition.WhenILoginAsTheClient() (5.2s)
Then I should be able to import a campaign
-> done: ClientStepDefinition.ThenIShouldBeAbleToImportACampaign() (17.8s)

Look! It’s telling me what it’s doing at each step, how long it took, and whether it worked or not! But what about if it fails?

Given I have created a new client
-> done: ClientStepDefinition.GivenIHaveCreatedANewClient() (0.3s)
And my clients have full access to their accounts
-> done: ClientStepDefinition.GivenMyClientsHaveFullAccessToTheirAccounts() (0.0s)
When I login as the client
-> error:
I tried to login with username: ‘clienttest2071812263’, and password ”
But the login failed because:
‘Please enter a password.’
Then I should be able to import a campaign
-> skipped because of previous errors

It tells me what it was trying to do at the time, and then tells me what it was going to try to do next, before it failed. I can tell at a glance what this test case was trying to do and that will cut my maintenance time down considerably. It’s the simple things in life that make me happy.

Three good reasons to start using it
1. Writing tests in this format encourages realistic test verification. If I think about some of the tests that can typically lurk in many an automated test suite, some of them start to look quite silly when written in a plain-English format. For example, how many of you have seen some version of this test?

Scenario : Verify clicking something loads a page
Given I have navigated to the foo page
When I click the bar button
Then the widget image should appear

What value is this test actually adding? What will it tell me if it fails? If the image doesn’t appear, will that really tell me if the page hasn’t loaded? Writing tests in this way make it more obvious when the test mission doesn’t match the verification method.

2. Developers! If you write tests in this way up-front, it will give you a better idea of what you’re aiming to achieve with your implementation task. Plus you’ll have a handy suite of tests to check your code against, so you’ll know when you’re done.

3. Nicer output and clearer test structure makes maintenance easier. And making maintenance quick and easy is one of the biggest challenges in test automation. In addition, it makes the purpose of each test and each test step clear enough so that it should help to prevent test degradation. By test degradation, I mean the phenomenon by which tests can seemingly degrade over months of continuous maintenance, so that they no longer seem to meet their original purpose, until one day somebody looks at the test and wonders “what was this test even trying to do?”

Some Specflow tips
Unfortunately, the documentation for Specflow is incomplete, and the best guide to it is the blog post I’ve linked to above. But it’s pretty easy to figure out how to use it from within Visual Studio. Here are a few things that helped me.

Executing test cases using TestDriven
You can run test cases from within Visual Studio using TestDriven, but there’s a sneaky trick to it. Instead of running it from the *.feature file, you have to run it from the *.feature.cs file. But it’s easy enough from there, just right-click in the section where you can see your test steps and select Run Test(s).

Adding Setup and Teardown
Using NUnit Setup and Teardown methods won’t work here, but there are Specflow versions that you can use instead. In your step definition file, just add the [ BeforeScenario ] and [ AfterScenario ] attributes to your Setup and Teardown methods, respectively. For example:

[ BeforeScenario ]
public void SetUp()
{
// Setup code
}
[ AfterScenario ]
public void TearDown()
{
// Cleanup code
}

How to get started
Steven Sanderson’s blog post should tell you all that you need to know to get started with Specflow.  But I would be remiss if I didn’t also mention that Bruce McLeod has implemented Specflow into his testing stack at testingstax.codeplex.com. If you’re just getting started with WatiN, I’d definitely recommend looking into Bruce’s automation stack.

What do you think of Specflow?
I’ve only been using it for a couple of weeks and clearly it’s made a good impression on me so far. If you’ve already run into some major pitfalls, I’ve love to know about them before I hit them too. Or, if you just have any good tips for using it, I’d be glad to hear them also.

9 thoughts on “Automating in Gherkin syntax with Specflow

  1. Pingback: Tweets that mention Automating in Gherkin syntax with Specflow | Purple Box Testing -- Topsy.com

  2. Rob Lambert

    Trish,

    Thanks for this post. I’ve been looking for something to do some BDD for a while now. :) Shame it doesn’t work with NUnit cos that’s what we’re using heavily, but I’ll spin up a test project and give it a go. It might be worth the transition and rewrite :)

    Thanks
    Rob..

  3. Trish Khoo Post author

    Glad it was helpful, Rob! Actually it does work with NUnit. My org has been using MbUnit heavily but I was able to use it side-by-side with Specflow running NUnit. Haven’t tried putting it into continuous integration yet, but these early results are encouraging!

  4. Rob Lambert

    Trish,

    thanks for responding. Excellent stuff – I’ll be sure to see how we can get it working. We use Selenium at the moment, but I’m sure we can work out a way of getting the whole thing working.

    I’ve been trying to find a “simple” way to bring our user stories directly in to the code base. this could be the ideal solution. All others have required just a little “too” much refactoring of tests.

    Thanks
    Rob..

  5. Nathan Bain

    I was lucky enough to go to a workshop by Gaspar Nagy and Jonas Bandi, 2 of the creators of SpecFlow, at last years Progressive .NET Tutorials. You can see a video of the session on the Skillsmatter website – I am sure you can pick up a few tips there.

    Part 1) http://skillsmatter.com/podcast/agile-testing/driving-an-aspdot-net-mvc-application-outside-in-with-specflow
    Part 2) http://skillsmatter.com/podcast/open-source-dot-net/advanced-topics-of-behavior-driven-development-with-specflow-examples

    Enjoy..

  6. Angus McDonald

    Hi Trish,

    Very interesting post. I’ve now picked up SpecFlow and played with it a bit, and it looks like a great way of capturing business requirements. Much easier to use than Fitnesse.

    I would like to pick your brains about who are great testers in the Sydney marketplace as my company is looking to hire a great tester. Could you please give me a ring on (02) 8064 0945?

    Thanks,
    Angus McDonald

  7. Pingback: SpecFlow Strategy

Leave a Reply