Why automate your test cases?

Where this, maybe even very basic question came from? I talked with my old schoolmate and he had had a hard time to prove to his boss why they should automate some of their test cases. They are doing regression tests that people are executing before every release. To me it seemed like a very simple case, to automate what they were doing.

So why should test cases be automated? Feedback time is the key. With testing you are trying to find (and also trying to avoid) defects as soon as it has been presented to the code. With manual regression testing, this feedback loop is long and regression testing itself is time consuming which causes a problem in test coverage. With automation, you can have bigger set of test executed with shorter time and after creating good Continuous Integration system, there is no need of human interaction for regression test execution.

You can sleep your nights better, with good test automation. Because with it, you know all the time where you are with your Software.

Ps. You can find some Automation testing tools here.

Automation frameworks: Robot framework, Cucumber and Fitnnesse.
Continuous Integration Servers: Hudson and BuildBot

Incentives are a bad substitute for leadership

Managers are using incentives in order to achieve things they think are important instead of good leadership, that is, discussing, interacting and collaborating with people, why they should do like this. Money is a carrot. Problem here is that those bonus targets must be somehow measurable and IF people are trying to get their carrot they are trying to get those meters green, instead of focusing on the root causes. People must feel that they own the problem and be motivated to solve them. Also if problems are discussed, instead of stated, there are probably popping up better ideas than the original one man's idea.

Btw. Good video siding the issue by Daniel Pink

Our show in XP2010 and more

We had quite a good warm up band. Mark Streibeck from Google introduced us their Continuous Integration system and it was amazing! Everybody always committing to same head and it must be green all the time. Pretty mind blowing if you think that they have 1500+ product, 10000 developers and 60M+ test cases.

Anyway. My feeling about our presentation was pretty good. If you were there, please give me feedback. It was first time for me and Ran to write something for conference and it was an extremely good experience. I must continue with this path little bit further, because my opinion is that companies should do more scientific research.

The conference was facilitated greatly, everything from preparation of the conference itself to evening programs, or what can you say that we had Jazz musicians improvising and telling the theory about that (which is very close to pair programing) and later on there was a gig by an extreme metal band called Keep of Kalessin (you should check out this).

So, thank you very much and see ya!

Viking Laws

I was in Trondheim for the XP2010 conference (btw I had a good and educational time, I'll write more details later) and afterwards we visited Oslo with my wife and daughter. I found this extremely neat postcard. It says:

§1 Be brave and aggressive.
Be direct.
Grab all opportunities.
Use varying methods of attack.
Be versatile and agile.
Attack one target at a time.
Don't plan everything in detail.
Use top quality weapons.

§2 Be prepared.
Keep weapons in good condition.
Keep in shape.
Find good battle comrades.
Agree on important points.
Choose one chief.

§3 Be a good merchant.
Find out what the market needs.
Don't promises what you can't keep.
Don't demand overpayment.
Arrange things so that you can return.

§4 Keep the camp in order.
Keep things tidy and organized.
Arrange enjoyable activities which strengthen the group.
Make sure everybody does useful work.
Consult all members of the group for advice.

In the future I wont refer anymore to the Agile Manifesto, the Viking Laws will be the thing ;)

Experience Report in XP2010: Automated Acceptance Testing of High Capacity Network Gateway

I'm going to be in XP2010 and be one of the presenters of our Experience Report. First time for me to publish something. If you are around, please come to say hello :). Here is our abstract:

Abstract.
In this paper we will explore how agile acceptance testing is applied in testing a high capacity network gateway. We will demonstrate how the organisation managed to grow agile acceptance testing testing from two co-located teams to 20+ multi-site team setup and how acceptance test driven development is applied to complex network protocol testing. We will also cover how the initial ideas that we had of agile acceptance testing evolved during product development. At the end of paper we give recommendations to future projects using agile acceptance testing based on feedback that we have collected from our first customer trials.

What Key Performance Indicators (KPI) should Agile Team(s) use?

It would be nice to measure how we are doing and also check, has our newly implemented improvements hit the spot. Metrics are always nice, but (un)fortunately only few of them are usable. Here is a list of those KPIs what I think should be used.

Velocity
How many Story Points(SP) has team consumed during previous iterations and use this information to forecast. Keep your Story Points uncorrupted and here you have good and handy tool. If you are wondering what is SP, you should read Mike Cohn's excellent book Agile Estimating And Planning.

Cycle Time
Cycle Time (or Lead Time), means a time from when a customer request comes into a process to a time when it has actually delivered to a customer. Good way to measure the whole process, not only R&D efficiency. With this, you can check how good your flow is.

Boomerangs
Boomerangs are things, which are coming back to the process after they have declared as done and delivered. Because those things are defects like bugs and misunderstandings in customer requirements, this is a measurement for quality and communication. You can read more about this from Gojko Adzic's blog.

Customer Satisfaction
It's good to check, time to time, what people who are paying our salary think about us :)

Agile Testing vs. Waterfall Testing

We had this small conversation with my colleague, what Agile Testing really is and is there a conceptional difference between these two?

One thing should be common. Both should verify that the customer expectations are satisfied. But when in waterfall way of testing, the testing phase is long and in the beginning of testing condition of software is uncertain, in Agile the emphasis is always on working software, situation where we are each moment should be clear. This is done by using Continuous Integration, with Automated Regression tests.

Finding bugs is one of the main reasons of testing in waterfall model. In Agile, on the other hand, and especially when using Engineering practices like Test Driven Development (TDD) and Acceptance Test Driven Development (ATDD), tests are telling when things are done and bugs are avoided in the first place.

Also iterative way of doing makes planning different. In waterfall, testers have a long period of test planning, but in Agile test planning is made in the same way as everything else, in the last moment (of course not too late :) ) and also it's short term.