For some time now I have been wanting to share some of our own experiences with the different agile development methodologies we have used during the past few years. This post will be about the realities "on the ground", unfiltered and hopefully be another piece in the puzzle to make agile methodologies become further accepted in the corporate business world.

We have worked with different agile development methodologies in the past. We've never been religious about implementing everything these frameworks recommend but rather chose what we felt really worked. We started to work with ExtremeProgramming (XP) in Seattle in 2001 at Amazon.com . I had the honour to manage the web-development technologies platform team there, which had just started using XP shortly before I arrived in the States. More recently we added parts of Scrum to our mix and have been applying it since the beginning of 2007.

I get surprised time and again that not many people seem to be aware of the concept of agile software development. Even worse, when people have heard about it they don't seem to have had very good experiences. There seems to be suppliers out there who use it to sell their services without really knowing what agile development entails. When a new client believes that we are no different, it can be very frustrating.

Let's do a reality check. Based on my personal experience, the application of agile methodologies delivers results much more consistent with the customer’s expectations than waterfall methodologies. All while increasing overall quality and lowering the likelihood of breaking stuff when introducing changes and additions to existing functionality. There is no doubt that projects with agile teams don't always succeed. But what is the percentage and overall customer satisfaction of agile-driven projects? And how does it compare with standard waterfall methodology?

I strongly feel that when it comes to building web and software applications, the only way to allow a company to increase innovation, but not downtime and bugs, is to become agile. This will mean a change in internal company culture. Implementing agile teams is much more a change management project than anything else. All of a sudden, business owners and software developers have to meet and talk. Instead of (often not so useful) business and technical specification documents that can become longer than a standard-length novel, internal customers have to slice and dice up their big ideas into many little, more digestible pieces, which will in turn be prioritised against all other internal and external demands by their software development team.

After an initial phase-in time for the business analysts that now have to create compelling and easy to understand so-called user-stories that are only "reminders for conversations" with other people within the company, the prioritisation and waiting in line until a project is started can be difficult to get used to. Although, when compared with the world before, there are no more false "Yes I understand what you want", "Yes, I read your specification document" or "Yes, we'll get it done next week, promise". Instead, the software development team should have become much more customer-centric and service-orientated. True, they may not promise to get it done immediately, but the more they learn how much they can get done in a certain duration of time (agile really helps development teams in tracking their time and estimations) the happier they will be to give deadlines for project deliveries. And those deadlines will stand the test much better I am sure than deadlines before the "dawn of agile".

With time, the internal customers of the software development team will understand that they actually get what they wanted in the first place. More and better communication will lead to a teaming-up of the technical and business teams. There will be no side-picking any longer but an understanding that both "sides" need to work together to create the best results.

All of this may sound a bit utopian to the ears of someone not accustomed to working with real agile teams. I recall very vividly an experience not so long ago with an outsourcing company that we had hired to do some product-development. Stupidly, we sent them specification documents. Didn't meet with them on a daily, sometimes not even weekly, basis by Skype. Had heated exchanges by e-mail on what we "actually meant to say“ in a certain paragraph of our specification document. And ended up receiving results far below our expectations. Why did we not do what we preach to our clients when consulting with them? Well, I guess any organisation needs to learn. We did!
Now contrast this with a new day, a new team and an agile development framework. Daily meetings to catch up on what was done the previous day, what will be done the following day and any blockers they have encountered that we might need to get out of their way (processes, tools, etc). We are getting what we want, are able to test and code-review what they build. And most importantly, everyone just seems darn happy to work with everyone else. So maybe not so utopian after all. I swear to never go back to the twilight zone of waterfall project-management.

Last but not least: A senior level buy-in - as is true with any other larger change management project in any organisation - is absolutely mandatory for the overall health of the transformation from pre-historic to 21st century development methodologies. There is no way that a team manager all by himself will be able to push through all he needs and make everyone happy at the same time without an approval from the highest ranks within the organisation. If such a buy-in exists though, there is no doubt in my mind that overall happiness and productivity will increase as a direct result from the change in software development practice.