From “waterfall” to “agile”, a bit of history.

In the 1960s, software development began to play an increasingly important role in the world, especially in the United States Department of Defense or the DoD (Department Of Defense). Software development for large systems was still in its infancy and even then software development was considered chaotic and difficult to control. The software community had to be disciplined!

The DoD, therefore, started the initiative to come up with a standard with which they could execute verifiable software projects and with which they could also set requirements for suppliers. The DoD appointed someone to draw up this standard way of software development. This person had little knowledge of software development and therefore started a literature study. Soon he came across a paper by the famous computer scientist Winston Royce. This 1970 paper called Managing The Development Of Large Software Systems is considered to be the original “waterfall” paper. 

In this paper, Winston Royce does indeed describe in the first pages a process that involves working in stages from idea to delivery. He continues with the observation that this linear approach is very risky and that you have to go through the entire process from analysis to production at least TWO times.

Despite the advice in the paper, the DoD has chosen to do development right the first time. To ensure that things can go right the first time, it was decided to build in a formal acceptance step after each phase, in which the requesting party must sign for agreement. The most famous standard that the DoD has produced from this is the so-called MIL-STD (Military Standard) standard, which consists of a large collection of books. The waterfall approach became the standard in software development and is still widely used today.

Consequences of the Waterfall approach

The waterfall approach means that project risks remain very high into the later phases. We have the following risks to take into account:

• Cost: can we implement the project at an acceptable cost and within an acceptable period of time?
• Business: is it indeed the solution to our problem?
• Technology: can we implement it technically?
• Knowledge: can the people currently working on it implement it?

These risks remain high for a long time because it is only clear during the implementation phase what the progress is. Once the test phase arrives, the risks become a real problem. In all previous stages, progress is measured by documents and other abstractions. All reports are usually green until testing. It often turns out that things do not work well, functionality is intended differently or new insights have been obtained, so that the solution from a while ago is no longer the right one.

When the system is finally put into production, it remains to be seen whether the customers will indeed use and buy it. And that while 100% of the investment has been made!

Not everyone is exposed to these risks

The outlined risks are mainly for the requesting party: the customer. This can be a good model for the supplying party! Each phase is concluded with a signed document. Costs are charged for each phase. In case the customer wants to make changes, that is of course possible. Every change goes through a Change Control Board in which approval takes place. However, a change is usually not without cost. It even happens that most turnover within a waterfall project is obtained from the “change requests”.

Agile development

The absolute core of agile development is the reduction of risks. Everything that has arisen around agile is all aimed at reducing the risks of development and innovation as early and as well as possible.

In agile development, for example, with a project of 12 months, delivery can be made every month to production. This does not require an entire investment, but an investment risk of 1/12 of the total. After 1 month you really know where you stand as a project. After all, you have delivered a real product that is ready for production. The delivery to production allows you to receive feedback from your customers. Is what you delivered indeed what the customer expects? Does it indeed solve the customer’s problem? Are the estimates of the number of interested customers correct?

Answers to these questions provide you with relevant information at an early stage. This allows you to make better decisions at a time when there is maximum time available. For example, if more customers are interested than expected, you can decide to invest more and get the product to market faster. If the customer is not completely satisfied with the result, you can immediately discover what would be a good solution. If progress is below expectations, you can, for example, decide to add more people or reduce the scope.

Agile manifesto

The thought leaders in software product development came together at the beginning of this century to construct a response to the waterfall process. All attendees were successful with an iterative way of working, such as Scrum, DSDM or Feature Driven Development. Although the details of these approaches differ, they have decided to capture the core of all these approaches, which they all agreed on. The result is the Agile Manifesto. 

The agile manifesto says that they are discovering better ways of software development by putting it into practice and by helping others apply it. It should no longer be the case that the theorists determine how things should be handled in practice. In practice, we find out what works and the academic world is there to study us and explain why it works. We mainly listen to people who actually do software development in practice!

Individuals & interactions over processes and tools

Software is made most successful by people who go through the entire process and work in an environment of trust and respect. An environment in which people have excellent interactions and relevant information is free and transparent. Processes and tools are less important to success than good people and excellent interaction.

Working software over comprehensive documentation

Creating value can only happen after the software has been delivered. The focus should, therefore, be on working software. While documentation is valuable for success, working software is conditional. Working software is also a way to measure progress and validate assumptions.

Customer collaboration over contract negotiation

The goal is to create value for the customer. We do this by working very closely with the customer during development. When the customer gains new insights, we process them as quickly and effectively as possible. The formal contractual process is important, but success depends even more on the fact that you can work effectively with your customer to realize maximum value.

Responding to change over following a plan

You will learn a lot during the project. At the start of the project, you know the least. Early and frequent validation of assumptions provides new knowledge that we welcome and use. The plan is important to create a common direction but to be successful, it is much more important you can effectively deal with changes and new insights.