Agile and Iterative when properly implemented can be a powerful tool. I have been involved in many projects that implemented a partial Scrum methodology and a few that were 90% of the way there.
Iterative Development
Who has not been on some project which had feature creep extend the release date for months even years? If there is one mantra of iterative development it is release early, release often. This intended to stave off so many of the longer-term errors that are made in Software Development Life Cycles (SDLC). The idea is that by releasing working software frequently that software can not only be reviewed by a client or internal team but also can be reviewed by the team for architectural, design and interface concerns. This should both make clients happy and prevent developers or development groups from getting so far off the same page that a robust round of integration refactoring (further extending deadlines) is required.
Agile Development
If iterative development is release early and often, agile development is expect change early and often. Again who has not worked on some release, have it ready to go, only to have management come down and declare that there is another must have feature that must be in the software which has nothing to do with the feature set that has been worked on. Part of the agile process is to expect this and to engage the client, sales group or internal management to be a part of the day to day operations so that any changes or feature requests (especially of the must have variety) are captured early and then integrated into the next iteration. The agile and iterative methods (esp. XP and Scrum) both seek to protect their developers from immediate change so as to prevent ‘context switching’ where a developer cannot stick with code long enough to achieve anything meaningful due to a lack of ability to get into ‘flow’ mode. [DeMarco]
Scrum
The following are a few of the attributes of Scrum that really speak to me as a means to a more productive team:
Planning
Whether story creation, backlog grooming or estimation, without some planning the team will not be set up for success. Some like to say that “in Agile there are no requirements” – this is not true. It’s that the requirements are flexible and allowed to change.
Scrum Standup
Effective way to get everyone on the same page and to prevent divergent development and resolve blockers quickly. Psychologically good way to keep people honest (ahem). Nice way to keep team unity and lines of communications open that might have more difficulty being established in a less frequent forum.
Uninterrupted Focus
Scrum mandates that management is not empowered to alter feature sets or tasking during the three week Scrum cycle. Again this establishes accountability and self direction through the elimination of ‘context switching’ whereby a developer cycles through a series of deliverables and meetings never being able to enter into ‘flow’ [DeMarco].
Scrum Master Role
The Scrum Master is more a coach then a project manager. This allows the scrum master to adopt a relationship with the developers that more reflects that of a team with a leader then workers with a manager. It is the role of the Scrum Master (coach) to ensure the highest productivity (or performance) of his developers (players).
Demo
The demo is the team’s time to shine – on a regular basis. It keeps moral up and product and engineering on the same page. If you are not looking forward to demoing the software you built in the last couple of weeks you are not doing Agile right!
Retrospective
A time to grieve and complement. The retro is what keeps the team’s relationship healthy. Just like with a spouse or child, if you keep your thoughts to yourself all the time and your thoughts only drive your actions behind a curtain, the net result will be distrust and missed opportunities. The retro is what keeps the relationship between the team strong as well as providing an opportunity for improvement. Issues raised in the retro can be addressed rather than ignored. Lessons learned can apply to everyone no just the single developer who experienced the pain that the lesson was born out of.
Timeboxing
Why Timeboxing?
One short answer is again client & developer morale. One of the great irritations in software development is either waiting for a release if you are a client or waiting for users to see your stuff as a developer. I mean, after all, one of the reasons we are engineers and not say investment bankers is that we desire to create things which are used in the world (fancy financial derivative products notwithstanding).
A client that receives a steady reliable stream of product improvements is more apt to feel comfortable with the software team than if they are repeatedly told (just wait for this next release – it will blow your socks off). Even if the next release with everything but the kitchen sink is delivered often a trust in timelines has been so broken that the political situation has become hostile.
Timeboxing ensures a steady flow of productive features and steady results that everyone can point to. A bird in the hand is worth two in the bush and all that. On top of this regular time boxing and short release cycles ensures that features that are quick and easy to develop get in the users hands as fast as possible rather than being held up by that must have feature which takes 10x as long. Further, as software developers are not mind readers, early incremental feature release ensures that the development and client groups are on the same page.
Timebox Length
Timeboxes are usually most effective when they have a length of three to six weeks [Larman]. This gives sufficient time for a design phase (if used) as well as the generation of any associated work products complimentary but tangential to the code itself (e.g. JavaDoc or as built architectural diagrams).
Feature Reduction not Extended Deadlines
From Larman:
If it eventually appears that the chosen requests (the scope) for the iteration can’t be met within the timebox, then rather than slip the iteration end date, the scope is reduced (pacing lower priority requests back on the wish-list) so that the partial, growing system always ends in a stable and tested state on the original planned iteration end date.
This is a really big point and the one that requires the most moral courage on the part of the iteration leader. You can’t extend dates. The date is real and nothing can change it. I have worked on projects that both suffered from both missed dates and angry clients as well as fake dates and angry developers. The dates should be sacrosanct and form a sort of contract between the developers and the client. Whatever is in the release can change but the client and the developers can know that the release will go out.
Larman additionally notes that:
It is important that timeboxing is not used to pressure developers to work linger hours to meet the soon-coming deadline. If the normal pace of work is insufficient, do less.
Also key, undoubtedly code quality goes down and bug rates go up when people start working 12 hour days and forgo personal life. The rush to get the job done invariably introduces quick fixes and hacks. If these are right before a release it is unlikely that they will get caught and fixed at that time. An additional tenant of Agile and Iterative Development is that bugs and design flaws caught early are on an order of magnitude cheaper to fix then down the road.
Summary
Writing now in 2019 it has been a long time (10 years) since I have worked on a project that was not using some form of agile. Sadly, few of these projects gained the acceleration from Agile that is possibile. In all cases this was due to the Agile process not being implemented properly or egregious violations of the Agile principles.
However some of the projects, and more specifically, at certain phases of some of the projects Agile performed beautifully and the team amazed itself with what it could accomplish.