Using Scrum Wisely – Where does Design Fit?

The Scrum development method is a useful set of guidelines to help teams scope, plan, status and demonstrate their work. When done well, it allows teams to get early feedback on the product they are building and provides a reasonably disciplined method of managing changes in scope.

Scrum includes some guidance for planning, estimation, requirements definition, testing and project statusing. For the subject of design, Scrum advocates “design by discovery,” or “emergent design.” That is, as each series of functions is developed, the design emerges. The code is refactored (cleaned up) to stay consistent with current design ideas. There is no specific up front design or analysis phase in Scrum and no specific design guidelines. Agile refers to “just enough design,” but that has many definitions.

In many engineering projects, it is difficult to fully specify a design since the team might not know what will and won’t work until it tries it. In less complex, smaller and self-contained projects, the risk of not having a design before coding might not kill the project since problems that are encountered can be fixed with acceptable cost. However, when more people and project locations are involved, and the cost of fixing design errors in the code is greater than fixing design errors before coding, then consideration should be given to some design activity up front. An emerging design is not necessarily a good one.

In the Agile world, a lot of people debate the “traditional” big up front design over allowing the design to emerge through coding. It is commonly seen as an all-or-nothing choice. There is the rationalization that coding involves thinking, and thinking is design, so coding will suffice for design.

What is missing is the understanding of what design is, why it is done, and how to do it in a way that is useful to the team. Fixing the design process is hard; skipping it is easier.

Mature teams that have developed design practices:

• don’t have debates on whether they are “traditional,” “agile” or “fill-in-the-blank” — they don’t care about the label
• focus their design effort on architecture-level design, higher-risk components, and areas that would be expensive to fix if they had design flaws
• don’t do all of the design up front; they know they can’t
• do design iterations to help them make design decisions that can be made now and identify areas of technical risk that need research
• use prototypes and mockups to feed into the next iteration of the design
• don’t abandon design just because it is hard or less fun than coding
• capture the design so that it can be reviewed for mistakes and referred to later

In this tidbit, we describe some ideas for hitting a balance*.

One of the characteristics of Scrum is that functionality is developed during every sprint (which is typically a 1-4 week period). This focus on developing functionality from the outset of the project can lead the team to skip (or minimize) non-development activities such as requirements analysis, design and system testing. When they are skipped, the result can be the creation of many parts that don’t fit together, or the development of a system that works well on the first release, but cannot easily be changed.

Design is hard to do. When it is historically done poorly (e.g., the design document is too large to manage, unreadable, or drowned out by non-design information) then the baby is often thrown out with the bath water and replaced by declarations such as, “Our code is the design, and any novice can understand our code base of 100,000 non-commented lines of brilliant code.” When Scrum is adopted, which has no explicit design phase, then the jettisoned baby and the bath water are promptly forgotten, until the team hits a large roadblock caused by the lack of design. The pendulum is swung back to previous bad design practices and the cycle repeats. This is by no means an agile-only issue. It has been around for decades.

If you research the topic of “design and Agile” or “design and Scrum,” you will see a very large range of ideas of what design might mean. We found definitions including, “Developing the features in order of risk,” to “Do iteration modeling for a few minutes at the beginning of an iteration,” and advice of “Minimizing, and in some cases eliminating, up-front analysis and design activities saves both time and money.”

What is design?
Here is a short summary of “design.” It isn’t perfect since it is an ambiguous word and there are numerous opinions to choose from.

Design is a map of the system that can be used to:

• clarify and communicate concepts and definitions to other people involved in the project; this can cover architecture and detailed design issues
• identify potential sticky areas that need to be investigated and researched to learn what works and what does not
• find many (but not all) of the errors in a project earlier at less cost than developing the final product
• provide a common reference point for design decisions
• assess the impact of changes

The information that can result from design includes:

• an architecture showing how the main components relate to each other
• interfaces to the user, to other systems and between components
• data definitions stating what data will be stored and where
• how constraints will be handled, such as: design for testing, expansion, security, portability and technology (e.g., PC-based, cloud-based)

Design information can be captured:

• in textual design notes (e.g., a series of one-sentence statements such as, “The server responds every 3 seconds with X output,” “Component Y will handle all system errors,” or “3 databases will be used and synced every 5 minutes”)
• in graphical form (e.g., modeling languages, event tables, flow/timing diagrams)
• as pseudo code – a textual description of what the code will do so that issues and defects can be found and resolved before time is spent developing code
• in one document or split up based on the area being defined (e.g., placed in the header of each code file)

Design documentation is usually a hassle to keep up-to-date when design changes are made in the coding phase. You might consider developing architecture and design notes up front, reviewing them for defects and issues, and then updating changes to the design when the code is complete. The level of design documentation to keep, and where it is kept (e.g., separate document and/or code file header section) can be decided based on maintenance needs, ease-of-learning for new teams and future enhancement plans.

Incorporating design activities
Incorporating design activities into Scrum while maintaining the benefits of Scrum is not difficult. The intent of Scrum is to deliver working code every sprint. This benefit can be maintained by changing the percentage of coding and non-coding activities in each sprint. Instead of each sprint being 100% coding and feature testing, start earlier sprints with a small allocation of coding time and front-load these sprints with other tasks, such as architecture design. For example:

Sprint 1: X% architecture design, 100-X% other sprint activities
Sprint 2: Y% architecture design, 100-Y% other sprint activities
Sprint 3: Z% architecture design, 100-Z% other sprint activities

For example, X = 80, Y = 40, Z = 0.

The earlier sprints can use the coding time for high-risk features, core product components and features that will be needed to support later functionality. You can also add more time to later sprints for system test, performance testing and reliability testing when the time needed for coding decreases.

You can select the percentage allocation for the sprints based on the need for that activity and the risk of not doing it. You might decide that design takes zero percent for low risk projects or features, or that design has a large percentage of time allocated for the first 3 sprints to work on architecture, and much less from sprint 4 onwards.

Summary
Don’t abandon design because it is not called out specifically in Scrum. Don’t get sucked into the near-worthless discussion of “traditional” versus “new.” Determine for yourself what design activities you need and when they will get done. It’s your product and your money!

*Thanks to Venkat, Brian and Alan for their valuable inputs.

For Scrum assistance, see Scrum services

For all Scrum-related articles, see Scrum articles