Agile Planet

Coding Standards, this week's Art of Agile Development release, is now available.

In our discussion of coding standards, Shane and I say that one of the most important things you may learn from coding standards is just how to disagree constructively. I think we were right on target with that one. Coding standards themselves have very little value, I've come to believe. You only really need them if you're having repeated design disagreements--especially if they occur in the guise of repeated refactoring/reversion cycles.

If you're having those sorts of disagreements, coding standards won't resolve the problem. But discussing coding standards might help. It's the discussion that has value, not the standard. The standard merely serves to record the result. If people use the standard as a club, that's your clue that the discussion failed.

Next Week

I'll post a practice from the Releasing chapter next week. There are only two choices left:

• Version Control
• Continuous Integration

Which one should I post? As usual, make your vote in the comments.

Comments

• Next: Iteration Demo
• Previous: Stand-Up Meetings
• Up: Chapter 6: Collaborating

in 99 words

In team software development, we create a collective work that is greater than any individual could create on his own. Arguing about style gets in the way.

When creating a coding standard, your most important achievement will be learning how to disagree constructively. To succeed, create the minimal set of standards you can live with. Focus on consistency and consensus over perfection. Remember that few decisions are irrevocable in agile development.

Assume your colleagues are professional and well-meaning. If they deviate from the standard, discuss reasons rather than placing blame. No coding standard can substitute for professional judgement.

as haiku

Fists fly, dust billows--
Tomatoes die, peas cry, as
we choose rose's hue.

Rant

Full Text

The following text is excerpted from The Art of Agile Development by James Shore and Shane Warden, published by O'Reilly. Copyright © 2008 the authors. All rights reserved.

Coding Standards

Audience

Programmers

We embrace a joint aesthetic.

Back in the days of the telegraph, as the story goes, telegraph operators could recognize each other on the basis of how they keyed their dots and dashes. Each operator had a unique style, or fist, that experts could recognize easily. Programmers have style, too. We each have our own way of producing code. We refine our style over years until we think it's the most readable, the most compact, or the most informative it can be.

Individual style is great when you're working alone. In team software development, however, the goal is to create a collective work that is greater than any individual could create on his own. Arguing about whose style is best gets in the way; it's easier if we work together in a single style.

XP suggests creating a coding standard: guidelines to which all developers agree to adhere when programming.

Beyond Formatting

I once led a team of four programmers who had widely differing approaches to formatting. When we discussed coding standards, I catalogued three different approaches to braces and tabs. Each approach had its own vigorous defender. I didn't want us to get bogged down in arguments, so I said that people could use whatever brace style they wanted.

The result was predictable: we had three different approaches to formatting in our code. I even saw two different ways of indenting within a single, short method.

You know what surprised me? It wasn't that bad. Sure, the layout was ugly, and I would have preferred consistency, but the code was still readable. In the end, the rest of our coding standard mattered much more than formatting.

We all agreed that clearly named variables and short methods were important. We agreed to use assertions to make our code fail fast, not to optimize without measurements, and never to pass null references between objects. We agreed on how we should and shouldn't handle exceptions, what to do about debugging code, and when and where to log events. These standards helped us far more than a consistent formatting style would have because each one had a concrete benefit. Perhaps that's why we were able to agree on them when we couldn't agree on formatting styles.

Don't get me wrong: a consistent formatting standard is good. If you can agree on one, do! However, when you're putting together your coding standard, don't fall into the trap of arguing about formatting. There are more important issues.

How to Create a Coding Standard

The most important thing you will learn is how to disagree.

Creating a coding standard is an exercise in building consensus. It may be one of the first things that programmers do as a team. Over time, you'll amend and improve the standards. The most important thing you may learn from creating the coding standard is how to disagree constructively.

To that end, I recommend applying two guidelines:

1. Create the minimal set of standards you can live with.

2. Focus on consistency and consensus over perfection.

Hold your first discussion of coding standards during the first iteration. The project will typically start out with some discussion of stories and vision, then some release planning and iteration planning (see Go! in Chapter 4). After iteration planning, customers and testers will continue working on the release plan. That's a great time for programmers to talk about coding standards.

The best way to start your coding standard is often to select a industry-standard style guide for your language. This will take care of formatting questions and allow you to focus on design-related questions. If you're not sure what it should encompass, starting points include:

• Development practices (start with the practices in Chapter 9 and Chapter 7)

• Tools, keybindings, and IDE

• File and directory layout

• Build conventions

• Error handling and assertions

• Approach to events and logging

• Design conventions (such as how to deal with null references)

Focus on agreements.

Limit your initial discussion to just one hour. Write down what you agree on. If you disagree about something, move on. You can come back to it later.

I like to write each item that we agree upon on a flip chart so we can tape it to a wall in our open workspace. If you can find a neutral observer to take notes (such as your project manager), so much the better.

If you have trouble, take a step back and talk about your goals for the software and the results you would like to see. Agree about these issues first, even if you disagree about specific approaches. You will have many opportunities to improve your standard. Make the most important decisions now, and move on.

Depending on your team, this may be a contentious discussion. In that case, consider bringing in a professional facilitator to redirect the discussion to your team goals when things gets heated. Your HR department might be able to provide someone, or you can use an outside consultant.

Plan to hold another one-hour coding standard meeting a few days later, and another one a few weeks after that. The long break will allow you to learn to work together and to try out your ideas in practice. If there's still disagreement, experiment with one approach or the other, then revisit the issue.

Ally

Retrospectives

Hold these initial meetings as often as they're useful. After that, change the standard at any time. Just stand up, announce your intention to the team, and, if everybody agrees, change the flip chart. Retrospectives are another good time to discuss changes to the coding standard.

The thumb vote is a quick way to check for consensus. Someone asks a question and everyone holds their thumb up (meaning "I agree"), sideways ("I'll go along with the group"), or down ("I disagree and want to explain why").

Over time, some of the items in the standard will become second nature. Cross them off to make room for more important issues. As you work together, you will recognize ways in which new standards can help. Add these new standards to the list in the same way as before, as long as everybody agrees to try them.

No matter what standards you choose, someone will be probably unhappy with some guideline even with a consensus-based approach. You'll probably find some practices jarring and grating at first. Over time, you'll get used to it. Coding standards are, in many ways, an aesthetic choice: it doesn't really matter what the standard is, as long as it's consistent and thoughtful. One of the marks of a professional is the willingness to put aside personal aesthetics for a team aesthetic.

Dealing with Disagreement

It's possible to pressure a dissenter into accepting a coding standard she doesn't agree with, but it's probably not a good idea. Doing so is a good way to create resentment and discord.

Instead, remember that few decisions are irrevocable in agile development; mistakes are opportunities to learn and improve. Ward Cunningham put it well:¹

¹http://en.wikiquote.org/wiki/Ward_Cunningham

It was a turning point in my programming career when I realized that I didn't have to win every argument. I'd be talking about code with someone, and I'd say, "I think the best way to do it is A." And they'd say, "I think the best way to do it is B. I'd say, "Well no, it's really A." And they'd say, "Well, we want to do B." It was a turning point for me when I could say, "Fine. Do B. It's not going to hurt us that much if I'm wrong. It's not going to hurt us that much if I'm right and you do B, because, we can correct mistakes. So [let's] find out if it's a mistake."

Go ahead and leave the contested item out of the standard. Maybe lack of standardization in that area will lead to a mess. If it does, the team will learn from the experience and you can change the standard.

Adhering to the Standard

Allies

Pair Programming

Collective Code Ownership

People make mistakes. Pair programming helps developers catch mistakes and maintain self-discipline. It provides a way to discuss formatting and coding questions not addressed by the guidelines. It's also an excellent way to improve the standard; it's much easier to suggest an improvement when you can talk it over with someone first.

Collective code ownership also helps people adhere to the standard, because many different people will edit the same piece of code. Code tends to settle on the standard as a result.

There are less effective approaches. Some teams use automated tools to check their source code for adherence to the coding standard. Others program their version control system to reformat files upon check-in. I don't like either approach: to me, the latter says that you don't trust people to make good decisions on their own, and the former tends to raise false warnings.

Assume your colleagues are professional and well-meaning.

I've also heard of teams who elevate their coding standards to requirements and punish infractions. The idea of enforcing a coding standard leaves a bad taste in my mouth. Your teammates are presumably professionals who pride themselves on doing good work. No coding standard can substitute for professional judgment. Try not to get too upset when you see people deviating from the standard.

Assume your colleagues are professional and well-meaning. If someone is not following the standard, assume that there's a good reason—even if all the evidence is to the contrary. Your challenge is to find that reason and address it. This approach shows respect for others and this will improve others' respect for you.

Before you do anything, ask yourself whether the coding standard was really a team effort. If everybody agreed to every item, they should have no problem following the standard.

Start by talking with your colleague alone to see if there's a disagreement. Take an attitude of collaborative problem solving: instead of saying "Why aren't you propagating exceptions like we agreed?" ask, "What do you think about the 'propagate exceptions' standard we agreed on? Should we keep it?" Give objections full consideration, raise them with the rest of the team, and consider changing the standard.

If the objector agrees with the standard but isn't applying it, it's possible that the standard isn't appropriate in every situation. Ask about specific cases you've noticed. Again, be collaborative, not confrontational. Say something like, "I think we're on the same page regarding the importance of propagating exceptions. In that case, can you explain what's happening in this method? I don't understand why this code doesn't propagate the exception here."

During this discussion, you may learn that the objector doesn't understand the standard. By this time, you should be in a good situation to discuss the standard and what it means. If he's a junior programmer and needs more help, coordinate with the rest of the team to make sure he gets plenty of pairing time with experienced developers.

There is another possibility for teams new to XP. Switching to XP is a big change and can make people feel like they've lost control. Sometimes they react by picking small things that they refuse to change. An obstinate desire to stick with a particular coding standard, regardless of the wishes of the rest of the team, might be a symptom of this reaction.

In this case, your best solution may be to let the infractions slide for several months. Over time, as team members become more comfortable with the changes in their environment, they'll relax and be more willing to compromise.

Questions

We have legacy code that doesn't fit our standard. Should we fix it?

Leave old code alone if it works and you don't need to read or touch it otherwise. It's expensive and risky to spend a lot of time fixing legacy code upfront. Instead, as you modify and refactor those sections of code, bring them up to the new coding standards. When you fix a bug, add a feature, or improve abstraction and factoring, use the new standards on everything you modify.

You can also use an automated tool to perform large-scale formatting changes. Don't spend too much time on this, but if you can do it easily, you might as well. I prefer to integrate immediately before and after such an operation, because reformatting changes tend to disguise other changes. Be aware that making such large-scale changes can render your version control system's change history much more difficult to read.

Results

When you agree on coding standards and conventions, you improve the maintainability and readability of your code. You can take up different tasks in different subsystems with greater ease. Pair programming moves much more smoothly, and you look for ways to improve the expressability and robustness of your code as you write it.

Contraindications

Don't allow coding standards to become a divisive issue for your team.

Alternatives

Some teams work so well together they don't need an written coding standard; their coding standard is implicit.

If you have a new team, create a written coding standard even if everybody gets along well. New teams often go through an initial honeymoon period in which team members are reluctant to disagree with each other. Eventually, disagreements will come out. It's much better to create a standard before problems escalate.

• Next: Iteration Demo
• Previous: Stand-Up Meetings
• Up: Chapter 6: Collaborating

Comments

• Next: Is XP Right for Us?
• Previous: XP Concepts
• Up: Table of Contents

Full Text

The following text is excerpted from The Art of Agile Development by James Shore and Shane Warden, published by O'Reilly. Copyright © 2008 the authors. All rights reserved.

Adopting XP

"I can see how XP would work for IT projects, but product development is different." —a product development team

"I can see how XP would work for product development, but IT projects are different." —an in-house IT development team

Before adopting XP, you need to decide whether it's appropriate for your situation. Often, people's default reaction to hearing about XP is to say, "Well, of course that works for other teams, but it couldn't possibly work for us."

XP's applicability is based on organizations and people, not types of projects.

Question that assumption. I've helped a wide variety of teams adopt XP: 20-person teams and one-person teams; huge corporations and small startups; shrinkwrap, in-house, and outsourced software vendors; proprietary and open source developers. Through these experiences, I've learned that software teams are more similar than they are different. XP's applicability has far more to do with your organization and the people involved than the type of project you're working on.

• Next: Is XP Right for Us?
• Previous: XP Concepts
• Up: Table of Contents

Comments

• Next: Test-Driven Development
• Previous: Incremental Requirements
• Up: Chapter 9: Developing

New Information

Although I still fully embrace using the Describe, Demonstrate, Develop process described for this practice, I've found that using tools such as Fit to automate those examples results in unacceptable maintenance burdens. The value is in the collaborative Describe, Demonstrate, Develop process, not the automated tests. I recommend ignoring the final section of this practice, "Automating the Examples." Everything else is still correct.

I describe the reasons for this change of heart in The Problems With Acceptance Testing. In Alternatives to Acceptance Testing, I explain what to do instead.

Full Text

The following text is excerpted from The Art of Agile Development by James Shore and Shane Warden, published by O'Reilly. Copyright © 2008 the authors. All rights reserved.

Customer Tests

Audience

Whole Team

We implement tricky domain concepts correctly.

Ally

Ubiquitous Language

Customers have specialized expertise, or domain knowledge, that programmers don't have. Some areas of the application—what programmers call domain rules—require this expertise. You need to make sure that the programmers understand the domain rules well enough to code them properly in the application. Customer tests help customers communicate their expertise.

Ally

Ten-Minute Build

Don't worry; this isn't as complicated as it sounds. Customer tests are really just examples. Your programmers turn them into automated tests, which they then use to check that they've implemented the domain rules correctly. Once the tests are passing, the programmers will include them in their ten-minute build, which will inform the programmers if they ever do anything to break the tests.

To create customer tests, follow the Describe, Demonstrate, Develop process outlined in the next section. Use this process during the iteration in which you develop the corresponding stories.

Describe

Customer tests are for communication.

At the beginning of the iteration, look at your stories and decide whether there are any aspects that programmers might misunderstand. You don't need to provide examples for everything. Customer tests are for communication, not for proving that the software works. (See No Bugs in Chapter 7).

For example, if one of your stories is "Allow invoice deleting", you don't need to explain how invoices are deleted. Programmers understand what it means to delete something. However, you might need examples that show when it's okay to delete an invoice, especially if there are complicated rules to ensure that invoices aren't deleted inappropriately.

If you're not sure what the programmers might misunderstand, ask. Be careful, though; when business experts and programmers first sit down to create customer unit tests, both groups are often surprised by the extent of existing misunderstandings.

Once you've identified potential misunderstandings, gather the team at a whiteboard and summarize the story in question. Briefly describe how the story should work and the rules you're going to provide examples for. It's okay to take questions, but don't get stuck on this step.

For example, if you decided to discuss invoice deletion, you might say:

Customer: One of the stories in this iteration is to add support for deleting invoices. In addition to the screen mockups we've given you, we felt some customer tests would be appropriate. Deleting invoices isn't as simple as it appears because we have to maintain an audit trail.

There are a bunch of rules around this issue. I'll get into the details in a moment, but the basic rule is that it's okay to delete invoices that haven't been sent to customers because presumably that kind of invoice was a mistake. Once an invoice has been sent to a customer, it can only be deleted by a manager. Even then, we have to save a copy for auditing purposes.

Programmer: When an invoice hasn't been sent and gets deleted, is it audited?

Customer: No—in that case, it's just deleted. I'll provide some examples in a moment.

Demonstrate

After a brief discussion of the rules, provide concrete examples that illustrate the scenario. Tables are often the most natural way to describe this information, but you don't need to worry about formatting. Just get the examples on the whiteboard.

Customer (continued): As an example, this invoice hasn't been sent to customers, so an Account Rep can delete it.

Sent	User	Okay to delete
N	Account Rep	Y

In fact, anybody can delete it—CSR's, managers, and admins.

Sent	User	Okay to delete
N	CSR	Y
N	Manager	Y
N	Admin	Y

But once it's sent, only managers and admins can delete it, and even then it's audited.

Sent	User	Okay to delete
Y	Account Rep	N
Y	CSR	N
Y	Manager	Audited
Y	Admin	Audited

Also, it's not a simple case of whether something has been "sent" or not. "Sent" actually means one of several conditions. If you've done anything that could have resulted in a customer seeing the invoice, we consider it "sent". Now only a manager or admin can delete it.

Sent	User	Okay to delete
Printed	Account Rep	N
Exported	Account Rep	N
Posted to web	Account Rep	N
Emailed	Account Rep	N

As you provide examples, be completely specific. It's tempting to create generic examples, such as "this invoice hasn't been sent to customers, so anybody can delete it", but those get confusing quickly and programmers can't automate them. Provide specifics. "This invoice hasn't been sent to customers, so an account rep can delete it." This will require you to create more examples—that's a good thing.

Your discussion probably won't be as smooth and clean as this example. As you discuss business rules, you'll jump back and forth between describing the rules and demonstrating them with examples. You'll probably discover special cases that you hadn't considered. In some cases, you might even discover whole new categories of rules that you need customer tests for.

One particularly effective way to work is to elaborate on a theme. Start by discussing the most basic case and providing a few examples. Next, describe a special case or additional detail and provide a few more examples. Continue in this way, working from simplest to most complicated, until you have described all aspects of the rule.

You don't need to show all possible examples. Remember, the purpose here is to communicate, not to exhaustively test the application. You only need enough examples to show the differences in the rules. A handful of examples per case is usually enough, and sometimes just one or two is sufficient.

Develop

When you've covered enough ground, document your discussion so the programmers can start working on implementing your rules. This is also a good time to evaluate whether the examples are in a format that works well for automated testing. If not, discuss alternatives with the programmers.

Programmer: Okay, I think we understand what's going on here. We'd like to change your third set of examples, though—the ones where you say "Y" for Sent. Our invoices don't have a "Sent" property. We'll calculate that from the other properties you mentioned. Is it okay if we use "Emailed" instead?

Customer: Yeah, that's fine. Anything that sends it works for that example.

Don't formalize your examples too soon. While you're brainstorming, it's often easiest to work on the whiteboard. Wait until you've worked out all the examples around a particular business rule (or part of a business rule) before formalizing it. This will help you focus on the business rule rather than formatting details.

In some cases, you may discover that you have more examples and rules to discuss than you realized. The act of creating specific examples often reveals scenarios you hadn't considered. Testers are particularly good at finding these. If you have a lot of issues to discuss, consider letting some or all of the programmers get started on the examples you have while you figure out the rest of the details.

Don't use customer tests as a substitute for test-driven development.

Ally

Test-Driven Development

Programmers, once you have some examples, you can start implementing the code using normal test-driven development. Don't use the customers' tests as a substitute for writing your own tests. Although it's possible to drive your development with customer tests—in fact, this can feel quite natural and productive—the tests don't provide the fine-grained support that TDD does. Over time, you'll discover holes in your implementation and regression suite. Instead, pick a business rule, implement it with TDD, then confirm that the associated customer tests pass.

Focus on Business Rules

One of the most common mistakes in creating customer tests is describing what happens in the user interface rather than providing examples of business rules. For example, to show that an account rep must not delete a mailed invoice, you might make the mistake of writing this:

1. Log in as an account rep

2. Create new invoice

3. Enter data

4. Save invoice

5. Email invoice to customer

6. Check if invoice can be deleted (should be "no")

What happened to the core idea? It's too hard to see. Compare that to the previous approach:

When an invoice has been emailed, an account rep may not delete it... or, as you might draw it on the whiteboard:

Sent	User	Okay to delete
Emailed	Account Rep	N

Good examples focus on the essence of your rules. Rather than imagining how those rules might work in the application, just think about what the rules are. If you weren't creating an application at all, how would you describe those rules to a colleague? Talk about things rather than actions. Sometimes it helps to think in terms of a template: "When (scenario X), then (scenario Y)."

It takes a bit of practice to think this way, but the results are worth it. The tests become more compact, easier to maintain, and (when implemented correctly) faster to run.

Ask Customers to Lead

Remember the "Customer" in "Customer Tests".

Team members, watch out for a common pitfall in customer testing: no customers! Some teams have programmers and testers do all the work of customer testing, and some teams don't involve their customer at all. In others, a customer is present only as a mute observer. Don't forget the "customer" in "customer tests." The purpose of these activities to bring the customers' knowledge and perspective to the team's work. If programmers or testers take the reins, you've lost that benefit and missed the point.

In some cases, customers may not be willing to take the lead. Programmers and testers may be able to solve this problem by asking the customers for their help. When programmers need domain expertise, they can ask customers to join the team as they discuss examples. One particularly effective technique is to ask for an explanation of a business rule, pretend to be confused, then hand a customer the whiteboard marker and ask him to draw an example on the board.

If customers won't participate in customer testing at all, this may indicate a problem with your relationship with the customers. Ask your mentor (see "Find a Mentor" in Chapter 2) to help you troubleshoot the situation.

Testers' Role

Testers play an important support role in customer testing. Although the customers should lead the effort, they benefit from testers' technical expertise and ability to imagine diverse scenarios. While customers should generate the initial examples, testers should suggest scenarios that customers don't think of.

On the other hand, testers should be careful not to try to cover every possible scenario. The goal of the exercise is to help the team understand the customers' perspective, not to exhaustively test the application.

Automating the Examples

Programmers may use any tool they like to turn the customers' examples into automated tests. Ward Cunningham's Fit (Framework for Integrated Test)¹, is specifically designed for this purpose. It allows you to use HTML to mix descriptions and tables, just as in my invoice auditing example, then runs the tables through programmer-created fixtures to execute the tests.

¹Available free at http://fit.c2.com/.

See http://fit.c2.com/ or [Mugridge & Cunningham] for details about using Fit. It's available in several languages, including Java, .NET, Python, Perl, and C++.

You may be interested in FitNesse at http://fitnesse.org/, a variant of Fit. FitNesse is a complete IDE for Fit that uses a Wiki for editing and running tests. (Fit is a command-line tool and works with anything that produces tables in HTML.)

Ally

Exploratory Testing

Fit is a great tool for customer tests because it allows customers to review, run, and even expand on their own tests. Although programmers have to write the fixtures, customers can easily add to or modify existing tables to check an idea. Testers can also modify the tests as an aid to exploratory testing. Because the tests are written in HTML, they can use any HTML editor to modify the tests, including Microsoft Word.

Programmers, don't make Fit too complicated. It's a deceptively simple tool. Your fixtures should work like unit tests, focusing on just a few domain objects. For example, the invoice auditing example would use a custom ColumnFixture. Each column in the table corresponds to a variable or method in the fixture. The code is almost trivial (see Example 9-1).

Example 9-1. Example fixture (C#)

  public class InvoiceAuditingFixture : ColumnFixture
  {
      public InvoiceStatus Sent;
      public UserRole User;
      public Permission OkayToDelete() {
          InvoiceAuditer auditer = new InvoiceAuditer(User, InvoiceStatus)
          return auditer.DeletePermission;
      }
  }

Ally

Ubiquitous Language

Using Fit in this way requires a ubiquitous language and good design. A dedicated domain layer with Whole Value objects² works best. Without it, you may have to write end-to-end tests, with all the challenges that entails. If you have trouble using Fit, talk to your mentor about whether your design needs work.

²See Domain-Driven Design [Evans] for a discussion of domain layers and http://c2.com/ppr/checks.html#1 [Cunningham] for information about Whole Value.

I often see programmers try to make a complete library of generic fixtures so that no one need write another fixture. That misses the point of Fit, which is to segregate customer tests from programmer implementation. If you make generic fixtures, the implementation details will have to go into the tests, which will make them too complicated and obscure the underlying examples.

Most tests can be expressed with a simple ColumnFixture or RowFixture.

Questions

When do programmers run the customer tests?

Ally

Ten-Minute Build

Once the tests are passing, make them a standard part of your ten-minute build. Like programmers' tests, you should fix them immediately if they ever break.

Should we expand the customer tests when we think of a new scenario?

Ally

Slack

Absolutely! Often, the tests will continue to pass. That's good news; leave the new scenario in place to act as documentation for future readers. If the new test doesn't pass, talk with the programmers about whether they can fix it with iteration slack or whether you need a new story.

What about acceptance testing (also called functional testing)?

Ally

No Bugs

Automated acceptance tests tend to be brittle and slow. I've replaced acceptance tests with customer reviews (see "Customer Reviews" later in this chapter) and a variety of other techniques (see "A Little Lie" in Chapter 3).

Results

Ally

Ubiquitous Language

When you use customer tests well, you reduce the number of mistakes in your domain logic. You discuss domain rules in concrete, unambiguous terms and often discover special cases that you hadn't considered. The examples influence the design of the code and help promote a ubiquitous language. When written well, the customer tests run quickly and require no more maintenance than unit tests do.

Contraindications

Ally

Test-Driven Development

Don't use customer tests as a substitute for test-driven development. Customer tests are a tool to help communicate challenging business rules, not a comprehensive automated testing tool. In particular, Fit doesn't work well as a test scripting tool—it doesn't have variables, loops, or subroutines. (Some people have attempted to add these things to Fit, but it's not pretty.) Real programming tools, such as xUnit or Watir, are better for test scripting.

Allies

Whole Team

Sit Together

In addition, customer tests require domain experts. The real value of the process is the conversation that explores and exposes the customers' business requirements and domain knowledge. If your customers are unavailable, those conversations won't happen.

Finally, because Fit tests are written in HTML, Fit carries more of a maintenance burden than xUnit frameworks do. Automated refactorings won't extend to your Fit tests. To keep your maintenance costs down, avoiding creating customer tests for every business rule. Focus on the tests that will help improve programmer understanding, and avoid further maintenance costs by refactoring your customer tests regularly. Similar stories will have similar tests: consolidate your tests whenever you have the opportunity.

Alternatives

Some teams have testers, not customers, write customer tests. Although this introduces another barrier between the customers' knowledge and the programmers' code, I have seen it succeed. It may be your best choice when customers aren't readily available.

Customer tests don't have to use Fit or FitNesse. Theoretically, you can write them in any testing tool, including xUnit, although I haven't seen anybody do this.

Further Reading

Fit for Developing Software [Mugridge & Cunningham] is the definitive reference for Fit.

"Agile Requirements" [Shore 2005a], online at http://www.jamesshore.com/Blog/Agile-Requirements.html, is a series of essays about agile requirements, customer testing, and Fit.

• Next: Test-Driven Development
• Previous: Incremental Requirements
• Up: Chapter 9: Developing

Comments

• Next: Estimating
• Previous: Slack
• Up: Chapter 8: Planning

Stories

Audience

Whole Team

We plan our work in small, customer-centric pieces.

Stories are not requirements.

Stories may be the most misunderstood entity in all of XP. They're not requirements. They're not use cases. They're not even narratives. They're much simpler than that.

Ally

Release Planning

Stories are for planning. They're simple, one-or-two line descriptions of work the team should produce. Alistair Cockburn calls them "promissory notes for future conversation"¹. Everything that stakeholders want the team to produce should have a story:

¹http://c2.com/cgi/wiki?UserStory, accessed 30 May 2007.

• "Warehouse inventory report"

• "Full-screen demo option for job fair"

• "TPS report for upcoming investor dog-and-pony show"

• "Customizable corporate branding on user login screen"

Some people find that using a template helps them write good stories. For example, [Cohn] recommends using the template "As a (role) I want to (goal) so that (reason)".

Ally

Incremental Requirements

This isn't enough detail for the team to implement and release working software, nor is that the intent of stories. A story is a placeholder for a detailed discussion about requirements. Customers are responsible for having the requirements details available when the rest of the team needs them.

Although stories are short, they still have two important characteristics.

1. Stories represent customer value and are written in the customers' terminology. (The best stories are actually written by customers.) They describe an end-result that the customer values, not implementation details.

2. Stories have clear completion criteria. Customers can describe an objective test that would allow programmers to tell when they've successfully implemented the story.

The following examples are not stories:

• "Automate integration build" does not represent customer value.

• "Deploy to staging server outside of the firewall" describes implementation details rather than an end-result, and it doesn't use customer terminology. "Provide demo that customer review board can use" would be better.

• "Improve performance" has no clear completion criteria. Similarly, "Make it fast enough for my boss to be happy" lacks objective completion criteria. "Searches complete within one second" is better.

Story Cards

Write stories on index cards.

I prefer 3x5-inch cards because they fit into my pocket.

This isn't the result of some strange Ludditian urge on the part of XP's creators; it's a deliberate choice based on the strengths of the medium. You see, physical cards have one feature that no conglomeration of pixels has: they're tactile. You can pick them up and move them around. This gives them power.

Ally

Release Planning

During release planning, customers and stakeholders gather around a big table to select stories for the next release. It's a difficult process of balancing competing desires. Index cards help prevent these disputes by visually showing priorities, making the scope of the work more clear, and directing conflicts towards the plan rather than personalities.

Ally

Informative Workspace

Story cards also form an essential part of an informative workspace. After the planning meeting, move the cards to the release planning board—a big, six-foot whiteboard, placed prominently in the team's open workspace (see Figure). You can post hundreds of cards and still see them all clearly. For each iteration, place the story cards to finish during the iteration on the iteration planning board—another big whiteboard; see Figure—and move them around to indicate your status and progress. Both of these boards are clearly visible throughout the team room and constantly broadcast information to the team.

Use index cards to be more responsive to stakeholder suggestions.

Index cards also help you be responsive to stakeholders. When you talk with a stakeholder and she has a suggestion, invite her to write it down on a blank index card. I always carry cards with me for precisely this purpose. Afterwards, take the stakeholder and her card to the product manager. They can walk over to the release planning board and discuss the story's place in the overall vision. Again, physical cards focus the discussion on relative priorities rather than contentious "approved / disapproved" decisions.

Ally

Estimating

If the product manager and stakeholder decide to add the story to the release plan, they can take it to the programmers right away. A brief discussion allows the programmers to estimate the story. Developers write their estimate—and possibly a few notes—on the card, and then the stakeholder and product manager place the card into the release plan.

Index cards are simpler and more effective than computerized tools.

Physical index cards enable these ways of working in a very easy and natural way that's surprisingly difficult to replicate with computerized tools. Although you can use software, index cards just work better: they're easier to set up and manipulate, make it easier to see trends and the big picture, and allow you to change your process with no configuration or programming.

Most people are skeptical about the value of index cards at first, so if you feel that way, you're not alone. The best way to evaluate the value of physical story cards is to try them for a few months, then decide whether to keep them.

Customer-Centricity

Ally

The Planning Game

Stories need to be customer-centric. Write them from the on-site customers' point of view and make sure that they provide something that customers care about. On-site customers are in charge of priorities in the planning game, so if a story has no customer value, your customers won't—and shouldn't—include it in the plan.

A good way to ensure that your stories are customer-centric is to ask your customers to write the stories themselves.

One practical result of customer-centric stories is that you won't have stories for technical issues. There should be no "Create a build script" story, for example—customers wouldn't know how to prioritize it. Although programmers can tell customers where the technical stories belong in the plan, that disrupts the balance of power over the scope of the project and can leave customers feeling disenfranchised.

Instead, include any technical considerations in the estimate for each story. If a story requires that the programmers create or update a build script, for example, include that cost when estimating for that story.

Ally

Incremental Design And Architecture

Including technical costs in stories, rather than having dedicated technical stories, requires incremental design and architecture.

Customer-centric stories aren't necessarily always valuable to the end-user, but they should always be valuable to the on-site customers. For example, a story to produce a trade-show demo doesn't help end-users, but it helps the customers sell the product.

Splitting and Combining Stories

Stories can start at any size, but it is difficult to estimate stories that are too large or too small. Split large stories; combine small ones.

Select story sizes such that you complete four to ten each iteration.

The right size for a story depends on your velocity. You should be able to complete four to ten stories in each iteration. Split and combine stories to reach this goal. For example, a team with a velocity of ten days per iteration might split stories with estimates of more than two days, and combine stories that are less than half a day.

Combining stories is easy. Take several similar stories, staple their cards together, and write your new estimate on the front.

Splitting stories is more difficult, because it tempts you away from vertical stripes and releasable stories. It's easiest to just create a new story for each step in the previous story. Unfortunately, this approach leads to story clumps. Instead, consider the essence of the story. Peel away all of the other issues and write them as new stories. [Cohn] has an excellent chapter on how to do this in his book Agile Estimating and Planning. He summarizes various options for splitting stories:

• Split large stories along the boundaries of the data supported by the story.

• Split large stories based on the operations that are performed within the story.

• Split large stories into separate CRUD [Create, Read, Update, Delete] operations.

• Consider removing cross-cutting concerns (such as security, logging, error handling, and so on) and creating two versions of the story: one with and one without support for the cross-cutting concern.

• Consider splitting a large story by separating the functional and nonfunctional [performance, stability, scalability, and so forth] aspects into separate stories.

• Separate a large story into smaller stories if the smaller stories have different priorities.

Special Stories

Most stories will add new capabilities to your software, but any action that requires the team's time but is not a part of normal work needs a story.

Documentation Stories

Ally

Documentation

XP teams need very little documentation to do their work (see Documentation), but you may need the team to produce documentation for other reasons. Create documentation stories just like any other: make them customer-centric and make sure you can identify specific completion criteria. An example of a documentation story is "Produce user manual".

"Non-Functional" Stories

Ally

Performance Optimization

Performance, scalability, and stability—so-called non-functional requirements—should be scheduled with stories too. Be sure that these stories have precise completion criteria. See Performance Optimization for more.

Bug Stories

Allies

No Bugs

"Done Done"

Ideally, your team will fix bugs as soon as they find them, before declaring a story as "done done". Nobody's perfect, though, and you will miss some bugs. Schedule these bugs with story cards: "Fix multiple-user editing bug". Schedule them as soon as possible to keep your code clean and reduce your need for bug-tracking software.

Bug stories can be difficult to estimate. Often, the biggest timesink in debugging is figuring out what's wrong, and you usually can't estimate how long that will take. Instead, provide a time-boxed estimate. "We'll spend up to a day investigating this bug. If we haven't fixed it by then, we'll schedule another story."

Spike Stories

Ally

Spike Solutions

Sometimes programmers won't be able to estimate a story because they don't know enough about the technology required to implement the story. In this case, create a story to research that technology. An example of a research story is "Figure out how to estimate 'Send HTML' story". Programmers will often use a spike solution (see Spike Solutions) to research the technology, so these sorts of stories are often called spike stories.

Word these stories in terms of the goal, not the research that needs to be done. When programmers work on a research story, they only need to do enough work to make their estimate for the real story. They shouldn't try to figure out all of the details or solve the entire problem.

Programmers can usually estimate how long it will take to research a technology even if they don't know the technology in question. If they can't even estimate how long the research will take, timebox the story as you do with bug stories. I find that a day is plenty of time for most spike stories, and half a day is sufficient for most.

Estimating

Other than spike stories, you normally don't need to schedule time for the programmers to estimate stories—you can just ask them for an estimate at any time. It's part of the overhead of the iteration, as are support requests and other unscheduled interruptions. If your programmers feel that estimating is too much of an interruption, try putting new story cards in a pile for the programmers to estimate when it's convenient.

Sometimes you'll have a large number of new stories to estimate. In this case, it might be worth creating a story card for estimating those stories.

Meetings

Like estimating, most meetings are part of the normal overhead of the iteration. If you have an unusual time commitment, such as training or an day-long off-site, you can reserve time for it with a story.

Architecture, Design, Refactoring, and Technical Infrastructure

Ally

Incremental Design And Architecture

Don't create stories for technical details. Technical tasks are part of the cost of implementing stories and should be part of the estimates. Use incremental design and architecture to break large technical requirements into small pieces that you can implement incrementally.

Questions

Are index cards really more effective than computerized tools? What about reporting?

See our discussion of reporting (Reporting) for more information.

What about backups? Won't the cards get lost?

If you exercise reasonable care, you're unlikely to lose cards. The only time I've seen a team lose their cards was when they created them and then left them in an ignored pile somewhere on their boss's desk for six months.

That said, there's always the possibility of fire or other disaster. If the risk of losing cards is too great for you, consider taking a digital photo of the release planning board every week or so. An eight megapixel camera has sufficient resolution to capture a six-foot whiteboard and all of the detail on the cards.

Why do you recommend sizing stories so that we can complete four to ten stories per iteration?

If you only have a few stories in an iteration, it's harder to see that you're making progress, which increases the risk that you won't see problems in time to correct them. In addition, if something goes wrong and you can't finish a story, it will represent a large percentage of your work. Too many stories, on the other hand, increases your planning and estimating burden. Each story becomes smaller, making it harder to see the big picture.

An average of six stories per iteration leads to a three-month release plan (the maximum I recommend) consisting of 78 stories. That's a nice number. It gives you flexibility in planning without overwhelming you with details.

Our customers understand programming. Can we create programmer-centric technical stories?

It's much more difficult to create customer-centric stories than programmer-centric stories, so it's tempting to find excuses for avoiding them. "Our customers don't mind if we have programmer-centric stories" is one such excuse. Try to avoid it.

Even if your customers really do have the ability to prioritize programmer-centric stories, customer-centric stories lead to better plans. Remember, your goal is to create stories that allow you to release the software at any time. Programmer-centric stories usually don't have that property.

If your customers are programmers—if you're writing software for programmers, such as a library or framework—then your stories should use a programmer-centric language (see Ubiquitous Language). Even so, they should reflect your customers' perspective. Create stories about your customers' needs, not your plans to fulfill their needs.

How can we encourage stakeholders to use stories for requesting features?

When a stakeholder asks you for a feature, take out an index card and invite him to write it down so it can be scheduled. For electronic requests, a customer should follow up, either by speaking to the requester in person or creating the story card himself.

If stakeholders refuse to use stories, the product manager can manage this relationship by providing stakeholders with what they want to see and translating stakeholders wishes into stories for the team.

Results

When you use stories well, the on-site customers understand all of the work that they approve and schedule. You work on small, manageable, and independent pieces and can deliver complete features frequently. The project always represents the best possible value to the customer at any point in time.

Contraindications

Stories are no replacement for requirements. You need another way of getting details, whether through expert customers on-site (the XP way) or a requirements document (the traditional way).

Ally

Incremental Design And Architecture

Be very cautious of using customer-centric stories without also using most of the XP development practices. Customer-centric stories depend on the ability to implement infrastructure incrementally with incremental design and architecture. Without this ability, you're likely to incur greater technical debt.

Ally

Sit Together

Physical index cards are only appropriate if the team sits together, or at least has a common area in which they congregate. Experienced distributed teams often keep physical index cards at the main site and copy the cards into the electronic system. This is an administrative headache, but for these teams, the benefits of physical cards make the added work worthwhile.

Some organizations are skittish about using informal planning tools. If important members of your organization require a formal Gantt chart, you may need to provide it. Your project manager can help you make this decision. As with a distributed team, you may find it valuable to use physical cards as well, then duplicate the information into the tool.

Alternatives

For teams that don't use stories, the main distinction between stories and the line items in most plans is that stories are customer-centric. If you can't use customer-centric stories for some reason, customers cannot participate effectively in the planning game. This will eliminate one of its primary benefits: the ability to create better plans by blending information from both customers and programmers. Unfortunately, no alternative practice will help.

Another distinctive feature of stories is the use of index cards. Physical cards offer many benefits over electronic tools, but you can still use an electronic tool if necessary. Some teams track their stories using spreadsheets and others use dedicated agile planning tools. None of these approaches, however, provide the benefits of physical index cards.

Further Reading

Agile Estimating and Planning, by [Cohn], discusses options for splitting stories in Chapter 12.

• Next: Estimating
• Previous: Slack
• Up: Chapter 8: Planning

Comments

• Next: Continuous Integration
• Previous: Version Control
• Up: Chapter 7: Releasing

in 99 words

Build, test, and deploy your entire product at any time with the push of a button.

Your build should be comprehensive but not complex. Make it compile source code, run tests, configure registry settings, initialize database schemas, set up web servers, launch processes, build installers, and deploy. Your IDE won't do all this, so learn to use a dedicated build tool. Make sure your build works when disconnected from the network, too.

Builds should be fast. If not, look at your tests. End-to-end integration tests are the typical culprit. Replace them with faster, more maintainable unit tests.

Commentary

Living in the Punch Card Era

Full Text

The following text is excerpted from The Art of Agile Development by James Shore and Shane Warden, published by O'Reilly. Copyright © 2008 the authors. All rights reserved.

Ten-Minute Build

Audience

Programmers

We eliminate build and configuration hassles.

Here's an ideal to strive for. Imagine that you've just hired a new programmer. On the programmer's first day, you walk him over to the shiny new computer you just added to your open workspace.

"We've found that keeping everything in version control and having a really great automated build makes us a lot faster," you say. "Here, I'll show you. This computer is new, so it doesn't have any of our stuff on it yet."

You sit down together. "Okay, go ahead and check out the source tree." You walk him through the process and the source tree starts downloading. "This will take a while because we have all of our build tools and libraries in version control, too," you say. "Don't worry—like any good version control system, it brings down changes, so it's only slow the first time. We keep tools and libraries in version control because it allows us to update them easily. Come on, let me show you around the office while it downloads."

After giving him the tour, you come back. "Good, it's finished," you say. "Now, watch this—this is my favorite part. Go to the root of the source tree and type build."

The new programmer complies, then watches as build information flies by. "It's not just building the source code," you explain. "We have a complex application that requires a web server, multiple web services, and several databases. In the past, when we hired a new programmer, he would spend his first couple of weeks just configuring his workstation. Test environments were even worse. We used to idle the whole team for days while we wrestled with problems in the test environment. Even when the environment worked, we all had to share one environment and we couldn't run tests at the same time.

"All that has changed. We've automated all of our setup. Anybody can build and run all of the tests on their own machine, any time they want. I could even disconnect from the network right now and it would keep building. The build script is doing everything: it's configuring a local web server, initializing a local database... everything.

"Ah! It's almost done. It's built the app and configured the services. Now it's running the tests. This part used to be really slow, but we've made it much faster lately by improving our unit tests so we could get rid of our end-to-end tests."

Suddenly, everything stops. The cursor blinks quietly. At the bottom of the console is a message: BUILD SUCCESSFUL.

"That's it," you say proudly. "Everything works. I have so much confidence in our build and tests that I could take this and install it on our production servers today. In fact, we could do that right now, if we wanted to, just by giving our build a different command.

"You're going to enjoy working here." You give the new programmer a friendly smile. "It used to be hell getting even the simplest things done. Now, it's like butter. Everything just works."

Automate Your Build

What if you could build and test your entire product—or create a deployment package—at any time, just by pushing a button? How much easier would that make your life?

Producing a build is often a frustrating and lengthy experience. This frustration can spill over to the rest of your work. "Can we release the software?" "With a few days of work." "Does the software work?" "My piece does, but I can't build everything." "Is the demo ready?" "We ran into a problem with the build—tell everyone to come back in an hour."

Automating your build is one of the easiest ways to improve morale and increase productivity.

Sadly, build automation is easy to overlook in the rush to finish features. If you don't have an automated build, start working on one now. It's one of the easiest things you can do to make your life better.

How to Automate

There are plenty of useful build tools available, depending on your platform and choice of language. If you're using Java, take a look at Ant. In .NET, NAnt and MSBuild are popular. Make is the old standby for C and C++. Perl, Python, and Ruby each have their preferred build tools as well.

Your build should be comprehensive but not complex. In addition to compiling your source code and running tests, it should configure registry settings, initialize database schemas, set up web servers, launch processes—everything you need to build and test your software from scratch without human intervention. Once you get the basics working, add the ability to create a production release, either by creating an install file or actually deploying to servers.

Construct your build so that it provides a single, unambiguous result: SUCCESS or FAILURE. You will run the build dozens of times per day. Manual checks are slow, error-prone, and—after the umpteenth time—seriously annoying.

You should be able to build even when disconnected from the network.

A key component of a successful automated build is the local build. A local build will allow you to build and test at any time without worrying about other people's activities. You'll do this every few minutes in XP, so independence is important. It will also make your builds run faster.

Be cautious of IDEs and other tools that promise to manage your build automatically. Their capability often begins and ends with compiling source code. Instead, take control of your build script. Take the time to understand exactly how and why it works and when to change it. Rather than starting with a pre-made template, you might be better off creating a completely new script. You'll learn more, and you'll be able to eliminate the complexity a generic script adds.

The details are up to you. In my build scripts, I prefer to have all auto-generated content go into a single directory called build/. The output of each major step (such as compiling source code, running tests, collating files into a release, or building a release package) goes into a separate directory under build/. This structure allows me to inspect the results of the build process and—if anything goes wrong—wipe the slate clean and start over.

When to Automate

At the start of the project, in the very first iteration, set up a bare-bones build system. The goal of this first iteration is to produce the simplest possible product that exercises your entire system. That includes delivering a working—if minimal—product to stakeholders.

Use the build script to configure the integration machine. Don't configure it manually.

Because the product is so small and simple at this stage, creating a high-quality automated build is easy. Don't try to cover all of the possible build scenarios you need in the future. Just make sure that you can build, test, and deploy this one simple product—even if it does little more than "Hello, world!" At this stage, deployment might be as simple as creating a .zip file.

Ally

Continuous Integration

Once you have the seed of your build script, it's easy to improve. Every iteration, as you add features that require more out of your build, improve your script. Use your build script every time you integrate. To make sure it stays up-to-date, never configure the integration machine manually. If you find that something needs configuration, modify the build script to configure it for you.

Automating Legacy Projects

If you want to add a build script to an existing system, I have good news and bad news. The good news is that creating a comprehensive build script is one of the easiest ways to improve your life. The bad news is that you probably have a bunch of technical debt to pay off, so it won't happen overnight.

As with any agile plan, the best approach is to work in small stages that provide value as soon as possible. If you have a particularly complex system with lots of components, work on one component at a time, starting with the one that's most error-prone or frustrating to build manually.

Once you've picked the right component to automate, start by getting it to compile. That's usually an easy step, and it allows you to make progress right away.

Next, add the ability to run unit tests and make sure they pass. You probably compile and run unit tests in your IDE already, so this may not seem like a big improvement. Stick with it; making your build script able to prove itself is an important step. You won't have to check the results manually any more.

Your next step depends on what's causing you the most grief. What is the most annoying thing about your current build process? What configuration issue springs up to waste a few hours every week? Automate that. Repeat with the next-biggest annoyance until you have automated everything. Once you've finished this, congratulations! You've eliminated all of your build annoyances. You're ahead of most teams: you have a good build script.

Now it's time to make a great build script. Take a look at how you deploy. Do you create a release package such as an installer, or do you deploy directly to the production servers? Either way, start automating the biggest annoyances in your deployment process, one at a time. As before, repeat with the next-biggest annoyance until you run out of nits to pick.

This won't happen overnight, but try to make progress every week. If you can solve one annoyance every week, no matter how small, you'll see noticeable improvement within a month. As you work on other things, try not to add new debt. Include all new work in the build script from the beginning.

Ten Minutes or Less

A great build script puts your team way ahead of most software teams. After you get over the rush of being able to build the whole system any time you want, you'll probably notice something new: the build is slow.

With continuous integration, you integrate every few hours. Each integration involves two builds: one on your machine and one on the integration machine. You need to wait for both builds to finish before continuing on because you can never let the build break in XP. If the build breaks, you have to roll back your changes.

A ten-minute build leads to a twenty-minute integration cycle. That's a pretty long delay. I prefer a ten or fifteen-minute integration cycle. That's about the right amount of time to stretch my legs, get some coffee, and talk over our work with my pairing partner.

The easiest way to keep the build under five minutes (with a ten-minute maximum) is to keep the build times down from the beginning. One team I worked with started to look for ways to speed up the build whenever it exceeded 100 seconds.

Many new XP teams make the mistake of letting their build get too long. If you're in that boat, don't worry. You can fix long build times in the same agile way you fix all technical debt: piece by piece, focusing on making useful progress at each step.

Slow tests are the most common cause of slow builds.

For most teams, their tests are the source of a slow build. Usually it's because their tests aren't focused enough. Look for common problems: are you writing end-to-end tests when you should be writing unit tests and integration tests? Do your unit tests talk to a database, network, or file system?

Ally

Test-Driven Development

You should be able to run about 100 unit tests per second. Unit tests should comprise the majority of your tests. A fraction (less than ten percent) should be integration tests, which check that two components synchronize properly. When the rest of your tests provide good coverage, only a handful—if any—need to be end-to-end tests. See Test-Driven Development in Chapter 9 for more information.

Although tests are the most common cause of slow builds, if compilation speed becomes a problem, consider optimizing code layout or using a compilation cache or incremental compilation. You could also use a distributed compilation system or take the best machine available for use as the build master. Don't forget to take advantage of the dependency evaluation features of your build tool: you don't need to rebuild things that haven't changed.

In the worst-case scenario, you may need to split your build into a "fast" build that you run frequently and a "slow" build that an integration server runs when you check in (see Continuous Integration later in this chapter). Be careful—this approach leads to more build failures than a single, fast build. Keep working on making your build faster.

Questions

Who's responsible for maintaining the build script?

Ally

Collective Code Ownership

All of the programmers are responsible for maintaining the script. As the codebase evolves, the build script should evolve with it.

At first, one person will probably be more knowledgeable about the script than others. When you need to update the script, pair with this person and learn all you can.

The build script is the center of your project automation universe. The more you know about how to automate your builds, the easier your life will become and the faster you'll be able to get work done.

We have a configuration management (CM) department that's responsible for maintaining our builds. We aren't allowed to modify the script ourselves. What do we do?

You need to be able to update your scripts continuously to meet your specific needs. It's unlikely that anybody can be more responsive to your needs than you are. If the CM department is a bottleneck, ask your project manager for help. He may be able to give you control over the scripts.

Alternatively, you might be able to use a two-stage build in which you run your own scripts privately before handing control over to the CM department.

How do we find time to improve our build?

Ally

Slack

Improving your build directly improves your productivity and quality of life. It's important enough to include in every iteration as part of your everyday work. The best way to do this is to include enough slack in your iteration for taking care of technical debt such as slow builds. If a particular story will require changes to the build script, include that time in your story estimate.

Should we really keep all of our tools and libraries in version control?

Yes, as much as possible. See Version Control earlier in this chapter for details.

Does the build have to be under ten minutes? We're at eleven.

Ten minutes is a good rule of thumb. Your build is too long when pairs move on to other tasks before the integration cycle completes.

We use an IDE with an integrated build system. How can we automate our build process?

Many IDEs use an underlying build script that you can control. If not, you may be better off using a different IDE. Your other alternative is to have a separate command line-based build system, such as Ant, NAnt, or make. You risk duplicating information about dependencies, but sometimes that cost is worthwhile.

We have different target and development environments. How do we make this build work?

If possible, use a cross compiler. If that doesn't work, consider using a cross-platform build tool. The benefits of testing the build on your development platform outweigh the initial work in creating a portable system.

How can we build our entire product when we rely on third-party software and hardware?

Even if your product relies on yet-to-be-built custom hardware or unavailable third-party systems, you still need to build and test your part of the product. If you don't, you'll discover a ton of integration and bug-fixing work when the system becomes available.

A common solution for this scenario is to build a simulator for the missing system, which allows you to build integration tests. When the missing system becomes available, the integration tests help you determine if the assumptions you built into the simulator were correct.

Missing components add risk to your project, so try to get your hands on a test system as soon as possible.

How often should we build from scratch?

At least once per iteration. Building from scratch is often much slower than an incremental build, so it depends on how fast the build is and how good your build system is. If you don't trust your build system, build from scratch more often. You can set up a smoke-testing system that builds the project from scratch on every check-in.

My preference is to reduce build times so that incremental builds are unnecessary, or to fix the bugs in the build system so I trust the incremental builds. Even so, I prefer to build from scratch before delivering to customers.

Results

With a good automated build, you can build a release any time you want. When somebody new joins the team, or when you need to wipe a workstation and start fresh, it's a simple matter of downloading the latest code from the repository and running the build.

When your build is fast and well-automated, you build and test the whole system more frequently. You catch bugs earlier and, as a result, spend less time debugging. You integrate your software frequently without relying on complex background build systems, which reduces integration problems.

Contraindications

Every project should have a good automated build. Even if you have a system that's difficult to build, you can start chipping away at the problem today.

Some projects are too large for the ten-minute rule to be effective. Before you assume that this is true for your project, take a close look at your build procedures. You can often reduce the build time much more than you realize.

Alternatives

If the project truly is too large to build in ten minutes, it's probably under development by multiple teams or sub-teams. Consider splitting the project into independent pieces that you can build and test separately.

If you can't build your system in less than ten minutes (yet), establish a maximum acceptable threshhold and stick to it. Drawing this line helps identify a point beyond which you will not allow more technical debt to accumulate. Like a sink full of dishes two hours before a dinner party, the time limit is a good impetus to do some cleaning.

• Next: Continuous Integration
• Previous: Version Control
• Up: Chapter 7: Releasing

Comments

• Next: XP Concepts
• Previous: The XP Lifecycle
• Up: Chapter 3: Understanding XP

in 99 words

XP teams are self-organizing and cross-functional. This has two important consequences: first, they're responsible for their own success. This means teams define success (by interviewing stakeholders and sponsors), create plans to achieve success, and execute on those plans without explicit management direction.

Second, XP teams include all the expertise necessary to do so.

In practice, XP teams are composed of business experts ("customers"), implementation experts ("programmers"), and quality experts ("testers"). The whole team works together to create its own plans and deliver successful software. No single person is "in charge." Instead, leadership shifts fluidly with the situation.

as haiku

The soil is dry--
the irrigator has gone
I start to water

Commentary

TANSTAAFL

Full Text

The following text is excerpted from The Art of Agile Development by James Shore and Shane Warden, published by O'Reilly. Copyright © 2008 the authors. All rights reserved.

The XP Team

Working solo on your own project—"scratching your own itch"—can be a lot of fun. There are no questions about which features to work on, how things ought to work, if the software works correctly, or whether stakeholders are happy. All the answers are right there in one brain.

Team software development is different. The same information is spread out among many members of the team. Different people know:

• How to design and program the software (programmers, designers, and architects).

• Why the software is important (product manager)

• The rules the software should follow (domain experts)

• How the software should behave (interaction designers)

• How the user interface should look (graphic designers)

• Where defects are likely to hide (testers)

• How to interact with the rest of the company (project manager)

• Where to improve work habits (coach)

All of this knowledge is necessary for success. XP acknowledges this reality by creating cross-functional teams composed of diverse people who can fulfill all of the team's roles.

The Whole Team

XP teams sit together in an open workspace. At the beginning of each iteration, the team meets for a series of activities: an iteration demo, a retrospective, and iteration planning. These typically take two to four hours in total. The team also meets for daily stand-up meetings, which usually take five to ten minutes each.

Other than these scheduled activities, everyone on the team plans his own work. That doesn't mean everybody works independently; they just aren't on an explicit schedule. Team members work out the details of each meeting when they need to. Sometimes it's as informal as somebody standing up and announcing across the shared workspace that he would like to discuss an issue. This self-organization is a hallmark of agile teams.

On-Site Customers

On-site customers—often just called customers—are responsible for defining the software the team builds. The rest of the team can and should contribute suggestions and ideas, but the customers are ultimately responsible for determining what stakeholders will find valuable.

Customers' most important activity is release planning. This is a multi-faceted activity. Customers need to evangelize the project's vision; identify features and stories; determine how to group features into small, frequent releases; manage risks; and create an achievable plan by coordinating with programmers and playing the planning game.

On-site customers may or may not be real customers, depending on the type of project. Regardless, customers are responsible for refining their plans by soliciting feedback from real customers and other stakeholders. One of the venues for this feedback is the weekly iteration demo, which customers lead.

In addition to planning, customers are responsible for providing programmers with requirements details upon request. XP uses requirements documents only as memory aids for customers. Customers themselves act as living requirements documents, researching information in time for programmer use and providing it as needed. Customers also help communicate requirements by creating mock-ups, reviewing work-in-progress and creating detailed customer tests that clarify complex business rules. The entire team must sit together for this communication to take place effectively.

Typically, product managers, domain experts, interaction designers, and business analysts play the role of the on-site customer. One of the most difficult aspects of creating a cross-functional team is finding people qualified and willing to be on-site customers. Don't neglect this role; it's essential to increasing the value of the product you deliver. A great team will produce technically excellent software without on-site customers, but to truly succeed, your software must also bring value to its investors. This requires the perspective of on-site customers.

Include exactly one product manager and enough other on-site customers for them to stay one step ahead of the programmers. As a rule of thumb, start with two on-site customers (including the product manager) for every three programmers.

[Coffin] describes an experience with two nearly identical teams, one that did not have on-site customers and one that did. The team without on-site customers took fifteen months to produce a product with mediocre value:

The total cost of the project exceeded initial expectations and the application under delivered on the user's functional expectations for the system... real business value was not delivered until the second and third [releases] and even then the new system was not perceived as valuable by its users because it required them to change while providing no significant benefit.

A team composed of many of the same developers, at the same company, using the same process, later produced a product with compelling value in less than three months:

The first production release was ready after 9 weeks of development... it surpassed scheduling and functional expectations, while still coming in on-budget... In the first two months of live production usage over 25,000 citations were entered using the new system. The application development team continued to deliver new releases to production approximately every six weeks thereafter. Every release was an exciting opportunity for the team of developers and users to provide value to the company and to improve the user's ability to accomplish their jobs.

One of the primary reasons for this success was customer involvement.

Many of the shortcomings of the [first] system stemmed from a breakdown in the collaborative atmosphere that was initially established. Had users been more involved throughout the project, the end result would have been a system that much more closely aligned with their actual needs. They would have had a greater sense of ownership and communication between the various groups would have been less tense.

...

The success of the [second] system caused many people in the organization to take note and embrace the lessons learned in this project... other projects teams restructured their physical arrangements into a shared project room as the [second] team had done.

If the customers won't move to the team, move the team to the customers.

Customer involvement makes a huge difference in product success. Make an extra effort to include customers. One way to do so is to move to their offices rather than asking them to move to your office. Make sure the customers agree and that there's adequate space available.

Why So Many Customers?

Two customers for every three programmers seems like a lot, doesn't it? I initially started with a much smaller ratio, but I often observed customers struggling to keep up with the programmers. I eventually arrived at the two-to-three ratio after trying different ratios on several successful teams. I also asked other XP coaches about their experiences. The consensus was that the two-to-three ratio was about right.

Most of those projects involved complex problem domains, so if your software is fairly straightforward, you may be able to have fewer customers. Keep in mind that customers have a lot of work to do. They need to figure out what provides the most value, set the appropriate priorities for the work, identify all of the details that programmers will ask about, and fit in time for customer reviews and testing. They need to do all this while staying one step ahead of the programmers, who are right behind them, crunching through stories like a freight train. It's a big job. Don't underestimate it.

The Product Manager (aka Product Owner)

The product manager only has one job on an XP project, but it's a doozy. That job is to maintain and promote the product vision. In practice, that means documenting the vision, sharing it with stakeholders, incorporating feedback, generating features and stories, setting priorities for release planning, providing direction for the team's on-site customers, reviewing work in progress, leading iteration demos, involving real customers, and dealing with organizational politics.

In addition to maintaining and promoting the product vision, product managers are also often responsible for ensuring a successful deployment of the product to market. That may mean advertising and promotion, setting up training, and so forth. These ordinary product management responsibilities are out of the scope of this book.

The best product managers have deep understandings of their markets, whether the market is one organization (as with custom software) or many (as with commercial software). Good product managers have an intuitive understanding of what the software will provide and why it's the most important thing their project teams can do with their time.

A great product manager also has a rare combination of skills. In addition to vision, she must have the authority to make difficult trade-off decisions about what goes into the product and what stays out. She must have the political savvy to align diverse stakeholder interests, consolidate them into the product vision, and to effectively say "no" to wishes that can't be accommodated.

Product managers of this caliber often have a lot of demands on their time. You may have trouble getting enough attention. Persevere. Theirs is one of the most crucial roles on the team. Enlist the help of your project manager and remind people that software development is very expensive. If the software isn't valuable enough to warrant the time of a good product manager—a product manager who could mean the difference between success and failure—perhaps it isn't worth developing in the first place.

Make sure your product manager is committed to the project full-time. Once a team is running smoothly, the product manager might start cutting back on his participation. Although domain experts and other on-site customers can fill in for the product manager for a time, the project is likely to start drifting off-course unless the product manager participates in every iteration. [Rooney] experienced that problem, with regrettable results:

We weren't sure what our priorities were. We weren't exactly sure what to work on next. We pulled stories from the overall list, but there was precious little from the Customer [product manager] in terms of what we should be working on. This went on for a few months.

Then, we found out that the Gold Owner [executive sponsor] was pissed—really pissed. We hadn't been working on what this person thought we should.

In a predictable environment, and by delegating to a solid set of on-site customers, a product manager might be able to spend most of her time on other things, but she should still participate in every retrospective, iteration demo, and most release planning sessions.

Some companies have a committee play the role of product manager, but I advise against this approach. The team needs a consistent vision to follow, and I've found that committees have trouble creating consistent, compelling visions. When I've seen committees succeed, it's been because one committee member acted as de facto product manager. I recommend that you explicitly find a product manager. Her role may be nothing more than consolidating the ideas of the committee into a single vision, and that's likely to keep her hands full. Be sure to choose a product manager with plenty of political acumen in this case.

Domain Experts (aka Subject Matter Experts)

Most software operates in a particular industry, such as finance, that has its own specialized rules for doing business. To succeed in that industry, the software must implement those rules faithfully and exactly. These rules are domain rules and knowledge of these rules is domain knowledge.

Most programmers have gaps in their domain knowledge, even if they've worked in an industry for years. In many cases, the industry itself doesn't clearly define all its rules. The basics may be clear, but there are nitpicky details where domain rules are implicit or even contradictory.

The team's domain experts are responsible for figuring out these details and having the answers at their fingertips. Domain experts, also known as subject matter experts, are experts in their field. Examples include financial analysts and PhD chemists.

Domain experts spend most of their time with the team, figuring out the details of upcoming stories and standing ready to answer questions when programmers ask. For complex rules, they create customer tests (often with the help of testers) to help convey nuances.

On small teams, product managers often double as domain experts.

Interaction Designers

The user interface is the public face of the product. For many users, the UI is the product. They judge the product's quality solely on their perception of the UI.

Interaction designers help define the product UI. Their job focuses on understanding users, their needs, and how they will interact with the product. They perform such tasks as interviewing users, creating user personas, reviewing paper prototypes with users, and observing usage of actual software.

Don't confuse graphic design with interaction design. Graphic designers convey ideas and moods via images and layout. Interaction designers focus on the types of people using the product, their needs, and how the product can most seamlessly meet those needs.

You may not have a professional interaction designer on staff. Some companies fill this role with a graphic designer, the product manager, or a programmer.

Interaction designers divide their time between working with the team and working with users. They contribute to release planning by advising the team on user needs and priorities. During each iteration, they help the team create mock-ups of UI elements for that iteration's stories. As each story approaches completion, they review the look and feel of the UI and confirm that it works as they expected.

The fast, iterative, feedback-oriented nature of XP development leads to a different environment than interaction designers may be used to. Rather than spending time researching users and defining behaviors before development begins, interactions designers must iteratively refine their models concurrently with iterative refinement of the program itself.

Although interaction design is different under XP than in other methods, it is not necessarily diminished. XP produces working software every week, which provides a rich grist for the interaction designer's mill. Designers have the opportunity to take real software to users, observe their usage patterns, and use that feedback to effect changes as soon as one week later.

Business Analysts

On non agile teams, business analysts typically act as liaisons between the customers and developers, by clarifying and refining customer needs into a functional requirements specification.

On an XP team, business analysts augment a team that already contains a product manager and domain experts. The analyst continues to clarify and refine customer needs, but the analyst does so in support of the other on-site customers, not as a replacement for them. Analysts help customers think of details they might otherwise forget and help programmers express technical tradeoffs in business terms.

Programmers

A great product vision requires solid execution. The bulk of the XP team consists of software developers in a variety of specialties. Each of these developers contributes directly to creating working code. To emphasize this, XP calls all developers programmers.

Include between four and ten programmers. In addition to the usual range of expertise, be sure to include at least one senior programmer, designer, or architect who has significant design experience and is comfortable working in a hands-on coding environment. This will help the team succeed at XP's incremental design and architecture.

If the customers' job is to maximize the value of the product, then the programmers' job is to minimize its cost. Programmers are responsible for finding the most effective way of delivering the stories in the plan. To this end, programmers provide effort estimates, suggest alternatives, and help customers create an achievable plan by playing the planning game.

Programmers spend most of their time pair programming. Using test-driven development, they write tests, implement code, refactor, and incrementally design and architect the application. They pay careful attention to design quality, and they're keenly aware of technical debt (for an explanation of technical debt, see XP Concepts later in this chapter) and its impact on development time and future maintenance costs.

Programmers also ensure that the customers can choose release the software at the end of any iteration. With the help of the whole team, the programmers strive to produce no bugs in completed software. They maintain a ten-minute build that can build a complete release package at any time. They use version control and practice continuous integration, keeping all but the most recent few hours' work integrated and passing its tests.

This work is a joint effort of all of the programmers. At the beginning of the project, the programmers establish coding standards that allow them to collectively share responsibility for the code. Programmers have the right and responsibility to fix any problems they see, no matter which part of the application it touches.

Programmers rely on customers for information about the software to build. Rather than guessing when they have a question, they ask one of the on-site customers. To enable these conversations, programmers build their software to use a ubiquitous language. They assist in customer testing by automating the customers' examples.

Finally, programmers provide for the long-term maintainability of the product by providing documentation at appropriate times.

Designers and Architects

Everybody codes on an XP team, and everybody designs. Test-driven development combines design, tests, and coding into a single, ongoing activity.

Expert designers and architects are still necessary. They contribute by guiding the team's incremental design and architecture efforts and helping team members see ways of simplifying complex designs. They act as peers—that is, as programmers—rather than teachers, guiding rather dictating.

Technical Specialists

In addition to the obvious titles (programmer, developer, software engineer), the XP "programmer" role includes other software development roles. The programmers could include a database designer, a security expert, or a network architect. XP programmers are generalizing specialists. While each person has his own area of expertise, everybody is expected to work on any part of the system that needs attention. (See Collective Code Ownership in Chapter 7 for more.)

Testers

Testers help XP teams produce quality results from the beginning. Testers apply their critical thinking skills to help customers consider all possibilities when envisioning the product. They help customers identify holes in the requirements and assist in customer testing.¹

¹This discussion of tester responsibilities is part of my variant of XP (see the sidebar "A Little Lie," earlier in this chapter). Classic XP doesn't include testers as a distinct role.

Include enough testers for them to stay one step ahead of the programmers. As a rule of thumb, start with one tester for every four programmers.

Testers also act as technical investigators for the team. They use exploratory testing to help the team identify whether it is successfully preventing bugs from reaching finished code. Testers also provide information about the software's nonfunctional characteristics, such as performance, scalability, and stability, by using both exploratory testing and long-running automated tests.

However, testers don't exhaustively test the software for bugs. Rather than relying on testers to find bugs for programmers to fix, the team should produce nearly bug-free code on their own. When testers find bugs, they help the rest of team figure out what went wrong so that the team as a whole can prevent those kinds of bugs from occurring in the future.

These responsibilities require creative thinking, flexibility, and experience defining test plans. Because XP automates repetitive testing rather than performing manual regression testing, testers who are used to self-directed work are the best fit.

Some XP teams don't include dedicated testers. If you don't have testers on your team, programmers and customers should share this role.

Why So Few Testers?

As with the customer ratio, I arrived at the one-to-four tester-to-programmer ratio through trial and error. In fact, that ratio may be a little high. Successful teams I've worked with have had ratios as low as one tester for every six programmers, and some XP teams have no testers at all.

Manual script-based testing, particularly regression testing, is extremely labor-intensive and requires high tester-to-programmer ratios. XP doesn't use this sort of testing. Furthermore, programmers create most of the automated tests (during test-driven development), which further reduces the need for testers.

If you're working with existing code and have to do a lot of manual regression testing, your tester-to-programmer ratio will probably be higher than I've suggested here.

Coaches

XP teams self-organize, which means each member of the team figures out how he can best help the team move forward at any given moment. XP teams eschew traditional management roles.

Instead, leaders lead by example, helping the team reach its potential rather than creating jobs and assigning tasks. To emphasize this difference, XP leaders are called coaches. Over time, as the team gains experience and self-organizes, explicit leadership becomes less necessary and leadership roles dynamically switch from person to person as situations dictate.

A coach's work is subtle; it enables the team to succeed. Coaches help the team start their process well by arranging for a shared workspace and making sure that the team includes the right people. They help set up conditions for energized work, and they assist the team in creating an informative workspace.

One of the most important things the coaches can do is to help the team interact with the rest of the organization. They help the team generate organizational trust and goodwill, and they often take responsibility for any reporting needed.

Coaches also help team members remember to maintain their self-discipline, helping them remain in control of challenging practices such as risk management, test-driven development, slack, and incremental design and architecture.

The coach differs from your mentor (see "Find a Mentor" in Chapter 2). Your mentor is someone outside the team who you can turn to for advice.

The Programmer-Coach

Every team needs a programmer-coach to help them with XP's technical practices. Programmer-coaches are often senior developers and may have titles such as "technical lead" or "architect". They can even be functional managers. While some programmer-coaches make good all-around coaches, others require the assistance of a project manager.

Programmer-coaches also act as normal programmers and participate fully in software development.

The Project Manager

Project managers help the team work with the rest of the organization. They are usually good at coaching nonprogramming practices. Some functional managers fit into this role as well. However, most project managers lack the technical expertise to coach XP's programming practices which necessitates the assistance of a programmer-coach.

Project managers may also double as customers.

Include a programmer-coach and consider including a project manager.

Other Team Members

The preceding roles are a few of the most common team roles, but this list is by no means comprehensive. The absence of a role does not mean the expertise is inappropriate for an XP team; an XP team should include exactly the expertise necessary to complete the project successfully and cost-effectively. For example, one team I worked with included a technical writer and an ISO 9001 analyst.

The Project Community

Projects don't live in a vaccuum; every team has an ecosystem surrounding it. This ecosystem extends beyond the team to the project community, which includes everyone who affects or is affected by the project.² Keep this community in mind as you begin your XP project, as everybody within it can have an impact on your success.

²Thanks to David Schmaltz and Amy Schwartz of True North pgs, Inc., for this term.

Two members of your project community that you may forget to consider are your organization's Human Resources and Facilities departments. Human Resources often handles performance reviews and compensation. Their mechanisms may not be compatible with XP's team-based effort (see Trust in Chapter 6). Similarly, in order to use XP, you'll need the help of Facilities to create an open workspace (see Sit Together in Chapter 6).

Stakeholders

Stakeholders form a large subset of your project community. Not only are they affected by your project; they have an active interest in its success. Stakeholders may include end users, purchasers, managers, and executives. Although they don't participate in day-to-day development, do invite them to attend each iteration demo. The on-site customers—particularly the product manager—are responsible for understanding the needs of your stakeholders, deciding which needs are most important, and knowing how to best meet those needs.

The Executive Sponsor

The executive sponsor is particularly important: he holds the purse strings for your project. Take extra care to identify your executive sponsor and understand what he wants from your project. He's your ultimate customer. Be sure to provide him with regular demos and confirm that the project is proceeding according to his expectations.

Filling Roles

The exact structure of your team isn't that important as long as it has all the knowledge it needs. The makeup of your team will probably depend more on your organization's traditions than on anything else.

In other words, if project managers and testers are typical for your organization, include them. If they're not, you don't necessarily need to hire them. You don't have to have one person for each role—some people can fill multiple roles. Just keep in mind that someone has to perform those duties even if no one has a specific job title saying so.

At a minimum, however, I prefer to see one person clearly designated as "product manager" (who may do other customer-y things) and one person clearly defined as "programmer-coach" (who also does programmer-y things).

The other roles may blend together. Product managers are usually domain experts and can often fill the project manager's shoes, too. One of the customers may be able to play the role of interaction designer, possibly with the help of a UI programmer. On the programming side, many programmers are generalists and understand a variety of technologies. In the absence of testers, both programmers and customers should pick up the slack.

Team Size

The guidelines in this book assume teams with four to ten programmers (five to 20 total team members). For new teams, four to six programmers is a good starting point.

Applying the staffing guidelines to a team of six programmers produces a team that also includes four customers, one tester, and a project manager, for a total team size of 12 people. Twelve people turns out to be a natural limit for team collaboration.

XP teams can be as small as one experienced programmer and one product manager, but full XP might be overkill for such a small team. The smallest team I would use with full XP consists of five people: four programmers (one acting as coach) and one product manager (who also acts as project manager, domain expert, and tester). A team of this size might find that the product manager is overburdened; if so, the programmers will have to pitch in. Adding a domain expert or tester will help.

On the other end of the spectrum, starting with ten programmers produces a 20-person team that includes six customers, three testers, and a project manager. You can create even larger XP teams, but they require special practices that are out of the scope of this book.

Prefer better to bigger.

Before you scale your team above twelve people, however, remember that large teams incur extra communication and process overhead, reducing individual productivity. The combined overhead might even reduce overall productivity. If possible, hire more experienced, more productive team members rather than scaling to a large team.

A 20-person team is advanced XP. Avoid creating a team of this size until your organization has had extended success with a smaller team. If you're working with a team of this size, continuous review, adjustment, and an experienced coach are critical.

Full-Time Team Members

All of the team members should sit with the team full-time and give the project their complete attention. This particularly applies to customers, who are often surprised at the level of involvement XP requires of them.

Some organizations like to assign people to multiple projects simultaneously. This fractional assignment is particularly common in matrix-managed organizations. (If team members have two managers, one for their project and one for their function, you're probably in a matrixed organization.)

Fractional assignment is dreadfully counterproductive.

If your company practices fractional assignment, I have some good news. You can instantly improve productivity by reassigning people to only one project at a time. Fractional assignment is dreadfully counterproductive: fractional workers don't bond with their teams, they often aren't around to hear conversations and answer questions. and they must task switch, which incurs a significant hidden penalty. "[T]he minimum penalty is 15 percent... Fragmented knowledge workers may look busy, but a lot of their busyness is just thrashing." [DeMarco 2002] (p.19-20)

If your team deals with a lot of ad hoc requests, you may benefit from using a batman, discussed in Iteration Planning in Chapter 8.

That's not to say that everyone needs to work with the team for the entire duration of the project. You can bring someone in to consult on a problem temporarily. However, while she works with the team, she should be fully engaged and available.

• Next: XP Concepts
• Previous: The XP Lifecycle
• Up: Chapter 3: Understanding XP

Comments

• Next: Risk Management
• Previous: Release Planning
• Up: Chapter 8: Planning

in 99 words

Customers have the most information about value: what best serves the organization. Programmers have the most information about cost: what it takes to implement and maintain those features. And remember, every decision to do something is a decision to not do something else.

The planning game brings together customers and programmers so that they may maximize value while minimizing costs. Anybody may create stories. Programmers estimate the stories, and customers prioritize them. Programmers and customers may question each others' decisions, but each group has final say over its area of expertise.

The end result is a single prioritized list.

Commentary

Coulda, Shoulda, Woulda

Full Text

The following text is excerpted from The Art of Agile Development by James Shore and Shane Warden, published by O'Reilly. Copyright © 2008 the authors. All rights reserved.

The Planning Game

Audience

Whole Team

Our plans take advantage of both business and technology expertise.

You may know when and what to release, but how do you actually construct your release plan? That's where the planning game comes in.

In economics, a game is something in which "players select actions and the payoffs depend on the actions of all players."¹ The study of these games "deals with strategies for maximizing gains and minimizing losses... [and are] widely applied in the solution of various decision making problems."²

¹Deardorff's Glossary of International Economics, http://www-personal.umich.edu/~alandear/glossary/g.html

²Dictionary definition of "game theory", http://dictionary.reference.com/search?q=game theory&x=0&y=0

That describes the planning game perfectly. It's a structured approach to creating the best possible plan given the information available.

The planning game is most notable for the way it maximizes the amount of information contributed to the plan. As a result, it is a strikingly effective way to plan. Althought it has limitations, if you work within them, I know of no better way to plan.

How to Play

XP assumes that customers have the most information about value: what best serves the organization. Programmers have the most information about costs: what it will take to implement and maintain those features. To be successful, the team needs to maximize value while minimizing costs. A successful plan needs to take into account information from both groups, as every decision to do something is also a decision not to do something else.

Accordingly, the planning game requires the participation of both customers and programmers. (Testers may assist, but they do not have an explicit role in the planning game.) It's a cooperative game; the team as a whole wins or loses, not individual players.

Because programmers have the most information about costs—they're most qualified to say how long it will take to implement a story—they estimate.

Because customers have the most information about value—they're most qualified to say what is important—they prioritize.

Neither group creates the plan unilaterally. Instead, both groups come together, each with their areas of expertise, and play the planning game:

Allies

Stories

Estimating

1. Anyone creates a story or selects an unplanned story.

2. Programmers estimate the story.

3. Customers place the story into the plan in order of its relative priority.

4. Repeat until all stories have been estimated and placed into the plan.

The planning game doesn't always follow this neat and orderly format. As long as programmers estimate and customers prioritize, the details aren't important. For example, the programmers may estimate a stack of stories all at once for the customers to prioritize later. Typically, most stories are created at the beginning of each release, during initial release planning sessions, as the team brainstorms what to include.

During the planning game, programmers and customers may ask each other questions about estimates and priorities, but each group has final say over its area of expertise.

The result of the planning game is a plan: a list of stories in priority order. Even if two stories are of equivalent priority, one must come before the other. If you're not sure which to put first, pick one at random.

Overcoming Disagreements

Release planning is always a difficult process because there are many more stories to do than there is time available to do them. Also, each stakeholder has his own priorities, and balancing these desires is challenging. Tempers rise and the discussion gets heated—or worse, some people sit back and tune out, only to complain later. This struggle is a natural and happens on any project, XP or not, that tries to prioritize conflicting needs.

Ally

Stories

My favoriate way to plan is to gather the team, along with important stakeholders, around a large conference table. Customers write stories on index cards, programmers estimate them, and customers place them on the table in priority order. One end of the table represents the stories to do first and the other end represents stories to do last. The plan tends to grow from the ends towards the middle, with the most difficult decisions revolving around stories that are neither critical nor useless.

Use index cards to focus disagreements away from individuals.

Using index cards and spreading them out on a table allows participants to point to stories and move them around. It reduces infighting by demonstrating the amount of work to be done in a visible way. The conversation focuses on the cards and their relative priorities rather than vague discussions of principles or on "must have / not important" distinctions.

On Disappointment

The planning game is certain to give you information that makes you unhappy. You may feel tempted to blame the messenger and stop playing the planning game, or stop using XP altogether. That would be a mistake. As David Schmaltz of True North pgs says, every project has a fixed amount of disappointment associated with it. You can either use the planning game to dole out the disappointment in measured doses... or save it all up for the end.

How to Win

When customers and programmers work directly together throughout this process, something amazing happens. I call it the miracle of collaboration. It really is a miracle because time appears out of nowhere.

Like all miracles, it's not easy to achieve. When programmers give an estimate, customers often ask a question that causes every programmer's teeth to grind: "Why does it cost so much?"

The instictive reaction to this question is defensive. "It costs so much because software development is hard, damn it! Why are you questioning me!?"

Programmers, there's a better way to react. Reword the customer's question in your head into a simple request for information: "Why is this expensive?" Answer by talking about what's easy and what's difficult.

For example, imagine that a product manager requests a toaster to automatically pop up the toast when it finishes. The programmers reply that the feature is very expensive, and when the product manager asks why, the programmers calmly answer, "Well, popping up the toast is easy; that's just a spring. But detecting when the toast is done—that's new. We'll need an optical sensor and some custom brownness-detecting software."

This gives the product manager an opportunity to ask, "What about all those other toasters out there? How do they know when the toast is done?"

The programmers respond: "They use a timer, but that doesn't really detect when the toast is done. It's just a kludge."

Now the product manager can reply, "That's okay! Our customers don't want a super toaster. They just want a regular toaster. Use a timer just like everyone else."

"Oh, okay. Well, that won't be expensive at all."

When you have honest and open dialog between customers and programmers, the miracle of collaboration occurs and extra time appears out of nowhere. Without communication, customers tend not to know what's easy and what's not, and they end up planning stories that are difficult to implement. Similarly, programmers tend not to know what customers think is important, and they end up implementing stories that aren't very valuable.

With collaboration, the conflicting tendencies can be reconciled. For example, a customer could ask for something unimportant but difficult, and the programmers could point out the expense and offer easier alternatives. The product manager could then change direction and save time. Time appears out of nowhere. It's the miracle of collaboration.

Questions

Won't programmers pad their estimates or slack off if they have this much control over the plan?

In my experience, programmers are highly educated professionals with high motivation to meet customer expectations. [McConnell 1996] (pp. 255-256) validates this experience: "Software developers like to work. The best way to motivate developers is to provide an environment that makes it easy for them to focus on what they like doing most, which is developing software... [Developers] have high achievement motivation: they will work to the objectives you specify, but you have to tell them what those objectives are."

Although programmer estimates may be higher than you like, it's most likely because they want to set realistic expectations. If the estimates do turn out to be too high, the team will achieve a higher velocity and automatically do more each iteration to compensate.

Won't customers neglect important technical issues if they have this much control over the plan?

Customers want to ship a solid, usable product. They have to balance that desire with the desire to meet crucial market windows. As a result, they may sometimes ask for options that compromise important technical capabilities. They do so because they aren't aware of the nuances of technical trade-offs in the same way that programmers are.

Programmers, you are most qualified to make decisions on technical issues, just as the customers are most qualified to make decisions on business issues. When the customers ask for an explanation of an estimate, don't describe the technical options. Instead, interpret the technology and describe the options in terms of business impact.

That is, rather than describing the options like:

We're thinking about using a Mark 4 Wizzle-Frobitz optical sensor here for optimal release detection. We could use a Mark 1 spring-loaded countdown timer, too. We'd have to write some custom software to use the Mark 4, but it's very sophisticated, cutting edge stuff and it will allow us to detect the exact degree of brownness of the bread. The Mark 1 is ancient tech without dynamic detection abilities, but it won't take any extra time to implement. Which would you prefer?

... try instead:

We have two choices for popping up toast. We can either use an optical sensor or a timer. The optical sensor will allow us to toast the bread to the user's exact preference, but it will increase our estimate by three days. The timer won't take any extra time but the user is more likely to have undercooked or burned toast. Which would you prefer?

If a technical option simply isn't appropriate, don't mention it, or mention your decision in passing as part of the cost of doing business:

Because this is the first iteration, we need to install a version control system. We've included that cost in the estimate for our first story.

Our product manager doesn't want to prioritize. He says everything is important. What can we do?

Be firm. Yes, everything is important, but something has to come first and something will come last. Someone has to make the tough schedule decisions. That's the product manager's job.

Results

Ally

Vision

When you play the planning game well, both customers and programmers feel that they have contributed to the plan. Any feelings of pressure and stress focus on the constraints of the plan and possible options, rather than on individuals and groups. Programmers suggest technical options for reducing scope while maintaining the project vision. Customers ruthlessly prioritize the stories that best serve the vision.

Contraindications

The planning game is a simple, effective approach that relies on many of XP's simplifying assumptions, such as:

Allies

Stories

The XP Team

Estimating

1. Customer-centric stories

2. Story dependencies that customers can manage effectively (in practice, this means no technical dependencies and simple business dependencies)

3. Customers capable of making wise prioritization decisions

4. Programmers capable of making self-consistent estimates

If these conditions are not true on your team, you may not be able to take advantage of the planning game.

Ally

Incremental Design And Architecture

The planning game also relies on the programmers' abilities to implement design and architecture incrementally. Without this capability, the team will find itself creating technical stories or strange story dependencies that make planning more difficult.

Finally, the planning game assumes that the team has a single dominant constraint. (For more information about the Theory of Constraints, see XP Concepts in Chapter 3.) It's very rare for a system to exhibit two constraints simultaneously, so this shouldn't be a problem. Similarly, the planning game assumes that the programmers are the constraint. If this isn't true for your team, discuss your options with your mentor.

Alternatives

There are a wide variety of project planning approaches. The most popular seems to be Gantt charts that assume task-based plans and schedule what each individual person will do.

In contrast to that approach, the planning game focuses on what the team produces, not on what individuals do. The team has the discretion to figure out how to produce each story and organizes itself to finish the work on time.

This focus on results, rather than tasks, combined with the planning game's ability to balance customer and programmer expertise makes it the most effective approach to software planning I've experienced. However, if you wish to use another approach to planning, you can do so. Talk with your mentor about how to make your preferred approach to planning work with the rest of the XP practices.

• Next: Risk Management
• Previous: Release Planning
• Up: Chapter 8: Planning

Comments