Russell Bateman
November 2016
I don't want to "introduce" Agile to anyone. That's too much work. I have specific views about Agile and Scrum. I favor Scrum, probably because it's the first thing I did and my subsequent exposure to other forms of Agile I judged with some perjoration, whether or not there was really anything wrong with their practices. For example, I find pair-programming to be mostly (but not always) a way to double the number of resources needed to accomplish a task, though I see mentoring, collaboration and peer reviewing as super important.
So, this document is a collection of observations and comments I've been invited to make upon Agile in general, and Scrum in particular, that are meaningful to what I'm experiencing today.
However, I do wish to make some statements about religiosity. New initiates to Agile and Scrum tend to see the ritual on the face because it's different, it's unexpected, etc. without understanding the reasoning behind it. They implement what's not important while what is important completely eludes them. Let's look at some bad examples on both sides of the problem that illustrate this.
A team is preoccupied with the practice of standing up during the daily meeting. What's behind the idea of a stand-up is the brevity of the meeting that should differentiate it strongly from earlier, interminable meeting practices. The team stands up, yet continues to violate the essence of the Scrum stand-up by conversational wandering that wastes time and isn't part of the stand-up even when well intended. If the purpose of the stand-up is abused, most team members want to sit down because it's going to be long. Why not?
In early Scrum days, some authors suggested minor fines imposed upon team members for tardiness or absence in the spirit of promoting the seriousness of the daily meeting. The fines were to be put into a kitty for a later party budget. I saw this silliness pass quickly away and never hear of it anymore.
Sprint planning means giving points to stories. There is a tool for this—a deck of cards. Some teams get those cards out and play the game. It can be a useful thing, but since they don't really understand the correlation between story points and what they're doing, they assign value to the points to equate accomplishing a story with a number of man days or hours. The point of the cards really is a) the Fibonacci-like sequence of only indirectly correlatable point values and that b) those values are intentionally chosen to prevent their direct use to evoke hours, days or weeks.
Demonstrations and retrospectives are often seen as unimportant and routinely skipped over. This is a counter-example in the religiosity argument. There are things to be done in Scrum that should be religiously adhered to while others not so much.
Retrospectives are often abused. These are for the critique of the team's Scrum process not congratulations for things finished in the sprint nor self-flagellation for what didn't get done.
Seriously though, ...
What do I really want to talk about? I'll spend the rest of this document focusing on the bits of Scrum and Agile I find really useful to answer questions I really think need answers.
These are a good place to start. Developing software using Scrum requires
Where these roles break down is where there is
One of the greatest challenges in switching to Agile is balancing the need of managers to do what they do in part because there's a real need to do it, since they must, to varying degrees, manage upward to senior managers, to a board, to investors and/or to customers, and in part, their upbringing in an industry with traditional practices that Agile threatens, with the Agile way.
The solution to this is not easy. It requires management to soften and to risk exposure to failure. Two big contributors to the success of this are time and experience with Agile, and proven trustworthiness of the Agile team.
The product manager, more frequently referred to as the product owner today, is responsible for a relationship with a customer and the market realities of the product to be developed. His is the responsibility to
The Scrum master isn't a manager at all, but a facilitator. This is what makes product owners and engineering managers utterly unsuitable for the task of Scrum master since, though they can facilitate, they represent accountability and threaten the ability of...
...which takes on much greater responsibility and accountability in Agile than development teams have traditionally. Each individual is a contributor. Each owns tasks and responsibilities. Each, according to experience, becomes a trustworthy member that enables the smooth functioning of the Agile process. The success of the team is what eventually consoles traditional management in their abandonment of traditional management concerns (and opens them to doing more important things for the company than micro-managing development teams).
The threefold roles in Agile development have been summarized as individuals who belong to one of two different groups: pigs and chickens. This is a cartoon of the narrative that started it all:
The metaphor is a useful one, and kind of obvious, but perhaps it merits some categorization to illustrate. Let's divide the individuals we've discussed so far in our propos above:
It's important to understand that, because they have direct flesh in the daily game, pigs are allowed to talk during stand-up while chickens are only allowed to attend since they're donating replaceable resources to the undertaking. Like all metaphors, this barnyard simile is in trouble because all the stakeholders risk losing their jobs if software development is generally unsuccessful.
This is an off-hand comment.
As will become evident later, Agile teams will typically be very small because very dedicated and pointed to very precise targets. Epics and stories crafted by product owners for multiple teams must be groomed specifically for the talents, abilities and domain knowledge of each team in a larger undertaking.
Now we get down to the first of many practical observations.
Remember that this treatise isn't an introduction to Agile/Scrum, nor is it a book about Scrum, nor a chapter out of a book. It's only a collection of observations which may sound like it's trying to be something bigger. In the immediate context, it's about what I think important with comments germane to what I'm seeing today in my own work life.
The stand-up is the central work meeting in Scrum. It's practiced, religiously,
...whether or not anyone actually stands or sits.
The stand-up observes an additional, religious trapping in its litany. What happens in stand-up makes it the short meeting that there's no time or need to sit down for. The purpose of this litany is to inform other team members very generally as to what's happening, what the progress is, what problems are appearing if any, and reassure them that their teammates are contributing to the goals of the sprint. Three questions are answered by each team member:
The stand-up is what provides the cadence to the sprint even if it's not done face-to-face, which it need not be, given Slack®, messengers, or even just e-mail (though e-mail can contribute to a loss of cadence). So, cadence? It's like marching troops. Each soldier knows how to put one foot forward after another, but to march in time is the point. The stand-up helps everyone stick to the sprint's goals. It provides a cooperative gauge that this is happening and a recourse for when it does not.
...is often as important as what it is. The answers to these questions typically involve references to stories or tasks that were created as a result of sprint planning and probably have id numbers in the Agile software tool being used to manage them.
Stories and tasks should not be referred to in so vague a way that other teammates don't even know what the team member reporting is talking about. Obviously, this also indicts the size of the team and scope of the sprint. If the team members can't realistically get their arms clear around the trunk of what's being developed by their mates, then the team is too big.
Stand-up is not a time to discuss story, task or problem details. Story and task details are discussed in sprint planning. They and problems are discussed in another meeting if need be, but not in the stand-up. Doing otherwise turns the stand-up back into a sit-down staff meeting from non-Agile days gone by.
This said, most teams tolerate obvious, quick-fire questions by other team members during a member's recitative. Good judgment is exercised. Eye-rolling is a sign it's not. And, when the stand-up becomes other than the rapid-fire 1-2-3 answer to questions, it damages the advantage of cadence.
Most Scrum teams adopt a concept of a meeting after the stand-up, that is optional and only attended by invested stakeholders in the problem(s) to be discussed.
The backlog is created, groomed, prioritized and otherwise maintained by the product owner. Let's take backlog content one by one:
Epics are not really part of Scrum. Their existence arose long after Scrum was invented and as a nod to the need to express concepts larger than would fit into stories and also hierarchical organization of stories in a larger project. And projects are almost always much larger than one story. So, epics are good.
A story might really be an epic if it's difficult to
Of course, this may vary according to a team's size and ability to execute. Some teams are able to achieve greater velocity than others. (More on velocity another time.)
The story is the fundamental unit of work in Scrum and other Agile idioms. There's much written about how big stories can be, how many subtasks, etc. However, there is a simple formalism that's useful in which to cast the definition of a story to help corral what it is and what it is not. This is to state it in the following terms:
"As a role , I want feature , so that ability ."
An example is deserved.
"As a user of this object-drawing program, I want the option to print any object in various formats, ink and paper, PDF document, etc., so that I can isolate the object for various purposes, like micro engineering it as a component or simply debugging it."
This expression need not be the name of the story nor its summary because succinctness is important. But, it should figure prominently, probably as the first thing, in the story description. It's the product owner's job to couch this statement, whatever trivial name is given to the story.
It's important, whether or not this formalism is adhered to, that a story not be simply a summary, i.e.: "Print object," with description of what the story is to accomplish left to that. That's an egregiously brief treatment of this most fundamental object in Agile. The story should contain every detail that needs to be tackled by any member, even the newest, on the team. But, it doesn't reach to the task level.
Next up in the story description (real Agile software tools have a separate section for this) is what's known as a list of acceptance criteria. This is the list of observations of the product of a story that can and must be made to call it accomplished. For our example,
All the drawn object information should be visible including its title, id, description and relationship to other objects. On the preview page, there should be an option to print or download in different formats (PDF and XML).
Most of what is listed in acceptance criteria should be demonstrated at a special time at sprint's end (retrospective). More on this later. Specifics about unit-test coverage may figure as an acceptance criterium if the product owner has concerns about stability and knows enough about what's going to be testable.
Last, the number of story points that characterize the story's relative complexity and effort-required in comparison to other stories the executing team has done. This brings us to...
0 ½ 1 2 3 5 8 13 20 40 100 ∞
These are another of Scrum's trappings that are strange to newcomers and often misunderstood.
When estimating a story's complexity and effort required, it's natural to think in terms of how many team members, doing what and how long it would take them. However, this is not the purpose of story points. Instead, and this is important to note, story points are less useful for the current than for future sprints.
Story points are what allow a team to offer handfuls of what's needed to accomplish a story's work. Handfuls are measurable in sand, marbles, water—or in this case, sheer oompf. Is this a really big story that we can't complete in one sprint? Maybe it's multiple stories. Maybe it's an epic. Product owner: rethink this as a series of smaller, less ambitious steps toward the goal.
The value of the story points is retrospective: after a team's been together a year, it has something called velocity. This measurement, completely arbitrary, results in a history that can be consulted in terms of how many points a team might complete in a sprint. It helps know what to pull from the backlog in sprint planning. It helps to see a team's improvement in productivity over time.
Refusing to allow story points to have an association with hours or days is what makes a team's velocity less coupled to equivocation. Indeed, this decoupling makes it so that points will be more honest since they do not relate to time. Relate them to time and immediately there's going to be padding and expectations.
Most Agile teams that start out insisting on equating points and hours end up abandoning hours once they gain experience. One Agile acolyte explains how he reasons this with new teams uncomfortable with points like this:
Make a list of 8 dogs. Ask the manager to estimate with you how long it will take to groom the 8 dogs. The manager won't want to because she doesn't know much about dog grooming. Clarify that isn't the point and that taking a reasonable guess is fine. Have a few pictures handy. As soon as the manager says "that fluffy poodle with the French cut will take two hours," you then say, "I should have mentioned I'm an expert groomer—put myself through college dog grooming in fact—and I know that cut would take me two hours and would take you much longer since you're inexperienced." The point is you are (at least in pretend) a great groomer. Your manager isn't. As soon as the manager agrees it will take you two and her eight, ask how you can now solve that problem. You are both right—you two, her eight. The manager may say to go with your number since it's the lowest. OK, onto the next dog. You're still the expert. [The point is that we] can't put your number on everything—you can't do all the grooming as that isn't realistic.
Now offer to repeat it with points. See if she can agree that perhaps this labrador really only needs a shampoo—so let's call that a 2. (In case we have a small, shampoo-only dog to call a 1.) You can and she can agree on that. Then get her to agree that the poodle will take 4 times longer to cut. So call it an 8. And so on. Point out the intractable problem that your time and hers are not the same. Yet you can [always] agree on a more abstract value: points.
Where hours do come into it, however, is...
Tasks are conscious of hours because assigned to specific developers who estimate how long they will take. Stories should not be broken down into tasks that are impossibly huge to accomplish. Each task should take no more than 16 hours to complete; an 8-hour maximum would be better, a 4-hour even better.
Task break-down happens during sprint planning. It happens before the fist of fingers the team give at the end of sprint planning to express acceptance of the work and confidence in the sprint's success.
Task break-down requires experience. It's really something a team lead or senior engineers do. Some teams split sprint planning into two small meetings across two days. On the first day, stories are estimated and swag is done to accept stories from the backlog. Then, the meeting breaks and developers go back to their cubes to break stories down into tasks and assign hours (with the understanding that a junior developer, or one unfamiliar with the needed technologies of the solution might take longer). The next day, everyone comes back together and, if really necessary, a story might be dropped from the sprint as impossible to do in the sprint. However, the story would already be in a state of being "groomed" and it could easily brought back in by team members who happen to finish their sprint work early.
We've seen in the oblique remarks above, how a product owner and a team come together to plan a sprint:
Once planned and launched, the sprint is in execution. Developers may pursue low-hanging fruit, but if important features are in jeopardy, priority rules.
At this point comes a difficult problem to which few if any have a solution: how do developers and testers interact? How can a developer spend the entire sprint on his tasks knowing his tester won't have enough time to complete her own?
Various remedies to this obvious problem have been tried. None are perfect. In summary, they are:
This said, and no matter how the problem is solved, one Agile principle is certain: development must write their own unit tests and these must be thorough. Testers will validate acceptance criteria. More on this later.
At the end of the sprint, two more things happen before the next sprint's planning. First comes the demonstration. This is when the acceptance criteria of each story completed in the sprint is shown to an audience made up of pigs, chickens and also as many interested if uninvested attendees as wish to see what's been done. Often, the best demonstrator is the tester.
Sometimes, in the absence of a UI aspect to the product, the demonstration may wax a bit arcane. It may even come down to an assurance by the team members that the acceptance criteria have been met.
The retrospective is never about what's been accomplished. It answers these questions:
Never is the answer to the first two questions above something about, "we didn't get that component finished" or "good thing we got those APIs done."
The sprint retrospective is a scorecard about the Agile process itself or how Agile the team was. It's about what went right (as compared to the team's history of performing in Agile), what went wrong (what principles could we have respected more with benefit?), and is there something the team can surely do in the next and future sprints to be better at what they're doing.
Obvious good points to make might be how the team was able to make testers successful or, failing that, how they torpedoed testers and what the team must do to adapt their work processes to making testing succeed.
Again, the retrospective isn't for pointless back-patting or breast-beating over what happened in the sprint to the stories the team implemented. It's about how to ensure subsequent sprints go better.
Deep in sprint execution, it may become apparent to the Scrum master and/or senior developers that the sprint will fail to meet the planned objectives. Sprint means agile: It's time to stop, declare a broken arrow (a term from the United States Air Force that means that a weapon was dropped accidentally from an aircraft), and re-engage the sprint planning process. If a team's sprint must correspond chronologically with other sprints (how unfortunate), then an extra-long or an extra-short sprint might be planned in order to honor that cadence.
My initiation to Scrum came in a day before tools were available. We wrote our own and these worked remarkably like the third-party tools that would come later.
Scrum tools should
Over my years, I've seen some good ones:
If you're still working on "tickets," that's not Scrum.
My assessment is that Agile tools do not make good bug trackers; bug trackers never make good Agile tools. The best bug tracker I've ever worked with is Bugzilla (but, it requires substantial Perl expertise). JIRA is a good issue tracker too, but it's a very bad Agile tool.
For me, the most important notions to associate with Agile development revolve around testing.
Test- or behavior-driven development (TDD, BDD) is often touted whenever Agile is discussed. This is because this sort of development is an effective deterrent of technical debt.
Unit tests, whether driven by feature testing or blatant coverage, can save development bacon time and time again. It's wrong and stupid to spend time in the 20% just to get full code-line coverage, but having complete, logical coverage of components (i.e. classes, data and methods in Java) means that a component will survive heavy refactoring when, without unit-test coverage, it would be too dangerous to attempt—this is mounting technical debt.
At a minimum, full, unit-test coverage for low-hanging fruit should exist: what does (this class, this code being tested) produce and what should it not produce that I can test for?
Beyond low-hanging fruit, full coverage for boundary errors.
If possible, a nod to likely pathology (what are likely catastrophes to happen in this code: can they be identified and "tested against"?).
Integration-, functional-, stress-, etc. testing: all these, as automated as possible, are key to a product's developers being able to "sleep through a storm."
Continuous deployment.
These are many and subtle, I have already mentioned a few, associated with the following list. What I say here could use some more work, but it's a start.
The backlog translates the product owner's vision in coarse grain. If the product owner is too busy to define it, he likely defines enough out for the next sprint or two and calls it good. This results in the team not having a good, long-term understanding of the product.
An anti-pattern for the backlog is the product owner doing the task break-downs. This risks taking a sense of ownership away from the development team.
Formally, as already alluded to, there are two planning meetings for the sprint. The first is to go through the groomed backlog with the product owner to determine what stories or tasks of stories the team will accept for the spring. This gives context and vision to the team and they give first thoughts on how much effort is required.
The second part of sprint planning is where the team dives into the stories and breaks them up into tasks. Some wait to do story points attribution until break-downs have been done.
The antipattern is the team thinking they can skip planning because they already know what they're doing.
An anti-pattern here is the meeting becoming a status meeting with news, technical discussion, etc. Originally, the religious principle of standing up for it was to help keep it from this.
Another antipattern is giving status update to the product owner or development manager. The meeting is really for the team so that its members know what each is doing and are able to offer help. Minimal identification of the stories/tasks worked on is sufficient; there's no reason to "educate" or inform the rest of the team on them; that's a different meeting.
A third anti-pattern associated with the stand-up is talking about tasks that aren't in the task-management software. If other tasks are being worked on, they aren't part of the sprint and that should be corrected.
Incremental value, warts and all, must be shown to the stakeholders. It's a mini-handover. No showcasing? This is an anti-pattern. There's no understanding that we're progressing or accomplishing anything. The chickens should be disappointed.
This is about not what can be improved in what we're doing, but in how we're going about it. It's difficult to know what can be improved if no one knows what Agile really is. A coach observing what's going on is a good idea. Education of team members is a good idea. Then retrospective becomes meaningful. Skipping the retrospective is an anti-pattern. Turning a retrospective into a list of things accomplished or not accomplished during the sprint is also an anti-pattern.
Management must resist the strong temptation to leap in with traditional command and control. I'm not a manager. I feel for them. I can only say that this is a hard thing for them and that it can limit or destroy the benefits of the Agile process. There is much to read about this out there. Anti-patterns abound.
Learn what Scrum is, learn its history, learn how it's been implemented in other industries, learn what started it (Jeffrey Liker, The Toyota Way). Not understanding deeply enough leads to a cargo-cult practice of Scrum, i.e.: the assumption that because we're doing two-week sprints or standing up every day, we will reap the benefits of Agile. My own deemphasis of the religious aspects of Scrum injects risk too: be sure you know what you're doing and why. Learn Scrum.
Agile doesn't imply effortless quality. Reproducible testing must be done at every level—testing that will tell you at any moment when your efforts are about to cave in. Code reviews or adequate collaboration keeps good practices moving forward. The anti-pattern of not elevating testing to the same level of importance as design and development is a killer.
The code review is a positive pattern for Agile, however, if misused, it can be an anti-pattern. Code reviews are good for a) helping juniors not make mistakes (to put it crudely) by failing to use good pattern techniques, good organization, etc., but they are also b) good for senior developers to avoid inadvertent mistakes through sanity checks, i.e.: an extra pair of eyes. What's to avoid, however, is thinking, especially in (a), but also for (b) that code reviews find bugs. The best way to find and forever more guard against bugs is the unit test. Code reviews are about a) attention to quality, b) knowledge sharing, and c) collective ownership.
It is an anti-pattern to fail to define, in each story, just what "done" means and it risks the developer not finishing his tasks and the story not solving the problem it was meant to solve. Acceptance criteria cannot be left off stories.
There are many anti-patterns in communication. Communication is one of the softer areas of process. It's something that's very culture-based. Standards and practices must be establish that best serve the team. Assumptions that stand-ups, water-cooler conversations keeps everyone informed are bad. The process should leave enough of a documentation trail—and a discoverable one—that even the least experienced team member can be productive. Much of this is solved using tools, most with organization. Ad hoc document associations weaken the success of the team. Documents should be tied in—tied into version control, tied into code, tied into whichever Agile management tool is used.