Why is transparency so important?

Because

• it makes inspection possible, which is needed for adaption
• it enables visibility, to observe and understand what is happening
• it leads to more communication
• it establishes trust (in the process, as well as among team members)

This is why I think transparency is at the core of every good development process.

How does Scrum create transparency?

• A visual (physical) task board is a great tool to raise the team’s visibility of the status of the current Sprint. For me the instant status report for manager type people is a mere side effect. Much more important is the constant feedback for the team – to be aware of the progress within the Sprint. Only with this knowledge can the team be truly self-organizing. This is why I am usually against all these software tools that try to replace a physical task board. Yes, there is a slight overhead to produce index cards and print out the burndown chart, but in my opinion it is minor in comparison to the raised level of transparency. Advocates of Kanban say that the typical Scrum task board with its columns for Backlog, In Progress and Done is not transparent enough. The column In Progress is a black box, hiding states like Analysis or Development. What is the right granularity for your team depends on your context, like the complexity of your development process, team size and others. My advice is to try keeping it as simple as possible without sacrificing transparency.
• Early feedback is essential to make sure you are doing the right thing and that you are doing it right. It enables the team to discover and correct any errors and misconceptions as soon as possible. Feedback cycles can be applied at every level of development: pair programming, unit tests, acceptance tests, Daily Scrums, Sprints and releases. The time frames of these cycles range from seconds to months and the feedback is coming from different sources: your colleagues, automated tests, the Product Owner and most importantly the users of your product.
• Future work seems to be predictable. This is achieved by the Product Backlog, which can be examined by anyone anytime. This creates a sense of predictability during the project. Even though most team members barely ever look at it, its existence creates the illusion of anticipation of where the project is heading.
• Cross-functional teams work together on a common task. In an ideal (software development) world there is no them only an us. More often than not there is a bit of rivalry and tension when developers, testers and DBAs  are in separate teams. This is because they don’t understand each others pain points, caused by a lack of communication and transparency between teams. In big organizations they might not even sit in the same office or know each other personally. People tend to be so much more understanding and forgiving when they communicate face to face rather than via email. By putting all needed specialists within the same team, Scrum enables communication between them and makes their needs transparent to each other.

Nowhere left to hide

Even though transparency is so important and brings the above described advantages your team might be reluctant about working in a glass house process. To happily do so requires the team to be comfortable with admitting mistakes. Mistakes are inevitable and they are not necessarily a bad thing either. Making mistakes is quite possibly the best learning technique that is out there, as long as you don’t forget about the learning part.

Are you ready for the truth?

This can of course only be achieved if the people leaders of the team have the above understanding about mistakes and react appropriately on them. If your team is scared to admit mistakes, they won’t. They also won’t adopt a transparent process. So be aware that the truth might not look as nice as the filtered reports you are used to. It’s kind of a red pill – blue pill thing.

What did I miss?

• What other reasons are there why transparency is important?
• What other things make Scrum transparent?
• What makes Scrum less transparent?
• What are other processes using to create transparency?

Image by *SΛM via Flickr

• A list of goals that describe objectives to satisfy the costumers needs.
• A fixed length (7 ± 2) queue (Product Backlog). This is based on the idea that 7 is the number of things a person can keep in their mind at once and prioritize. It ensures that customers are making value prioritized decisions at all times.
• The 7 slots in the queue are to be filled with Minimal Marketable Features (MMF) by the customer (Product Owner). The customer uses the list of goals to generate new MMFs once the development team has eaten away the top priority MMF.
• The customer prioritizes the MMFs purely by value. There is no upfront estimation of MMF’s. This is based on the idea that cost is distributed linearly and therefore approximately constant, whereas value is distributed exponentially and therefore outweighs cost. Estimation itself is considered a wasteful task (see arguments below).
• There is one (and only one) slot for urgent tasks, which the costumer can use to bypass the queue. The development team tries to keep this slot empty at all times.
• The development team has a fixed amount of slots to pull MMFs from the queue; just like the Work in Progress (WIP) limit in Kanban. This depends on the size of your development team, but there should be about two slots.
• A MMF is considered done when everyone included in the process thinks it’s done, including the customer. In the done state, the MMF is already part of an automatic deployment, so unless someone takes action to prevent it, it will end up in production.
• Throughput time (time between ‘MMF gets added to the queue’ and ‘MMF is in production’) is projected by previous measurements. Since there are no estimates this is the only way to do this. These measurement-based projections are supposed to be more accurate than estimation-based predictions. Arlo puts a dotted line on the bottom of the queue with “Your approximate time to wait is x days”. This is what he calls the “Disneyland wait time”.

This is what I gathered from the above mentioned screencast, Arlo’s comment on this blog entry by Aaron Sanders explaining Naked Planning, this podcast from Agile 2008 and this short video from Agile 2007. I don’t think there is any “official” documentation about the process, just videos from conferences.

Estimates deliver negative business value

Arlo argues that when customers prioritize the MMFs purely by value without considering the cost at all, they actually make better decisions. Let’s say we’ve got a two by two matrix with cost on the horizontal axis and value on the vertical axis and MMFs scattered throughout this matrix. The common behavior is to choose the cheap MMFs on the left – first the ones on the top with high value, then the ones further down with low value. These low value / low cost MMFs are what Lister and DeMarco label with ‘Death March’ features. If you take away the cost axis, people only choose high value MMFs – some with low some with high cost. Arlo calls these short term value (high value / low cost) and long term value (high value / high cost) items. According to his studies, it turns out that this is a much better portfolio of MMFs than just low cost MMFs. Consequently, not giving your customers estimates to consider when prioritizing their MMFs actually let’s them make better decisions. Having estimates is therefore a negative business value proposition or as Arlo puts it: “It’s positive cost, it’s slightly dishonest and it delivers negative business value.“

He further explains that not doing estimates also saves a lot of time:

I never spend any time tracking estimates, improving the team’s ability to estimate, applying math to counter the effects of holidays & sick days, etc. This estimation normalization stuff takes a surprising amount of time – as I only discovered when I stopped trying to make halfway-decent estimates. Possibly most importantly, eliminating estimates eliminates conversations like “you promised that, this week, you’d do X, Y, and Z. Where’s Z?” Instead, the conversations become “What is the relative importance of X, Y, and Z? Well, X, then Y, then, eventually Z. Can we just release X+Y as soon as they’re done? Sure. We’ll give you some warning when the release is coming close.” It opens up a lot more success opportunities, and shuts out several types of bad behavior.

Points to debate over

I find the idea from a developer’s perspective very appealing, because it takes away the dreadful task of having to come up with an estimate, but there are a couple of issues with this approach that I am not quite sure about.

• Team motivation: How do you achieve a commitment from the team and keep the motivation up, without the looming deadline of the next sprint demo and without the team trying to achieve the estimates? One possible way to keep the motivation up is to keep regular meetings in which the team demonstrates the done MMFs to customers. Naked Planning does this by keeping demos and retrospectives as a heartbeat for the team. However, I am questioning if the motivation will be as high as with the commitment to a certain number of MMFs prior to the sprint. But maybe I am wrong. There is a study mentioned in Peopleware by Lister and De Marco, that suggests that “the highest [software development] average productivity was on those projects that didn’t estimate at all.”
• Customer buy-in: You need to find a customer that is able and willing to make business decisions independently from the cost of the wanted MMF. From my experience with Scrum, it is hard enough to convince customers that they don’t need a detailed project plan with a big design up front. How can you justify abandoning release and sprint planning (and sprints for that matter) all together? Here Arlo draws from his experience by using the Naked Planning process with his team. In his experience, the ‘Disneyland wait time’ (the time for the 7th item on the queue to go into production) is usually never longer than 90 days. This means the customer can plan with the seven most valuable MMFs within 3 months.

Image via Wikipedia

Beyond Scrum and project execution

Before you start screaming, bear with me for a moment and read on. We all know a project doesn’t just start when someone has a brilliant idea for a product and writes down some requirements in a Product Backlog. There is more to it, much more I am afraid. Things like Visioning, Project Approval and Staffing. Also, the engagement with your customer might not end with the deployment of the product into their environment. You are likely to also provide training and support for the product that you delivered. This is where Lean Principles come into play and can provide valuable guidelines beyond Scrum. One of the principles in Lean Thinking is to “optimize across organizations“. Lean considers products from concept to cash. It focuses on the entire organization. This is where management plays a key role. To achieve this, managers have to coordinate teams as part of the whole, but they don’t have to tell the team members what to do from start to completion of the development effort. Empowered (Scrum) teams will handle this just fine.

So, the reason why managers are not mentioned in Scrum, is because Scrum operates on a level below where management is needed. Managers don’t have to get wound up with trying to micro-manage teams. Instead they can concentrate on creating an organizational structure for teams to work in.

Appropriate project plans

For the very same reason, there is no place for a traditional project plan in Scrum. Think about it. Who is asking you for a project plan? Is it your development teams or is it your customers? That’s what I thought. Your customers want to have a project plan so they know when the product will be finished and how much it is going to cost them. The reality (in most cases) is dead simple: No project plan, no project. Here comes the BUT. This project plan needs to be based on estimates from the team and on a high enough level to give them and your customer some wiggle room. You know your customer’s requirements will change. So, why tie them all down before even starting the project? This usually requires you to educate your customers to be comfortable with a certain level of uncertainty. Unfortunately, many customers would still rather have a detailed project plan with the certainty that it will be wrong.

Trust is the foundation, again

Comfort with uncertainty  is based on trust. The trust of your customer in your organization. I mentioned trust before. The trust your empowered teams need to self-organize and to achieve the given product visions. This trust ultimately needs to come from your customers. Trust that your organization will act in their best interest and also in your capability to do so.

Project Managers have to advance too

All of this presents challenges to both Project Managers and the organizations they work for. It requires a change in the way they used to work, a change in mindset. The business is accustomed to completely defining cost, time and scope at the beginning of a project. To deal with uncertainty and risk, Project Managers have to acquire new skills in techniques such as rolling-wave planning or progressive elaboration.

So, yes, I do think there is a place for Project Managers in agile organizations, but like the rest of us, they need to keep learning new skills to keep up with the industry standards of developing software.

Project Manager (PM)

The PMI Project Management Body of Knowledge recognizes five basic process groups and nine knowledge areas being typical of almost all projects. These are the domains of a Project Manager. The five basic process groups identified are

1. Initiating,
2. Planning,
3. Executing,
4. Monitoring and Controlling and
5. Closing.

Some or all of these processes are contained in each of the nine knowledge areas consisting of

1. Integration,
2. Scope,
3. Time,
4. Cost,
5. Quality,
6. Human Resource,
7. Communications,
8. Risk and
9. Procurement.

Scrum Master (SM)

The Scrum Guide defines the Scrum Master role as follows:

“The ScrumMaster is responsible for ensuring that the Scrum Team adheres to Scrum values, practices, and rules. The ScrumMaster helps the Scrum Team and the organization adopt Scrum. The ScrumMaster teaches the Scrum Team by coaching and by leading it to be more productive and produce higher quality products. The ScrumMaster helps the Scrum Team understand and use self-management and cross-functionality. However, the ScrumMaster does not manage the Scrum Team; the Scrum Team is self-organizing.” [emphasis is mine]

Comparison

Both SMs and PMs try to create an environment that enables the team to do their work. They are facilitators. However, they are doing this in completely different ways. SMs manage the development process and remove obstacles identified by the team, whereas traditional PMs manage resources (including the team members) and do all of the planning, monitoring and controlling involved.

For Scrum teams, the project planning consists of

• the Product Backlog,
• the Release Backlog (both owned by the Product Owner) and
• the Sprint Backlog (picked and estimated during the Sprint Planning Meeting by the development team)  together with
• the corresponding Release and Sprint Burndown Charts (kept up do date by the SM).

The SM monitors the team’s achievements during the sprint using the Sprint Burndown Chart and raises discrepancies to the Product Owner, who then controls the outcome of the sprint by adjusting its scope. The development team also monitors and controls its own progress daily during the Daily Scrums. In Scrum terms this is known as inspect and adapt. This means the PM tasks of planning, monitoring and controlling, are shared by the SM, the Product Owner and the development team.

A traditional PM would do all these tasks by himself after gathering the status from the team in regular meetings. This also means that he is held ultimately accountable for the success or failure of the entire project.

Conclusion

And so, I would answer the question, if Scrum Masters are the new Project Managers with a definite ‘No’. Scrum Masters fulfill some of the tasks of a traditional Project Manager, but so do the other members of a Scrum team, i.e. the Product Owner and the development team. Should PMs become SMs then? Possibly. However, it requires a lot of discipline of the PM not to try to manage the team, and not to tell them which tasks to tackle next during the Daily Scrums. Since it is the SM’s responsibility to enforce the Scrum process there is no-one to correct the PM in case he falls back into the old rut. It is more common that SMs are recruited from within (while possibly still being part of) the development team.

Consequently, is there no place for Project Managers in Scrum? I shall cover this question in my next entry.

Vision

Imagine two European explorers in the fifteenth century hiring crews to sail to India. Explorer A sends his crew into the woods to chop trees without any further explanations.

Explorer B, on the other hand, gathers his crew and tells them about their goal to be the first to sail to India to gain easier access to their riches (mainly spices). He instills in them a vision of something they can be proud of if they can achieve it.

Which crew do you think is more motivated? Which one will finish their ship first? Exactly, that’s my point. Teams don’t need step by step instructions of how to develop a software product. They’ve done it before (at least some team members should have). They know what it takes. What they need is a common goal to motivate them and the trust that they can do it. If the team members are not entrusted to determine their own work, they will never fully commit to the product vision, nor will they ever feel the desire to make the key decisions that they should be responsible for.

Leadership

A lot has been written about Scrum sprints replacing the traditional waterfall model, but what is less talked about is that Scrum is also about leadership versus management. According to Seth Godin, “The secret about leadership is simple: Do what you believe in. Paint a picture of the future. Go there. People will follow.” “Leaders have followers. Managers have employees.“

Of course this shines a bad light on management. We don’t even let anyone outside the development team speak at our daily Scrums because we don’t trust them not to tell us how to do our job. The team is, therefore, entrusted to deliver the Product Backlog on their own, and managers are (only) expected to assist the team to build the best product possible.

There is no doubt that each group of people needs leadership to achieve a goal. But this is just another one of the skills I mentioned above. There are undiscovered leaders within your team. They might not have the title on their business cards, but they are there. You don’t have to appoint them either, people will turn to them automatically when problems arise.

Responsibility

There is an issue that remains unresolved. I like to call it the “Spiderman Dilemma“: “With great power comes great responsibility“. What if members on your team don’t want to have this responsibility? What if they are too used to their factory life, where they have tasks assigned to them and they can go home after their eight-hour day without worrying about the product they are producing? I am afraid, I don’t have the answer. After all, I guess Scrum is not for everyone. I do believe, however, that these people are the exception. Generally team members like to have some authority over their time and effort and some input into what they do, because it gives them the chance to be proud of the product they create.

So, give your team a vision and the trust that they can achieve it, not a step by step instruction.

The above metaphor was inspired by a quote from Antoine de Saint-Exupery: “If you want to build a ship, then don’t drum up men to gather wood, give orders, and divide the work. Rather, teach them to yearn for the far and endless sea.”

Drawings by Nicole Day.