I have spoken about this topic in several conferences in India in the last year. After my talk, I would get several requests to share a “template” of the Upstream process! The people who heard me in the conference understood the context and the problem that it was trying to solve. However, it is clear this is a common problem that many teams face and hence, I decided to write a blog about it.
Upstream Kanban for Backlog Grooming
One of the common issues in most teams undergoing Lean-Agile transformation is that their effort and focus of the transformation process is on the development team. Most Agile literature starts with “groomed stories”. Agile practices are primarily Development team focused. Once developed, with the growing maturity and adoption of CI/ CD practices, it is becoming feasible to quickly deploy “fully tested” code to production via automated pipelines.
What is unfortunately talked about very little is – how do we get groomed stories? This has been largely left to the domain of the Product Owner. Somehow, quite magically, groomed stories arrive in the backlog – that are functionally clear, small, testable, independent and most importantly, implementable (that last one is not part of the INVEST criteria but is equally important).
Unfortunately, the development teams suffer when this magic fails – and it’s not their fault. They start the Sprint Planning process with the prioritized User Stories and the team realizes that there are functional questions that go unanswered. We all understand that the PO (Product Owner) is supposed to have a ready inventory of groomed user stories. Reality is often different! Either they are confused or there are other stakeholders they need to check with. Then, there are questions around the wire-frame, there are design questions – take approach X versus Y. All these open questions mean that the team does not know what to do as they are in the Sprint Planning process. They can proceed with assumptions but that will potentially result in rework and often, result in slippage in the Sprint scope.
Further, the nature of work is such that the cycle time needed to groom stories is far longer than developing those stories. If stories can be developed in 2 weeks, grooming an epic into fully groomed stories can take a couple of months, specifically if there is some level of wire-framing and iterating with the business involved. Talk about who is the bottleneck!
Given this challenge, I have often aligned the focus of teams and management from downstream to upstream. I have done this in the following manner:
- Define upstream value stream distinctly separate from the downstream.
- Pull some senior people (often, technical) from downstream, who are aware of the product and how it works, to collaborate with the PO. The PO (Product Owner) still continues to be the overall person to decide what gets done, what’s its priority. The senior people ask questions like, have you thought of this, what happens in this case? In some cases, they can even suggest, if we do it like this, it would be a lot easier and less time consuming – would that meet the business need? In other words, the PO and senior team members collaborate and think upstream to finally converge on what needs to be built downstream by the Development team.
As I said above and I want to repeat, step 2 is very iterative and collaborative. It does not work by the PO handing-off to the senior technical person in some manner and then expecting something back. It does not undermine the role and responsibility of the PO. Its only objective is to support the PO (Product Owner) in defining better groomed stories where the need, the acceptance criteria, are well defined with less ambiguity.
With this context, I am sharing some examples of how we have approached this in a couple of different scenarios:
- Scenario I: Greenfield development of a product. In this example, you will see a process of grooming the epic until it is well understood and then, using a process like Story Mapping to generate groomed stories. The Value Stream has been further detailed to cover Entry/ Exit criteria, applicable practices for each step, etc.
In the above snapshot (we will elaborate below), you see a Planning Value Stream, which is the Upstream process. The key objective of Upstream is to generate Groomed Stories. There are two downstream lanes – one for Kanban teams and one for Scrumban teams. The Value Stream for converting Groomed Stories into production-worthy code is the same.
The Upstream has three parts:
a) The grooming process of the EPIC. The Value Stream looks like this:
b) Taking a Groomed EPIC and generating User Stories. I strongly recommend Story Mapping for the same. The Value Stream looks like this:
c) Taking Developed Stories forward. This part can vary significantly for every team. If you are going to deploy the Stories into Production, then you can do this as part of Downstream Value Stream (which is demonstrated in Scenario II below). However, if you are part of an environment where you need to submit the EPIC for some kind of formal documented acceptance (for the e.g., regulatory environment), then you will need to do something like what we have shown above.
A note of caution: Do not try to copy the template – understand what was attempted to be done and then build or adapt to your context.
The downstream is what you would have commonly experienced in any Scrumban/Kanban team. It looks like this:
- Scenario II: A product in lights-on mode. In most cases here, Enhancements are small independent functionalities and hence, tagged as a Story, or a slightly larger chunk of functionality that is classified as a Feature and then broken into independent stories without a formal Story Mapping process. This is the context of my own Product Development team that builds the Nimble product.
I have not put a detailed explanation for this. I assume it is pretty self-explanatory in the context of what is explained for Scenario I. Just to highlight 2 key differences:
a) You will see the absence of the Story Map process because most of the Enhancements are fairly small in size. An occasional Feature can be broken into Stories without an elaborate Story Mapping process.
b) The Planning Value Stream (called the Story Grooming Value Stream) ends with the generation of the Stories for the downstream.
I have elaborated two examples of how this process can pan out. I hope this helps open your mind on how upstream process can be Value Streamed to get better “Groomed Stories” that your downstream can focus on and execute without impediments and in full flow. Feel free to share others if they work for you in a different context or environment.