The Importance of Backlog Management from a Developer

The speed at which software developers can deliver features within a given time period is increasing rapidly, particularly within the last decade. However, their ability to generate and collect ideas often becomes compromised, creating an organizational impediment to organization.

A software development backlog describes the product’s future, but it often becomes overloaded. This tendency can delay a product’s time to market and reduce its quality. Restoration of the backlog can be a key requirement for getting a project back on track and fostering the innovation that’s so important for successful software development. This post explains the importance of the backlog and how it affects software development. It also discusses how to prevent the backlog from becoming overloaded, which is essential for maintaining a productive development environment through a clear backlog management strategy.

Overview of Product Backlog

A backlog is a list of actions that a development team may complete to achieve a desired outcome for a software application. It can include bug fixes, new features, changes to existing features and infrastructure changes among other possibilities. The backlog is the most authoritative source for team activities, meaning that the team shouldn’t work on anything that isn’t on the backlog. However, the mere fact that a task is on the backlog doesn’t mean that it will ever be delivered, since it’s only an option for delivering an outcome rather than a commitment. Adding an item to the backlog should be a fast, easy process, as should the removal of an item.

The development team owns the backlog and may assign a specific team member with responsibility for maintaining it. Product backlogs may have a variety of formats, although user stories are the most common. Story maps and information radiators are also common tools that team members use to describe backlog items.

The development team determines the specific format that will best serve as a reminder of tasks they may work on. Backlog items vary considerably in size and detail, primarily depending on when the team expects to work on them. An item that the team will work on soon should be smaller, but contain enough detail to complete the task. Items that lack an expected start date may be broader in scope with less detail.

The sequence of items in a backlog routinely changes as the development team gains a better understanding of the desired outcome and the best solution for achieving an outcome. In addition, the frequent removal and deletion of backlog items makes the backlog highly dynamic. Team members may also use the backlog to communicate their current tasks with other members.

Index cards and sticky notes are common methods of representing a backlog physically. However, it can also be represented digitally with generic text files or spreadsheets as well as dedicated backup management tools. Physical backlogs are most useful for keeping the backlog continuously visible and facilitating group discussion. Digital backlogs are more effective when team members work in physically separate locations or backlog items require a large amount of documentation.

Workflow

The figure below shows the most important processes for managing a backlog. Each process will have its own specific goals.

The entire backlog management process must be properly configured before software development begins, as it can be quite difficult to do so after the project starts. Building the pipeline needed to manage project requirements solves many problems by ensuring this process remains transparent and providing the project’s intended vision to all stakeholders. It also helps to reduce the cost of maintaining the backlog.

Balance

The process of balancing the backlog can be visualized by imagining a series of containers, each of which corresponds to a particular action. In this model, balancing the backlog requires you to add the same amount of fluid as you remove, keeping the fluid in each container at a constant level. The diagram below illustrates this process:

Assume for this example that you have a total of 1,200 hours' worth of work already in progress. You have three crews, each of which can perform 400 hours of work in a week. The work in progress will therefore require one week to complete, resulting in no backlog since all work is currently in progress. However, you then add another 1,200 hours' worth of work, increasing your backlog to one week. Assuming a maximum allowable backlog of one week, you wouldn't be allowed to accept more work for another week.

User Stories Activity

The workflow of software development is typically neat and simple at the beginning of a project. Everything appears to be under control at this point, but the backlog is usually larger and more complex over time. This trend is the result of actions such as running multiple sprints at the same time, fixing many bugs from these sprints, managing multiple work items and grooming the backlog for new requirements. Product backlog refinement, also referred to as backlog grooming, is essential for organizing and prioritizing tasks effectively for future development. The following chart illustrates this process:

It’s essential to track backlog activity regularly to ensure it doesn’t get out of hand. In particular, you need to remain aware of the number of work items, their type and who is responsible for each item.

Understanding the Product Backlog

The product backlog is a crucial component of Agile product development, serving as a dynamic list of features, user stories, bug fixes, and process improvements that are prioritized and refined over time. It is a strategic asset that guides the development team toward delivering high-quality products that meet customer needs and expectations. The product backlog is owned by the Product Owner or Product Manager, who is responsible for ensuring its accuracy, relevance, and alignment with the product vision and goals.

In essence, the product backlog acts as a roadmap for the development team, providing a clear direction on what needs to be done and in what order. It is a living document that evolves as new information becomes available, ensuring that the team is always working on the most valuable tasks. By maintaining a well-organized and prioritized product backlog, the Product Owner can help the team stay focused and aligned with the overall product strategy.

Problems Caused by Overloaded Backlogs

The backlog is the main tool for managing and implementing the roadmap in the Agile development methodology. An overloaded backlog can endanger the project, especially when it’s the result of uncontrolled growth. A large, unkempt backlog creates stress on developers, and can eventually cause them to abandon the backlog. Collecting customer feedback is crucial to ensure the backlog aligns with customer needs and expectations.

A proactive vision is needed to avoid reactive planning for each sprint, which eventually results in an unmanageable, relevant backlog. In these cases, the development teams can lose sight of their long-term goals and resort to simply completing individual tasks. The specific problems of overloaded backlogs include increased maintenance costs, reduction of information value and inhibited innovation.

Maintenance Costs

An overloaded backlog has a high maintenance cost because the number of user stories that each team member is working on increases with the backlog size. When the team becomes over committed, it also increases the time needed to complete a particular item. The portion of a backlog that is committed acts like a queue, meaning it follows the rules of queue behavior. Specifically, each committed item requires some attention on the part of the developer to remain valid. Developers can only provide meaningful attention to a finite number of items, which is particularly detrimental in the case of cues.

Information Value Reduction

Grooming an overloaded backlog can appear to be an overwhelming process, especially when most items in the backlog appear to be insolvable in practical terms. This process is the most common reason for backlogs to become obsolete. Furthermore, an individual item appears insignificant when it's part of a backlog containing thousands of items, no matter how brilliant that one idea may be. In this case, the backlog can become a trash can containing all the work that developers want to complete that never will.

Inhibited Innovation

An oversized backlog impairs innovation because reorganizing it is a tedious chore. It causes developers to always add brilliant ideas at the very beginning or end of the backlog, neither of which is good practice. Adding items to the front effectively invalidates the rest of the backlog, while adding them to the back virtually ensures they will never be completed. Furthermore, team members working under these conditions will lose their incentive to contribute their best ideas, resulting in declining productivity.

Backlog Size

A development team needs to know how big their backlog can become before it becomes a problem. An experienced team lead can often answer this question with acceptable precision, but specific formulas also exist to make this determination. These methods include estimating the number of backlogged items per quarter and the time needed to review each item.

Backlogged Items per Quarter

This method estimates the maximum number of items that a team can backlog each quarter, although the actual recommended value can vary considerably depending on the organization, product owner and team composition. Assume for this example that a product owner is responsible for maintaining the backlogs for the next quarter. The team performs six to eight sprints in each quarter, with each sprint requiring a maximum of two weeks. Past behavior shows that the development team averages six to eight user stories in each sprint, meaning the number of user stories in the backlog for each quarter should not exceed 7 × 7 = 49. The project manager should therefore investigate the backlog any time it has substantially more than 50 stories per quarter.

Time Needed to Review Items

Estimating the maximum backlog size based on the time needed to review each item requires you to divide the number of items in the backlog by the number of product owners on the development team. This quotient represents the average number of backlog items that each product owner is responsible for. Assume for this example that each backlog item requires five minutes of discussion for each team member to grasp it. You can then calculate the amount of time in minutes that will be required to discuss the entire backlog by multiplying the number of backlog items by five. Divide this figure by 60 to convert the discussion time into hours.

The next step is to determine if it's feasible to spend this amount of time discussing the backlog given the availability of the team members and the other demands on their time. If this time commitment isn't even remotely feasible, then reducing the backlog size should become a priority.

Causes of an Overloaded Backlog

An overloaded backlog usually has many causes that intertwine, but they're generally driven by human nature. The most common causes include the following:

• Hoarding
• Misusing granularity
• Excessive dependency resolution
• Over commitment by product owners

Hoarding

Many people are reluctant to discard anything, but it's especially difficult to do so for things that may be valuable. This is also true of backlog items because a larger backlog can make it seem that the project is protected from failure. However, the opposite is more likely to be true, making it important to remove backlog items when it will increase the chances of a successful product. If an item has been in a backlog for more than two years, you should strongly consider retiring it. If it truly is a great idea, it will eventually make its way back into the backlog.

Misusing Granularity

Project managers often believe that tracking backlog items at a highly granular level allows them to predict and control law-term problems with scope creep. This tendency is especially strong in conservative managers and those who are just beginning to adopt an Agile methodology. However, a granular backlog is actually against Agile development because it assumes the development team knows exactly what it will do in the distant future.

The granularity of backlog management should reflect the fact that software development can be a highly uncertain process beyond the near future. There isn't a hard and fast rule to granularity but the DEEP acronym coined by Roman Pichler provides a useful description for an effective backlog. The D in this acronym stands for “detailed appropriately”, the first E stands for “estimated”, the second E stands for “emergent” and the P stands for “prioritized”. In other words, the granularity of backlog items should remain as coarse as possible, especially for items in the distant future. Use a granular approach only when absolutely necessary.

Excessive Dependency Resolution

Project managers often work hard to resolve dependency chains in the backlog, typically when the project is not yet completely Agile. This process involves breaking down larger backlog items into their components and assigning more specific goals for each task.

This may sound like an effective strategy for backlog management, but overdoing it can compromise the quality of the backlog in two ways. First, it greatly increases the number of items in the backlog, which also increases the management overhead. Second, it changes the developers' focus from providing value and achieving goals to completing tasks. Project managers who are tempted to resolve every task dependency in the backlog should trust the Agile process rather than trying to manage the backlog for the distant future.

Over Commitment by Product Owner

Product owners are often pulled in many directions during software development. For example, they must decide which customer ideas they will transform into deliverables. These decisions also involve achieving some consensus among the project's many stakeholders, each of which may have a different definition of a successful project. In addition, a product owner must understand the client's business, its customers and the overall market when deciding which features to implement.

Unfortunately, most software development firms don't appoint enough product owners to accommodate the size of their development teams. As a result, backlog quality decreases, causing both the product and the team to suffer. The solution to this problem is for organizations to take the product owner role more seriously by increasing the ratio of owners to team members. The higher this ratio is, the more manageable the backlog will become, making the team more productive.

Prioritization Techniques

Prioritizing the product backlog is a critical task that requires careful consideration of various factors, including customer needs, business value, strategic alignment, risk, and dependencies. There are several prioritization techniques that can be used, each offering a unique approach to determining the most important tasks:

• Kano Model: This technique plots customer delight against implementation investment to identify the most valuable features. It helps in distinguishing between basic needs, performance needs, and excitement needs, ensuring that the development team focuses on features that will truly delight customers.
• MoSCoW Model: This technique categorizes features as must-haves, should-haves, could-haves, and won’t-haves to prioritize them based on their importance. It provides a clear framework for making tough decisions about what to include in each sprint.
• Cumulative Voting: This technique involves allocating a fixed number of points or “dollars” to stakeholders, who can then spend them on the features they consider most important. It ensures that the most critical features receive the attention they deserve.
• Stack Ranking: This technique involves ranking features against each other to determine their relative priority. It is a straightforward method that helps in identifying the most important tasks at a glance.
• Smallest Effort First: This technique prioritizes features based on their ease of implementation, allowing the team to achieve quick wins. It is particularly useful for maintaining momentum and demonstrating progress to stakeholders.
• Cost of Delay: This technique prioritizes features based on their opportunity cost, ensuring that the most valuable features are implemented first. It helps in making informed decisions about which tasks will deliver the highest return on investment.

By using these prioritization techniques, the Product Owner can ensure that the product backlog remains focused on delivering maximum value to customers and stakeholders.

Backlog Grooming

Refining the backlog often begins by investigating the Design in Process (PIP) inventory, which includes all the backlog items currently in progress. Donald Reinertsen popularized the use of this term in describing the development of lean workflows, particularly for determining the amount of inventory an organization should stockpile at any given time. This concept is also used in software development by treating backlog items as inventory. Selecting the right backlog management tool that aligns with team workflows and enhances productivity is crucial for effective backlog management.

In other words, the DIP helps determine the maximum number of backlog items you should have based on the number of items you’re currently working on. There is no specific number that answers this question, but the DIP does provide a starting point for measuring the maximum backlog for each product owner. The primary limitation is the number of items that a product owner can effectively administrate. For example, doubling the number of product owners doubles the maximum recommended size of the backlog, all other conditions being equal.

Furthermore, the DIP inventory shouldn’t include work that won’t be completed for the foreseeable future. If the development team doesn’t expect to complete a task for years, even in the best case, that task shouldn’t be considered “in progress” for the purposes of its backlog status. Factors such as market maturity and product type help determine the maximum completion time for items in the DIP inventory.

In general, project managers should use Dunbar’s number as a guide to the maximum backlog size for each product owner. Anthropologist Richard Dunbar first used this number during the 1990s to describe the maximum number of relationships that a person can maintain at any given time. Dunbar’s number can be applied to backlog size because each product owners must maintain a relationship with all tasks in their respective backlogs. In this model of backlog management, product owners should limit their respective backlogs to 150 items.

Backlog Management Tools and Best Practices

Effective product backlog management requires the use of specialized tools and techniques. Some of the best backlog management tools include:

• Jira: A comprehensive project management tool that offers features such as prioritization, task management, collaboration, and reporting. Jira is widely used in Agile development for its robust capabilities and flexibility.
• Trello: A visual project management tool that uses boards, lists, and cards to organize and prioritize tasks. Trello’s intuitive interface makes it easy for teams to collaborate and stay on top of their work.
• Asana: A work management platform that offers features such as task management, collaboration, and reporting. Asana helps teams coordinate their efforts and track progress towards their goals.
• ClickUp: A lifecycle management software that offers features such as project planning, task management, and collaboration. ClickUp’s versatility makes it suitable for teams of all sizes and industries.

Best practices for backlog management include:

• Regular Backlog Grooming: This involves reviewing and refining the backlog on a regular basis to ensure it remains accurate and relevant. Regular grooming sessions help in identifying and addressing any issues before they become major problems.
• Prioritization: This involves prioritizing the backlog based on customer needs, business value, and other relevant factors. By focusing on the most important tasks, the development team can deliver high-impact results.
• Collaboration: This involves involving the development team, stakeholders, and customers in the backlog management process to ensure everyone is aligned and informed. Collaborative efforts lead to better decision-making and more effective backlog management.
• Transparency: This involves maintaining open lines of communication and providing visibility into the backlog management process. Transparency fosters trust and ensures that everyone is on the same page.

By following these best practices and using the right tools, teams can ensure effective product backlog management that drives successful product development and delivery.

Collaboration and Communication

Effective collaboration and communication are critical components of successful product backlog management. This involves:

• Involving the Development Team: Engaging the development team in backlog grooming and prioritization ensures they are aligned and informed. Their input is invaluable in assessing the feasibility and impact of backlog items.
• Communicating with Stakeholders and Customers: Regular communication with stakeholders and customers helps in understanding their needs and expectations. This ensures that the product backlog reflects the most current and relevant requirements.
• Maintaining Open Lines of Communication: Keeping communication channels open ensures that everyone is informed and aligned. This can be achieved through regular meetings, updates, and the use of collaboration tools.
• Using Collaboration Tools: Tools such as Slack, Microsoft Teams, or email facilitate communication and collaboration among team members. These tools help in sharing information, discussing issues, and making decisions in real-time.

By following these best practices and using the right tools and techniques, teams can ensure effective product backlog management that drives successful product development and delivery. Effective collaboration and communication not only enhance the backlog management process but also contribute to a more cohesive and productive development team.

Summary

Managing a backlog generally consists of keeping it large enough to keep the development team busy, but not so large that it becomes irrelevant. This challenge requires a simple strategy that uses the development team to periodically review the backlog and remove unneeded items. These items will primarily consist of tasks that are unlikely to be completed within a reasonable time frame. Ensuring that backlog items have the right degree of granularity is also a major part of ensuring the backlog remains at a useful size. Finally, the entire development team should regularly participate in refining the backlog.  

About Baytech

Baytech is passionate about the technology we use to build custom business applications, especially enterprise solutions that optimize business processes. We've been delivering software solutions in a variety of technologies since 1997. Our success is due to the skill and efficiency of our senior staff, which includes software engineers, project managers, and DevOps experts. All of our engineers are onshore, salaried staff members.

We focus on the quality, usability, and scalability of our software and don't believe in mitigating cost at the risk of quality. We manage project costs by implementing an efficient development process that's completely transparent and uses the latest standards and practices to build software right the first time. Contact us today to learn more about how we can help your business.