Case study

Building The Big Picture At ATB Financial

How Agile By Design brought macro transparency and forecasting practices to a large scale agile delivery initiative.

ATB Financial

Banking and Financial Services

Founded 1938

5,000 employees

Revenue (2023): $1.9 billion (CDN)

Building The Big Picture At ATB Financial

The Challenge

ATB’s banking group had successfully stood up 10+ concurrent agile teams, with talented team members, great culture, and excellent technical practices (full-stack development, full automated test coverage, CI/CD). Each team was capably managed, and displayed all the basic behaviors and practices that you’d expect from a well-functioning team.

One of ATB’s major delivery initiatives had some challenges; however, those challenges had nothing to do with the six assigned delivery teams, how they planned, or how they went about their work. Instead, they were more macroscale: the leadership team didn’t have a clear picture of what was going on within the initiative, and did not have a clear roadmap to the end of what to this point had been a multi-year commitment.

The Approach

Agile By Design took on leadership roles for this major initiative and started with consolidating information. Several feature lists were in use (Jira, Excel, PPT, Technical vs Customer roadmaps), and needed consolidating into one consistent source of truth. Working with key ATB personnel, the team co-created a program level Kanban containing the single list of customer facing and technical features, as well as outlining a high-level delivery process that was purposefully agnostic of team delivery framework (Scrum, XP, Kanban).

To address the long-term delivery forecasting challenge, the team worked closely with ATB team leads to relatively flash size the ~80 remaining identified features and use existing throughput data to construct delivery forecasts using Monte Carlo simulation techniques. These forecasts were progressively refined as individual features were themselves refined into workable user stories for each team. Forecasts were also updated every two weeks based on the latest scope and productivity information and actively shared with both divisional and senior leadership as a part of steering committee sessions.

Additional internal communication hurdles were cleared by establishing lightweight recurring cadences for things such as new feature intake, risk tracking and action planning, and exploratory testing sessions for new features with key stakeholders.

The Impact

The implementation of a program Kanban immediately brought clarity to current delivery status and future feature sequencing and could also easily be shared and understood by a wide-ranging audience. Churn decreased considerably, status reporting effectively vanished, and time could be spent instead on addressing challenges escalated by team members.

Executive confidence in the initiative rose significantly as a result of the increased clarity and clear methodology for producing delivery forecasts. Interim milestones were also able to be forecast (with greater accuracy) and this data was used to set expectations and trigger operational readiness activities. Initial forecasts based on flash sizing also proved to be quite robust, even after additional refinement (within 15%), demonstrating the power of the law of large numbers.

Exploratory testing efforts also drastically improved the optics of the program with key stakeholders, and provided great opportunities for hands-on feedback along with drastically slashing customer-reported bugs.

Although we were not able to see the initiative through to absolute completion, 10+ releases occurred during the 18 month engagement, and a well-functioning operating model was handed over to internal personnel for the final leg of what had been a long and arduous road.

‍

Key Concepts

Kanban

Literally, from the Japanese, 'visual signal.' Aphysical representation of a human system wherein work items are posted ascards on a board to visualize progress within a process.

Monte Carlo Simulation

Monte Carlo Simulation is a statistical technique used to understand the impact of risk and uncertainty in prediction and forecasting models. It was formally developed in 1946 by physicists working on the Manhattan Project and was named after the Monte Carlo Casino in Monaco, reflecting the random nature of the method similar to roulette.

The Monte Carlo method involves building a model of the scenario of interest and using random numbers to simulate a range of possible outcomes. This process is repeated many times to generate a distribution of results. The technique is widely used in various fields, including business risk modeling, weather forecasting, telecommunications, and artificial intelligence.

In agile, Monte Carlo simulations can be used for forecasting completion dates or understanding potential delivery times by simulating various scenarios based on historical data and variability in throughput. This method provides a probabilistic approach to project planning, offering insights with different confidence levels, which is more robust than linear forecasting models.

‍

Flash Sizing

Flash sizing is an agile estimation technique designed to quickly estimate the size of work items by comparing them to previously completed items. This method allows teams to rapidly assign relative sizes to new tasks by leveraging their collective experience and knowledge. Instead of detailed analysis, flash sizing focuses on quick, relative assessments, enabling teams to maintain momentum and make swift decisions about their work backlog.

The approach emphasizes simplicity and speed, using past work as reference points to gauge the complexity and effort required for new items. By asking questions like "Is this task bigger or smaller than a similar task we completed before?", teams can efficiently size their work without getting bogged down in detailed estimation processes.

Exploratory Testing

Exploratory testing is an approach to software testing where testers actively explore the application without predefined test cases. Instead, they use their creativity, intuition, and knowledge of the application to identify potential issues. This method emphasizes learning, test design, and test execution simultaneously, making it highly adaptable to finding edge cases and unexpected behavior.

In an agile context, exploratory testing can be particularly effective as it allows testers to quickly adapt to changes and uncover issues that might not be anticipated by scripted tests. It is often used in combination with other testing methods, such as automated and regression testing, to ensure comprehensive coverage and high-quality software.

‍

Books

Our Podcast

Agility with Jeff