Scaled Agile Integration Testing

A guest post for Janet Gregory's 'Agile Testing' blog by Carey Fletcher, Qual IT Test Manager

My first exposure to agile testing had me “diving in the deep end”, attempting to define a test strategy for a complex insurance system which was transitioning from a waterfall to agile methodology. At the time there was minimal discussion in the testing community on how you would approach system integration/UAT testing within an organisation with a scaled agile approach.

After a discussion with Janet Gregory at a conference (co-author of “Agile Testing”) and armed with the “Agile & QA organisation structure” used by Hartford Group, I decided the most effective approach was to convert the existing waterfall testing team into a centralised QA team.

The Central Test team was a hybrid waterfall and “post-agile development” team. At the beginning of the month we were delivered a large release from the external vendor’s sprint teams. This release was functionally tested from an end to end perspective over a 5-week period. During this period the internal agile teams, working at a 2-week sprint iteration, were deploying “Potentially Shippable” releases into the Central Test team for system integration testing (SIT). The 6th week was a dedicated regression, at a system integration/UAT level. Using this approach, we had regular 6-week releases deployed into production, for all components requiring system integration.

The central testers and agile teams worked together to define the scope of the additional “post-agile development” testing, including targeted regression, system integration, UAT, non-functional or any additional “…ibility” testing. The central and agile team’s one-page test plans and the “definitions of done” has been met, provided the business owners and production release management with a clear understanding of the quality of each release.

There were, of course, significant improvements that could be made to increase speed to production, by adopting further agile practices within the Central Test team, but we had at this point proved the testing organisational structure and testing flow in an agile environment worked.

How we changed and adapted the processes:

Initially, we were repeating some of the agile team’s testing in the central environment because the quality of the releases was low but as the quality improved we moved to the agile team smoke testing their own releases and the Central Test team executing the agreed “Post development” testing.
The business assigned a risk rating factor to all the end to end business processes and scenarios (positive and negative) to assist with defining a risk-based approach to the targeted regression required.
Introduction of automation, initially the smoke tests; API verification; and business process with a high-risk rating.
Introduced scrum of scrum of scrums, which was a nationwide view of all development and included the Dev Ops managers, production release management and the Central Test manager. The scrum of scrums had a project level view, whereas the scrum of scrum of scrums had a release management view.
A Test Environment Manager was appointed, to ensure the central test environment is managed as a “Production like” environment.
This Central Test team approach was successfully applied again, by Qual IT, during the system integration and UAT of a Government system which was used by every person in New Zealand. The agile development approach allowed for functional testing; integration testing with adjacent components and an early view of performance testing, but a central team was required to verify the complex data flows across all the components of the system, from a functional and non-functional perspective.

I am often challenged when applying a Central Test team strategy, in a project which has adopted the agile methodology, claiming that I am “old school”, applying waterfall principles which will provide a barrier to delivering working software frequently. This would be true if there are minimal integrations between potential shippable releases but in a complex system with multiple business process integrations, the Central Test team approach works effectively in parallel with the agile development teams, providing that additional testing support, which is not possible within an individual scrum team.