Why team autonomy is the key to scaling in regulated environments

Building a bank from scratch requires dealing with regulations around financial institutions. But this does not mean that we cannot apply modern software development practices and state-of-the-art software engineering approaches to be fast in developing new features and products.

Especially in a regulated environment, more complex processes and responsibilities are required. This can end up in a scenario where complicated manual processes and organizational dependencies slow the development down. And as a company grows this problem increases exponentially.

Autonomous teams to the rescue

We can address complicated processes by simplification or by a high level of automation. But for organizational dependencies the solution is autonomous teams.

Team autonomy is easy to measure: for every feature, measure the number of teams that are required to develop, ship, and run it. Ideally, this number is close to 1. This approach to autonomous teams is great for scaling a company and it works in a regulated environment as well.

Modular architecture

The architecture of a software system usually follows organizational structures. In an environment of autonomous teams, this usually leads to a modular architecture. The typical patterns are modular monoliths and microservices.

Modular architecture allows each team to implement their domain in a separate module. This gives each team its own space to develop features and makes them more independent of each other. Microservices even add the additional benefit of being independently deployable and observable.

But every modular system comes with a common part: a platform on which modules can be built. This platform can include tools for observability or deployment. It also needs to define what a module or service looks like in order to be considered a part of the system.

Many companies keep this platform as lean as possible. But for a regulated bank, this is barely achievable. Therefore, our requirements on a platform are way higher than in other companies. This would require us to scale up our platform team significantly and increase the dependencies of other teams on the platform team. Over time the resulting communication overhead would notably slow us down.

Each team builds its own platform

As Solarisbank is using AWS to run their software systems, we can follow a simple alternative to a big central platform by letting each team build their own platform. That means that each team gets its own set of AWS accounts for stages like testing and production. The whole AWS account is owned by the team and works as a boundary for the ownership of the team. Everything inside the AWS account is the team’s responsibility.

Of course, this introduces redundancy because every team needs to build the same common parts. We address this issue by using Terraform modules to share the automation of infrastructure setups. This pays off in multiple ways. On one hand, teams can reuse solutions and be more effective. On the other hand, it allows us to standardize the setup in a way that is compliant with regulations. We force ourselves to automate as much of the setup as possible and reduce the risk of human errors.

Another benefit is that each AWS account only includes the required parts and is isolated from others. Some regulatory requirements, for example, need to be applied only to specific products. Therefore, we can separate the special requirements in the corresponding accounts without messing up a centrally shared platform. Also, the roll-out of updates can be done account by account. This way, issues can be limited to a specific account instead of impacting the whole company.

You Build it — You Run it

Since teams are not only responsible for their software, but also for the infrastructure and everything inside their AWS accounts, this also requires a different approach when it comes to operational tasks such as on-call. Our engineering teams are on-call themselves, because they know what to do best in the case of an incident. But it also ensures the right alarms are set and fine-tuned.

Developers being on-call also introduces a nice side-effect: when one team depends on another team by using their APIs, the team that owns the API will automatically be on-call for the other team’s product as well. So, teams have an incentive to avoid dependencies.

Splitting the monolith

Like many companies applying a modular architecture, we started with a monolith first. And of course, it still exists. The clear separation of AWS accounts per team pays off once again, when it comes to splitting the remaining parts. We can now ask a simple question: how do we make all teams independent of the monolith?

This allows us to define which parts need to be extracted and where they need to be moved to.

Conclusion

For us, AWS accounts are a core tool in our architecture. They allow us to go beyond the microservice approach and apply the same principles to infrastructure as well. As a result, we can extend the level of team autonomy significantly.

This setup allows us to scale up the company at a high pace, even when operating in a regulated environment.