Building Resilience: How Atlassian Untangled Dependencies to Fortify its Cloud Platform
disaster recovery is a critical concern for any cloud-based service, and Atlassian recently undertook a meaningful effort to bolster its platform’s resilience. A comprehensive tabletop exercise revealed vulnerabilities stemming from complex interdependencies within their infrastructure.This led to a basic re-architecting of their systems, prioritizing stability and faster recovery times.
The Problem: A Tangled Web of dependencies
In 2023, atlassian simulated 6.5 days of recovery efforts to proactively identify potential weaknesses. The exercise highlighted a concerning reality: a single point of failure could cascade through numerous services, bringing down a significant portion of the platform. Initial testing showed only three services functioning, while a subsequent “after” scenario revealed dozens remained offline due to intricate dependency chains.
These dependencies, notably circular ones, create a fragile system where a problem in one area can quickly propagate and amplify. This is a challenge many large organizations face, and recent high-profile outages at providers like Cloudflare and AWS underscore the importance of addressing them.
The Solution: A Layered Architecture
Atlassian’s response was to move away from a monolithic structure and embrace a layered architecture, often described as a “layer cake.” This approach strategically organizes the cloud infrastructure, minimizing the impact of failures.
Here’s how the new system works:
* Lower Layers: These foundational layers have the fewest dependencies, providing a stable base.
* Upper Layers: These layers, built on top, naturally have more dependencies but are isolated from the core stability of the lower layers.
It’s important to understand that complete elimination of dependencies isn’t realistic or even desirable. Rather, Atlassian focused on managing them effectively with these key principles:
- A component in layer (N) should only depend on lower layers (N → N-1 = Good).
- Dependencies within the same layer (N → N = Bad) are strictly avoided.
- dependencies on higher layers (N → N+1 = Bad) are also prohibited.
Key infrastructure Changes
Beyond the architectural shift, Atlassian implemented several crucial infrastructure changes. They migrated Artifactory from Micros to Kubernetes, successfully breaking a critical circular dependency. Furthermore, they developed a new provisioning system called Atlassian Platform Deployer (APD).
APD leverages AWS CloudFormation for deployment orchestration, offering a more streamlined and reliable process. This new system proved instrumental in launching Atlassian’s recently announced Government Cloud and eventually, migrating Micros itself to APD.
Results and Future Outlook
the effort has yielded significant improvements. While internal circular dependencies still exist, Atlassian has eliminated hundreds, resulting in a more robust and easily recoverable platform. This is particularly vital as the company transitions towards an all-cloud model, encouraging all customers to migrate from on-premise solutions.
You can rest assured that Atlassian is prioritizing platform stability.By proactively addressing dependency issues, they’re building a more resilient future for their services and ensuring a smoother transition for your buisness. A reliable platform is essential, and Atlassian’s commitment to untangling its infrastructure demonstrates a dedication to providing just that.
Worth a look