Context
The platform processes more than 10 billion messages a year, around 30 million a day, for enterprise customers. A single cloud provider was a single failure domain for the business. The plan was to move workloads from AWS EKS to Oracle Cloud OKE and re-architect the messaging infrastructure to run across both.
Constraint
Around 40 applications and 20 database servers (16 vCPU and 64 GB RAM each) were owned by about ten teams. Messaging customers notice minutes, not hours. Every step needed an agreed rollback position.
Decisions
- Wrote Terraform modules once and reused them for every cluster, node pool, ingress and observability stack, so nobody hand-built an environment.
- Put Rancher on top of both EKS and OKE, so onboarding a team to the new cloud looked like what they already knew.
- Sequenced by blast radius: stateless services first, then anything with a queue, then the databases.
- Replicated databases into OCI, caught them up and verified them before moving writes. Cutovers were rehearsed on non-production workloads until the runbook was boring.
- Shifted traffic gradually with lowered DNS TTLs and weighted routing, holding between steps while watching error and latency dashboards.
- Gave every batch an owner on the app team and an owner on the platform side. With ten teams, coordination was the hard part.
Result
- Finished on schedule with zero customer-facing downtime.
- Cloud spend dropped by $15–20K a month (10–15%) while reliability improved.
- The messaging infrastructure now runs across AWS and OCI, with a real failure domain between them.
- The Terraform modules became the organization’s standard way to provision infrastructure.
What I would do differently
Build the IAM and networking translation layer earlier; it blocked more batches than any Kubernetes issue. Publish a shared migration-progress dashboard per team from week one. And delete the temporary dual-run tooling as soon as the last batch lands, because temporary infrastructure has a way of becoming permanent.