Have you ever heard about DB Clustering (eg MySQL Clustering)? If not, I would give you a quick summary of it.
Before, I worked at a big company based in Munich, Germany (Still working in Vietnam tho haha). Oh boy, they do have a big monolith monster project, the size, code,...
Man, imagine you have to scale a project like that, millions of users (from public to internal), the DB write operations are insane.
There is no way a traditional 1 write - N replica would work. Write instance will and always be a bottleneck.
Then, here I welcome you to the DB Clustering!
DB Clustering details and the Pros
Just like how you deploy your backend project and put it under a Load Balancer, then share the traffic between the instances.
DB Clustering is just as same as that:
- Shared-nothing: my machine, my usage
- Sync replication: crazy right? replication for replica is async
- Automatic failover
- High availability
In Tech, there will be no silver bullet for everything. Same as clustering, it would have its own Cons, eg:
- Self-managed (yeah, seems like cloud services still prefer the traditional way - write replica)
- But once you perfect the setup (self-healing, fail-over,...), you're done.
- Some limitations, check out MySQL
But the PROs here are way better to go with.
Apply to a Monolith Project
With the PROs above, for example, I have an App with MAU (monthly active users) of around 1M. Here would be my perfect setup:
- Load Balancer: 4 instances, using the round-robin algorithm.
- 5 clusters:
- 4 active, using the round-robin algorithm.
- 1 standby, will jump in if any active instance is down.
- 5 clusters:
If I needed a Queue Worker, then I would give it a dedicated cluster as well 🥳 Thus, it can work independently without taking all the resources from the HTTP layer.
Then I don't have to bother about the monolith structure (yes I know it is bad), but extracting a big app into multiple services aka microservices would take years without new features.
And I can sleep in peace without worrying about spiking in the database when the write operations are high as fk.
- Small/medium app (even big): let's go with the DB Clustering to help you scale while you're only focusing on the feature development.
- Super big app: DB Clustering for your own microservices.