Should You Be Sharding Your Database At All?
In my last blog, I talked about database sharding, exploring some of the reasons for doing so, as well as the challenges inherent to the approach. Given the many challenges with database sharding, it's no wonder that organizations and vendors are seeking alternatives.
For those who need an ACID-compliant relational database, you may want to consider an entirely new architecture, one designed around the idea that elastic scale-out is an essential requirement right from the start. We hear consistently from customers and prospects that they want to be able to start a database on one machine and scale out as they need additional capacity, cache, parallelism, or support for more clients.
In order to achieve that goal, however, the fundamental structure of the database needs to be reconsidered. When it comes to relational databases, many consider it incontrovertible that a database is a single unit with transaction consistency and data durability lying intertwined. But to effectively achieve a scale-out database, these two areas - one related to processing power and the other to data replication - need to be considered separately.
By doing so, you can provide a system that scales horizontally; that is always consistent; that replicates data correctly and consistently; and in the face of failure, always fails correctly. You still get the view of a single logical database, no matter how many hosts you’re running on, how many processes you have going, or how many datacenters you’re running in. You’re still programming to a single logical database. You’re still working as if there’s a single logical database. Which means that your developers are free to write their applications without having to think about operational concerns. Plus, your operations people are free to scale the database however they need to, to take advantage of their resources. Then, when you need to bring things online on demand, or when you no longer need additional resources and you want to shed them, you can do that at no effect to the application itself.
On the other hand, it’s entirely possible to use traditional scale-up database management systems (DBMSs) that are not cloud-native. These historical RDBMSs were designed 30 years ago, long before the advent of the cloud. They can be engineered to support cloud applications, and sharding/replication is certainly part of that. Making these traditional models meet the promise of the cloud involves time and effort that can delay companies and independent software vendors (ISVs) from reaping cloud benefits.
Check out our tech blog to better understand the NuoDB alternative to database sharding.