Elastic Scale-out is Everything

Klint Finley, in his piece about Most Over-Hyped Cloud Technologies, reports that readers identifed “NoSQL” as the #1 most hyped technology.

That’s interesting, but to be fair it’s not really fair.  To start with NoSQL isn’t a technology.  People sometimes refer to it as a “movement”, but even that isn’t really right.  NoSQL really reflects a rebellion, an appropriate one, against the anachronistic limitations of corporate database policies, against the historical limitations of the LAMP stack at the data management level, and against the generally sclerotic state of database systems, particularly as relates to architectural innovation.

In short NoSQL is a rallying cry for senior technical folk who are rightly pounding the table and declaring that database technologies of the 20th century just don’t cut it when it comes to 21st century data management challenges. Web-facing systems contend with extreme data volumes, extreme concurrency, extreme unpredictability and scalable but unreliable commodity infrastructure.  NoSQL is the wake-up call for database technologists: Your 30-year-old designs don’t cut it.

The traditional “OldSQL” databases are antithetical to cloud architectures.  Call up your database rep and ask them for a database system that can be run on 1, 10, 100 or 1000 machines according to demand.  Even better, try getting a system that can scale elastically in real time from 1-1000 machines and back again many times per day.  You can do that with the presentation layer.  You can do it with the application layer.  You can do it with the storage layer.

The NoSQL message is not the question of why databases don’t do this, but the bold assertion that we’ll do whatever it takes so that databases can do this.  Dynamic scale-out is non-negotiable.  NoSQL says we’ll jettison anything and everything if we have to in order to get elastic scalability, and that includes powerful query languages, database consistency guarantees, durability, atomic operations and just about everything else.

But there’s always a But.  What happens when we work out how to design elastically scalable databases?  Most obviously what happens is that the requirement for elastic scalability goes from being the only thing that matters to being the baseline requirement.  The stuff I had before (application-independent content, a powerful query language, data consistency guarantees, etc) suddenly become more important again.  And at that point the pieces of a database system that people have had to build into their applications (at gigantic cost) will get pushed back into the database server where we would all like to see them reside.

At NuoDB we love the fact that NoSQL has shouted so loudly that we need to  stop the train, that the database track we have been on leads in a very different direction to clouds, utility computing, dynamic scale-out, commodity datacenters etc.  The message is clearly correct – it is time to change.

We agree that elastic scalability is the highest priority.  NoSQL and NewSQL agree that scale-out is non-negotiable but where the two diverge is that the former stops at saying elasticity is everything while the latter argues that it is the table-stake.  NoSQL says stop the train, while NewSQL points at new tracks that connect the last generation of databases to the next generation of databases.

Whether or not NoSQL is the #1 Most Over-Hyped Cloud Technology it has served the vital purpose of highlighting the bottleneck in the cloud computing stack.  In future anyone developing cloud-style database applications will demand elastic scalability, and that is a huge step forward.

Add new comment