NuoDB as a Docker Database
This video demonstration scales out a NuoDB database across three Docker containers, showcasing how engine processes can fail without database interruption and how minimum service levels are enforced in NuoDB.
This tour is designed to showcase the scale out, continuous availability, and administrative features of NuoDB. The tour was built on Docker container technology and demonstrates how easy it is to install a NuoDB database, load data, and leverage distributed processing capabilities across multiple containers, with the ability to withstand failure.
We are currently running on an Ubuntu VM with Docker installed on a MacBook.
To start off the tour let’s navigate to the NuoDB_TGT directory that we had downloaded.
The first we’ll run the setup.sh script. This is setting up our environment, and downloading NuoDB so it can be installed on the containers that we will run during the next step. Now that the download has completed, let’s run the deploy.sh script. To do this we have to enter the credentials that we’ll want to use to manage the NuoDB Domain going forward.
As you can see, the script is now starting 3 different containers: NuoDB1, NuoDB2, and NuoDB3. Each container has the NuoDB software installed and is part of the NuoDB domain. A domain a secure environment for deploying and managing NuoDB distributed Databases. A Domain can contain one or more Databases and each Database can run on one or more hosts.
Now that we have deployed our containers, let’s take a deeper look at our environment.
We can now see is that the we have:
- Three Docker instances, which are our three separate NuoDB hosts.
- A Java API, which is the NuoDB REST service providing a REST API for NuoDB domain administrative tasks.
- Three Java Nuoagent.jar processes, which are the NuoDB brokers, one running on each of the three NuoDB hosts.
What we will do next is start using the NuoDB manager to interact with our database.
The NuoDB Manager is a command line tool that lets you monitor, analyze, and control a NuoDB domain. What we’ll do first is a domain summary. The domain summary gives us a high level view of what is going on in our domain. As you can see by the output, this confirms that we have 3 separate hosts, which are represented by the Docker containers we installed earlier.
Now that we’ve seen our domain, let’s actually do something interesting by creating a database. I’ll use the NuoDB manager tool to specify my database name, username, and password. From there we have multiple template options. For this case, we’ll choose a Single host template. Finishing up, we can set other database parameters, such as memory allocation and what host we would like to start the database on. Now that we have created our database, let’s see the domain summary again.
Now our domain summary shows two additional processes running, one transaction engine and one storage manager, giving us an active single host database.
In just a couple simple keystrokes, we’ve got a new database up and running and ready for action. Now that we have this database running, let’s execute some simple SQL commands to create a table, insert some data, and query it.
For this next section, I’ll take some SQL statements that are in the NuoDB Tour instructions and copy them in. The first will create a table user, while the second will enter data into the table. We can run a quick select star statement from the users table to see all the data we just entered. If we want to drill down further, we can run a show table to see the fields and their data types.
So what we’ve seen so far is pretty standard for a SQL database. But it’s worth getting the basics of a relational database out of the way before we show the cool abilities that come with being a distributed system.
Now that we’ve played around with some basic SQL and a single host database, let’s make things more interesting by updating our demo database.
Let’s say I’m not happy with the lack of resilience in my database and I want to configure some redundancy. How do I do that?
We’ll need to log back into the NuoDB manager to connect with our domain.
Changing the processes that we have running is as simple as updating the template from single host to “Minimally Redundant.” With this change to the “Minimally Redundant” template, several important capabilities of NuoDB come to play. First, there are now two TEs and two SMs in operation, which means that if any process goes down the database remains up and running. Also, there are now two copies of the data on disk, which provides data redundancy. Finally, with extra TEs and SMs in operation on additional hosts, the overall performance of the database would increase. This is called "Horizontal Scale Out", best of all we accomplished this with no interruption of service.
Now that we’ve scaled out our database let’s make things more interesting by importing some more data. We can copy some SQL files that are part of the NuoDB tour package and run them to insert some sample hockey data sets into our database. Now that we’ve inserted our data, let’s take a look at what is in there. First the hockey schema that we just created. In there, we can see our hockey, players, scoring, and team tables.
Now let’s drill down further into the players table. As you can see, this table contains all the information that would normally be listed as roster information, such as position, height, and weight. We will do a quick query to ensure that these tables will behave just as we would expect, even though we have an extra TE and SM running.
As you can see, the results are what we would expect from a query with a limit of 10.
Now that we’ve imported some data into our database, let’s drive some load to make this more interesting. We’ll start some sample JDBC clients that will continually query the tables in the hockey schema. To start these clients, we will simply run the three clients that were shipped with the NuoDB TGT package. Now that we’ve got these three clients running, let’s see how they are interacting with the database.
We’ll do this by going back to the Nuo SQL tool, and querying the SQL strength and nodeId from our systems.connections table. This will output the SQL strength and what node each JDBC connection is accessing.
The interesting thing to pay attention to here is the changing SQL strengths and node IDs. NodeId 2 and 4 are the TEs, and we can see that our transactions are being evenly distributed across both TEs. You can also see my query of SQL strength. It is just another connection to be managed.
This required no application tuning or database tuning, besides making the update to the database template. If your database ever becomes bottlenecked, simply scaling out to more TEs can give you the transaction numbers you are looking for pretty much instantaneously and without an interruption to the workload.
Let’s take a look at our domain summary before we go forward with our next step, which will be unexpectedly killing a TE that is running. What we will want to pay attention to here is the nodeId. To shutdown a process, I will have to enter the host tag and specific nodeId that I want to kill.
Let’s now check the domain summary to see what’s happened. As you can see there is no longer a nodeId 2 and a nodeId 5 has been added to the domain. This is because the demo database has a service level agreement enforced by the “Minimally Redundant” template. If any process goes down, NuoDB will automatically start and sync a new process to meet the redundant set up of two TEs and two SMs. If I had ran the domain summary quicker, I might have caught the database before the new TE was started, but I would have had to be fast.
Let’s go back and use the NuoSQL tool to check that the new TE, nodeId 5, is processing transactions like it should be. As you can see by the output of the query, nodeId 5 is processing transactions as expected.
Not only can a NuoDB database by updated without downtime, but it can adapt to failures as well. Once again, without any changes to your application.
That’s the end of the demo! We’ve seen install, set up, initial testing in one virtual host, scale out to multiple hosts, and resilience to failure all in less than 10 minutes. And now I am cleaning out my installation, just to reassure you that if, and I hope you will, try the tour yourself, it is not going to leave anything lying around when you are done. Thanks for watching and visit the NuoDB website to take the tour for yourself.