Tag Archives: nosql

For the reader who is not familiar; the title of this article reads: NoSQL not equal NoDBA. And what I mean by it is that while the traditional function of the DBA is different in the NoSQL environment; one still needs a subject matter expert (SME) on the payroll in order to keep the “engine” running smoothly. NoSQL is just another specialty.

Many years ago I was caught-up in SleepyCat’s BDB libraries. They worked, they were fast, and as they promised; you could forgo a DBA. I developed a few proof of concept applications using BDB and they worked great. They included speed, big data, ACID and everything they promised. Luckily for me, at the time, the projects never ran long enough for a disaster to occur. I know now that, at the time, I did not know enough about BDB to recover from even a moderate system failure.

Today we are inundated with NoSQL alternatives. Riak, MongoDB, Redis, Cassandra, Volt, Orient; just to name a few. To my knowledge, none of them actually state that a DBA is not required, however, they all seem to imply that your developers are going to assume the responsibility. At least Riak and MongoDB have enterprise consoles for the NOC (network operations center) suggesting that they realize otherwise.

Let’s start with the schema. Most developers will knock out their first or second iteration of the schema over lunch. And in most cases it’s probably pretty simple. It’s not until you get into production that “you” realize the warts when your perfect parochial schema. I’ve implemented several payment systems. The first holds 12B active accounts and processes 12M sale transactions a day(333TPS). The second had a hard time at 25TPS. The first contained only 5 tables and the second was a beautiful 100 table constraint nightmare.

And then there is “real world” data. For example, when you’re doing 12M transactions a day Oracle it’s still a challenge to export the data so that it can be warehoused and reported upon. ETL is going to take time. That’s when one might consider sharding and other approaches to optimization; even normalization (all functions that should be performed by a DBA). However, in the NoSQL/NoDBA world, this function is going to fall on the developer… who is no longer working on new functions or revenue generating opportunities but is instead sandbagging the dam.

As far as SME’s go. They tend to know vertical markets or applications very well. They tend not to know every last detail about the data store.

For example, there was a time when my DOS based PC would crash and I’ve have to fix my harddisk. There was a time when I could and would repair the filesystem by hand, however, after Norton Utilities performed that function in a fraction of the time I had to turn in my keys. And now, when that type of failure occurs on my Linux machine I simply reinstall. I do not have the time or the inclination to repair the data.

That function was always left to the DBA when it came to traditional RDBMS and the sysadmin when the filesystem went bad. I just cannot imagine that anyone would want to perform that function when there are people who specialize in it.

So just because you have read the docs for the client libraries and maybe the source code. None of that makes you a SME. And there is nothing that is going to replace the SME. Just because you’re not calling him/her a DBA does not mean that the function is not being performed.