MapReduce, NoSQL, andWhy Stick with SQL?

Suppose you have a heap of data and you want to plow through it with MapReduce at Hadoop speed. EnterpriseDB offers a MapReduce adaptor.

But, if you are going to implement NoSQL or processing of data on the scale that MapReduce and Hadoop offer and you need a rewrite, why not just migrate to Riak or any of the other NoSQL engines?

In answer, Padir said, "Most people aren't going to migrate their application to a database that has a completely different data store on the back end because that's a whole application rewrite."

Despite the number of new apps and startups appearing, there's a still a heap of legacy code out there. As I found talking to Xeround, the slowest moving part in many organizations is the codebase. Padir:

So, I think it has to do with: What is your application trying to do? What are the skills that you have? Because at the end of the day, most developers know how to write SQL, and if you're taking an application that already exists, it's just the flexibility you have to change the application.

I want to be able to take my data that's stored in Hadoop and put it in my relational database and be able to use some of my tools that I already have to run queries – more structured queries on that data – and vice versa. I have data that's in my relational database and I want to put it in my Hadoop cluster.

People are rewriting their applications in steps, but they want to start by having both worlds. So, again, database is a requirement rather than a technology. Instead of installing Oracle or MySQL, architects are choosing several database technologies to solve each of the problems they have. However, because other database technologies offer a few more features, you get a features race to satisfy developers that the database platform they're running on offers everything they need.

Future of PaaS

EnterpriseDB has made the move from a few products to a cloud service, which coexist. These products solve different problems and, unlike the other people I've spoken to (so far), DBaaS is just one way that the database problem can be solved.

Even though EnterpriseDB is clearly positioned at the database admin end of the market, with the configuration options and the ability to tweak, users do still expect it to work out of the box.

DBaaS has one strong argument in its favor: Chances are they have better database admins and better defaults than you because that's what the DBaaS provider does. For companies replacing their self-maintained database hardware or services, DBaaS removes a chunk of the admin. If you get to the tipping point at which the defaults just aren't good enough, bring it back in-house, but don't give yourself unnecessary pain by doing that from the beginning.

"Amazon and HP have lots of people who are completely dedicated to the security of their systems and keeping them up and running [compared with] a small company with a small data center that just happens to think they're secure because nobody knows about them," added Padir.

Most CTOs and developers have a lot of experience with database management, but many of them don't have a great deal of expertise when compared with the concentrated expertise used in the management of EnterpriseDB, Amazon's RDS, Rackspace's upcoming cloud database, and other DBaaS options. Even on security, Padir noted, "We have two additional products …. Both of these combined help protect you against SQL injection attacks and other bad things that can happen to your data."

Outsourcing the detail and management of your database sounds like a good idea. I for one am always in favor of paying someone else if they can do it better, so I concentrate on my product. But outsourcing can go wrong. Data is the most important part of your company, and if you are outsourcing control of it, you need to know what control is left to you, and you need to test the expertise of the platform or the support.

Anecdotally, the company I work for just migrated a lot of hosting to Rackspace-managed cloud to reduce the pointless sys admin tasks the developers were doing. Although they enjoyed the management and problem solving, and although this stuff was all essential to our service, they were spending their time doing something they didn't have to do.

"Pointless" sys admin tasks are very pointful tasks, but they are not what differentiates us from our competition. The ability to recover from backups is pretty basic and not what our devops team needs to spend its time on.

This move is no silver bullet. We had to learn Rackspace's support limits, and we tested at every stage. The most boring and menial tasks up to complex performance issues were sent their way so that, just like a new member of the dev team, we know how the service performs.

DBaaS is the same. It isn't a silver bullet, it's a new member of your stack, and it might do some stuff well, but you have to understand where its expertise ends.