Cutting Edge vs Bleeding Edge

I was reading some article about how facebook designed their new data center. The photo of their data center suddenly reminded me the company I worked for more than 10 years ago: Terraspring.

Terraspring was ahead of its time. What did we do? We did utility computing. All the resources (compute, storage, network) in a data center can be put into a pool managed by Terraspring, they can be dynamically allocated, and users are only charged by their usage. Does it sound familiar? Yes, many of the ideas are now part of the cloud computing.

Our original plan was to build our own data centers and provided services to other companies. As the 2000 internet bubble busted, the demand for data center went for a nose dive. We were forced to change our business model to become a software provider. That didn't went well either. Soon we were sold to Sun Microsystems and the whole team disbanded in 2004*.

Looking at this new wave of data center boom, I was reminded one comment I heard from this week's Big Data Analytics Conference. Somebody asked the panel speakers what's their advice for building a successful company. One of them replied that you didn't have to start from the bleeding edge. That's too risky. You should start from the cutting edge. That's good enough**. When Terraspring started in late 90s and built its data centers, it was on the bleeding edge. Nobody ever heard of utility computing, nor did the market exist. Would our fate be different if we started at a different time or lasted a bit longer? I wish I knew.

*Interestingly, that's also about the time Amazon started their AWS effort.

**But you also have to move really really fast.

Capacity Planning for Virtualized Datacenters - Sun Network 2003

Terraspring became part of Sun's N1. On slide 35 you can still see the name Terraspring.

The Magician's Trick

There are tons of info on the web about cloud computing and highly scalable systems. However, rarely the whole architecture of a product is illustrated. The architecture of a product is like the magician's trick: a trade secret.

That's why I got very excited when I stumbled upon the Netflix slides. For the first time, a product of such scalability and availability was showed in details.  For those who have always pondered how the magician pulls it off, this is treasure. Better yet, the SVForum Cloud and Virtualization SIG is doing a Netflix Night on March 27th. There is also another meetup on the same night focused on how Netflix migrated from Oracle to Cassandra. Are things too good to be true?

Global Netflix Platform

View more presentations from Adrian Cockcroft

Bridging the Cloud with Delphix

For two years I worked at a wonderful startup called Delphix. Delphix is the leader in database virtualization and our technology allows customers to provision a database as easily as provisioning a VM. With Delphix, CIOs finally have the tool to provide the agility and cost saving their IT divisions have long enjoyed with server virtualization technology.

My focus had since shifted to cloud computing and other areas after Delphix. Recently, while reading about the AWS Storage Gateway announcement and the RDS support, I started ruminating on the Delphix's solution. Delphix has positioned its product as an enterprise/private cloud solution (because of the sensitivity of data, database is always the last thing an enterprise customer wants to move to the cloud). Can Delphix work with a public cloud such as Amazon AWS? Can it cross the chasm to enable DBaaS in a hybrid cloud environment? Most importantly, can it be done NOW?

The answer immediately appeared in my mind: using V2P*. With Delphix's V2P feature, a user can create an EC2 instance, attach an EBS volume, then run the Delphix V2P. With just a few clicks, a user then have a new database created, data loaded, in the cloud. The concept is illustrated in the following diagram**.

Fig 1. Use Delphix V2P to create a database instance in AWS.

The approach has several advantages over the AWS RDS approach. First, Delphix does all the bookkeeping, so the user can provision a new database to any time point without worrying about the backup schedules and how to find the right data set from the backup. Second, the whole operation is applied to the Delphix appliance only, so no load is applied to the production database (which the RDS export or dump approach requires). Third, the user gets a solution that works for both private and public cloud.

In short, cloud computing opens many new opportunities. How to bridge the gap between the private and public cloud is an interesting challenge for all solution providers. Delphix already has a strong starting with its current V2P offering. It will be really interesting to see what new features Delphix will provide to make the cloud transition even easier in the future.


* How did I know? Because the feature was implemented by yours truly.
** Thanks to my coworker Fred for contributing the diagram and validating the info.