Big Data, Bare Metal
MCA Article Feed.
I have put together a couple blog entries reviewing some cost analysis that I did 2-3 years ago around Hadoop and Azure storage/server architectures–specifically how we worked with customers to reduce the costs of these environments (in part) with enterprise-class storage. It goes without saying—but I will anyway—the focus of these economic models and case studies was on the deployment and costs of the storage infrastructure. Some of these new cloud/big data environments do not use RAID overhead or distribute data across hundreds of nodes and disk clusters to perform the work. As I did this work, we took a myopic view of just the storage hardware aspect of these environments. I guess you would expect that from an HDS employee.
Earlier this week I had an interesting call with Ramon Chen of Rainstor, and compared notes on how they reduce DB costs, and therefore storage with their product offering. After our conversation, it was clear to me that big data cost reductions can happen on at least 2 levels:
- Bare metal infrastructure optimization
- Software and database optimization
Take a look at Ramon’s article, which is a very compelling story on how their compression technology had a massive impact on the total Hadoop storage and server infrastructure cost. His blog can be read here: http://rainstor.com/how-much-is-that-hadoop-cluster-really-costing-you/
We hope to collaborate on a joint effort in the near future to show the compounded impact of SAN/storage optimization and database/compression optimization with large and very large Hadoop environments.
I am not apologizing for my views and recommendations to reduce the infrastructure costs of large, analytic or cluster architectures. There are certainly many factors to reduce the costs of the hardware on the floor. But a wider view to look beyond the bare-metal costs can be just as valuable.
I will resume my blog series next week with a few more (bare metal) big data/cloud cost reduction case studies. If you missed case study #1, you can read it here.
