Azure

MCA Article Feed.

I have put together a couple blog entries reviewing some cost analysis that I did 2-3 years ago around Hadoop and Azure storage/server architectures–specifically how we worked with customers to reduce the costs of these environments (in part) with enterprise-class storage. It goes without saying—but I will anyway—the focus of these economic models and case studies was on the deployment and costs of the storage infrastructure. Some of these new cloud/big data environments do not use RAID overhead or distribute data across hundreds of nodes and disk clusters to perform the work. As I did this work, we took a myopic view of just the storage hardware aspect of these environments. I guess you would expect that from an HDS employee.

Earlier this week I had an interesting call with Ramon Chen of Rainstor, and compared notes on how they reduce DB costs, and therefore storage with their product offering. After our conversation, it was clear to me that big data cost reductions can happen on at least 2 levels:

MCA Article Feed.

I have been developing a small mini-series on the economics of big data, with a focus on the storage approach used in Hadoop and Azure architectures. The intro blog, case study #1, and a review of bare-metal analysis have been posted to this blog over the past few weeks. I will wrap up this series with another case study, this time with a Hadoop environment.

Like so many organizations that embark on a new IT infrastructure/architecture, the start-up investments are such that planners and procurement are price sensitive. Getting a Hadoop environment genned-up can be very easy, and relatively low in cost. This is true for our case study #2 client, who is a large online retailer.

MCA Article Feed.

There is plenty of talk in the press today about big data, analytics and our next new wave for IT. I would like to present 2-3 blogs on a small but important subset of the big data world: storage infrastructure (and more importantly, optimal storage architectures). I will use our storage economics approach for the definitions of “optimal”, meaning you can address optimized storage from other dimensions as well (resiliency, scale, performance, etc.) as you develop big data strategies.

I had a period of time—2-3 years ago—where I was measuring and costing large Hadoop and Azure systems environments. I became very excited about these new distributed architectures, but at that time was not able to dedicate effort and resource for further research on their cost behaviors. Now that these systems have a new moniker (big data), the demand for big-data-economics conversations is here again. Good thing I have the models and methodology all sorted-out.

MCA Article Feed.

Last week I posted an introductory blog about big data and some work I had done a few years back in this space (before it was called big data). I have a couple of these large TCO assessments in my library, but will just share 2 or 3 of these that have the easiest story to tell, and make the point around price and cost of big data storage infrastructures.

This first case study (circa 2009) is a large retailer with most of the revenue coming from web transactions. They used the Azure cloud platform, and had (at the time) 1,500 hosts, about 4PB of JBOD and rack mount storage. They were convinced that the seemingly low priced disk architecture connected to the server/node architecture was meeting their price and cost objectives. The fact that the sprawl of the cloud and big data systems (analytics) was on a rapid pace to overthrow their data center, and they were on track to triple the data center size to meet the growing storage/rack space sprawl.