Member Article

Big data infrastructure seem trickier with cloud

Cloud computing has definitely made big data infrastructure discussions a bit trickier, especially for those working with existing infrastructure. Now that the cloud has gained prominence in the IT infrastructure landscape, there are more approaches and solutions that businesses can choose from to support business intelligence (BI) projects.

IT heads have been bombarded by these questions: Should the business shift to the cloud? What will they do with the existing infrastructure? Is the cloud even secure?

No wonder many system administrators have been freaking out wondering if they will still have jobs, as cloud technologies and services seek to streamline deployments and job descriptions. Cloud computing has definitely thrown a monkey wrench into the old system.

Big data demands have evolved

The processes involved in doing BI today are still pretty much similar – data needs to be collected, stored, crunched, and reported. However, consumer behavior driven primarily by the increased role of technology in our lives has increased the pace for turning data to insights.

The ways data can be collected today have also diversified. In retail alone, technologies such as beacons, trackers, and smart point-of-sale systems have added more data points for analysis. Once the adoption of internet-of-things (IoT) spikes, there’s even more data to be collected.

This increase then impacts the whole process. More data means a bigger need for storage and computing power for processing. The demand for quicker insights also places added burden to the infrastructure. Reports aren’t confined to monthly board meetings anymore. Many organizations now appreciate the value of shared dashboards displayed in office lobbies and common areas, offering real-time information on-demand.

Dealing with all of this through traditional infrastructure can be a challenge.

Cloud has advantages

Speed and scalability are among the key advantages of cloud computing for doing contemporary BI. If ever there’s increased resource demand, a cloud deployment allows you to provision instances in a matter of a few clicks. With traditional IT, this usually involves a tedious process of purchasing appliances and integrating the hardware into the existing infrastructure.

Cloud pricing is now even competitive against running traditional IT, if not cheaper. The intensified competition among resource providers like Amazon Web Services (AWS), Microsoft Azure, and Google has driven prices down. Since you only pay for what you use, it’s easier and faster to define the ROI of these resources for big data projects.

This can also be done through a hybrid deployment, mixing on-premises infrastructure, cloud servers all tied together through platform-as-a-service offerings that can bridge across datacenters and cloud deployments. Network appliances like firewalls and load balancing solutions can now be availed through subscription such as services like Incapsula’s cloud-based load balancing and WAF, which can rather than through outright purchase of on-premises appliance.

One potential drawback might be the perceived advantage of traditional IT, particularly with the belief that data has to be located in a physically secure site (e.g., on-premises) and you can’t guarantee that with the cloud due to its distributed nature. However, one can also argue that having only a single point of failure would be detrimental to any enterprise deployment. In comparison, a distributed infrastructure can ensure better disaster recovery and failover capabilities.

It appears that those who are just getting into data analytics have it easier since they can readily avail of end-to-end cloud-based business intelligence (BI) services and not even worry about infrastructure as opposed to many established organizations.

Managing the shift

Let’s be realistic. Organizations with existing or large investments in traditional IT systems can face stagnancy. For some, it would even be a challenge to convince conservative finance people how the cloud could bring quicker ROI when it might still be years before their existing traditional IT realizes returns on earlier investments.

Today’s BI projects seems to be a great match for the capabilities of cloud computing, any CIO worth his paycheck would be making strides in using the technology. In addition, it’s not as if existing traditional infrastructures suddenly lose value. The tricky thing is integrating and maximizing the resources available to get BI projects done.

Hybrid cloud setups – where traditional and cloud resources are integrated – are the option for many such organizations. These setups could maximize the use of existing infrastructure and leverage cloud computing for a variety of purposes. IaaS providers can also handle the necessary connectivity requirements thus eliminating the need for additional network appliances and hardware.

At its core, business intelligence is about using data to aid business decisions. But if an organization can leverage cloud computing, it could mean an added competitive advantage, which can be critical in today’s business environment.

This was posted in Bdaily's Members' News section by Joydeep Bhattacharya .