Energy use and Ecological Impact of Large scale Data Centres

The Energy use and Ecological Impact of Large scale Data Centres can be described by analysis of the concept of Energy proportion system.

50

100

0

10

50

90

Typical Operating Region

Percentage of System Utilization

Percentage of Power Usage

Power

Energy Efficiency

The above diagram shows power requirements scale linearly with the load, the energy efficiency of a computing system is not a linear function of the load; even when idle, a system may use 50% of the power corresponding to the full load. Data collected over a long period of time shows that the typical operating region for the servers at a data centre is from about 10% to 50% of the load.

This is a very important concept of resource management in a cloud environment in which load is concentrated in a subset of servers and rest of the servers are kept in standby mode whenever possible. An energy proportional system consumes on power when idle, very little power under light load and gradually more power as load increases.

            An ideal energy proportion system is always operating at 100% efficiency. Operating efficiency of a system is captured by an expression of performance per watt power. Energy proportion Network consumes energy proportional to their communication load. An example of energy proportion network is infiniband.

Strategy to reduce energy consumption is to concentrate the load on a small no. Of disk and allow others to operate in low power mode.

Another technique is based on data migration. The system uses data storage in virtual nodes managed with distributed hash table, the migration is controlled by two algorithms, a short term optimization algorithm used for gathering or spreading virtual nodes according to the daily variation of workload so that the number of active physical node is reduced to a minimum and a long term optimization algorithm, used for coping with changes in the popularity of data over a longer period i.e. a week. 

Dynamic resource provisioning is also necessary to minimize power consumption. Two critical issues related to it are i) amount of resource allocated to each application ii) placement of individual work load.

 

Cloud Interoperability

Since the internet is a network of network, so Inter cloud seems plausible.

A network offers high level services for transportation of digital information from a source or a host outside the network to destination, another host or another network that can deliver the information to its final destination. This transportation of information is possible because there is agreement on a) how to uniquely indentify the source and destination of the information b)how to navigate through a maze of networks  and c) how to actually transport the data between a source and a destination. The three elements on which agreement is in place are IP address, IP protocol, and transportation protocol such as TCP/UDP.

In case of cloud there are no standards for storage or processing, the clouds have different delivery models which are not only large but also open. But following the lines of internet cloud interoperability seems feasible.  Standards are required for items such as naming, addressing, identity, trust, presence, message, multicast, time, means to transfer, store and process information etc.  An inter cloud would require the development of an ontology for cloud computing and then each service provider would have to create a description of all resources and services using this ontology. Each cloud would then require an interface which may be termed as an Inter cloud Exchange to translate the common language describing all object and action included in a request originating from another cloud in terms of internal objects and action.

 The above diagram show communication among disparate clouds

The Open Grid Forum has created a working group to standardize such an interface called as the Open Cloud Computing Interface (OCCI).  The OCCI has adopted resource oriented architecture (ROA) to represent key components comprising cloud infrastructure service.  Each resource identified by a URI (Unique resource identifier) can have multiple representations that may or may not be hypertext. The OCCI working group plans to map API to several formats. Thus a single URI entry point defines an OCCI interface.

The API implements CRUD operations: Create, Retrieve, update and delete. Each is mapped to HTTP POST, GET, PUT and DELETE respectively. OCCI provides the capabilities to govern the definition, creation, deployment, operation and retirement of infrastructure services. 

Cloud clients can invoke a new application stack, manage its life cycle and manage the resources that it uses using the OCCI.  The SINA association has established a working group to address the standardization of cloud storage.  The cloud Management Data Interface (CDMI) serves the purpose. In CDMI the storage space exposed by various type of interface is abstracted using the notion of a container.   The container also serves as a grouping of the data storage in the storage space and also forms a point of control for applying data services in aggregation. The CDMI provider data object interface with CRUD semantics and can also be used to manage containers exported for use by cloud infrastructure.

The above diagram shows OCCI and CDMI in an integrated cloud environment

To achieve interoperability, CDMI provides a type of export that contains information obtained via the OCCI interface. In addition, OCCI provide a type of storage that corresponds to exported CDMI containers.  

Cloud Storage Diversity and vendor lock in

There are several risk involved when a large organization relies solely on a single cloud provider.

A solution to guarding against vendor lock is to replicate the data into multiple cloud service providers. But this straight forward replication is costly and poses technical challenges at the same time. The overhead to maintain data consistency could drastically affect the performance of the virtual storage system consisting of multiple full replicas of the organization’s data spread over multiple vendors.

Another solution is based on extension of design principle of a RAID-5 system used for reliable data storage. A RAID 5 system uses block level striping with distributed parity over a disk array. The disk controller distributes the sequential block of data to the physical disk and computes a parity block by bitwise XORing the data block. The parity block is written on a different disk for each file when all parity is written to a dedicated disk as in RAID 4. This technique allows us to recover data after a single disk is lost.

    The above diagram show a RAID 5 controller. If disk 2 is lost file 3 still has all its blocks and the missing blocks can be for by applying XOR operation (eg: a2=a1XORapXORa3) 

The same process can be applied to detect and correct error in a single block. We can replicate this system in cloud, data may be stripped across four clouds, the proxy cloud provides transparent to data. Proxy carries out function of the RAID controller. The RADI controller allows multiple accesses to data. For example block a1, a2, a3 can be read and written concurrently.

The above diagram shows the concept of RAID 5 applied in Cloud to avoid vendor lock in

Proxy carries out function of RADI controller as well as authentication and other security related functions. The proxy ensures before and after atomicity as well as all or nothing atomicity for data access. The proxy buffers the data, possible converts the data manipulation command, optimizes the data access.  For example aggregates multiple write operation, convert data to formats specific to each cloud and so on. This Model is used by Amazon. Performance penalty due to proxy overhead is minor and cost increase is also minor.