EMC ViPR - Software Defined Storage - Architecture

1. Introducing EMC ViPR

EMC ViPR provides a central point of access to all management functions, translating requests into specific calls to the underlying storage, while offering storage services to multiple users or tenants, with different access roles through a single common portal. This approach standardizes operations, reduces complexities, and improves an organizations efficiency and agility in deliver storage when and where needed. It offers universal APIs to facilitate rapid application development and integration with the existing storage management point solutions. ViPR enables RainPole for the first time to manage heterogeneous storage environments, including third party arrays and commodity storage like Amazon S3 or even DropBox. RainPole developers are also able to build an API interface to any standard storage to be managed by ViPR.
But what makes EMC Software-Defined Storage unique, is that unlike previous attempts at storage virtualization, ViPR decouples the control path from the data path. By abstracting the control path, storage management operates at the virtual layer, which gives customers the ability to partition a storage pool into virtual storage arrays. This is analogous to partitioning a server into a number of virtual machines. Control path data services provide multitenancy, service cataloging, metering and monitoring across all arrays. Unlike previous attempts at storage virtualization, EMC ViPR does not sit in the data path for file and block stores. This ensures applications can access storage and all its underlying value and data services embedded in the storage arrays. It also enables administrators to centralize data provisioning and data management tasks, and allow any applications to access file and block data.
To summarize, ViPR enables RainPole to deliver:

•  Heterogeneous storage array management

•  Policy based management of storage pools abstracted from hardware

•  Leverage the capabilities of the underlying hardware platforms

•  Centralized Storage network management with Unified storage views across the Data Center

•  New interface support

ViPR Software-Defined Storage

EMC ViPR provides a central point of access to all management functions, translating requests into specific calls to the underlying storage, while offering storage services to multiple users or tenants, with different access roles through a single common portal. This approach standardizes operations, reduces complexities, and improves an organizations efficiency and agility in deliver storage when and where needed. It offers universal APIs to facilitate rapid application development and integration with the existing storage management point solutions.
ViPR enables RainPole – for the first time – to manage heterogeneous storage environments, including third party arrays and commodity storage like Amazon S3 or even DropBox. RainPole developers are also able to build an API interface to any standard storage to be managed by ViPR.
But what makes EMC Software-Defined Storage unique, is that unlike previous attempts at storage virtualization, ViPR decouples the control path from the data path. By abstracting the control path, storage management operates at the virtual layer, which gives customers the ability to partition a storage pool into virtual storage arrays.
This is analogous to partitioning a server into a number of virtual machines. Control path data services provide multitenancy, service cataloging, metering and monitoring across all arrays. Unlike previous attempts at storage virtualization, EMC ViPR does not sit in the data path for file and block stores. This ensures applications can access storage and all its underlying value and data services embedded in the storage arrays. It also enables administrators to centralize data provisioning and data management tasks, and allow any applications to access file and block data.

ViPR Storage Asset Discovery
ViPR discovers the arrays and all their corresponding storage pools and ports. Once the Fibre Channel switches are added, ViPR automatically discovers and maps the Fibre Channel networks. And ViPR can accomplish this virtualization and mapping for EMC and non-EMC arrays, including EMC VMAX, EMC VNX, EMC Isilon, EMC VPLEX, EMC Atmos and NetApp. And ViPR will support additional EMC and commodity disks in addition to publishing the APIs.
 

ViPR Abstraction layer
ViPR hides the complexity of all the underlying storage arrays and exposes their core functionality as data services while retaining the unique attributes of the arrays. Storage administrators then create Virtual Storage Pools in ViPR that represent sets of capabilities required by unique application workloads. For example, a transactional workload would be best served by a Virtual Storage Pool that features the characteristics of high-performance block storage such as EMC VMAX. A cloud application such as online file and content sharing is not performance-sensitive and would work just fine on commodity hardware that more economically provides the requisite level of data protection and availability. In either case, a user subscribes to a Virtual Storage Pool that meets their workload’s demands.

2. ViPR Controller

ViPR controller:

•  Discovers and registers all ViPR-supported arrays

•  Abstracts physical storage into a virtual pool where storage services such as provisioning, metering, cataloging, and delivery can be automated

•  Delivers storage through a self-service catalog, and

•  Centralizes management across the virtual storage environment

•  Virtualizes your physical storage and configures your virtual storage arrays,

•  Automates storage tasks and centralizes management across physical and virtual environments

•  Discovering and registering arrays is the first of 3 easy steps to virtualize, automate, and centralize storage.

Through an easy-to-use portal, the storage administrator defines the storage environment that it wants ViPR to manage. They point to storage arrays, SAN switches, and data protection devices. ViPR then discovers and abstracts physical storage arrays with all their unique capabilities into a single pool of virtual storage. This is a very simple process as you will learn in this lab. The storage administrator simply points to the array and ViPR discovers the rest. This step only needs to be done at the beginning, when ViPR is first deployed, or at anytime the administrator wants to add or change the configuration.

ViPR Controller Architecture

•  Distributed as virtual appliance (software-only package)

•  Load balancer service evenly distributes workloads across VMs and maximize throughput

•  Workflow automator and controller services that execute workflow steps on storage infrastructure

•  Coordinator service coordinates tasks between distributed processes ViPR

ViPR Software Architecture

ViPR makes a multi-vendor storage environment look like one, big virtual array. ViPR uses software adapters that connect to the underlying arrays, similarly to how device drivers enable universal device compatibility with a PC. ViPR  exposes the APIs so any vendor, partner or customer can build new adapters to add new arrays. This creates an extensible “plug and play” storage environment that can automatically connect to, discover and map arrays, hosts and SAN fabrics

3. ViPR Data Services

With the separation of the physical from the logical (data plane from the control plane) many new capabilities are possible for providing cloud-scale data operations over traditional storage and for enabling partners and customers to extend and customize the ViPR platform.
ViPR Data Services are cloud-scale storage services implemented in software and layered over traditional storage - ViPR-managed arrays. ViPR Data Services are hardware agnostic. It unlocks storage and data services from vendorspecific arrays, and extends them across heterogeneous storage.
The ViPR object data service is similar to Amazon S3 model

ViPR Data Services Architecture
The ViPR object data service features a scale-out node-based architecture with full failover between nodes. Data nodes handle all object requests and are virtualized to facilitate easy deployment.
Providing object and HDFS as data services enables new uses for legacy applications and data and supporting nextgeneration applications and workflows without heavy infrastructure lifting.
Object-on-file preserves investments in existing software and workflows while adding new object capabilities. Hadoop supports new use cases where there is no legacy solution

4. Docs