EMC ViPR - Software Defined Storage - Fundamental

1. Overview

ViPR has four major components. Although storage arrays are central to the use of ViPR, they are not a component of ViPR. The major components are the ViPR controller, ViPR object service, ViPR integration, and ViPR monitoring and reporting (M&R). Each of these components is installed separately.

• ViPR controller abstracts, pools, and automates physical storage resources into policy-based virtual pools with self-service access to a catalog of resources. It also provides block and file control services. This is the first and most fundamental component of ViPR. There can be no ViPR deployment without the ViPR controller.

• ViPR object service delivers a scalable storage platform that provides object storage capabilities. This component of ViPR is optional. If the ViPR object service is deployed, the ViPR controller must also be deployed. ViPR 2.2 is a transitional release that separates controller services from object data services. EMC Elastic Cloud Storage and ViPR Commodity software represent ViPR object data services and will follow a different development track and schedule. 

• ViPR integration with cloud platforms provides an alternative to provisioning storage from the ViPR UI. The ViPR integration features allow users of VMware, Microsoft, and OpenStack to stay in their preferred management tool and initiate storage provisioning without switching to another UI. These components are optional and, like the ViPR object service, they cannot exist without the ViPR controller.

• ViPR monitoring and reporting. ViPR software distribution includes the Solution Pack for ViPR. The Solution Pack leverages the ViPR monitoring and metering REST API bulk feeds to expose availability and usage of ViPR managed storage. When used in combination with Storage Resource Management (SRM) Suite, this Solution Pack connects the dots from physical to ViPR managed volumes.

Installing the ViPR controller requires installing multiple VMs for block and file virtualization, a load balancer, the REST API, and support for the command line. The controller is delivered as a VMware vApp (OVF file) that must be installed on VMware ESXi.
The vApp for the controller has two configurations, either 3 VMs or 5 VMs. Both configurations are
able to handle expected ViPR workloads with the same level of performance.

• The ViPR controller 3 VM configuration can handle the loss of a single VM without impacting users.

• The ViPR controller 5 VM configuration can handle the loss of two VMs without impacting users. The choice of configuration should be determined by the level of tolerance to ViPR VM loss. The load balancer included with the ViPR controller allows each controller VM to behave as the  entrypoint, so that customers can connect to any VM and the workload will be balanced across all VMs.

The ViPR object service provides the object-on-file features of ViPR, and the object API support for S3, Swift, and Atmos. To enable the object service, the customer must install one or more service VMs.
The service VMs are used by the ViPR controller to hold the object metadata used by the ViPR object service. Like the controller, multiple VMs provide tolerance to VM loss without impacting users. The VM for the ViPR object service is also delivered as an OVF, but it installs a single VM. If multiple VMs are desired for object service scalability, the service VM is simply installed multiple times.

Using the ViPR object-on-file service, you can optionally toggle the access mode of a bucket  between the standard REST access (default) and filesystem access (via NFS) using the file access mode feature. The file access mode determines whether a resource can be accessed as a file on a file system or as an object via a REST API.
When filesystem access is enabled for a bucket, all existing objects in the bucket are no longer accessible via REST, but are mountable using an NFS client. Mount points for each object can be obtained with a GET call.
Users can access objects as files using ViPR extensions to Amazon S3, OpenStack Swift, and EMC Atmos APIs. The ViPR extensions can be used to specify the file access mode of resources within a bucket such as videos, images, or documents.

2. Physical to Virtual

With physical storage, each switch, array, and connection must be individually managed. Most enterprise IT and managed service provider environments contain many of each, and often have multiple models from multiple manufacturers. Managing each resource individually is time consuming and error prone

By abstracting storage from the physical arrays, ViPR does much of the management of the individual components, allowing administrators and users to treat storage as a large resource focusing just on the amount of storage needed and the performance and protection characteristics required.
ViPR exposes the storage infrastructure within its control through a simplified model, hiding and handling the details of array and disk selection, LUN creation, SAN zoning, LUN masking, and the differences between one storage device and another.
ViPR is aware of and leverages intelligence such as FAST, snapshots, and cloning capabilities within individual models of storage arrays. The same applies to protection technologies such as RecoverPoint and VPLEX. ViPR provides abstractions to take full advantage of these technologies

The virtual array is a ViPR abstraction for the physical arrays and the network connectivity between hosts and these arrays. The virtual array provides a more abstract view of the storage environment for use in either applying policy or provisioning.
All physical arrays participating in a virtual array should be connected to the same fabrics or VSANs (virtual storage area networks) to ensure that they all have equivalent network connectivity to the environment. When a storage administrator adds physical arrays to ViPR, ViPR discovers their storage pools, ports, and configuration. After FC switches are added, ViPR automatically discovers and maps the FC networks. When populating a virtual array with physical arrays and networks, the administrator must ensure that when storage is presented from the virtual array to a host, the host must be able to physically reach the storage presented to it.
Having examined the connectivity between hosts and arrays, the administrator can build the virtual arrays. When all hosts can reach all arrays, the entire storage infrastructure can be grouped into a single virtual array; however, physical arrays may need to be placed into separate virtual arrays to accommodate different physical configurations and different requirements for fault tolerance, network isolation, or tenant isolation.
In the typical physical environment there are multiple arrays, each with their own management tools, processes, and best practices. With the ViPR virtual array, all of the unique capabilities of the physical arrays are available, but ViPR automates the operations of the tools, processes, and best practices to simplify provisioning storage across a heterogeneous storage infrastructure. In this way ViPR can make a multi-vendor storage environment look like one big virtual array. ViPR can accomplish these tasks for specific types of block and file storage, including: VMAX, VNX, Isilon, VPLEX, and NetApp. With the physical arrays configured into ViPR virtual arrays, the administrator can now build ViPR policies that are automatically applied across heterogeneous arrays.

In ViPR, a virtual pool represents a standardized storage service offering out of which storage may be provisioned. Virtual pools are abstractions. As part of configuring ViPR, the administrator maps the physical array pools into virtual pools. Virtual pools expose performance and protection levels from the disks to the user. When defining the virtual pool, the administrator will separate block from file pools; then the administrator will select the pools that deliver the tiers of performance characteristics they wish to expose to users. Finally, the administrator will identify the level of data protection available to each pool. 

The name assigned to the virtual pool should reflect the attributes that the pool represents. If the virtual pool contains Flash disks, a name such as “High Performance Pool” might be appropriate. When provisioning storage, the user will identify the type of storage they desire using this name, so ensure that this name clearly identifies the capabilities of the storage within the virtual pool. In this way the virtual pool becomes the definition used by ViPR to enable policy-based storage provisioning within virtual arrays. For example:

•  Block storage pools on flash drives protected by VPLEX Metro can represent a tier 1 virtual pool.

•  Block storage pools on Fibre Channel that can be replicated with RecoverPoint may represent a second virtual pool for tier 2 block storage.

•  Block storage pools on Fibre Channel that are not replicated define a tier 3 virtual pool.

•  File storage pools could all be grouped together into a single virtual pool.

Later, when storage is provisioned, the user will identify which virtual pool they want their storage to use. ViPR will then apply its built-in best practices to select the best physical array and storage pool that meets the provisioning request.

3. ViPR Features and Capabilities 

 

In the ViPR open cloud platform, hardware is abstracted. It’s easy for enterprises, service providers, and third parties to write connectors or code that understands the underlying arrays and exposes them to the ViPR platform.
This approach is unique because typically these southbound APIs are proprietary. But with ViPR, they are open. Most importantly, when third-party arrays are added to ViPR, they keep their unique attributes, even though their management and provisioning are centralized in ViPR.

Clusters can be associated with hosts. If a Windows or ESX host is discovered, and, if that host is in a cluster, then ViPR will automatically detect this and add an entry to the ViPR Cluster tab. Windows or ESX hosts manually associated with clusters in which they do not belong, will be corrected by ViPR if ViPR automatically discovers those hosts.
As new hosts are added to ViPR, if those hosts are in a cluster, ViPR will recognize this. If the cluster is one it already knows, the new host will be added to that existing cluster, if not, a new cluster will be created. A host cannot be manually placed in one cluster when it actually belongs to different windows cluster, ViPR will discover this error and automatically correct it. The windows clustering support requires the Microsoft Windows clustering software to be set up and enabled.

ViPR defines file, block, and object storage in software as services. Specifically, ViPR file and blo ck services provide all the functionality of physical block and file storage arrays, including advanced protection services such as snapshots, cloning, and replication.
Because ViPR file and block services do not operate in the data path, users are able to retain and leverage all the unique attributes of the underlying block and file arrays. This means that VMAX users can continue to use FAST and the Unisphere element manager with ViPR as they do today, plus enjoy all the benefits of centralized provisioning, management, reporting, self-service access, and more. Applications access file and block data directly.
Over time, EMC will continue to build and deliver greater services and foster a community of services that continue to extend and add value to the ViPR platform.