EMC - iSilon - SmartQuota

•SmartQuotas is an integrated management tool that monitors and enforces storage limits on an EMC Isilon cluster. SmartQuotas is an optional feature and requires a license. It provides an administrator the ability to track storage usage by individual user, as well as restrict the amount of storage an individual user can consume.
•SmartQuotas monitors and enforces capacity utilization.
•SmartQuotas provides alerting and reporting functionality

There are five types of quotas that can be configured, directory, user, default user, group and default group.

• Directory quotas are placed on a directory, and account for all directories and files within that directory, regardless of user or group. Directory quotas are useful for shared folders where a number of users store data, and the concern is that the directory will grow unchecked because no single person is responsible for it.

• User quotas are applied to individual users, and track all data that is written to a specific directory. User quotas allow the administrator to control how much data any individual user stores in a particular directory.

• Default user quotas are applied to all users, unless a user has an explicitly defined quota for that directory. Default user quotas allow the administrator to apply a quota to all users, and not on a user-by-user basis.

• Group quotas are applied to groups, and limit the amount of data that the collective users within a group can write to a directory. Group quotas function in the same way as user quotas, except for a group of people and not individuals.

• Default group quotas are applied to all groups, unless a group has an explicitly defined quota for that directory. Default group quotas operate like default user quotas, except on a group basis. You should not configure any quotas (user, group or directory) on the root of the file system (/ifs), as it could result in significant performance degradation

Default Quotas are baseline quotas that are associated with a large number of objects, such as users and groups. For example, creating a default user quota will automatically associate that quota with every existing and new user, unless that user already has an individual quota established. When a user first writes data to the file system, an explicit entry for that user is generated and linked to the default quota. The default quota can be changed by the administrator, but such changes will propagate to all users or groups that are linked to the default quota.
Users who need to be assigned a different quota must first be unlinked from the default quota before a custom quota can be configured

Usage accounting and limits determine how OneFS enforces quotas on the EMC Isilon cluster. Quotas can be configured to only perform accounting, or to performing accounting and enforcing. The accounting option tracks storage usage, but does not impose any limits or perform any notification. Accounting is useful for environments that require usage data for chargeback or showback, for analyzing usage trends and capacity planning. The enforcing option tracks usage as well as takes actions when the limits are reached. There are three settings for the enforcing option: hard, soft and advisory. They behave as follows:

• •Hard: The quota limit cannot be exceeded.

• •If a write causes the quota to be exceeded, the write will fail and the user will receive a quota error on their screen.

• •An alert is logged in the cluster logs, and an email notification is sent to the configured recipients. System level notifications are used to notify the appropriate people

• •Writes will be allowed once the usage falls below the threshold.

• •Soft: Allows the limit to be exceeded but establishes a grace period.

• •Once the grace period expires, if the threshold is still exceeded, it will enforce the limit as a hard limit, including generating alerts and sending notifications.

• •Advisory: Establishes an informational limit.

• •If the limit is exceeded, an alert is logged and a notification is sent, but no prohibitive actions are taken.

When defining a quota, the administrator must choose the types of data that are included in calculating the capacity a user has consumed. The options are:

• 1.Default: The default setting is to only track user data, that is just the data that is written by the user. It does not include any data that the user did not directly store on the EMC Isilon cluster.

• 2.Snapshot Data: This option tracks both the user data and any associated snapshots. This setting cannot be changed once the quota is defined. To disable snapshot tracking, the quota must be deleted and recreated.

• 3.Data Protection Overhead: This option tracks both the user data and any associated FEC or mirroring overhead. This option can be changed after the quota is defined.

• 4.Snapshot Data and Data Protection Overhead: Tracks all data – user, snapshot and overhead – with the same restrictions. It is critical to understand how these options will impact the users, especially with hard quotas. 

For example, assume a user quota with a hard limit of 100 GB has been established. A user writes a 10 GB file to a directory that has a 3x protection level set. The user’s capacity usage reports 10 GB, or 10% of his/her available capacity. Take that same quota, but with data protection overhead tracking enabled. When the user writes the same 10 GB file to the same directory with 3x protection, the quota system will now report that the user has written 30 GB of data, or 30% of his/her capacity. It is possible to create multiple similar quotas to accomplish different goals. In the previous example, a hard quota could be established that does not include data protection overhead to restrict the amount of storage that the user consumes, while also having an advisory quota that does track data protection overhead for capacity management or chargeback/showback purposes.

• •Quotas can be configured to track different types of capacity utilization.

• •Snapshot data and protection overhead can be included in the quota consumption in any combination in addition to the file data itself.

• •Nesting quotas refers to having multiple quotas within the same directory structure. In the example shown, all quotas are hard enforced. At the top of the hierarchy, we have the /ifs/data/media folder which has a directory quota of 1 TB. Any user can write any amount of data into this directory, or the /ifs/data/media/temp directory, up to a combined total of 1 TB. The /ifs/data/media/photo directory has a user quota assigned, which restricts the total amount any single user can write into this directory to 25 GB. Even though the parent directory (media) is below its quota restriction, a user is restricted within the photo directory. The ifs/data/media/video directory has a directory quota of 800 GB, which restricts the capacity of this directory to 800 GB. However, if users place a large amount of data in the /ifs/data/media/temp directory, say 500 GB, then only 500 GB of data can be placed in this directory, as the parent directory (media) cannot exceed 1 TB.

• •SmartQuotas allows quotas to overlap to provide greater flexibility in implementation.

• •Directory level quotas for a
  
  department with default user quotas underneath is a common example of nested quotas

Each type of quota has different events that can be notified, as shown in the table.

• Limit Exceeded is triggered when a quota threshold is exceeded for advisory or soft quotas, and when a threshold is reached with a hard quota.

• Limit Remains exceeded will generate an alert on a recurring basis while the quota is exceeded.

• Grace Period Expired is triggered when a soft quota has been exceeded and has not been corrected before the grace period has elapsed.

• Write Access Denied occurs with a soft quota if the grace period has elapsed and a user attempts to write data. It is also triggered with a hard quota when the threshold is reached and the user attempts to write data.

• SmartQuotas will send notifications when quotas have been exceeded to both the user and an administrative email.

• Notification can be enabled by default for all quotas.

• Each quota enforcement type, advisory, soft, hard, can have a customized notification
• Isilon Student Guide Foundations
  Useful Link
  
  
  Isilon scaleout nas overview 
  
  
  Innovative File Data Storage Strategies to Power Your Enterprise, Syncplicity & Isilon 
  
  
  Scale out NAS for VMware Virtualization 
  
  
  emc isilon hdfs enterprise storage for hadoop 
  
  
  Building Hadoop-as-a-Service with Pivotal Hadoop Distribution, Serengeti, & Isilon