Cloudian HyperStore S3 Compliance Guide
This document is for Cloudian HyperStoreR software version 6.0. For more information about this release, see What’s New in the Cloudian HyperStore Administrator’s Guide.
1. Introduction
The Cloudian HyperStore system supports the great majority of the Amazon S3 REST API, including advanced features.
This document provides the details of the HyperStore system’s compliance with the S3 REST API. The organization of this document parallels that of the Amazon S3 API Reference. Links are provided to specific parts of the Amazon S3 API Reference so you can easily view additional information about individual API operations.
This document takes the approach of specifying in detail the things that the HyperStore system does support from the Amazon S3 REST API — from service, bucket, and object operations, down to the level of particular request parameters, request headers, request elements, response headers, response elements, and special errors. If it’s not listed in this document, the HyperStore system does not currently support it.
This document also describes ways in which the HyperStore system extends the Amazon S3 API, to support additional functionality. These extensions are in the form of additional request parameters or request headers that add enhanced functionality to standard Amazon S3 operations on buckets or objects. These extensions are described within the sections that document HyperStore compliance with standard Amazon S3 operations. The extensions are always identified by a sub-heading that says HyperStore Extension to the S3 API.
1.1. General Guidance on Developing S3 Applications for HyperStore
In nearly every way, developing a client application for the Cloudian HyperStore storage service is the same as developing a client application for Amazon S3. Consequently, when designing and building S3 applications for the HyperStore service you can leverage the wealth of resources available to Amazon S3 developers.
The best place to turn for resources for developing Amazon S3 and Cloudian HyperStore S3 applications is the Amazon S3 web site. Through that site, Amazon Web Services (AWS) Developer Centers are available for a variety of development technologies:
AWS Developer Centers include SDKs, community libraries, "Getting Started" guides, and tips and tricks.
Another good Amazon resource is the archive of Articles & Tutorials relating to S3.
Yet another helpful Amazon resource is the archive of Sample Code & Libraries relating to S3.
1.1.1. What’s Distinct About Developing for the HyperStore Service
In practice, the main differences between developing for developing for the Cloudian HyperStore service and developing for Amazon S3 are:
-
As detailed in this document, the HyperStore service supports the great majority of but not the entire Amazon S3 API.
-
Also as detailed in this document, the HyperStore service supports a small number of extensions to the Amazon S3 API.
-
HyperStore client applications must use the Cloudian HyperStore service endpoint rather than the Amazon S3 service endpoint.
-
You will need to use either the Cloudian Management Console (CMC) or the Cloudian HyperStore Administration API to create and manage service user accounts.
2. Error Responses
From the "Error Responses" section of the Amazon S3 API, the HyperStore system supports the error codes listed below. If an error code from the Amazon S3 API specification is not listed below, the HyperStore system does not support it. For error descriptions and for mapping of these errors to HTTP status codes, refer to the "Error Responses" section of the Amazon S3 API.
-
AccessDenied
-
AccountProblem
-
AmbiguousGrantByEmailAddress
-
BadDigest
-
BucketAlreadyExists
-
BucketAlreadyOwnedByYou
-
BucketNotEmpty
-
CrossLocationLoggingProhibited
-
EntityTooLarge
-
IllegalVersioningConfigurationException
-
IncorrectNumberOfFilesInPostRequest
-
InternalError
-
InvalidAccessKeyId
-
InvalidArgument
-
InvalidBucketName
-
InvalidBucketState
-
InvalidDigest
-
InvalidLocationConstraint
-
InvalidObjectState
-
InvalidPart
-
InvalidPartOrder
-
InvalidPolicyDocument
-
InvalidRange
-
InvalidRequest
-
InvalidSecurity
-
InvalidTargetBucketForLogging
-
InvalidURI
-
KeyTooLong
-
MalformedACLError
-
MalformedPOSTRequest
-
MalformedXML
-
MaxMessageLengthExceeded
-
MaxPostPreDataLengthExceededError
-
MetadataTooLarge
-
MethodNotAllowed
-
MissingContentLength
-
NoSuchBucket
-
NoSuchKey
-
NoSuchLifecycleConfiguration
-
NoSuchReplicationConfiguration
-
NoSuchUpload
-
NoSuchVersion
-
NotImplemented
-
PermanentRedirect
-
PreconditionFailed
-
Redirect
-
RestoreAlreadyInProgress
-
RequestIsNotMultiPartContent
-
RequestTimeTooSkewed
-
SignatureDoesNotMatch
-
ServiceUnavailable
-
SlowDown
-
TemporaryRedirect
-
TooManyBuckets
-
UnresolvableGrantByEmailAddress
-
UserKeyMustBeSpecified
2.1. REST Error Format
The HyperStore system supports the same REST error format specified in the "REST Error Responses" section of the Amazon S3 API.
3. Common Request Headers
From the "Common Request Headers" section of the Amazon S3 REST API, the HyperStore system supports the headers listed below. If a common request header from the Amazon S3 API specification is not listed below, the HyperStore system does not support it. For header descriptions, refer to the "Common Request Headers" section of the Amazon S3 REST API.
-
Authorization
-
Content-Length
-
Content-Type
-
Content-MD5
-
Date
-
Expect
-
Host
-
x-amz-date
4. Common Response Headers
From the "Common Response Headers" section of the Amazon S3 REST API, the HyperStore system supports the headers listed below. If a common response header from the Amazon S3 API specification is not listed below, the HyperStore system does not support it. For header descriptions, refer to the "Common Response Headers" section of the Amazon S3 REST API
-
Content-Length
-
Content-Type
-
Connection
-
Date
-
ETag
-
Server
-
x-amz-delete-marker
-
x-amz-request-id
-
x-amz-version-id
5. Authenticating Requests (AWS Signature Version 4)
The HyperStore system supports AWS Signature Version 4 for authenticating inbound API requests. The HyperStore implementation of this feature is compliant with Amazon’s specification of the feature. For example, you can express authentication information in the HTTP Authorization header or in query string parameters; and you can compute a checksum of the entire payload prior to transmission, or for large uploads, you can use chunked upload.
For more information on this Amazon S3 feature, refer to the "Authenticating Requests (AWS Signature Version 4)" section of the Amazon S3 REST API.
|
|
The HyperStore system continues to support AWS Signature Version 2 as well.
6. Operations on the Service
From the "Operations on the Service" section of the Amazon S3 REST API, the HyperStore system supports the operations listed below. If a service operation from the Amazon S3 API specification is not listed below, the HyperStore system does not support it. For details of HyperStore support for specific operations — such as which parameters and headers are supported — click on the links.
6.1. GET Service
For the Amazon S3 "GET Service" operation, the HyperStore system supports the S3 common request headers and common response headers and also the operation-specific items listed below. If a parameter, header, element, or special error from the Amazon S3 specification for this operation is not listed below, the HyperStore system does not support it.
For operation usage information and for descriptions of request and response items, see GET Service in the Amazon S3 REST API specification.
6.1.1. Response Elements
-
Bucket
-
Buckets
-
CreationDate
-
DisplayName
-
ID
-
ListAllMyBucketsResult
-
Name
-
Owner
7. Operations on Buckets
From the "Operations on Buckets" section of the Amazon S3 REST API, the HyperStore system supports the operations listed below. If a bucket operation from the Amazon S3 API specification is not listed below, the HyperStore system does not support it. For details of HyperStore support for specific operations — such as which parameters and headers are supported — click on the links.
7.1. DELETE Bucket
For the Amazon S3 "DELETE Bucket" operation, the HyperStore system supports the S3 common request headers and common response headers. For operation usage, information see DELETE Bucket in the Amazon S3 REST API specification.
7.2. DELETE Bucket cors
For the Amazon S3 "DELETE Bucket cors" operation, the HyperStore system supports the S3 common request headers and common response headers. For operation usage information see DELETE Bucket cors in the Amazon S3 REST API specification.
7.3. DELETE Bucket lifecycle
For the Amazon S3 "DELETE Bucket lifecycle" operation, the HyperStore system supports the S3 common request headers and common response headers. For operation usage information see DELETE Bucket lifecycle in the Amazon S3 REST API specification.
7.4. DELETE Bucket policy
For the Amazon S3 "DELETE Bucket policy" operation, the HyperStore system supports the S3 common request headers and common response headers. For operation usage information see DELETE Bucket policy in the Amazon S3 REST API specification.
7.5. DELETE Bucket replication
For the Amazon S3 "DELETE Bucket replication" operation, the HyperStore system supports the S3 common request headers and common response headers. For operation usage information see DELETE Bucket replication in the Amazon S3 REST API specification.
7.6. DELETE Bucket website
For the Amazon S3 "DELETE Bucket website" operation, the HyperStore system supports the S3 common request headers and common response headers. For operation usage information see DELETE Bucket website in the Amazon S3 REST API specification.
7.7. GET Bucket (List Objects)
For the Amazon S3 "GET Bucket (List Objects)" operation, the HyperStore system supports the S3 common request headers and common response headers and also the operation-specific items listed below. If a parameter, header, element, or special error from the Amazon S3 specification for this operation is not listed below, the HyperStore system does not support it.
For operation usage information and for descriptions of request and response items, see GET Bucket (List Objects) in the Amazon S3 REST API specification.
7.7.1. Request Parameters
-
delimiter
The HyperStore system does not support %c2%85(U+0085) as a delimiter value -
encoding-type
-
marker
-
max-keys
-
prefix
HyperStore Extension to the S3 API
The HyperStore system supports the following Request Parameter as an extension to the "GET Bucket (List Objects)" operation:
Parameter |
Description |
Required |
||
---|---|---|---|---|
meta |
You can use the request parameter meta=true to return user-defined object metadata with the GET Bucket response. Without this HyperStore extension, the standard S3 GET Bucket method returns only the system metadata associated with each object, not user-defined metadata. Using standard S3 API calls you would have to do a GET Object on individual objects to retrieve the user-defined object metadata. (For an overview of the HyperStore system’s support for object metadata, see Object Metadata). If you use the HyperStore S3 extension request parameter meta=true option for GET Bucket, then within the XML response body the user-defined metadata associated with each object (if any) will be presented as metadata elements nested in the Contents element for the object. In the example below, the three lines that follow the Owner element show how user-defined metadata for the object my-image.jpg would be included in the GET Bucket response body. HTTP/1.1 200 OK ... <ListBucketResult xmlns="http://s3.cloudian.com/2013-10-01/"> ... <Contents> <Key>my-image.jpg</Key> <LastModified>2014-10-12T17:50:30.000Z</LastModified> <ETag>"fba9dede5f27731c9771645a39863328"</ETag> <Size>434234</Size> <StorageClass>STANDARD</StorageClass> <Owner> <ID>75aa57f09aa0c8caeab4f8c24e99d10f8e7faeebf76c078efc7c6caea54ba06a</ID> <DisplayName>mtd@amazon.com</DisplayName> </Owner> <metadata name="name1">value1</metadata> <metadata name="name2">value2</metadata> <metadata name="name3">value3</metadata> </Contents> ... ... </ListBucketResult>
|
No (defaults to "false") |
7.7.2. Response Headers
-
x-amz-bucket-region
HyperStore Extension to the S3 API
The HyperStore system supports the following Response Header as an extension to the "GET Bucket (List Objects)" operation:
Parameter | Description | Required |
---|---|---|
x-gmt-policyid |
This header specifies the unique ID of the storage policy assigned to the bucket. For more information see PUT Bucket. |
No |
7.7.3. Response Elements
-
Contents
-
CommonPrefixes
-
Delimiter
-
DisplayName
-
Encoding-Type
-
ETag
-
ID
-
IsTruncated
-
Key
-
LastModified
-
ListBucketResult
-
Marker
-
MaxKeys
-
Name
-
NextMarker
-
Owner
-
Prefix
-
Size
-
StorageClass (values STANDARD and GLACIER only)
HyperStore Extension to the S3 API
The HyperStore system supports a metadata response element as described in the Request Parameters section above.
7.8. GET Bucket acl
For the Amazon S3 "GET Bucket acl" operation, the HyperStore system supports the S3 common request headers and common response headers and also the operation-specific items listed below. If a parameter, header, element, or special error from the Amazon S3 specification for this operation is not listed below, the HyperStore system does not support it.
For operation usage information and for descriptions of request and response items, see GET Bucket acl in the Amazon S3 REST API specification.
7.8.1. Response Elements
-
AccessControlList
-
AccessControlPolicy
-
DisplayName
-
Grant
-
Grantee
-
ID
-
Owner
-
Permission
7.9. GET Bucket cors
For the Amazon S3 "GET Bucket cors" operation, the HyperStore system supports the S3 common request headers and common response headers and also the operation-specific items listed below. If a parameter, header, element, or special error from the Amazon S3 specification for this operation is not listed below, the HyperStore system does not support it.
For operation usage information and for descriptions of request and response items, see GET Bucket cors in the Amazon S3 REST API specification.
7.9.1. Response Elements
-
CORSConfiguration
-
CORSRule
-
AllowedHeader
-
AllowedMethod
-
AllowedOrigin
-
ExposeHeader
-
ID
-
MaxAgeSeconds
7.10. GET Bucket lifecycle
For the Amazon S3 "GET Bucket lifecycle" operation, the HyperStore system supports the S3 common request headers and common response headers and also the operation-specific items listed below. If a parameter, header, element, or special error from the Amazon S3 specification for this operation is not listed below, the HyperStore system does not support it.
For operation usage information and for descriptions of request and response items, see GET Bucket lifecycle in the Amazon S3 REST API specification.
7.10.1. Response Headers
HyperStore Extension to the S3 API
The HyperStore system supports the following Response Headers as extensions to the "GET Bucket lifecycle" operation:
Parameter | Description | Required |
---|---|---|
x-gmt-tieringinfo |
See PUT Bucket lifecycle. |
No |
x-gmt-compare |
See PUT Bucket lifecycle. |
No |
7.10.2. Response Elements
-
Date
-
Days
-
Expiration
-
ID
-
LifecycleConfiguration
-
Prefix
-
Rule
-
Status
-
StorageClass
-
Transition
7.10.3. Special Errors
-
NoSuchLifecycleConfiguration
7.11. GET Bucket policy
For the Amazon S3 "GET Bucket policy" operation, the HyperStore system supports the S3 common request headers and common response headers and also the operation-specific items listed below. If a parameter, header, element, or special error from the Amazon S3 specification for this operation is not listed below, the HyperStore system does not support it.
For operation usage information and for descriptions of request and response items, see GET Bucket policy in the Amazon S3 REST API specification.
7.11.1. Response Elements
The response contains the (JSON) policy of the specified bucket.
7.12. GET Bucket location
For the Amazon S3 "GET Bucket location" operation, the HyperStore system supports the S3 common request headers and common response headers and also the operation-specific items listed below. If a parameter, header, element, or special error from the Amazon S3 specification for this operation is not listed below, the HyperStore system does not support it.
For operation usage information and for descriptions of request and response items, see GET Bucket location in the Amazon S3 REST API specification.
7.12.1. Response Elements
-
LocationConstraint
7.13. GET Bucket logging
For the Amazon S3 "GET Bucket logging" operation, the HyperStore system supports the S3 common request headers and common response headers and also the operation-specific items listed below. If a parameter, header, element, or special error from the Amazon S3 specification for this operation is not listed below, the HyperStore system does not support it.
For operation usage information and for descriptions of request and response items, see GET Bucket logging in the Amazon S3 REST API specification.
7.13.1. Response Elements
-
BucketLoggingStatus
-
Grant
-
Grantee
-
LoggingEnabled
-
Permission
-
TargetBucket
-
TargetGrants
-
TargetPrefix
7.14. GET Bucket replication
For the Amazon S3 "GET Bucket replication" operation, the HyperStore system supports the S3 common request headers and common response headers and also the operation-specific items listed below. If a parameter, header, element, or special error from the Amazon S3 specification for this operation is not listed below, the HyperStore system does not support it.
For operation usage information and for descriptions of request and response items, see GET Bucket replication in the Amazon S3 REST API specification.
7.14.1. Response Headers
HyperStore Extension to the S3 API
HyperStore will return these headers if and only if they were included in the PUT Bucket replication request that created the bucket’s replication configuration:
-
x-gmt-crr-endpoint
-
x-gmt-crr-credentials
These headers would have been used in the PUT request if the replication destination bucket is not in the HyperStore system in which the source bucket is located (such as in the case of replicating from a HyperStore bucket to Amazon S3 as the destination). For more information about these headers see the section regarding HyperStore support for PUT Bucket replication.
7.14.2. Response Elements
-
ReplicationConfiguration
-
Role
-
Rule
-
ID
-
Status
-
Prefix
-
Destination
-
Bucket
-
StorageClass
7.14.3. Special Errors
-
NoSuchReplicationConfiguration
7.15. GET Bucket Object versions
For the Amazon S3 "GET Bucket Object versions" operation, the HyperStore system supports the S3 common request headers and common response headers and also the operation-specific items listed below. If a parameter, header, element, or special error from the Amazon S3 specification for this operation is not listed below, the HyperStore system does not support it.
For operation usage information and for descriptions of request and response items, see GET Bucket Object versions in the Amazon S3 REST API specification.
7.15.1. Request Parameters
-
delimiter
-
encoding-type
-
key-marker
-
max-keys
-
prefix
-
version-id-marker
HyperStore Extension to the S3 API
The HyperStore system supports an optional request parameter meta=true. This extension works exactly as described for GET Bucket (List Objects), except that for "GET Bucket Object Versions", in the response body the metadata element will be nested in the Version element and DeleteMarker element of the ListVersionsResult object.
7.15.2. Response Elements
-
DeleteMarker
-
DisplayName
-
Encoding-Type
-
ETag
-
ID
-
IsLatest
-
IsTruncated
-
Key
-
KeyMarker
-
LastModified
-
ListVersionsResult
-
MaxKeys
-
Name
-
NextKeyMarker
-
NextVersionIdMarker
-
Owner
-
Prefix
-
Size
-
StorageClass
-
Version
-
VersionId
-
VersionIdMarker
HyperStore Extension to the S3 API
The HyperStore system supports a metadata response element, which presents user-defined metadata. This extension works exactly as described for GET Bucket (List Objects), except that for "GET Bucket Object Versions", in the response body the metadata element will be nested in the Version element and DeleteMarker element of the ListVersionsResult object.
7.16. GET Bucket versioning
For the Amazon S3 "GET Bucket versioning" operation, the HyperStore system supports the S3 common request headers and common response headers and also the operation-specific items listed below. If a parameter, header, element, or special error from the Amazon S3 specification for this operation is not listed below, the HyperStore system does not support it.
For operation usage information and for descriptions of request and response items, see GET Bucket versioning in the Amazon S3 REST API specification.
7.16.1. Response Elements
-
Status
-
VersioningConfiguration
7.17. GET Bucket website
For the Amazon S3 "GET Bucket website" operation, the HyperStore system supports the S3 common request headers and common response headers and also the operation-specific items listed below. If a parameter, header, element, or special error from the Amazon S3 specification for this operation is not listed below, the HyperStore system does not support it.
For operation usage information and for descriptions of request and response items, see GET Bucket website in the Amazon S3 REST API specification.
7.17.1. Response Elements
The response XML includes the same elements that were uploaded when the bucket was configured as a website.
7.18. HEAD Bucket
For the Amazon S3 "HEAD Bucket" operation, the HyperStore system supports the S3 common request headers and common response headers and also the operation-specific items listed below. If a parameter, header, element, or special error from the Amazon S3 specification for this operation is not listed below, the HyperStore system does not support it.
For operation usage information and for descriptions of request and response items, see HEAD Bucket in the Amazon S3 REST API specification.
7.18.1. Response Headers
-
x-amz-bucket-region
HyperStore Extension to the S3 API
The HyperStore system supports the following Response Header as an extension to the "HEAD Bucket" operation:
Parameter | Description | Required |
---|---|---|
x-gmt-policyid |
This header specifies the unique ID of the storage policy assigned to the bucket. For more information see PUT Bucket. |
No |
7.19. List Multipart Uploads
For the Amazon S3 "List Multipart Uploads" operation, the HyperStore system supports the S3 common request headers and common response headers and also the operation-specific items listed below. If a parameter, header, element, or special error from the Amazon S3 specification for this operation is not listed below, the HyperStore system does not support it.
For operation usage information and for descriptions of request and response items, see List Multipart Uploads in the Amazon S3 REST API specification.
7.19.1. Request Parameters
-
delimiter
-
encoding-type
-
max-uploads
-
key-marker
-
prefix
-
upload-id-marker
7.19.2. Response Elements
-
ListMultipartUploadsResult
-
Bucket
-
KeyMarker
-
UploadIdMarker
-
NextKeyMarker
-
NextUploadIdMarker
-
Encoding-Type
-
MaxUploads
-
IsTruncated
-
Upload
-
Key
-
UploadId
-
Initiator
-
ID
-
DisplayName
-
Owner
-
StorageClass
-
Initiated
-
ListMultipartUploadsResult.Prefix
-
Delimiter
-
CommonPrefixes
-
CommonPrefixes.Prefix
7.20. PUT Bucket
For the Amazon S3 "PUT Bucket" operation, the HyperStore system supports the S3 common request headers and common response headers and also the operation-specific items listed below. If a parameter, header, element, or special error from the Amazon S3 specification for this operation is not listed below, the HyperStore system does not support it.
For operation usage information and for descriptions of request and response items, see PUT Bucket in the Amazon S3 REST API specification.
|
For HyperStore S3 bucket capacity information, see Maximum Objects Per S3 Bucket and Mass Deletes. |
7.20.1. Request Headers
-
x-amz-acl
-
x-amz-grant-read
-
x-amz-grant-write
-
x-amz-grant-read-acp
-
x-amz-grant-write-acp
-
x-amz-grant-full-control
HyperStore Extension to the S3 API
The HyperStore system supports the following Request Header as an extension to the "PUT Bucket" operation:
Parameter |
Description |
Required |
||
---|---|---|---|---|
x-gmt-policyid |
This header specifies the unique ID of the storage policy to assign to the newly created bucket. The storage policy determines how data in the bucket will be distributed and protected through either replication or erasure coding. System administrators can create multiple storage policies — through either the CMC or the Admin API — and the system automatically assigns each a unique policy ID that becomes part of the policy definition. (To obtain a list of storage policies for your system and their policy IDs, you can use the Admin API’s GET /bppolicy/listpolicy method). With the "x-gmt-policyid" request header for "PUT Bucket", you specify the ID of the desired storage policy when you create a new bucket. Note however that some policies may not be available to all user groups — a policy’s availability is specified by system administrators at the time of policy creation, and this information becomes part of the policy definition. When you specify an "x-gmt-policyid" value with a "PUT Bucket" request, the policy ID must be for a policy that is available to the group to which the bucket owner belongs. Also the policy ID must be for a storage policy from the service region that is specified in the "PUT Bucket" request’s LocationConstraint element. If the "PUT Bucket" request does not include the "x-gmt-policyid" request header, then the system will automatically assign the system default storage policy to the bucket during bucket creation.
Example header: x-gmt-policyid: 1bc90238f9f11cb32f5e4e901675d50b For more information on storage policies, see Storage Policies. |
No |
7.20.2. Request Elements
-
CreateBucketConfiguration
-
LocationConstraint
|
The HyperStore system enforces the same bucket naming restrictions as does Amazon S3. |
7.21. PUT Bucket acl
For the Amazon S3 "PUT Bucket acl" operation, the HyperStore system supports the S3 common request headers and common response headers and also the operation-specific items listed below. If a parameter, header, element, or special error from the Amazon S3 specification for this operation is not listed below, the HyperStore system does not support it.
For operation usage information and for descriptions of request and response items, see PUT Bucket acl in the Amazon S3 REST API specification.
7.21.1. Request Headers
-
x-amz-acl
-
x-amz-grant-read
-
x-amz-grant-write
-
x-amz-grant-read-acp
-
x-amz-grant-write-acp
-
x-amz-grant-full-control
7.21.2. Request Elements
-
AccessControlList
-
AccessControlPolicy
-
DisplayName
-
Grant
-
Grantee
-
ID
-
Owner
-
Permission
7.22. PUT Bucket cors
For the Amazon S3 "PUT Bucket cors" operation, the HyperStore system supports the S3 common request headers and common response headers and also the operation-specific items listed below. If a parameter, header, element, or special error from the Amazon S3 specification for this operation is not listed below, the HyperStore system does not support it.
For operation usage information and for descriptions of request and response items, see PUT Bucket cors in the Amazon S3 REST API specification.
7.22.1. Request Headers
-
Content-MD5
7.22.2. Request Elements
-
CORSConfiguration
-
CORSRule
-
ID
-
AllowedMethod
-
AllowedOrigin
-
AllowedHeader
-
MaxAgeSeconds
-
ExposeHeader
7.23. PUT Bucket lifecycle
For the Amazon S3 "PUT Bucket lifecycle" operation, the HyperStore system supports the S3 common request headers and common response headers and also the operation-specific items listed below. If a parameter, header, element, or special error from the Amazon S3 specification for this operation is not listed below, the HyperStore system does not support it.
For operation usage information and for descriptions of request and response items, see PUT Bucket lifecycle in the Amazon S3 REST API specification.
|
With the HyperStore system, only the bucket owner can create Lifecycle rules. |
7.23.1. Request Headers
-
Content-MD5
HyperStore Extension to the S3 API
The HyperStore system supports the following Request Headers as extensions to the "PUT Bucket lifecycle" operation:
Parameter |
Description |
Required |
||||
---|---|---|---|---|---|---|
x-gmt-tieringinfo |
This extension header enables you to configure a bucket for schedule-based automatic transitioning (also known as "auto-tiering") of objects from HyperStore storage to Amazon S3 storage or Amazon Glacier storage. For background information on the HyperStore auto-tiering feature, see Auto-Tiering. The x-gmt-tieringinfo header is formatted as follows: x-gmt-tieringinfo: url-encode(S3/S3GLACIER|EndPoint:url-encode(s3-endpoint), [Action:stream/nostream/redirect])
For objects tiered to Glacier, using the POST Object Restore operation is the only supported object retrieval method. Streaming and redirects are not supported. # Example 1 (before URL encoding) - Tiering to Amazon S3 x-gmt-tieringinfo: S3|EndPoint:http://s3.amazonaws.com. # Example 2 (before URL encoding) - Tiering to Glacier x-gmt-tieringinfo: S3GLACIER|EndPoint:http://s3.amazonaws.com. # URL encoding of Example 2 x-gmt-tieringinfo: S3GLACIER%7CEndPoint%3Ahttp%253A%252F%252Fs3.amazonaws.com.
|
No |
||||
x-gmt-compare |
If you include this extension header in your "PUT Bucket lifecycle" request and set the header value to "LAT", then in lifecycle rules that you configure with the "Days" comparator the rule will be implemented as number of days since the object’s Last Access Time. If you do not use this extension header, or if you include the header but assign it no value or any value other than "LAT", then "Days" based lifecycle rules will be implemented as number of days since the object’s creation (the default Amazon S3 behavior). You can use this header to create:
|
No |
A sample PUT Bucket Lifecycle request/response pair is below. This rule transitions objects in the user’s HyperStore S3 storage bucket to Amazon S3 90 days after Last Access Time. Note that the StorageClass element specifies GLACIER even though the tiering target is actually Amazon S3.
# Request PUT /?lifecycle HTTP/1.1. Host: bucket1.cloudians3.com:80. Accept-Encoding: identity. Content-Length: 216. User-Agent: Boto/2.24.0 Python/2.6.6 Linux/2.6.32-358.23.2.el6.x86_64. x-gmt-tieringinfo: S3%7CEndPoint%3Ahttp%253A%252F%252Fs3.amazonaws.com. x-gmt-compare: LAT Date: Sun, 16 Nov 2014 17:54:16 GMT. Content-MD5: Nip5xNP0P41djj608bRHNQ==. Content-Type: text/xml. Authorization: AWS a-key:NfPRnsSbTcxBZ2vN2MX4ZsArJAQ=. . <?xml version="1.0" encoding="UTF-8"?> <LifecycleConfiguration> <Rule> <ID>Transition to Amazon S3</ID> <Prefix></Prefix> <Status>Enabled</Status> <Transition> <StorageClass>GLACIER</StorageClass> <Days>90</Days> </Transition> </Rule> </LifecycleConfiguration> # Response HTTP/1.1 200 OK. Date: Sun, 16 Nov 2014 17:54:16 GMT. x-amz-request-id: AF5C7C2098C511E3. x-gmt-usage: 0,1,623,89,0. Content-Length: 0. Server: CloudianS3.
A second sample PUT Bucket Lifecycle request/response pair is below. This rule transitions all objects in the user’s HyperStore S3 storage bucket to Amazon Glacier one year after object creation, and then expires (deletes) the objects five years after object creation. Note that the EndPoint is an Amazon S3 endpoint even though the ultimate tiering target is Glacier.
# Request PUT /?lifecycle HTTP/1.1. Host: bucket1.cloudians3.com:80. Accept-Encoding: identity. Content-Length: 235. User-Agent: Boto/2.24.0 Python/2.6.6 Linux/2.6.32-358.23.2.el6.x86_64. x-gmt-tieringinfo: S3GLACIER%7CEndPoint%3Ahttp%253A%252F%252Fs3.amazonaws.com. Date: Tue, 18 Feb 2014 17:54:16 GMT. Content-MD5: Nip5xNP0P41djj608bRQHN==. Content-Type: text/xml. Authorization: AWS a-key:NfPRnsSbTcxBZ2vN2MX4ZsArJAQ=. . <?xml version="1.0" encoding="UTF-8"?> <LifecycleConfiguration> <Rule> <ID>Transition to Glacier and later delete</ID> <Prefix></Prefix> <Status>Enabled</Status> <Transition> <StorageClass>GLACIER</StorageClass> <Days>365</Days> </Transition> <Expiration> <Days>1825</Days> </Expiration> </Rule> </LifecycleConfiguration> # Response HTTP/1.1 200 OK. Date: Tue, 18 Feb 2014 17:58:16 GMT. x-amz-request-id: AF5C7C2098C522F4. x-gmt-usage: 0,1,623,89,0. Content-Length: 0. Server: CloudianS3.
7.23.2. Request Elements
-
Date
-
Days
-
Expiration
-
ID
-
LifecycleConfiguration
-
Prefix
-
Rule
-
Status
-
StorageClass
If you are using the HyperStore extension request header x-gmt-tieringinfo, then for the request element "StorageClass" you must specify "GLACIER". Set the Storage Class to GLACIER even if your final tiering destination is Amazon S3. -
Transition
7.24. PUT Bucket policy
For the Amazon S3 "PUT Bucket policy" operation, the HyperStore system supports the S3 common request headers and common response headers and also the operation-specific items listed below. If a parameter, header, element, or special error from the Amazon S3 specification for this operation is not listed below, the HyperStore system does not support it.
For operation usage information and for descriptions of request and response items, see PUT Bucket policy in the Amazon S3 REST API specification. For policy construction guidance see also the Amazon documentation on Using Bucket Policies, but note that the HyperStore system supports only the policy types indicated below.
7.24.1. Request Elements
The body is a JSON string containing the policy contents containing the policy statements.
The HyperStore system supports only these bucket policy types:
|
The system does not check the bucket policy if the user performing an operation in the bucket (such as a PUT Object or GET Object) is the bucket owner. In the case of GET Object, the system does not check the bucket policy if the user doing the GET is the object owner. For example, if a bucket policy is set so that only certain IP addresses are allowed to access the bucket, this will not apply to an object owner doing a GET on the object. |
Restricting Access to a Specific HTTP Referrer
The only accepted format is the below that allows GetObject on the bucket from only the specified referrer URIs.
{ "Version":"2012-10-17", "Id":"http referer policy example", "Statement":[ { "Sid":"Allow get requests originated from URI-1 and URI-2", "Effect":"Allow", "Principal":"*", "Action":"s3:GetObject", "Resource":"arn:aws:s3:::examplebucket/*", "Condition":{ "StringLike":{ "aws:Referer":["URI-1"] }, "StringLike":{ "aws:Referer":["URI-2"] } } } } ] }
-
Multiple "StringLike" conditions can be specified.
-
URI value (e.g., URI-1 and URI-2) is compared to HTTP Referer header with case-insensitive matching and multi-character wildcard (*) and single-character wildcard (?).
Restricting Access to Specific IP Addresses
The HyperStore system supports restricting bucket access to specific source IP addresses, by using the "IpAddress" and/or "NotIpAddress" conditions and the "aws:SourceIp" condition key. The example below allows authenticated users from source address range 54.240.143.* to perform any S3 action in the bucket — except for users from origin IP address 54.240.143.188, which is forbidden access.
{ "Version": "2012-10-17", "Id": "S3PolicyId1", "Statement": [ { "Sid": "IPAllow", "Effect": "Allow", "Principal": "*", "Action": "s3:*", "Resource": "arn:aws:s3:::examplebucket/*", "Condition" : { "IpAddress" : { "aws:SourceIp": "54.240.143.0/24" }, "NotIpAddress" : { "aws:SourceIp": "54.240.143.188/32" } } } ] }
Policy for Server-Side Encryption
The HyperStore systems supports bucket policies that mandate server-side encryption (so that object upload requests are rejected if they omit the server-side encryption request header) or that forbid server-side encryption (so that object upload requests are rejected if they include the server-side encryption request header).
For example, the policy below requires all PUT Object requests to include the x-amz-server-side-encryption request header:
{ "Version":"2012-10-17", "Id":"PutObjPolicy", "Statement":[{ "Sid":"DenyUnEncryptedObjectUploads", "Effect":"Deny", "Principal":"*" "Action":"s3:PutObject", "Resource":"arn:aws:s3:::YourBucket/*", "Condition":{ "StringNotEquals":{ "s3:x-amz-server-side-encryption":"AES256" } } } ] }
|
For more information about HyperStore’s support for server-side encryption, including how to enable HyperStore to support the AES-256 encryption standard that Amazon-compliant server-side encryption requires, see Server-Side Encryption. |
Policy for Allowing Object Read Access for Specified Users
HyperStore supports the creation of bucket policies that allow object read access to specified users. The users are specified with the "Principal" element, and each user must be specified in format "CanonicalUser":"<canonicalUserId>". The only supported Actions are GetBucketLocation, ListBucket, and GetObject. Configuring support for all three actions will allow the specified user(s) to list and read all objects in the bucket. This applies to future objects as well as current objects.
For example:
{ "Version": "2012-10-17", "Statement": [ { "Sid": "Allow read access to two specified users", "Effect": "Allow", "Principal": {"CanonicalUser":"3c60500e4c20208909b332b1bcdd3752", "CanonicalUser":"8773d93a05a663e5be8294afe8bd3652"}, "Action": [ "s3:GetBucketLocation", "s3:ListBucket", "s3:GetObject" ], "Resource": [ "arn:aws:s3:::examplebucket/*" ] } ] }
Policy for Public Access to Buckets Configured as Websites
If you have configured a bucket as a static website (using PUT Bucket Website), you can establish a bucket policy that allows public access to the website:
{ "Version":"2012-10-17", "Statement":[{ "Sid":"PublicReadForGetBucketObjects", "Effect":"Allow", "Principal": "*" "Action":["s3:GetObject"], "Resource":["arn:aws:s3:::example-bucket/*" ] } ] }
For further information, see the Amazon documentation on Setting Up a Website.
|
Public access to buckets configured as static websites is through HTTP not HTTPS. The static website feature does not support HTTPS (SSL). |
7.24.2. Response Elements
PUT response elements return whether the operation succeeded or not.
7.25. PUT Bucket logging
For the Amazon S3 "PUT Bucket logging" operation, the HyperStore system supports the S3 common request headers and common response headers and also the operation-specific items listed below. If a parameter, header, element, or special error from the Amazon S3 specification for this operation is not listed below, the HyperStore system does not support it.
For operation usage information and for descriptions of request and response items, see PUT Bucket logging in the Amazon S3 REST API specification.
|
If you are supporting bucket logging in your service, and if you use a load balancer in front of your S3 Service nodes, you should configure your S3 Service to support the HTTP X-Forwarded-For header. This will enable bucket logs to record the true originating IP address of S3 requests, rather than the load balancer IP address. By default the S3 Service does not support the X-Forwarded-For header. You can enable support for this header using the system configuration file s3.xml.erb. See s3.xml.erb in the Cloudian HyperStore Administrator’s Guide. |
7.25.1. Request Elements
-
BucketLoggingStatus
-
EmailAddress
-
Grant
-
Grantee
-
LoggingEnabled
-
Permission
-
TargetBucket
-
TargetGrants
-
TargetPrefix
7.26. PUT Bucket replication
For the Amazon S3 "PUT Bucket replication" operation, the HyperStore system supports the S3 common request headers and common response headers and also the operation-specific items listed below. If a parameter, header, element, or special error from the Amazon S3 specification for this operation is not listed below, the HyperStore system does not support it.
For operation usage information and for descriptions of request and response items, see PUT Bucket replication in the Amazon S3 REST API specification. For an overview of this Amazon S3 feature see Cross-Region Replication in the Amazon S3 Developer’s Guide.
|
Unlike Amazon S3, HyperStore does not require that you set up an IAM Role (or anything analogous) in order to use bucket replication. Also, HyperStore does not require that the destination bucket be in a different region than the source bucket. With HyperStore you can replicate to a destination bucket that’s in the same region as the source bucket, if you want to. Note however that replicating from a source bucket to another bucket in the same region will not provide the geographical dispersion of data copies that replicating across regions does. Apart from the above exceptions, HyperStore bucket replication has the same requirements as Amazon S3 bucket replication, including that versioning must be enabled (using the PUT Bucket versioning operation) on both the source bucket and the destination bucket in order to use bucket replication. In addition, HyperStore has this requirement in order to use bucket replication: To replicate to a destination bucket in a given service region, there must be an entry for that region in the system configuration file tiering-regions.xml. This is true regardless of whether that service region is part of your HyperStore system or is in an external system like Amazon S3. By default tiering-region.xml is pre-configured with all the standard Amazon S3 service regions. However if you are replicating to a region within your HyperStore system, or to region in an external S3 system other than Amazon’s, you must manually configure an entry for the region in tiering-region.xml before you can use the bucket replication feature. |
7.26.1. Request Headers
-
Content-MD5
7.26.2. HyperStore Extension to the S3 API
The HyperStore system supports the following Request Headers as extensions to the "PUT Bucket replication" operation. These headers are required only if you are replicating to a destination bucket that is not in the same HyperStore system as the source bucket (for example, if you are replicating from a HyperStore source bucket to a destination Amazon S3 bucket; or from a HyperStore source bucket to a destination bucket in a completely independent HyperStore system).
Parameter | Description | Required |
---|---|---|
x-gmt-crr-endpoint |
Sevice endpoint of the destination S3 service, in format <protocol>://<endpoint>:<port>. For example https://s3.amazonaws:443. Since security credentials will be transmitted in this request (see "x-gmt-crr-credentials" below), https is the recommended protocol rather than regular http. HyperStore does not force https use here, but for security https is advisable. This header is required if the destination bucket is not in the same HyperStore system as the source bucket. Do not use this header if the destination bucket is in the same HyperStore region as the source bucket, or if it is in a different region within the same HyperStore system as the source bucket. |
See description |
x-gmt-crr-credentials |
Access key and secret key for the user account that
HyperStore should use to write to the destination bucket in the destination S3
system, in format This header is required if the destination bucket is not in the same HyperStore system as the source bucket. Do not use this header if the destination bucket is in the same HyperStore region as the source bucket, or if it is in a different region within the same HyperStore system as the source bucket. |
See description |
7.26.3. Request Elements
-
ReplicationConfiguration
-
Role
As with the Amazon S3 API specification, for HyperStore the "Role" element must be included in the PUT Bucket replication request. However, HyperStore ignores the "Role" element’s value (so, you can use any random string as its value). HyperStore does not use an IAM role or anything analogous when implementing cross-region replication. -
Rule
-
ID
-
Status
-
Prefix
-
Destination
-
Bucket
Use the same "Bucket" value formatting as in the Amazon S3 API spec, i.e.
arn:aws:s3:::<bucketname>. -
StorageClass
If you include this optional element in the request, HyperStore ignores its value.
7.27. PUT Bucket versioning
For the Amazon S3 "PUT Bucket versioning" operation, the HyperStore system supports the S3 common request headers and common response headers and also the operation-specific items listed below. If a parameter, header, element, or special error from the Amazon S3 specification for this operation is not listed below, the HyperStore system does not support it.
For operation usage information and for descriptions of request and response items, see PUT Bucket versioning in the Amazon S3 REST API specification.
7.27.1. Request Elements
-
Status
-
VersioningConfiguration
7.28. PUT Bucket website
For the Amazon S3 "PUT Bucket website" operation, the HyperStore system supports the S3 common request headers and common response headers and also the operation-specific items listed below. If a parameter, header, element, or special error from the Amazon S3 specification for this operation is not listed below, the HyperStore system does not support it.
For operation usage information and for descriptions of request and response items, see PUT Bucket website in the Amazon S3 REST API specification.
7.28.1. Request Elements
-
WebsiteConfiguration
-
RedirectAllRequestsTo
-
HostName
-
Protocol
-
WebsiteConfiguration
-
IndexDocument
-
ErrorDocument
8. Operations on Objects
From the "Operations on Objects" section of the Amazon S3 REST API, the HyperStore system supports the operations listed below. If an operation is not listed here, the HyperStore system does not support it. For details of HyperStore support for specific operations — such as which parameters and headers are supported — click on the links.
8.1. DELETE Object
For the Amazon S3 "DELETE Object" operation, the HyperStore system supports the S3 common request headers and common response headers and also the operation-specific items listed below. If a parameter, header, element, or special error from the Amazon S3 specification for this operation is not listed below, the HyperStore system does not support it.
For operation usage information and for descriptions of request and response items, see DELETE Object in the Amazon S3 REST API specification.
|
Do not attempt to delete more than 100,000 objects from a single bucket in less than an hour. Doing so will result in TombstoneOverwhelmingException errors in the Cassandra logs and an inability to successfully execute an S3 GET Bucket (List Objects) operation on the bucket. If the system is in this error condition, you can trigger a tombstone purge as described in Dealing with Excessive Tombstone Build-Up in the Cloudian HyperStore Administrator’s Guide. |
8.1.1. Response Headers
-
x-amz-delete-marker
-
x-amz-version-id
8.2. Delete Multiple Objects
For the Amazon S3 "Delete Multiple Objects" operation, the HyperStore system supports the S3 common request headers and common response headers and also the operation-specific items listed below. If a parameter, header, element, or special error from the Amazon S3 specification for this operation is not listed below, the HyperStore system does not support it.
For operation usage information and for descriptions of request and response items, see Delete Multiple Objects in the Amazon S3 REST API specification.
|
For deleting multiple objects from a single bucket, please follow Amazon’s recommendations in the "Efficiently Deleting Objects" section on this web page: https://aws.amazon.com/articles/1904/ |
|
Do not attempt to delete more than 100,000 objects from a single bucket in less than an hour. Doing so will result in TombstoneOverwhelmingException errors in the Cassandra logs and an inability to successfully execute an S3 GET Bucket (List Objects) operation on the bucket. If the system is in this error condition, you can trigger a tombstone purge as described in Dealing with Excessive Tombstone Build-Up in the Cloudian HyperStore Administrator’s Guide. |
8.2.1. Request Headers
-
Content-MD5
-
Content-Length
8.2.2. Request Elements
-
Delete
-
Quiet
-
Object
-
Key
-
VersionId
8.2.3. Response Elements
-
DeleteResult
-
Deleted
-
Key
-
VersionId
-
DeleteMarker
-
DeleteMarkerVersionId
-
Error
-
Key
-
VersionId
-
Code
-
Message
8.3. GET Object
For the Amazon S3 "GET Object" operation, the HyperStore system supports the S3 common request headers and common response headers and also the operation-specific items listed below. If a parameter, header, element, or special error from the Amazon S3 specification for this operation is not listed below, the HyperStore system does not support it.
For operation usage information and for descriptions of request and response items, see GET Object in the Amazon S3 REST API specification.
8.3.1. Request Parameters
-
response-content-type
-
response-content-language
-
response-expires
-
response-cache-control
-
response-content-disposition
-
response-content-encoding
8.3.2. Request Headers
-
Range
-
If-Modified-Since
-
If-Unmodified-Since
-
If-Match
-
If-None-Match
-
x-amz-server-side-encryption-customer-algorithm
-
x-amz-server-side-encryption-customer-key
-
x-amz-server-side-encryption-customer-key-MD5
8.3.3. Response Headers
-
x-amz-delete-marker
-
x-amz-expiration
-
x-amz-meta-*
-
x-amz-server-side-encryption
-
x-amz-restore
-
x-amz-version-id
-
x-amz-website-redirect-location
-
x-amz-server-side-encryption-customer-algorithm
-
x-amz-server-side-encryption-customer-key-MD5
8.4. GET Object ACL
For the Amazon S3 "GET Object ACL" operation, the HyperStore system supports the S3 common request headers and common response headers and also the operation-specific items listed below. If a parameter, header, element, or special error from the Amazon S3 specification for this operation is not listed below, the HyperStore system does not support it.
For operation usage information and for descriptions of request and response items, see GET Object ACL in the Amazon S3 REST API specification.
8.4.1. Response Elements
-
AccessControlList
-
AccessControlPolicy
-
DisplayName
-
Grant
-
Grantee
-
ID
-
Owner
-
Permission
8.5. HEAD Object
For the Amazon S3 "HEAD Object" operation, the HyperStore system supports the S3 common request headers and common response headers and also the operation-specific items listed below. If a parameter, header, element, or special error from the Amazon S3 specification for this operation is not listed below, the HyperStore system does not support it.
For operation usage information and for descriptions of request and response items, see HEAD Object in the Amazon S3 REST API specification.
8.5.1. Request Headers
-
Range
-
If-Modified-Since
-
If-Unmodified-Since
-
If-Match
-
If-None-Match
-
x-amz-server-side-encryption-customer-algorithm
-
x-amz-server-side-encryption-customer-key
-
x-amz-server-side-encryption-customer-key-MD5
8.5.2. Response Headers
-
x-amz-expiration
-
x-amz-meta-*
-
x-amz-restore
-
x-amz-server-side-encryption
-
x-amz-server-side-encryption-customer-algorithm
-
x-amz-server-side-encryption-customer-key-MD5
-
x-amz-version-id
8.6. OPTIONS object
For the Amazon S3 "OPTIONS object" operation, the HyperStore system supports the S3 common request headers and common response headers and also the operation-specific items listed below. If a parameter, header, element, or special error from the Amazon S3 specification for this operation is not listed below, the HyperStore system does not support it.
For operation usage information and for descriptions of request and response items, see OPTIONS object in the Amazon S3 REST API specification.
8.6.1. Request Headers
-
Origin
-
Access-Control-Request-Method
-
Access-Control-Request-Headers
8.6.2. Response Headers
-
Access-Control-Allow-Origin
-
Access-Control-Max-Age
-
Access-Control-Allow-Methods
-
Access-Control-Allow-Headers
-
Access-Control-Expose-Headers
8.7. POST Object
For the Amazon S3 "POST Object" operation, the HyperStore system supports the S3 common response headers and also the operation-specific items listed below. If a form field, parameter, header, element, or special error from the Amazon S3 specification for this operation is not listed below, the HyperStore system does not support it.
For operation usage information and for descriptions of request and response items, see POST Object in the Amazon S3 REST API specification.
8.7.1. Form Fields
-
AWSAccessKeyId
-
acl
-
Cache-Control, Content-Type, Content-Disposition, Content-Encoding, Expires
-
file
-
key
-
policy
-
success_action_redirect, redirect
-
success_action_status
-
x-amz-storage-class (STANDARD only)
-
x-amz-meta-*
The metadata values must be UTF-8 and must not contain control characters less than 0x20 except for \r, \n, and \t. Also, normal XML escaping is required where appropriate. -
x-amz-website-redirect-location
-
x-amz-server-side-encryption
The presence of the x-amz-server-side-encryption header triggers server-side encryption, but the HyperStore S3 Service disregards the value of this header (for example, "AES256"). Instead it uses the encryption algorithm specified by the system setting mts.properties: cloudian.s3.serverside.encryption.keylength. By default, AES256 is not enabled in the HyperStore system and AES128 is used. While AES128 may be acceptable for testing purposes, to be compliant with Amazon S3 you must first enable AES256 in the HyperStore system. For instructions see Enabling AES-256. -
x-amz-server-side-encryption-customer-algorithm
To use the SSE-C feature you must first enable AES256 in the HyperStore system. By default it is not enabled. For instructions see Enabling AES-256. -
x-amz-server-side-encryption-customer-key
-
x-amz-server-side-encryption-customer-key-MD5
8.7.2. Response Headers
-
x-amz-expiration
-
success_action_redirect, redirect
-
x-amz-server-side-encryption
-
x-amz-server-side-encryption-customer-algorithm
-
x-amz-server-side-encryption-customer-key-MD5
-
x-amz-version-id
8.7.3. Response Elements
-
Bucket
-
ETag
-
Key
-
Location
8.8. POST Object restore
For the Amazon S3 "POST Object restore" operation, the HyperStore system supports the S3 common request headers and common response headers and also the operation-specific items listed below. If a parameter, header, element, or special error from the Amazon S3 specification for this operation is not listed below, the HyperStore system does not support it.
For operation usage information and for descriptions of request and response items, see POST Object restore in the Amazon S3 REST API specification.
|
In the context of the HyperStore system, this standard S3 operation is for temporarily restoring a copy of an object that has been auto-tiered to Amazon S3 or Amazon Glacier. For information about the HyperStore auto-tiering feature, see Auto-Tiering. |
8.8.1. Request Headers
-
Content-MD5
8.8.2. Request Elements
-
RestoreRequest
-
Days
8.9. PUT Object
For the Amazon S3 "PUT Object" operation, the HyperStore system supports the S3 common request headers and common response headers and also the operation-specific items listed below. If a parameter, header, element, or special error from the Amazon S3 specification for this operation is not listed below, the HyperStore system does not support it.
For operation usage information and for descriptions of request and response items, see PUT Object in the Amazon S3 REST API specification.
|
For HyperStore S3 bucket capacity information, see Maximum Objects Per S3 Bucket and Mass Deletes. |
|
For more information about HyperStore’s support for server-side encryption, including how to enable HyperStore to support the AES-256 encryption standard that Amazon-compliant server-side encryption requires, see Server-Side Encryption. |
8.9.1. Request Headers
-
Cache-Control
-
Content-Disposition
-
Content-Encoding
-
Content-Length
-
Content-MD5
-
Content-Type
-
Expect
-
Expires
-
x-amz-meta-*
The metadata values must be UTF-8 and must not contain control characters less than 0x20 except for \r, \n, and \t. Also, normal XML escaping is required where appropriate. -
x-amz-storage-class
-
x-amz-website-redirect-location
-
x-amz-acl
-
x-amz-grant-read
-
x-amz-grant-write
-
x-amz-grant-read-acp
-
x-amz-grant-write-acp
-
x-amz-grant-full-control
-
x-amz-server-side-encryption
The presence of the x-amz-server-side-encryption header triggers server-side encryption, but the HyperStore S3 Service disregards the value of this header (for example, "AES256"). Instead it uses the encryption algorithm specified by the system setting mts.properties: cloudian.s3.serverside.encryption.keylength. By default, AES256 is not enabled in the HyperStore system and AES128 is used. While AES128 may be acceptable for testing purposes, to be compliant with Amazon S3 you must first enable AES256 in the HyperStore system. For instructions see Enabling AES-256.
+ - x-amz-server-side-encryption-customer-algorithm
+ NOTE: To use the SSE-C feature you must first enable AES256 in the HyperStore system. By default it is not enabled. For instructions see Enabling AES-256.
+ - x-amz-server-side-encryption-customer-key - x-amz-server-side-encryption-customer-key-MD5
HyperStore Restrictions on Object Names
The following control characters cannot be used anywhere in an object name and will result in a 400 Bad Request response: 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0A ("\n"), 0x0B, 0x0C, 0x0D ("\r"), 0x0E, 0x0F, 0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, 0x19, 0x1A, 0x1B, 0x1C, 0x1D, 0x1E, 0x1F
Also unsupported are:
-
0x09 ("\t") at the beginning of an object name
-
0xBF (inverted question mark) at the end of an object name
Also, an object name may result in 400 Bad Request or be stored as a different name if the supplied object name:
-
Consists only of "."
-
Consists only of ".."
-
Contains a combination of "." and "/", or ".." and "/"
Examples of character sequences that will result in 400 Bad Request:
. |
.. |
./ |
../ |
./. |
./.. |
../. |
../.. |
../a |
../a/ |
a/../../b |
Examples of character sequences that will be stored as a different name:
Supplied Object Name | Stored As |
---|---|
./a |
a |
./a/ |
a/ |
a//b |
a/b |
a/./b |
a/b |
a/../b |
b |
a/.././b |
b |
8.9.2. Response Headers
-
x-amz-expiration
-
x-amz-server-side-encryption
-
x-amz-server-side-encryption-customer-algorithm
-
x-amz-server-side-encryption-customer-key-MD5
-
x-amz-version-id
8.10. PUT Object acl
For the Amazon S3 "PUT Object acl" operation, the HyperStore system supports the S3 common request headers and common response headers and also the operation-specific items listed below. If a parameter, header, element, or special error from the Amazon S3 specification for this operation is not listed below, the HyperStore system does not support it.
For operation usage information and for descriptions of request and response items, see PUT Object acl in the Amazon S3 REST API specification.
8.10.1. Request Headers
-
x-amz-acl
-
x-amz-grant-read
-
x-amz-grant-write
-
x-amz-grant-read-acp
-
x-amz-grant-write-acp
-
x-amz-grant-full-control
8.10.2. Request Elements
-
AccessControlList
-
AccessControlPolicy
-
DisplayName
-
Grant
-
Grantee
-
ID
-
Owner
-
Permission
8.10.3. Response Headers
-
x-amz-version-id
8.11. PUT Object - Copy
For the Amazon S3 "PUT Object - Copy" operation, the HyperStore system supports the S3 common request headers and common response headers and also the operation-specific items listed below. If a parameter, header, element, or special error from the Amazon S3 specification for this operation is not listed below, the HyperStore system does not support it.
For operation usage information and for descriptions of request and response items, see PUT Object - Copy in the Amazon S3 REST API specification.
8.11.1. Request Headers
-
x-amz-copy-source
-
x-amz-metadata-directive
-
x-amz-copy-source-if-match
-
x-amz-copy-source-if-none-match
-
x-amz-copy-source-if-unmodified-since
-
x-amz-copy-source-if-modified-since
-
x-amz-storage-class
-
x-amz-website-redirect-location
-
x-amz-server-side-encryption
The presence of the x-amz-server-side-encryption header triggers server-side encryption, but the HyperStore S3 Service disregards the value of this header (for example, "AES256"). Instead it uses the encryption algorithm specified by the system setting mts.properties: cloudian.s3.serverside.encryption.keylength. By default, AES256 is not enabled in the HyperStore system and AES128 is used. While AES128 may be acceptable for testing purposes, to be compliant with Amazon S3 you must first enable AES256 in the HyperStore system. For instructions see Enabling AES-256. -
x-amz-server-side-encryption-customer-algorithm
To use the SSE-C feature you must first enable AES256 in the HyperStore system. By default it is not enabled. For instructions see Enabling AES-256. -
x-amz-server-side-encryption-customer-key
-
x-amz-server-side-encryption-customer-key-MD5
-
x-amz-copy-source-server-side-encryption-customer-algorithm
-
x-amz-copy-source-server-side-encryption-customer-key
-
x-amz-copy-source-server-side-encryption-customer-key-MD5
-
x-amz-acl
-
x-amz-grant-read
-
x-amz-grant-write
-
x-amz-grant-read-acp
-
x-amz-grant-write-acp
-
x-amz-grant-full-control
8.11.2. Response Headers
-
x-amz-expiration
-
x-amz-copy-source-version-id
-
x-amz-server-side-encryption
-
x-amz-version-id
8.11.3. Response Elements
-
CopyObjectResult
-
ETag
-
LastModified
8.12. Initiate Multipart Upload
For the Amazon S3 "Initiate Multipart Upload" operation, the HyperStore system supports the S3 common request headers and common response headers and also the operation-specific items listed below. If a parameter, header, element, or special error from the Amazon S3 specification for this operation is not listed below, the HyperStore system does not support it.
For operation usage information and for descriptions of request and response items, see Initiate Multipart Upload in the Amazon S3 REST API specification.
|
HyperStore has been tested for multipart uploads of objects up to 50GB in total size (number of parts X size-per-part = up to 50GB). |
8.12.1. Request Headers
-
Cache-Control
-
Content-Disposition
-
Content-Encoding
-
Content-Type
-
Expires
-
x-amz-meta-
-
x-amz-storage-class
-
x-amz-website-redirect-location
-
x-amz-acl
-
x-amz-grant-read
-
x-amz-grant-write
-
x-amz-grant-read-acp
-
x-amz-grant-write-acp
-
x-amz-grant-full-control
-
x-amz-server-side-encryption
The presence of the x-amz-server-side-encryption header triggers server-side encryption, but the HyperStore S3 Service disregards the value of this header (for example, "AES256"). Instead it uses the encryption algorithm specified by the system setting mts.properties: cloudian.s3.serverside.encryption.keylength. By default, AES256 is not enabled in the HyperStore system and AES128 is used. While AES128 may be acceptable for testing purposes, to be compliant with Amazon S3 you must first enable AES256 in the HyperStore system. For instructions see Enabling AES-256. -
x-amz-server-side-encryption-customer-algorithm
To use the SSE-C feature you must first enable AES256 in the HyperStore system. By default it is not enabled. For instructions see Enabling AES-256. -
x-amz-server-side-encryption-customer-key
-
x-amz-server-side-encryption-customer-key-MD5
8.12.2. Response Headers
-
x-amz-server-side-encryption
-
x-amz-server-side-encryption-customer-algorithm
-
x-amz-server-side-encryption-customer-key-MD5
8.12.3. Response Elements
-
InitiateMultipartUploadResult
-
Bucket
-
Key
-
UploadId
8.13. Upload Part
For the Amazon S3 "Upload Part" operation, the HyperStore system supports the S3 common request headers and common response headers and also the operation-specific items listed below. If a parameter, header, element, or special error from the Amazon S3 specification for this operation is not listed below, the HyperStore system does not support it.
For operation usage information and for descriptions of request and response items, see Upload Part in the Amazon S3 REST API specification.
8.13.1. Request Headers
-
Content-Length
-
Content-MD5
-
Expect
-
x-amz-server-side-encryption
The presence of the x-amz-server-side-encryption header triggers server-side encryption, but the HyperStore S3 Service disregards the value of this header (for example, "AES256"). Instead it uses the encryption algorithm specified by the system setting mts.properties: cloudian.s3.serverside.encryption.keylength. By default, AES256 is not enabled in the HyperStore system and AES128 is used. While AES128 may be acceptable for testing purposes, to be compliant with Amazon S3 you must first enable AES256 in the HyperStore system. For instructions see Enabling AES-256. -
x-amz-server-side-encryption-customer-algorithm
To use the SSE-C feature you must first enable AES256 in the HyperStore system. By default it is not enabled. For instructions see Enabling AES-256. -
x-amz-server-side-encryption-customer-key
-
x-amz-server-side-encryption-customer-key-MD5
8.13.2. Response Headers
-
x-amz-server-side-encryption
-
x-amz-server-side-encryption-customer-algorithm
-
x-amz-server-side-encryption-customer-key-MD5
8.13.3. Special Errors
-
NoSuchUpload
8.14. Upload Part - Copy
For the Amazon S3 "Upload Part - Copy" operation, the HyperStore system supports the S3 common request headers and common response headers and also the operation-specific items listed below. If a parameter, header, element, or special error from the Amazon S3 specification for this operation is not listed below, the HyperStore system does not support it.
For operation usage information and for descriptions of request and response items, see Upload Part - Copy in the Amazon S3 REST API specification.
8.14.1. Request Headers
-
x-amz-copy-source
-
x-amz-copy-source-range
-
x-amz-copy-source-if-match
-
x-amz-copy-source-if-none-match
-
x-amz-copy-source-if-unmodified-since
-
x-amz-copy-source-if-modified-since
8.14.2. Response Headers
-
x-amz-copy-source-version-id
-
x-amz-server-side-encryption
8.14.3. Response Elements
-
CopyPartResult
-
ETag
-
LastModified
8.14.4. Special Errors
-
NoSuchUpload
-
InvalidRequest
8.15. Complete Multipart Upload
For the Amazon S3 "Complete Multipart Upload" operation, the HyperStore system supports the S3 common request headers and common response headers and also the operation-specific items listed below. If a parameter, header, element, or special error from the Amazon S3 specification for this operation is not listed below, the HyperStore system does not support it.
For operation usage information and for descriptions of request and response items, see Complete Multipart Upload in the Amazon S3 REST API specification.
8.15.1. Request Elements
-
CompleteMultipartUpload
-
Part
-
PartNumber
-
ETag
8.15.2. Response Headers
-
x-amz-expiration
-
x-amz-server-side-encryption
-
x-amz-version-id
8.15.3. Response Elements
-
CompleteMultipartUploadResult
-
Location
-
Bucket
-
Key
-
ETag
8.15.4. Special Errors
-
InvalidPart
-
InvalidPartOrder
-
NoSuchUpload
8.16. Abort Multipart Upload
For the Amazon S3 "Abort Multipart Upload" operation, the HyperStore system supports the S3 common request headers and common response headers and also the operation-specific items listed below. If a parameter, header, element, or special error from the Amazon S3 specification for this operation is not listed below, the HyperStore system does not support it.
For operation usage information and for descriptions of request and response items, see Abort Multipart Upload in the Amazon S3 REST API specification.
8.16.1. Special Errors
-
NoSuchUpload
8.17. List Parts
For the Amazon S3 "List Parts" operation, the HyperStore system supports the S3 common request headers and common response headers and also the operation-specific items listed below. If a parameter, header, element, or special error from the Amazon S3 specification for this operation is not listed below, the HyperStore system does not support it.
For operation usage information and for descriptions of request and response items, see List Parts in the Amazon S3 REST API specification.
8.17.1. Request Parameters
-
encoding-type
-
uploadId
-
max-parts
-
part-number-marker
8.17.2. Response Elements
-
ListPartsResult
-
Bucket
-
Encoding-Type
-
Key
-
UploadId
-
Initiator
-
ID
-
DisplayName
-
Owner
-
StorageClass
-
PartNumberMarker
-
NextPartNumberMarker
-
MaxParts
-
IsTruncated
-
Part
-
PartNumber
-
LastModified
-
ETag
-
Size
9. Access Control List (ACL)
For the Amazon S3 "Access Control List (ACL)" functionality, the HyperStore system supports the items listed below. If a grantee group, permission type, or canned ACL type from the Amazon S3 documentation is not listed below, the HyperStore system does not support it.
For ACL usage information and for descriptions of ACL items, see Access Control List (ACL) Overview in the Amazon S3 documentation.
9.1. Amazon S3 Predefined Groups
-
Authenticated users group
-
All users group
-
Log delivery group
9.2. Permission Types
-
READ
-
WRITE
-
READ_ACP
-
WRITE_ACP
-
FULL_CONTROL
9.3. Canned ACL
-
private
-
public-read
-
public-read-write
-
authenticated-read
-
bucket-owner-read
-
bucket-owner-full-control
9.3.1. HyperStore Extension to the S3 API
The HyperStore system supports these additional canned ACLs:
Canned ACL | Applies to | Permissions added to ACL |
---|---|---|
group-read |
Bucket and object |
Owner gets FULL_CONTROL. All other members of the owner’s HyperStore service user group get READ access. |
group-read-write |
Bucket and object |
Owner gets FULL_CONTROL. All other members of the owner’s HyperStore service user group get READ and WRITE access. |
|
To grant access to groups other than the requester’s own group, you cannot use canned ACLs. Instead, when using standard Amazon S3 methods for assigning privileges to a grantee (via request headers or request body), specify "<groupID>|" as the grantee. The "<groupID>|" format (with vertical bar) indicates that the grantee is a group — for example, "Group5|". |
|
When access privileges have through separate requests been granted to a group and to a specific member of the group, the user gets the broader of the privilege grants. For example, if Group5 is granted read-write privileges and a specific user within Group5 is separately granted read privileges, the user gets read-write privileges. |
10. HyperStore Developer Topics
10.1. Maximum Objects Per S3 Bucket and Mass Deletes
In the Cassandra column family CLOUDIAN_METADATA, the HyperStore system stores S3 object metadata. In this column family there is one row for each S3 bucket. In each row there is one column for each object in the bucket.
This object metadata storage architecture has implications for S3 bucket capacity and client application design:
-
Because Cassandra has a maximum of two billion columns per row, the HyperStore system has a maximum of two billion S3 objects per bucket.
-
Rather than pushing the limits of capacity for a single bucket, which may negatively impact system performance, it’s best to create multiple buckets and spread S3 object uploads across the multple buckets.
-
For deleting multiple objects from a single bucket, please follow Amazon’s recommendations in the "Efficiently Deleting Objects" section on this web page: https://aws.amazon.com/articles/1904/
10.2. Implementing Single Sign-On for the CMC
To enable integration between a portal and the Cloudian Management Console (CMC), the Cloudian HyperStore system employs one-way hash based Single Sign-On (SSO) solution. It allows for cross-domain sign-ons from the portal to CMC.
User provisioning is beyond the scope of the provided SSO solution. The HyperStore provides an Admin API (see the Cloudian HyperStore Administration API Reference) for user provisioning but the implementation of user mapping is left to the portal application integrating with CMC.
|
The CMC’s SSO solution has been redesigned in the HyperStore version 5.0 The following changes have been made.
|
|
Backward compatibility and deprecated APIs This redesigned SSO solution provides backward compatibility. If you already have working SSO from a portal to an earlier version of the CMC, it should remain working. However, some of the SSO methods from earlier releases have been deprecated. Cloudian, Inc. recommends not using these methods, and in a future release support for them will be discontinued.
|
10.2.1. How SSO for the CMC Works
Cloudian HyperStore SSO is ideal for sites that already have an authentication model in place using a browser/login session and that want to incorporate the Cloudian Management Console into their web portal application.
The idea is that a portal application calculates an one-way hash (also known as a signature) based on Cloudian HyperStore user identification, timestamp and the shared key. Then the user’s browser accesses ssosecurelogin.htm with the signature. The CMC checks for this signature to determine whether a user is authenticated or not. If the signature is found valid, access to the CMC from the client will skip the login page and take the user directly to a CMC interior page such as the Data Explorer page.
|
To use the Cloudian HyperStore SSO feature, the following system configuration settings must be set to "true":
Also in mts-ui.properties, if you enable SSO functionality (by setting sso.enabled to true), then for security reasons you should set sso.shared.key and sso.cookie.cipher.key to custom values. Do not use the default keys. |
10.2.2. CMC SSO Secure Login Using One-Way Hash
The single sign-on with one-way hash method relies on a one-way hash of query string parameters (also known as a signature).
The following HTTP API, using a signature, prompts the CMC to create an authenticated session for the client that submitted the request:
https://<cmc_FQDN>:<cmc_port>/Cloudian/ssosecurelogin.htm?user=USERID&group=GROUPID ×tamp=TIMESTAMP&signature=SIG&redirect=RELATIVE_OR_ABSOLUTE_URL
-
user: Cloudian HyperStore userId of the user
-
group: Cloudian HyperStore groupId of the group to which the user belongs
-
timestamp: Current Epoch time in milliseconds (eg. "1346881953440"). The timestamp is used to implement the configurable request expiration (mts-ui.properties: sso.tolerance.millis; expiration defaults to one hour).
-
signature: This is the URI encoding of the base64 representation of the calculated signature. For further information see below.
-
redirect: This optional parameter can be used to redirect the client to the given URL upon successful sign-in. It is typically set to a CMC interior page such as explorer.htm or admin.htm.
Each value must be URL-encoded by the client. Order of the parameters does not matter.
If the signature is found valid, the CMC creates an authenticated session for the HyperStore user, allowing the client to skip the login page and access to a CMC interior page.
How to Create the Signature
The portal server can create the signature by the following steps.
-
Assemble the query string.
-
querystring = "user=USERID&group=GROUPID×tamp=TIMESTAMP"
When using the querystring to create the signature, do not URL-encode the querystring. Also do not reorder the items. (By contrast, when the client subsequently submits the SSO secure login request to the CMC, it’s desirable to URL encode the request querystring.)
-
-
Calculate one-way hash for the querystring using the standard HmacSHA1 and the CMC SSO shared key. The shared key is configured by mts-ui.properties: sso.shared.key.
-
hashresult = HmacSHA1(querystring, sharedkey)
-
-
Base64 encode the resulting hash.
-
base64string = Base64Encode(hashresult)
-
-
URI encode the base64 encoded hash result.
-
signature = encodeURIComponent(base64string)
-
For a sample of a Python script that uses the one-way hash login API, see Cloudian HyperStore SSO Sample Script.
Access to a CMC’s Interior Page
After creating the signature, the portal server can return an HTML page with a hyperlink to the CMC SSO secure login API. The following example will display CMC’s Data Explorer page (explorer.htm) embedded in the inline frame on the portal’s page.
<iframe src="https://<cmc_FQDN>:<cmc_port>/Cloudian/ssosecurelogin.htm ?user=USERID&group=GROUPID×tamp=TIMESTAMP&signature=SIG &redirect=explorer.htm"></iframe>
CMC SSO Secure Login HTTP Response
If redirect=RELATIVE_OR_ABSOLUTE_URL is given, the CMC’s SSO secure login API returns an HTTP redirect response.
-
If the request was successful, the redirect response will take the client to the URL specified by redirect.
-
If the request failed, the redirect response will take the client to the CMC’s Login panel.
If redirect=RELATIVE_OR_ABSOLUTE_URL is not given, the CMC’s SSO secure login API returns an HTTP response with content-type "text/plain".
-
If the request was successful, the HTTP response status is 200 OK.
-
If the request failed, a 400 BAD REQUEST status is returned, along with a plain text status description. Possible reasons for failure include:
-
Missing required parameters
-
SSO token already exists (request is ignored)
-
Timestamp in request is outside of configured tolerance range
-
Invalid signature
-
Invalid credentials (group ID and/or user ID is invalid)
-
CMC Logout
This API method allows for immediately invalidating the CMC session.
https://<cmc_FQDN>:<cmc_port>/Cloudian/logout.htm&redirect=RELATIVE_OR_ABSOLUTE_URL
-
redirect: This optional parameter can be used to redirect the client to the URL after logging out from the CMC. It is typically set to a portal page. The URL must be URL-encoded by the client.
CMC Logout HTTP Response
If redirect=RELATIVE_OR_ABSOLUTE_URL is given, the CMC’s logout API returns an HTTP redirect response to take the client to the given URL after logging out from the CMC.
If redirect=RELATIVE_OR_ABSOLUTE_URL is not given, the CMC’s logout API returns an HTTP redirect response to take the client to the CMC’s Login panel.
Logging Out from the CMC and Portal at Once
You may want the logout link on the portal page to also trigger logout from the CMC. You can achieve this by using redrect parameter.
For example, if you have the portal’s logout link like this:
<a href="/auth/logout">Logout</a>
You can change it to the following:
<a href="https://<cmc_FQDN>:<cmc_port>/Cloudian/logout.htm ?redirect=https:%2F%2F<portal_FQDN>:<portal_port>%2Fauth%2Flogout">Logout</a>
-
The redirect URL must be an absolute URL including the protocol (e.g. https://) and portal’s FQDN.
-
The redirect URL must be URL-encoded.
10.2.3. Cloudian HyperStore SSO Sample Script
This section shows a sample script for Cloudian HyperStore SSO integration, using the method described above.
Call the CMC SSO Secure Login API: Python Example
Below is a sample Python script that outputs a HyperStore SSO secure login URL for use with the one-way hash method of having the CMC create a cookie. The script also creates an SSO logout URL.
#!/usr/bin/python import time import hmac import hashlib import base64 import urllib # TODO: Move these config options to configuration file SSO_DOMAIN = 'cmc.cloudian.com' SSO_PORT = 8443 SSO_KEY = 'aa2gh3t7rx6d' # TODO: Dynamically choose user/group based on the user # and group you want to login using. SSO_USER = 'sso@group' SSO_GROUP = 'ssogroup' # Do Not Change SSO_PROTO = 'https://' SSO_PATH = 'Cloudian/ssosecurelogin.htm' SSO_LOGOUT_PATH = 'Cloudian/ssologout.htm' def sso_sig(user, group, timestamp): # query string with no urlencoding for signature signme = 'user=%s&group=%s×tamp=%s' % (user, group, timestamp) hmacsha1 = hmac.new(SSO_KEY, signme, hashlib.sha1).digest() return base64.b64encode(hmacsha1) def sso_url(user, group): timestamp = int(time.time() * 1000) signature = sso_sig(user, group, timestamp) params = {'user': user, 'group': group, 'timestamp': timestamp, 'signature': signature} query = urllib.urlencode(params) url = '%s%s:%d/%s?%s' % (SSO_PROTO, SSO_DOMAIN, SSO_PORT, SSO_PATH, query) return url def sso_logout_url(): url = '%s%s:%d/%s' % (SSO_PROTO, SSO_DOMAIN, SSO_PORT, SSO_LOGOUT_PATH) return url print 'login: ' + sso_url(SSO_USER, SSO_GROUP) print '\nlogout: ' + sso_logout_url()
|
The sample script hard-codes the SSO secret key, which is not advisable for actual practice. In practice, you should keep the secret key safely on the server side. |
Confidentiality Notice
The information contained in this document is confidential to, and is the intellectual property of, Cloudian, Inc. Neither this document nor any information contained herein may be (1) used in any manner other than to support the use of Cloudian software in accordance with a valid license obtained from Cloudian, Inc, or (2) reproduced, disclosed or otherwise provided to others under any circumstances, without the prior written permission of Cloudian, Inc. Without limiting the foregoing, use of any information contained in this document in connection with the development of a product or service that may be competitive with Cloudian software is strictly prohibited. Any permitted reproduction of this document or any portion hereof must be accompanied by this legend.