All StorageFabric configuration data is encrypted with the Master Encryption Key and stored in the Configuration Bucket in cloud.
For increased availability, in the event of failures at the backend storage provider, a Backup Bucket can also be configured for StorageFabric configuration data. In the event of the Configuration Bucket being inaccessible, StorageFabric components automatically access the configuration data from the Backup Bucket.
A Cloud Bucket is a regular S3 bucket, hosted either with an on-prem or a cloud storage provider. Clients’ data is stored in Cloud Buckets. However, since StorageFabric virtualizes you entire storage namespace, StorageFabric clients are not exposed to provider-side buckets directly. Clients instead use Virtual Bucket Names. Each Virtual Bucket is backed by a Cloud Bucket.
The Configuration Bucket is a specially designated bucket to store StorageFabric configuration data. All StorageFabric configuration data is encrypted with the Master Encryption Key and stored in the Configuration Bucket in cloud. Therefore, storage providers cannot access the data.
Storing configuration data with providers makes StorageFabric components stateless giving extreme flexibility to deployments. Components can be added and removed easily without affecting overall availability.
Instead of complex on-prem data management using relation databases, or other heavy-weight storage solutions, with StorageFabric, all you need to do is manage the Master Encryption Key. Entire deployments can be bootstrapped and restored using the Master Encryption Key.
A Data Vault represents an Amazon Glacier Vault. Using a StorageFabric Gateway, clients can directly access Glacier Vaults using standard S3 Buckets API.
Master Encryption Key¶
All StorageFabric configuration data is encrypted with the the Master Encryption Key.
Internally, StorageFabric uses hierarchical keying wherein lower layer keys are encrypted using higher layer keys. The Master Encryption Key is the root key in the hierarchy.
The Master Encryption Key must be kept safe. Loosing the Master Encryption Key is equivalent to loosing your data. The Master Encryption Key is never communicated to the outside by StorageFabric components.
StorageFabric Configuration Manager¶
StorageFabric Configuration Manager is a set of tools, web interface, and REST API to configure StorageFabric Gateways. Configuration data includes information about cloud providers’, buckets, encryption keys, access control, and client and cloud access keys.
Configuration Manager facilitates setup and management of information that enables StorageFabric Gateway services. For example, Configuration Manager is used to setup encryption keys that are then used by StorageFabric Gateways for transparent encryption and decryption of applications’ data.
The StorageFabric Gateway is the single-point-of-access to cloud storage services, such as Amazon S3, Google cloud storage, etc. The Gateway exposes a standard object storage interface to client applications.
Some of the Gateway functionality includes (but is not limited to) the following:
Transparent data encryption and decryption.
Enforcing access control.
Logging and audit trails.
Enforcing data governance.
The StorageFabric LogViewer is a web-based graphical interface to access StorageFabric Gateway statistics and logs.
StorageFabric Scheduler is a StorageFabric component that manages internal house keeping tasks for StorageFabric components. The scheduler runs on the StorageFabric Configuration Manager and on Gateways.
StorageFabric virtualizes you entire storage namespace across all your storage providers. No matter what actual bucket names are with storage providers, StorageFabric clients see a virtual namespace. Clients upload and download data to and from Virtual Buckets. A Virtual Bucket is identified by its Virtual Bucket Name.
View Encryption Key¶
An encryption key that encrypts configuration data for a Data View. StorageFabric Gateways have access to the View Encryption Keys only. Each Gateway is setup using the View Encryption Key of the Data View that the Gateway is part of.
Virtual Bucket Name¶
Name of a Virtual Bucket. This name can be different from actual bucket name with a cloud provider. Hence it is referred to as a Virtual Bucket Name. StorageFabric clients use the virtual bucket name in API requests to a StorageFabric Gateway.
In StorageFabric, there is a single global bucket namespace. That is, each Virtual Bucket has a globally unique virtual name.
StorageFabric Gateway Domain¶
The base domain where clients can send requests to the StorageFabric Gateway. StorageFabric Gateway supports both Virtual Host Style Addressing URLs and Path Style Addressing URLs. For example, if the StorageFabric Gateway Domain is s3.abccorp.com and the bucket name is amazon-data-bucket, then clients can send requests to the following URLS:
A StorageFabric Gateway Domain is necessary for:
Remote clients to access the StorageFabric Gateway.
SSL connection between clients and the StorageFabric Gateway.
A StorageFabric license governs the use of StorageFabric. A valid license is required to use StorageFabric. In the absence of a valid license, StorageFabric may operate in restricted modes.
To learn more about license management, refer to the full product documentation.
By default this is the path /etc/storagefabric/licenses/.
Access Key ID¶
Authentication to most cloud providers is done using an Access Key ID and a Secret Access Key. An Access Key ID acts like a user-name, uniquely identifying the requester.
Secret Access Key¶
Authentication to most cloud providers is done using an Access Key ID and a Secret Access Key. A Secret Access Key acts like a password corresponding to an Access Key ID.
An additional piece of information used by cloud providers for client authentication in addition to the Access Key ID and Secret Access Key. Typically used when the credentials are temporary, that is, with a brief expiration time.
A client is any entity that sends requests to the StorageFabric Gateway. Clients are authenticated by the StorageFabric Gateway using Client Access Keys.
In StorageFabric, user specifically refers to an entity performing configuration management via the StorageFabric Configuration Manager web or command line interfaces.
When using the web interface, users authenticate using their username and password or via single sign-on.
When using the configuration manager REST API, users authenticate using their Client Access Keys.
Cloud Access Keys¶
Client Access Keys¶
StorageFabric components are stateless by default. In the event that a component fails or the component setup or initialization information is lost, all that is needed to setup an identical component are credentials to access the Configuration Bucket and the Master Encryption Key. To make recovery convenient, a command is displayed by the the full product documentation at completion. Using just this command, and the Master Encryption Key, a new identical component can be set at a later time, such as for recovery.
Bucket Encryption Key¶
The encryption key used to encrypt each File Encryption Key for all files in a Virtual Bucket. Each Virtual Bucket can have multiple encryption keys.
File Encryption Key¶
The encryption key used to encrypt file data.
There are three provider settings for multipart mode as described next.
Disabled Multipart is not supported by provider.
Emulated Multipart is not supported by provider. However, StorageFabric can emulate multipart uploads and downloads. For example, with Google.
Native Multipart is natively supported by provider. For example, Amazon S3, DreamObjects.
Tail Range Mode¶
Indicates whether a storage provider supports HTTP Range request of the type Last X bytes.
Indicates whether a storage provider supports “encoding-type=url” argument.
indicates that Unsigned Payload for V4 instances is supported by the provider.
Signed URLs can be used to share objects between clients using simple url exchanges. A signed URL cannot be used to access an object that the signer does not have access to.
Role-Based Access Control¶
A permission specifies whether a particular operation is allowed or denied.
A context is a JSON document that specifies a set of resources. Under Role-Based Access Control, contexts are used to specify the resources for which permissions are to be granted.
Financial Archival Mode¶
Sheltered Harbor is a standard developed by the financial industry to protect consumer data, in the event that a financial institution becomes inoperable.
Under Sheltered Harbor, consumers’ account data is archived and stored in a standard format and can be recovered in the event of outage.
Archived data is regularly monitored to ensure compliance with the specifications.
For more information, visit shelteredharbor.org.
An assertion is a signed document generated by an Identity Provider, that contains one or more claims.
In StorageFabric, an identity provider refers to an ADFS instance. Identity provider manages user credentials, and issues signed assertions containing one or more claims.
Service provider relies on signed assertions from an Identity provider to login users and enforce access control. For our purpose, service provider is the StorageFabric Configuration Manager.
Security Assertion Markup Language (SAML) is an XML-based standard for exchanging assertions (and claims) between an Identity Provider and a Service Provider. In StorageFabric, assertions generated by an identity provider, must be in SAML format.
Relying Party Trust¶
In ADFS, a relying party is a trusted entity that will be receiving and processing assertions. For our purpose, relying party is the StorageFabric Configuration Manager. The trust established between relying party and an Identity Provider is referred to as relying party trust.
Microsoft Active Directory is a set of components to manage information about network resources such as users, systems, services, etc. For more information, see msdn Active Directory documentation
Active Directory Federation Services¶
Microsoft Active Directory Federation Services (ADFS) is a service that allows secure sharing of identity information within networks, for example, single sign on. For more information, see msdn ADFS documentation
AWS KMS ID¶
AWS key ID. See AWS docs for more information.
In StorageFabric, the full ARN should not be used, but only the UUID. For example, if the key ARN is arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab, then use only 1234abcd-12ab-34cd-56ef-1234567890ab as the AWS KMS ID.
Passthrough (Plaintext) Mode¶
If passthrough mode is enabled on a Virtual Bucket, StorageFabric Gateways will not encrypt data stored in the Virtual Bucket.
This mode is useful for example, to add Virtual Buckets that were already in use prior to the StorageFabric deployment. Moreover, data stored in passthrough Virtual Buckets can directly be downloaded from the storage provider without using StorageFabric.
Passthrough mode is recommended only for Virtual Buckets storing non-sensitive data.
StorageFabric File Browser¶
The StorageFabric File Browser is a StorageFabric component that provides a browser-based interface for users to access their data.
In Inline DNS resolution mode, Gateways resolves DNS (for providers) the time a client request is received. Gateways then cache the DNS entry for the duration of the server response TTL (time-to-live).
In Background DNS resolution mode, Gateways continue to resolve the DNS (for providers) in the background every TTL seconds. If no client requests are received for 60 seconds, background resolution is stopped until a new client request arrives for that provider.
Background mode offers better performance over DNS Inline when cloud provider has a low TTL (such as Amazon which has less than 5 seconds), because Gateways will continue to resolve DNS in the background. As a result, subsequent client requests don’t need to wait for DNS resolution after the TTL would have expired.
Path Style Addressing¶
Path-style (or V1) includes the bucket name in the path of the URI. For example, s3://s3.storageprovider.com/<bucketname>/key.
Virtual Host Style Addressing¶
Virtual-hosted style (or V2) uses the bucket name as part of the domain name. For example, s3://<bucketname>.s3.storageprovider.com/key.
Available Release 3.4.0 onwards.
The NULL bucket is a special Virtual Bucket similar to /dev/null. It is used to test the network conditions between clients and Gateways. The NULL bucket behavaes similar to a Virtual Bucket with the exception that the NULL bucket has no backend storage provider and does not consume storage.