How ContentGrid stores data

ContentGrid is designed to provide secure, scalable, and reliable data storage for your applications. This page explains the core principles and technologies behind how ContentGrid stores and manages your data.

Overview

ContentGrid separates your data into two main categories:

• Metadata: Information about your content, such as file names, types and the other attributes you define in your data model.
• Content: The actual files, documents, or binary data you upload and manage.

This separation allows ContentGrid to optimize for both performance and security.

Metadata Storage: PostgreSQL

All metadata is stored in a PostgreSQL database. PostgreSQL is a robust, open-source relational database system known for its reliability and advanced features. By using PostgreSQL, ContentGrid ensures:

• Data Integrity: Strong consistency and transactional guarantees.
• Query Performance: Fast and flexible querying of metadata.
• Scalability: Ability to handle large volumes of metadata efficiently.

Content Storage: S3 Compatible Storage with Application-Side Encryption

ContentGrid implements application-side encryption for all content. Every document is encrypted individually using its own unique encryption key, which is securely stored in the PostgreSQL database alongside the metadata. This approach ensures that even if the storage backend is compromised, your data remains protected.

ContentGrid uses an encryption algorithm that supports range requests, allowing efficient access to parts of large files without decrypting the entire file. By default, the algorithm used is AES-CTR (Advanced Encryption Standard in Counter mode).

The actual content (files, documents, etc.) is stored in an S3 (Simple Storage Service) compatible service, a highly durable and scalable object storage service. To protect your data, ContentGrid applies encryption to all content before it is stored in S3.

• Application-Side Encryption: Each document is encrypted with its own key before leaving your application environment.
• Key Management: Encryption keys are stored securely in PostgreSQL, separate from the content itself.
• Encryption Algorithm: AES-CTR is used by default, supporting range requests for efficient file access.
• Durability: Object storage systems (like S3) provide very high durability, minimizing the risk of data loss.
• Scalability: S3 can handle virtually unlimited amounts of data, making it suitable for projects of any size.

Security and Compliance

• End-to-End Encryption: Data is encrypted both in transit and at rest, with application-side encryption ensuring content is protected before it reaches storage.
• Per-Document Keys: Each document uses a unique encryption key, enhancing security and isolation.
• Access Controls: Fine-grained access controls ensure that only authorized users and services can access your data.
• Auditability: All access and changes to data are logged for compliance and auditing purposes.

Summary

• Metadata is stored in PostgreSQL for reliability and performance.
• Content is stored in encrypted form in S3 compatible storage for durability and security.
• ContentGrid is designed to keep your data safe, compliant, and accessible.

Permissions Reference

Permissions Reference

Permissions in ContentGrid operate on the level of entities. By default, all permissions are denied and you cannot read or write any data. In order to grant a permission, you must define a policy. A policy is created for a particular entity and states that certain operations are allowed on that entity under certain conditions. For instance, a policy might say that the read operation is allowed on the invoice entity if the user has the bookkeeping attribute.

Operations

The currently supported operations are:

• Create: The creation of an entity. Policies involving entity attributes are evaluated against the attributes that you send. This corresponds to http POST.
• Read: Covers both reading a specific entity referred by id, as well as listing all visible entities. If you’re getting an entity by id but you don’t meet all the conditions, you will get a “Not Found” error. If you’re listing all entities, the list will be filtered down to the entities for which you meet the conditions. This corresponds to http GET and http HEAD.
• Update: Updating the attributes and relations of an entity. If there are policies specifying conditions on the attributes, you must meet the conditions both before and after the update to the attributes. This corresponds to http PUT and http PATCH.
• Delete: Deleting an entity. This corresponds to http DELETE.

Conditions

A policy has a list of conditions. If this list is empty, the operation in the policy is simply allowed. Otherwise, all the conditions have to match for the policy to allow access.

For instance, you can have two policies for the blog entity. The first policy has the operation READ and no conditions. The second policy has the operations CREATE and UPDATE, but also has a condition stating that the user must have the blogger attribute equal to true. The consequences of these policies are that anyone can read (and list) blog entities, but only people marked as bloggers can write them.

A condition consists of three parts:

• A left value: This is the value that you want to check. It can be an entity attribute, a user attribute, or a constant.
• A comparison operator: This is the symbol that defines the type of comparison you want to make. For example, if you want to check whether the left value is equal to the right value, you would use the equals operator. If you want to check whether the left value is a list that contains the right value, you would use the contains operator.
• A right value: This is the value that you want to compare the left value to. It can be an entity attribute, a user attribute, or a constant.

Types of Values

There are three types of values that can be used in a condition:

• Entity attribute: This refers to an attribute of the entity being accessed, or to an attribute of an entity that has a relation to the entity being accessed. For example, if you have a policy that controls access to a report, you could use the report’s confidentiality level as a value. If you have a policy that controls access to an employee’s information, you can use the employee’s relation to their department to ensure that only people in a particular department are visible.
• User attribute: This refers to a characteristic of the user who is trying to access the resource. For example, if you have a policy that controls access to files private to a particular department, you can verify that the user is a member of the department to which the file belongs.
• Constant: This is a fixed value that you define in the condition. It can be of subtype boolean, number, or string. It is useful as the right value side of a comparison where the left value is an entity attribute or user attribute. For example, if you have a policy that says admins can see everything, you could use a constant value of true as the right value in a condition where the left value is user.is_admin.

Some examples

• Allowing access to users of certain department: user.department equals "Sales"
• Allow access when attribute category has a certain value: entity.category equals "Published"
• Allow access where attribute vat matches the user attribute: entity.vat equals user.vat
• Following the relation belongs_to to party, using the attribute department on party: entity.belongs_to.department equals "Bookkeeping"

Types of comparison operators

These are the comparison operators that you can use in a condition:

• equals (=)
  • Used for any type of value, to check that the left value is equal to the right value
• not equals (≠)
  • Used for any type of value, to check that the left value is different from the right value
• greater than (>)
  • Used for numeric types or date types, to check that the left value is greater (or later in time) than the right
• greater or equals (≥)
  • Used for numeric types or date types, to check that the left value is greater or equal (or later in time) than the right
• less than (<)
  • Used for numeric types or date types, to check that the left value is smaller (or earlier in time) than the right
• less than or equals (≤)
  • Used for numeric types or date types, to check that the left value is smaller or equal (or earlier in time) than the right
• contains (∋)
  • Used with a collection as the type of the left value, to check that the right value appears in the collection
• in (∈)
  • Used with a collection as the type of the right value, to check that the left value appears in the collection