How ContentGrid stores data

ContentGrid is designed to provide secure, scalable, and reliable data storage for your applications. This page explains the core principles and technologies behind how ContentGrid stores and manages your data.

Overview

ContentGrid separates your data into two main categories:

  • Metadata: Information about your content, such as file names, types and the other attributes you define in your data model.
  • Content: The actual files, documents, or binary data you upload and manage.

This separation allows ContentGrid to optimize for both performance and security.

Metadata Storage: PostgreSQL

All metadata is stored in a PostgreSQL database. PostgreSQL is a robust, open-source relational database system known for its reliability and advanced features. By using PostgreSQL, ContentGrid ensures:

  • Data Integrity: Strong consistency and transactional guarantees.
  • Query Performance: Fast and flexible querying of metadata.
  • Scalability: Ability to handle large volumes of metadata efficiently.

Content Storage: S3 Compatible Storage with Application-Side Encryption

ContentGrid implements application-side encryption for all content. Every document is encrypted individually using its own unique encryption key, which is securely stored in the PostgreSQL database alongside the metadata. This approach ensures that even if the storage backend is compromised, your data remains protected.

ContentGrid uses an encryption algorithm that supports range requests, allowing efficient access to parts of large files without decrypting the entire file. By default, the algorithm used is AES-CTR (Advanced Encryption Standard in Counter mode).

The actual content (files, documents, etc.) is stored in an S3 (Simple Storage Service) compatible service, a highly durable and scalable object storage service. To protect your data, ContentGrid applies encryption to all content before it is stored in S3.

  • Application-Side Encryption: Each document is encrypted with its own key before leaving your application environment.
  • Key Management: Encryption keys are stored securely in PostgreSQL, separate from the content itself.
  • Encryption Algorithm: AES-CTR is used by default, supporting range requests for efficient file access.
  • Durability: Object storage systems (like S3) provide very high durability, minimizing the risk of data loss.
  • Scalability: S3 can handle virtually unlimited amounts of data, making it suitable for projects of any size.

Security and Compliance

  • End-to-End Encryption: Data is encrypted both in transit and at rest, with application-side encryption ensuring content is protected before it reaches storage.
  • Per-Document Keys: Each document uses a unique encryption key, enhancing security and isolation.
  • Access Controls: Fine-grained access controls ensure that only authorized users and services can access your data.
  • Auditability: All access and changes to data are logged for compliance and auditing purposes.

Summary

  • Metadata is stored in PostgreSQL for reliability and performance.
  • Content is stored in encrypted form in S3 compatible storage for durability and security.
  • ContentGrid is designed to keep your data safe, compliant, and accessible.