ContentGrid Documentation

Introduction

ContentGrid is a modern highly-scalable Content Services Platform. ContentGrid allows you to design, deploy, manage and iterate on content-centric applications, enabling users to retrieve and work with content in a modern, seamless and secure way, across devices and organizational boundaries.

This documentation describes ContentGrid Console concepts, the problems it solves, and contains quick-start tutorials for using ContentGrid.

Subsections of ContentGrid Documentation

Concepts

Console

The ContentGrid Console is a management interface for the ContentGrid platform. It provides a web-based user interface to manage your ContentGrid Projects, Applications and more.

The Console is available at https://console.contentgrid.com

Organizations

An Organization is a top-level concept. Every operation within ContentGrid, happens in the context of an Organization.

ContentGrid Organizations are used to:

Projects

A Project belongs to an Organization and contains:

Your Users will use the deployed Applications.

Members

Members are project administrators for an Organization. They can create new Projects, collaborate on Blueprints, manage Applications and IAM Realms

Note that members are not Users. A User does not need to be a member of your Organization to access an Application.

At this time, members cannot be assigned different roles or permissions. Every member is effectively an organization and project administrator.

Blueprint

A blueprint is the design area of a Project. It describes the full model of an application, including the Data Model, Permissions and any Automations. This model determines how the data is stored in the database, how the REST API is structured and how automations are integrated.

A Release can be created from a Blueprint.

Creating a new version of an active application means making changes in the Blueprint and creating a new Release before deploying this release to a running Application.

A Project has a Blueprint called main.

A Blueprint does not contain any application- or user-data.

TIP: If you are wondering where application- & user-data is stored, see: storing-data.

Release

A Release is an immutable snapshot of a Blueprint and is the artifact that can be deployed to an Application.

A Release is created from the current Blueprint. It has a release-label, like v1.2.3 and optionally a short description. A Release follows a chronological previous Release and belongs to a single Project.

Data Model

The data model is a logical abstraction of concepts from the real world. The data model contains Entities with Attributes and Relations between Entities.

Entities

An Entity represents an object or concept in the digital world. It can represent a physical object, like a product or an invoice-document, or a virtual concept, like an online order.

Attributes

An attribute is a single characteristic of an Entity. An attribute has a data type, such as text, number or date. A file is stored as a special attribute type called content. For example, the attributes for an invoice could be number, amount and document.

Relations

A relation is a named link between two Entities in the Data Model.

Entities can have a one-to-one, one-to-many, many-to-many or many-to-one relationship with another entity. For example, an invoice has a many-to-one relationship with the customer entity: many invoices can have a reference to the same customer.

A relation can be optional or required.

Permissions

Permissions govern under what conditions Entities are visible for a principal (a User or Service Account). They are made up of a collection of Policies per Entity.

ContentGrid uses an Attribute Based Access Control (ABAC) system. This means that a set of rules define access, using attributes of the principal issuing a request and attributes of the Entities being acted upon.

Permissions are deny-by-default.

Policies

Policies are a set of rules, that specify a the conditions under which Operations are allowed on an Entity. A policy always applies to a single Entity.

Under normal circumstances, every Entity gets a Policy. In the absence of any Policies for an Entity, all Operations on this Entity are disallowed for everyone.

One Entity can have multiple Policies, that are inclusively disjunctive. In other words, the policies are combined with “OR” logic: a request is allowed if the conditions of Policy A OR Policy B are satisfied.

Operations

Operations are actions that a principal (a User or Service Account) can perform on an Entity.

The available actions are Read, Create, Update, Delete. A policy can cover one or more operations.

Policy Conditions

Policy Conditions specify additional requirements that must be met, before an operation is allowed. They are like extra security checks that ensure that only the right people can access your Entities or perform actions on them. For a policy to allow something, all conditions in that policy must pass.

A policy without conditions allows the selected operations for all logged-in users.

Automations

Webhooks

Webhooks provide a way to configure a ContentGrid Application to send notifications to external systems whenever certain changes occur.

When a mutating operation is performed (for example, a user created a document), an HTTP request is sent to the configured endpoint, with a payload describing the change. Every operation is handled individually and asynchronously. Delivery of webhook events are usually in chronological order, but in-order delivery is not guaranteed.

A webhook receiver needs Service Account credentials, if it wants to call the REST API of the Application.

Applications

Applications are running instances of a Project. An Application is created in a Zone. After creating the Application, a Releases can be Deployed to an Application.

A Project can have multiple active Applications, possibly representing different environments or to try out different Releases.

Each Application is linked to one IAM Realm. The first time an Application is created in a Zone, a default IAM Realm is created for the Organization in this Zone.

Zones

A Zone is the deployment target of an Application and maps to the geographical location of a cloud provider that runs ContentGrid Applications.

An Organization creates an Application in a given Zone.

Currently there is one zone available: Scaleway Paris

Deployments

A Deployment is the rollout of a specific Release to an Application.

Once a Release is deployed, the Application provides a REST API to store, retrieve and manipulate data.

IAM

IAM (Identity and Access Management) is the system that manages User-accounts for ContentGrid Applications.

Realms

A Realm or IAM Realm contains a collection of Users, Groups and their attributes. Applications are linked to an IAM Realm to authenticate Users and Service Accounts.

Multiple Applications, across Projects but within the same Organization, can be connected to the same IAM Realm.

When an Application is created in a Zone for the first time, an IAM Realm gets automatically provisioned.

Users

IAM Users are human principals that interact with Applications.

Users are identified by their email address. Users can be a assigned to one or more Groups. Groups can have attributes and are recommended way to organization Permissions.

NOTE: an Organization Member is not automatically a User of an Application.

Groups

IAM Groups allow you to organize Users based on shared characteristics.

Groups have a collection of user memberships and a set of Attributes. For the purpose of permission evaluation, users inherit attributes from the groups they are a member of. For example, you might create a group for all users in the accounting department, so you set the group’s department-name attribute to “accounting”, so that you can create a permission policy for invoices that checks what department a user belongs to.

These attributes are defined in the IAM Attribute Model.

Attribute Model

The IAM Attributes describe a set of properties for Groups. The purpose of this model is to describe the information that can be used in permissions. Users inherit properties from the groups they are member of.

It is similar to defining Attributes for an Entity in the Datamodel, but is not part of a Blueprint.

An attribute has a name, a data type and can be single- or multi-valued. For instance, if you want to be able to write permission policies that allow access to admins, you should define an is-admin attribute of type boolean.

Service Accounts

A Service Account is a non-human principal in an IAM Realm. A Service Account can acquire an access token, to interact with the REST API of an Application.

Service Accounts are useful for programmatic integrations with the REST API. For example, a Webhook receiver may call back into the REST API to update a status field, but it needs to be authenticated to do so.

At this moment, it is not possible to create Service Accounts in the ContentGrid Console.

Guides

ContentGrid has 2 main web interfaces. One of them for managing applications, is called the Console. The web UI for using ContentGrid applications is called Navigator.

Console

The Console enables you to create projects, model blueprints and deploy applications.

Navigator is the out-of-the-box end user interface for every ContentGrid applications and allows you to create, search, consult and delete entities.

Subsections of Guides

Subsections of Console

Getting Started

This guide will walk you through the initial steps of setting up your organization, creating a project, and starting with data modeling in ContentGrid.

1. Create an Organization

Before you can create a data model, you need to have an organization.

  1. On the main page, click the Create Organization button.
  2. Follow the prompts to set up your organization.

If you already have an organization, you can skip this step.

2. Create a Project

Once your organization is set up:

  1. Navigate to your organization page.
  2. Click the Create Project button.

Screenshot showing the Create Project button on the organization page Screenshot showing the Create Project button on the organization page

  1. In the dialog, enter a name for your project and confirm.

Screenshot of the Create Project dialog where you enter the project name Screenshot of the Create Project dialog where you enter the project name

3. Start Data Modeling

Now you can begin modeling your data:

  1. Click the Add Entity button to start creating your first entity.

Screenshot showing the Add Entity button in the Data Model menu Screenshot showing the Add Entity button in the Data Model menu

  1. In the Data model tool, you can model your business entities. Click Add Entity, provide a name, and optionally a description. Providing a useful description is a good practice and can help some of the AI features in ContentGrid work better.

Screenshot of the Add Entity dialog Screenshot of the Add Entity dialog

  1. After creating your first entity, you can edit its name or delete it if needed.

Entities, Attributes, and Relations

Entities can have attributes and relationships to other entities.

Add an Attribute

  1. Within an entity, click Add Attribute.
  2. A dialog will appear with the following options:

Screenshot of the Add Attribute dialog Screenshot of the Add Attribute dialog

  • Property Name: A unique name within the entity.
  • Type: The data type for this attribute. Options include:
    • Text
    • Integer
    • Decimal
    • Boolean
    • Date
    • Content (binary object or file)
    • Audit Metadata (automatically managed subfields: createdBy, createdAt, modifiedBy, modifiedAt)
  1. After creating the attribute, you can edit it further:

Screenshot of the Edit Attribute page Screenshot of the Edit Attribute page

  • Name: Rename the attribute.
  • Description: Add a description to help users and AI agents.
  • Search Options:
    • Search values exactly: Values must match exactly when searching (useful for status, category, etc.).
    • Search values starting with given prefix: Values can be searched by prefix (case- and accent-insensitive; useful for names, titles, etc.).
  • Constraints:
    • Value is required: Attribute must have a value for the item to be created.
    • Must be unique: Attribute value must be unique across all items.
    • Set allowed values: Provide a list of allowed values; only these can be used.

Add a Relation

You can model relationships to other entities (or to the same entity):

  1. Click Add Relation.
  2. In the pop-up dialog, configure the following:

Screenshot of the Add Relation dialog Screenshot of the Add Relation dialog

  • From: The current entity.
  • To: The target entity for the relationship.
  • Name: Name of the relationship.
  • Description: (Optional) Description for the relationship.
  • Cardinality: Cardinality of the relationship (e.g., one-to-one, one-to-many). Choose the checkboxes that apply.
  • Relationship is required: Check if this relationship must always exist.

Permission Policies

Permission Policies

ContentGrid takes a different approach to permissions than legacy content management systems. Instead of making a complex hierarchical tree structure of permissions with inheritance, a set of permission policies describes access for each entity. These policies are rules that contain logical expressions, making use of (a combination of) entity attributes and user attributes.

How to create a Policy

First, go to the Permissions modeler:

Permissions Permissions

To create our first policy, we first have to choose the entity for which we are going to create a policy:

Select Entity Select Entity

When clicking the “Create Policy” button you will see the configuration options for creating a new policy:

Create Policy Create Policy

First, you’ll have to choose for which operation this policy will be evaluated. The options are: Read, Create, Update, Delete. You can choose one or more operations.

By choosing the visibility setting, you can define if this policy is applicable for authenticated users only, or for all users.

The “Additional conditions” section is where you define the conditions for this policy, access to the entity is granted when the conditions are fulfilled.

Multiple conditions can be applied, and each rule has a left and a right side, that are compared to each other. Both left and right sides of can be a “user attribute”, “entity attribute” or constant. The possible comparisons between the left and the right side are:

  • equals
  • not equals
  • greater than
  • greater or equals
  • less than
  • less than or equals
  • contains
  • in

You can add more conditions with the “Add Condition” button. All conditions have to be satisfied before a policy grants access. Save the policy with the “Add Policy” button.

Now, you should see your policy for this entity in the overview.

Policy Overview Policy Overview

Webhooks

Tutorial on webhooks

Webhooks are one way that ContentGrid applications can send automated messages or information to external systems. They are almost always faster than polling, and require less work on your end.

Each mutating operation on an resource is handled individually and is asynchronously, but almost immediately delivered to the configured webhook endpoint.

A ContentGrid Webhook

  • is always attached to a ContentGrid entity and selected change triggers
  • is delivered to a configured HTTP endpoint
  • is delivered as a HTTP POST message with a body payload and specific ContentGrid HTTP headers

Create New Webhook?

First, go to the Webhooks modeler:

Webhooks Webhooks

To create our first Webhook, we first have to click on Create Webhook:

Create Webhook Create Webhook

When clicking the “Create Webhook” button you will see the configuration options for creating a new Webhook:

Create Webhook Options Create Webhook Options

  • First, you will have to give a satisfying Webhook name (or description).
  • Then you have to choose for which entity this Webhook will be applied
  • After the entity selection, select the notifications triggers, where the options are
    • create (whenever the selected entity type is created)
    • update (when each existing entity is being updated)
    • content (when a document content has been changed)
    • delete (when the enetity selected has been deleted)
  • As a last step, you have to provide the Webhook URL to be invoked by your ContentGrid application

Webhooks sample Webhooks sample

After clicking save, the Webhook is saved and is listed in the overview but will only be available to your ContentGrid runtime application once you create a new release.

Webhooks listing Webhooks listing

You can add more Webhooks with the “Create Webhook” button.

Webhook endpoint

A Webhook endpoint has some specific constraints and must respect the following implementation details:

  • it has to support the HTTP POST method
  • should accept a JSON body and application/json Content-Type
  • can use our ContentGrid HTTP headers in order to validate that the delivered message is coming from our platform
    • User-Agent value is ContentGrid-Slingshot/APP_VERSION
    • ContentGrid-Application-Id contains the application id
    • ContentGrid-Deployment-Id contains the deployment id
    • ContentGrid-Signature provides a JWT signed by the our platform that should be used in order that the message is actually sent by the ContentGrid platform

Security and validation

ContentGrid signs JWTs using asymmetric encryption (RS256), and publishes the public signing keys in a JWKS (JSON Web Key Set).

The signing keys are rotated on a regular basis.

  • We discourage doing manual JWT validation since it might be easy to improperly implement and miss some important details that will lead to serious security vulnerabilities. Most JWT libraries take care of JWT validation for you.
  • We also highly recommend to use a JWK library for the programming language of your choice.

One of the benefits of JSON Web Token (JWT) is that you can validate a token using an easy cryptographic operation.

What are JWKs?

A JSON Web Key (JWK) is a JSON data structure that represents a cryptographic key. JWKs are a set of keys shared between different services and are used to verify the JWT token from the authorization server.

You should only be validating the received JWT against ContentGrid Json Web Key (JWK) URL which is ${CONTENTGRID_URL}/.well-known/jwks.json

We use JWKS to expose the public keys used by the ContentGrid platform to all the clients required to validate signatures.

For more information you can check the JWK RFC

The example of a JWKS is something that looks like this:

{
  "keys": [
    {
      "use": "sig",
      "kty": "RSA",
      "kid": "UVelusmvyM2xScEu0F_xSNlhelC5jZTD77R_3mmOZXs",
      "alg": "RS256",
      "n": "...yjXzcFpbMJB1fIFam9lQBeXWbTqzJwbuFbspHMsRowa8FaPw44l2C9Q42J3AdQD8CcN...",
      "e": "AQAB"
    },
    {
        ...
    }
  ]
}

In the above example some important fields are

  • e: is the exponent for a standard pem
  • n: is the moduluos for a standard pem
  • alg: the signing algorithm.
  • kid: a unique id for every key in the set.

ContentGrid’s JWT headers and claims:

{
  "kid": "UVelusmvyM2xScEu0F_xSNlhelC5jZTD77R_3mmOZXs",
  "alg": "RS256"
}.{
  "aud": "https://webhooks-demo.rtp-scw-sandbox.contentgrid.cloud/broker-process",
  "exp": 1679044765,
  "iat": 1679044465,
  "jti": "df475d5a-fc0e-4fca-a03a-c279e86fe9ed"
}.[Signature]
  • kid: is Identifier of the static key used to sign the JWT
  • alg: Algorithm used to sign the key
  • aud: Recipients that the JWT is intended for (the CG Application URL)
  • iat: The issuing time of the token in seconds
  • exp: The expiration time of the token in seconds
  • jti: Unique identifier of the token

Get the signing keys

Here is a Java exmaple of how to validate a JWT using JWKs with Nimbus JOSE + JWT

  • To validate the token, first, you need to get the JSON web key set from the JWKs endpoint. The token will be received as JSON in the validation endpoint in the body.
ResourceRetriever jwkSetRetriever = new DefaultResourceRetriever();
JWKSource<SecurityContext> jwkSource = new RemoteJWKSet<>(URI.create("${CONTENTGRID_URL}/.well-known/jwks.json").toURL(), jwkSetRetriever);
JWSKeySelector<SecurityContext> jwsKeySelector = JWSAlgorithmFamilyJWSKeySelector.fromJWKSource(jwkSource);
  • Additional validation for token claims
ConfigurableJWTProcessor<SecurityContext> jwtProcessor = new DefaultJWTProcessor<>();
jwtProcessor.setJWSKeySelector(jwsKeySelector);

jwtProcessor.setJWTClaimsSetVerifier((claims, context) -> {
	final Date now = new Date();			
	final Date exp = claims.getExpirationTime();
	if (exp != null) {
		if (now.after(exp)) {
			throw new BadJWTException("expired");
		}
	}
});	
  • and the JWT validation
SignedJWT signedJWT = SignedJWT.parse(THE_JWT);	
jwtProcessor.process(signedJWT, null);

This short guide provides the basic steps required to locally verify an access or ID token signed by ContentGrid. It uses packages from Nimbus JOSE + JWT for key parsing and token validation, but the general principles should apply to any JWT validation library.

Navigator

ContentGrid Navigator is the out-of-the box frontend for you ContentGrid application. The interface will adapt automatically to the entities and attributes you have configured.

Overview

An overview of the different UI elements of Navigator.

Create entity

Guide on how to create new entities with metadata and relations.

Subsections of Navigator

Navigator overview

Navigator overview Navigator overview

(1) Entity menu

The entity menu allows you to select an entity type for search and creation. The selected entity is highlighted.

(2) Search form

In the search form you can search on the searchable metadata of the entity and it’s related entities, as defined in the Console.

(3) Results table

The results table shows up after search and displays the search results as a table with metadata. Clicking one of the lines in the table will open the document preview.

(4) Create entity button

Clicking the create entity button will bring you to create form where you can fill in the metadata, files and link the relations for this entity.

(5) Document preview

The document preview will display a preview of the line selected in the results table. If there are multiple files attached to that entity, there will be a dropdown selector to choose what file you want to preview.

Info

The previewer can natively show PDF’s and images. For other documents, ContentGrid will attempt to transform the file to PDF. Currently, these formats are supported for previewing:

  • pdf
  • jpg, png
  • doc, docx, ppt, pptx, xls, xlsx
  • odt, ods, odp

Of course, other file formats can still be used in ContentGrid, but they don’t have out-of-the-box previews.

Creating entities

Create entity Create entity

(1) Create form

In the create form you can fill in the metadata fields of the entity you are creating. Mandatory fields are indicated with an asterisk. Datetime fields have a datetime picker. Content fields support drag and drop and opening a file picker.

(2) Relation field

If your entity has a relation to another relation, you will see that in the form. The link data button will bring you to the link relation popover where you can link the correct entities.

(3) Extract button

The extract button will open a pop-up to start an automated metadata extraction using AI based on the files you already uploaded in the form.

(4) Create/cancel buttons

These buttons allow you to submit or cancel the entity creation.

(5) Document preview

The document preview allows you to see the documents added next to the form. This can be practical when you need to read data from the document to fill in the form. The extract feature will also overlay information in the preview when used.

After clicking the link data button in the create form a popover will appear that will help you link related entities.

Add relation Add relation

(1) Search form

The search form in this popover functions similarly to the entity search form and searches on the linked entity.

(2) Result table

After pressing the “SEARCH” or “SHOW ALL” button, the results table shows the candidates for linking. Just click a line to link the entity. It is possible to link multiple entities if it is a “TO MANY” relationship.

(3) Create button

This button will open a new tab on the create form of the entity type you are trying to link. Use this when the entity you want to link does not yet exist.

Application REST API

ContentGrid Applications expose a REST API that can be used to programmatically access the metadata and content stored in your ContententGrid application.

The API documentation gives examples using curl, but various tools or libraries can be used for making calls to the API.

Authentication

Authenticate to the ContentGrid Application using OpenID Connect

API Usage

Basic usage of the REST API

HAL Usage

Advanced usage of the REST API using HAL and HAL-FORMS as hypertext formats

Error handling

Handling of REST API errors using Problem Details

Subsections of Application REST API

Authentication

An access token is required to access a ContentGrid Application.

Access tokens are always obtained through OpenID Connect (OIDC). The authentication token contains user identity information and the IAM attributes assigned to the user.

OIDC clients

To identify the application that performs the authentication, an OIDC client needs to be registered in the Organization’s IAM Realm. An Organization can have multiple IAM Realms; ensure you are using the same Realm as the one used by the Application.

An OIDC client can be configured in the “Clients” tab of the IAM Realm.

Setting up an OIDC connection requires 3 configuration parameters that can be obtained from the OIDC client page:

  1. Issuer URI
  2. Client ID
  3. Client Secret

OIDC configuration can be discovered from the Issuer URI using OIDC Discovery. If OIDC discovery is not supported by your software, you can obtain the authorization and token endpoints manually from the discovery document, but this is not recommended.

Service account authentication

When using service account authentication, the access token is issued with its own privileges instead of with the privileges of a user.

To authenticate as a service account, you need to use the Client credentials grant.

By default, no IAM attributes are assigned to a service account. These need to be set up if the applicable permission policies require it to provide access.

curl https://auth.$REGION.contentgrid.cloud/realms/$REALM/protocol/openid-connect/token    \
  -u $CLIENT_ID:$CLIENT_SECRET       \
  -d grant_type=client_credentials

User authentication

When using user authentication, the access token is issued with the privileges of the user.

To use user authentication, you need to use the Authorization code grant. This flow will necessary include a pass through the user’s web browser as part of the flow to obtain an access token.

Supplying credentials to the application

The access token is supplied to the application by using bearer authorization.

Other methods of supplying access tokens (like an access_token query parameter or form-encoded body parameter) are not supported.

curl -i https://$APP_ID.$REGION.contentgrid.cloud/   \
    -H "Authorization: Bearer $TOKEN"
GET / HTTP/1.1
Authorization: Bearer $TOKEN

API Usage

ContentGrid applications expose a REST API using JSON as a response format. All JSON responses use HAL as a hypertext format, linking to related resources. Actions that can be taken on resources are specified using the HAL-FORMS format.

Before using the API, you need to authenticate and obtain an access token.

This documentation covers the basics of using the Application REST API, advanced usage with HAL is described in HAL Usage.

Because the specific API is generated from the configured data model, the examples will use an example datamodel.

OpenAPI spec

An OpenAPI spec for your application’s datamodel is available from the ContentGrid Console. Every Release has its own associated OpenAPI spec, which you can view or download.

For deployed applications, a copy of the OpenAPI spec is also available on /openapi.yaml (after authentication).

Resource types

A ContentGrid application can be configured with kinds of entities that have attributes and relations.

A single instance of such an entity is generically referred to as an entity-item. Multiple entity-items of the same kind together form an entity-collection.

These two form the main resource types in a ContentGrid application. To support these resource types, a couple of additional resource types exist. Their relation to the main resource types is explained in the Resource Types reference.

Basic operations

This section explains all the basic operations (create, read, update, delete) for the resource types.

The operations are explained in their minimal form, without any request modifiers applied.

Tip

You can follow along with the examples by creating an application with the Example datamodel set up. Define the following variables to make the examples easily runnable.

REGION="<Region where the application is deployed>"
APP_ID="<ID of the application>"
REALM="<Internal reference of your IAM Realm>"
CLIENT_ID="<IAM Service account Client ID>"
CLIENT_SECRET="<IAM Service account Client Secret>"

You will need to obtain an access token. This can be done easily with an IAM service account. The Authentication documentation contains more details on this.

TOKEN="$(curl -Ss https://auth.$REGION.contentgrid.cloud/realms/$REALM/protocol/openid-connect/token    \
 -u $CLIENT_ID:$CLIENT_SECRET       \
 -d grant_type=client_credentials | jq .access_token -r)"

An access token is valid for 5 minutes; you can obtain a new token by re-running the command above.

Next, set up some entities that will be used by the examples

INVOICE_ID="$(curl -Ss https://$APP_ID.$REGION.contentgrid.cloud/invoices   \
    -F "total_amount=15.95"   \
    -F "received=2024-07-15"   \
    -F "pay_before=2024-08-14"   \
    -F "document=dummy-invoice;filename=invoice.txt;type=text/plain"   \
    -H "Authorization: Bearer $TOKEN"   \
    | jq .id -r
)"
INVOICE2_ID="$(curl -Ss https://$APP_ID.$REGION.contentgrid.cloud/invoices   \
    -F "total_amount=123.4"   \
    -F "received=2020-01-01"   \
    -F "pay_before=2020-02-01"   \
    -H "Authorization: Bearer $TOKEN"   \
    | jq .id -r
)"
SUPPLIER_ID="$(curl -Ss https://$APP_ID.$REGION.contentgrid.cloud/suppliers   \
    -F "name=Test supplier"   \
    -F "telephone=test"   \
    -H "Authorization: Bearer $TOKEN"   \
    | jq .id -r
)"

entity-item operations

An entity-item resource is exposed on the path /<entity-name-plural>/{id} (e.g. /invoices/{id} for the invoice entity type). {id} is a placeholder for the actual ID of the entity-item.

The following operations are available:

Method Content-Type Description
GET application/hal+json Read the entity-item with the specified id
PUT application/json Replace the entity-item data with the data supplied in the request body
PATCH application/json Update the entity-item data, modifying only the fields supplied in the request body
DELETE N/A Remove the entity-item
Note

The difference between PUT and PATCH is how attributes that are not present in the request body are handled:

  • PUT: It is written to the entity-item as null, removing the stored value.
  • PATCH: It is skipped, the stored value is unchanged.

This rule also applies to attributes of type content. Omitting the field will remove the content.

Update entity-item using PUT/PATCH

To show the difference between updating an entity-item with PUT or PATCH, we first show the existing state

{
  "id": "3211be1d-1ed1-4850-8ea6-3fa3218031f6",
  "received": "2024-07-15",
  "document": {
     "size": 123456,
     "mimetype": "application/pdf",
     "filename": "example-invoice.pdf"
  },
  "pay_before": "2024-08-14",
  "total_amount": 15.95,
  "_links": {
    [...]
  },
  "_templates": {
    [...]
  }
}
curl -i -X PUT https://$APP_ID.$REGION.contentgrid.cloud/invoices/$INVOICE_ID   \
    --json "{\"pay_before\": \"2024-08-31\", \"received\": \"2024-07-15\", \"total_amount\": 15.95}"   \
    -H "Authorization: Bearer $TOKEN"
PUT /invoices/$INVOICE_ID HTTP/1.1
Authorization: Bearer $TOKEN
Content-Type: application/json

{"pay_before": "2024-08-31", "received": "2024-07-15", "total_amount": 15.95}
HTTP/1.1 204 No Content

Note that document was not present in the request body, and has now been set to null, removing the document.

curl -i https://$APP_ID.$REGION.contentgrid.cloud/invoices/$INVOICE_ID   \
    -H "Authorization: Bearer $TOKEN"
GET /invoices/$INVOICE_ID HTTP/1.1
Authorization: Bearer $TOKEN
HTTP/1.1 200 OK
Content-Type: application/json

{
  "id": "3211be1d-1ed1-4850-8ea6-3fa3218031f6",
  "received": "2024-07-15",
  "document": null,
  "pay_before": "2024-08-31",
  "total_amount": 15.95,
  "_links": {
    [...]
  },
  "_templates": {
    [...]
  }
}
curl -i -X PATCH https://$APP_ID.$REGION.contentgrid.cloud/invoices/$INVOICE_ID   \
    --json "{\"pay_before\": \"2024-08-31\", \"received\": \"2024-07-15\", \"total_amount\": 15.95}"   \
    -H "Authorization: Bearer $TOKEN"
PATCH /invoices/$INVOICE_ID HTTP/1.1
Authorization: Bearer $TOKEN
Content-Type: application/json

{"pay_before": "2024-08-31", "received": "2024-07-15", "total_amount": 15.95}
HTTP/1.1 204 No Content

Fields that are absent in the request body are not modified from their original values.

curl -i https://$APP_ID.$REGION.contentgrid.cloud/invoices/$INVOICE_ID   \
    -H "Authorization: Bearer $TOKEN"
GET /invoices/$INVOICE_ID HTTP/1.1
Authorization: Bearer $TOKEN
HTTP/1.1 200 OK
Content-Type: application/json

{
  "id": "3211be1d-1ed1-4850-8ea6-3fa3218031f6",
  "received": "2024-07-15",
  "document": {
     "size": 123456,
     "mimetype": "application/pdf",
     "filename": "example-invoice.pdf"
  },
  "pay_before": "2024-08-31",
  "total_amount": 15.95,
  "_links": {
    [...]
  },
  "_templates": {
    [...]
  }
}

entity-collection operations

The entity-collection resource is exposed on the path /<entity-name-plural> (e.g. /invoices for the invoice entity type).

The following operations are available:

Method Content-Type Description
GET application/hal+json Read the entity-collection, applying the specified filters and pagination
POST application/json Create a new entity-item. The response will be the created entity-item resource
POST application/x-www-form-urlencoded Create a new entity-item with a form submission. The response will be the created entity-item resource
POST multipart/form-data Create a new entity-item, and immediately upload content as well. The response will be the created entity-item resource
Create an entity using JSON
curl -i https://$APP_ID.$REGION.contentgrid.cloud/invoices   \
    --json "{\"received\": \"2024-07-15\", \"total_amount\": 15.95, \"pay_before\": \"2024-08-14\"}"   \
    -H "Authorization: Bearer $TOKEN"
POST /invoices HTTP/1.1
Authorization: Bearer $TOKEN
Content-Type: application/json

{"received": "2024-07-15", "total_amount": 15.95, "pay_before": "2024-08-14"}
HTTP/1.1 201 Created
Location: /invoices/3211be1d-1ed1-4850-8ea6-3fa3218031f6
Content-Type: application/prs.hal-forms+json

{
  "id": "3211be1d-1ed1-4850-8ea6-3fa3218031f6",
  "received": "2024-07-15",
  "document": null,
  "pay_before": "2024-08-14",
  "total_amount": 15.95,
  "_links": {
    [...]
  },
  "_templates": {
    [...]
  }
}
Create an entity with content using multipart/form-data
curl -i https://$APP_ID.$REGION.contentgrid.cloud/invoices   \
    -F received=2024-07-15   \
    -F total_amount=15.95   \
    -F pay_before=2024-08-14   \
    -F document=@example-invoice.pdf   \
    -H "Authorization: Bearer $TOKEN"
POST /invoices HTTP/1.1
Authorization: Bearer $TOKEN
Content-Type: multipart/form-data;boundary="delimiter123"

--delimiter123
Content-Disposition: form-data; name="received"

2024-07-15
--delimiter123
Content-Disposition: form-data; name="total_amount"

15.95
--delimiter123
Content-Disposition: form-data; name="pay_before"

2024-08-14
--delimiter123
Content-Disposition: form-data; name="document"; filename="example-invoice.pdf"

....Omitted contents of file example-invoice.pdf....
--delimiter123--
HTTP/1.1 201 Created
Location: /invoices/3211be1d-1ed1-4850-8ea6-3fa3218031f6
Content-Type: application/prs.hal-forms+json

{
  "id": "3211be1d-1ed1-4850-8ea6-3fa3218031f6",
  "received": "2024-07-15",
  "document": {
     "size": 123456,
     "mimetype": "application/pdf",
     "filename": "example-invoice.pdf"
  },
  "pay_before": "2024-08-14",
  "total_amount": 15.95,
  "_links": {
    [...]
  },
  "_templates": {
    [...]
  }
}

Collection query parameters

An entity-collection has 3 query parameters that control the collection itself:

Query Parameter Type Description
_size integer, 1-1000 or absent Page size, the number of entity-item resources that will be returned on a single page
_cursor string or absent Cursor for the page to retrieve. Absent to retrieve the first page
_sort string or absent Sort the collection by an attribute. Parameter can be repeated multiple times to sort on multiple attributes

Pagination

By default, every page contains 20 entity-items. This can be changed with the _size query parameter.

When there are multiple pages, the meta information will contain prev_cursor and/or next_cursor fields. A previous or next page can be retrieved by setting the _cursor query parameter to the value in the prev_cursor or next_cursor field.

Note

A cursor is not a fancy page number.

  • Cursors are opaque. A client should not parse or attempt to modify a cursor; they are only meaningful to the server.
  • Cursors are unique for every entity-collection. They can not be reused in a different context or used with different filter parameters.
  • Cursors are ephemeral. They should not be stored permanently to use at a later point in time, and they do not permanently identify a certain page
Example of navigating through pages with a cursor

curl -i https://$APP_ID.$REGION.contentgrid.cloud/invoices   \
    -H "Authorization: Bearer $TOKEN"
GET /invoices HTTP/1.1
Authorization: Bearer $TOKEN
HTTP/1.1 200 OK
Content-Type: application/prs.hal-forms+json

{
    "_embedded": {
        [...]
    },
    "page": {
        "size": 20,
        "next_cursor": "1wskufc1",
        "total_items_estimate": 48,
        "total_items_exact": 48
    }
}
Navigate to the next page. Note that a prev_cursor appears, which can be used to navigate to the previous page
curl -i https://$APP_ID.$REGION.contentgrid.cloud/invoices?_cursor=1wskufc1   \
    -H "Authorization: Bearer $TOKEN"
GET /invoices?_cursor=1wskufc1 HTTP/1.1
Authorization: Bearer $TOKEN
HTTP/1.1 200 OK
Content-Type: application/prs.hal-forms+json

{
    "_embedded": {
        [...]
    },
    "page": {
        "size": 20,
        "prev_cursor": "0dtcbvz0",
        "next_cursor": "0qk2h5e2",
        "total_items_estimate": 48,
        "total_items_exact": 48
    }
}

Navigate again to the next page. Note that this is the final page, and next_cursor is absent, indicating that there are no further pages.

curl -i https://$APP_ID.$REGION.contentgrid.cloud/invoices?_cursor=0qk2h5e2   \
    -H "Authorization: Bearer $TOKEN"
GET /invoices?_cursor=0qk2h5e2 HTTP/1.1
Authorization: Bearer $TOKEN
HTTP/1.1 200 OK
Content-Type: application/prs.hal-forms+json

{
    "_embedded": {
        [...]
    },
    "page": {
        "size": 20,
        "prev_cursor": "1wskufc1",
        "total_items_estimate": 48,
        "total_items_exact": 48
    }
}

Sorting

By default, the collection is not sorted. The collection can be sorted by using the _sort query parameter.

The _sort query parameter can be repeated multiple times to sort on multiple attributes.

The _sort value is composed of the attribute name, followed by ,asc or ,desc to sort ascending or descending.

Note that not all attributes support sorting. The OpenAPI spec lists the valid values for the _sort query parameter.

Example of collection sorting on multiple attributes

First sort ascending on received, then descending on total_amount

curl -i https://$APP_ID.$REGION.contentgrid.cloud/invoices?_sort=received,asc\&_sort=total_amount,desc   \
    -H "Authorization: Bearer $TOKEN"
GET /invoices?_sort=received,asc&_sort=total_amount,desc HTTP/1.1
Authorization: Bearer $TOKEN

Filtering

Search filters can be applied to the entity-collection. By default no filters are applied.

All filters are application-specific, and are mapped directly to query parameters. The OpenAPI spec lists all the valid filters as query parameters.

Note

Using a query parameter that is not a known search filter or one of the special query parameters will be ignored and have no effect.

Multiple search filters can be combined. Different search filters are AND’ed together; repeating the same search filter multiple times will perform an OR on the different values of that filter.

Example of collection filtering with multiple search filters

Show invoices with total_amount equal to 15.95

curl -i https://$APP_ID.$REGION.contentgrid.cloud/invoices?total_amount=15.95   \
    -H "Authorization: Bearer $TOKEN"
GET /invoices?total_amount=15.95 HTTP/1.1
Authorization: Bearer $TOKEN
Show invoices with total_amount equal to 15.95 OR 123.4
curl -i https://$APP_ID.$REGION.contentgrid.cloud/invoices?total_amount=19.95\&total_amount=123.4   \
    -H "Authorization: Bearer $TOKEN"
GET /invoices?total_amount=19.95&total_amount=123.4 HTTP/1.1
Authorization: Bearer $TOKEN
Show invoices with total_amount equal to 15.95 AND pay_before equal to 2024-08-14
curl -i https://$APP_ID.$REGION.contentgrid.cloud/invoices?total_amount=15.95\&pay_before=2024-08-14   \
    -H "Authorization: Bearer $TOKEN"
GET /invoices?total_amount=15.95&pay_before=2024-08-14 HTTP/1.1
Authorization: Bearer $TOKEN
Show invoices with (total_amount equal to 15.95 OR 123.4) AND pay_before equal to 2024-08-14
curl -i https://$APP_ID.$REGION.contentgrid.cloud/invoices?total_amount=15.95\&total_amount=123.4\&pay_before=2024-08-14   \
    -H "Authorization: Bearer $TOKEN"
GET /invoices?total_amount=15.95&total_amount=123.4&pay_before=2024-08-14 HTTP/1.1
Authorization: Bearer $TOKEN

relation operations

An entity can have relations to a different entity. These are exposed on the path /<entity-name-plural>/{id}/<relation-name>. (e.g. /invoices/{id}/supplier for the invoice entity type and a relation supplier)

Depending on the relation type, the available operations and responses are different.

to-one relation operations

These operations are applicable to one-to-one and many-to-one relations.

Method Content-Type Description
GET N/A Read the relation link. Redirects to the entity-item that the relation refers to
PUT text/uri-list Update the relation link. Requires a single URL to the entity-item that the relation should refer to
DELETE N/A Remove the relation link. No entity-items are deleted, only the link between the two is severed
Reading and updating a to-one relation

Initially, no supplier is linked to the invoice, and fetching the relation results in a 404 Not Found error.

curl -i https://$APP_ID.$REGION.contentgrid.cloud/invoices/$INVOICE_ID/supplier   \
    -H "Authorization: Bearer $TOKEN"
GET /invoices/$INVOICE_ID/supplier HTTP/1.1
Authorization: Bearer $TOKEN
HTTP/1.1 404 Not Found

Setting a relation requires sending the URL of the created entity-item in the body.

curl -i -X PUT https://$APP_ID.$REGION.contentgrid.cloud/invoices/$INVOICE_ID/supplier   \
    --data-binary "https://$APP_ID.$REGION.contentgrid.cloud/suppliers/$SUPPLIER_ID"   \
    -H 'Content-Type: text/uri-list'   \
    -H "Authorization: Bearer $TOKEN"
PUT /invoices/$INVOICE_ID/supplier HTTP/1.1
Authorization: Bearer $TOKEN
Content-Type: text/uri-list

https://$APP_ID.$REGION.contentgrid.cloud/suppliers/$SUPPLIER_ID
HTTP/1.1 204 No Content

Reading the relation afterwards will result in a redirect to the specific item that was linked.

curl -i https://$APP_ID.$REGION.contentgrid.cloud/invoices/$INVOICE_ID/supplier   \
    -H "Authorization: Bearer $TOKEN"
GET /invoices/$INVOICE_ID/supplier HTTP/1.1
Authorization: Bearer $TOKEN
HTTP/1.1 302 Found
Location: /suppliers/$SUPPLIER_ID

to-many relation operations

These operations are applicable to one-to-many and many-to-many relations.

Method Content-Type Description
GET application/json Read the relation collection. Redirects to the entity-collection containing all entity-items that the relation refers to
POST text/uri-list Adds entity-items to the relation. Accepts one or more URLs to the entity-items that should be added to the relation collection
DELETE N/A Clear the relation collection. No entity-items are deleted, only the relation collection is emptied.
Reading and updating a to-many relation

In this example, we will add 2 invoices to a supplier.

curl -i -X POST https://$APP_ID.$REGION.contentgrid.cloud/suppliers/$SUPPLIER_ID/invoices   \
    --data-binary "https://$APP_ID.$REGION.contentgrid.cloud/invoices/$INVOICE_ID
https://$APP_ID.$REGION.contentgrid.cloud/invoices/$INVOICE2_ID"   \
    -H 'Content-Type: text/uri-list'   \
    -H "Authorization: Bearer $TOKEN"
POST /suppliers/$SUPPLIER_ID/invoices HTTP/1.1
Authorization: Bearer $TOKEN
Content-Type: text/uri-list

https://$APP_ID.$REGION.contentgrid.cloud/invoices/$INVOICE_ID
https://$APP_ID.$REGION.contentgrid.cloud/invoices/$INVOICE2_ID
HTTP/1.1 204 No Content

Reading the relation will result in a redirect to a collection containing all items that are linked. Note that the query parameter used in the redirect is subject to change and is not part of the public API.

curl -i https://$APP_ID.$REGION.contentgrid.cloud/suppliers/$SUPPLIER_ID/invoices   \
    -H "Authorization: Bearer $TOKEN"
GET /suppliers/$SUPPLIER_ID/invoices HTTP/1.1
Authorization: Bearer $TOKEN
HTTP/1.1 302 Found
Location: /invoices?_internal_supplier=$SUPPLIER_ID

relation-item operations

A to-many relation also has a way to reference an individual item inside the relation; the relation-item resource. These are exposed on the path /<entity-name-plural>/{id}/<relation-name>/{itemId} (e.g. /suppliers/{id}/invoices/{itemId} for the supplier entity type and a relation invoices).

Method Description
GET Read the relation collection item. Redirects to the entity-item if it is part of the relation collection
DELETE Remove the relation collection item from the relation collection. No entity-items are deleted, only the link between the two is severed
Removing an item from a to-many relation

This will remove a single entity-item from the relation

curl -i -X DELETE https://$APP_ID.$REGION.contentgrid.cloud/suppliers/$SUPPLIER_ID/invoices/$INVOICE_ID   \
    -H "Authorization: Bearer $TOKEN"
DELETE /suppliers/$SUPPLIER_ID/invoices/$INVOICE_ID HTTP/1.1
Authorization: Bearer $TOKEN
HTTP/1.1 204 No Content

After the entity-item is removed from the relations, further attempts to remove the same item will result in a 404 Not Found error

curl -i -X DELETE https://$APP_ID.$REGION.contentgrid.cloud/suppliers/$SUPPLIER_ID/invoices/$INVOICE_ID   \
    -H "Authorization: Bearer $TOKEN"
DELETE /suppliers/$SUPPLIER_ID/invoices/$INVOICE_ID HTTP/1.1
Authorization: Bearer $TOKEN
HTTP/1.1 404 Not Found

entity-content operations

An entity-content resource is exposed on the path /<entity-name-plural>/{id}/<attribute-name> (e.g. /invoices/{id}/document for the invoice entity type and a content attribute document). {id} is a placeholder for the actual ID of the entity-item.

The following operations are available:

Method Content-Type Description
GET any Retrieve stored file. If no file is currently stored, responds with HTTP 404 Not Found
PUT any Overwrite file with contents of request body
PUT multipart/form-data Overwrite file with contents of the file form-field
DELETE N/A Remove stored file

For the GET and PUT with arbitrary Content-Type; the filename is provided in the Content-Disposition header. If no filename is provided during content upload, the filename field of the content attribute will be set to null

Read and write content

Read content of an invoice with id $INVOICE_ID

curl -i https://$APP_ID.$REGION.contentgrid.cloud/invoices/$INVOICE_ID/document   \
    -H "Authorization: Bearer $TOKEN"
GET /invoices/$INVOICE_ID/document HTTP/1.1
Authorization: Bearer $TOKEN
HTTP/1.1 200 OK
Content-Type: application/pdf
Content-Disposition: attachment;filename="example-invoice.pdf"

....Raw PDF body omitted....
Overwrite content of an invoice using direct upload, setting the filename as well.
curl -i -X PUT https://$APP_ID.$REGION.contentgrid.cloud/invoices/$INVOICE_ID/document   \
    --data-binary "@example-invoice.pdf"   \
    -H 'Content-Disposition: attachment;filename="example-invoice.pdf"'   \
    -H 'Content-Type: application/pdf'   \
    -H "Authorization: Bearer $TOKEN"
PUT /invoices/$INVOICE_ID/document HTTP/1.1
Authorization: Bearer $TOKEN
Content-Disposition: attachment;filename="example-invoice.pdf"
Content-Type: application/pdf

....Omitted contents of file example-invoice.pdf....
HTTP/1.1 204 No Content
Overwrite content of an invoice using multipart upload
curl -i -X PUT https://$APP_ID.$REGION.contentgrid.cloud/invoices/$INVOICE_ID/document   \
    -F file=@example-invoice.pdf   \
    -H "Authorization: Bearer $TOKEN"
PUT /invoices/$INVOICE_ID/document HTTP/1.1
Authorization: Bearer $TOKEN
Content-Type: multipart/form-data;boundary="delimiter123"

--delimiter123
Content-Disposition: form-data; name="file"; filename="example-invoice.pdf"

....Omitted contents of file example-invoice.pdf....
--delimiter123--
HTTP/1.1 204 No Content

Request modifiers

The basic operations explained above can be extended with additional functionality. Additional functionality is layered on top of the basic operations using HTTP headers.

Conditional requests

Conditional requests allow to check a precondition before applying the request to the target resource. They are an implementation of RFC9110 Conditional Requests.

The primary usage of conditional requests is to prevent the “lost update” problem, where one system overwrites the changes of another system that made a write between the read and write of the first system.

Comparing without/with conditional requests
Without conditional request With conditional request
sequenceDiagram
    autonumber
    participant a as System A
    participant s as ContentGrid API
    participant b as System B

    a ->>+ s: GET item
    s -->>- a: 200 OK<br>ETag: "abc"<br>original data

    b ->>+ s: PUT item<br>new data
    s -->>- b: 204 No Content<br>ETag: "def"

    a ->>+ s: PUT item<br>other data
    s -->>- a: 204 No Content<br>ETag: "mno"
    note over a, s: "new data" was accidentally<br>overwritten by "other data"<br>without System A having seen it
sequenceDiagram
    autonumber
    participant a as System A
    participant s as ContentGrid API
    participant b as System B

    a ->>+ s: GET item
    s -->>- a: 200 OK<br>ETag: "abc"<br>original data

    b ->>+ s: PUT item<br>new data
    s -->>- b: 204 No Content<br>ETag: "def"

    a ->>+ s: PUT item<br>If-Match: "abc"<br>other data
    rect red
    s -->>- a: 419 Precondition Failed
    note over a, s : request was rejected<br>System A did not accidentally<br>overwrite the "new data"
    end

Conditional requests can be used on the resources that have an ETag response header, which are these:

Name URL
entity-item /<entity-name-plural>/{id}
to-one relation /<entity-name-plural>/{id}/<relation-name>
entity-content /<entity-name-plural>/{id}/<attribute-name>

To perform conditional requests, a previously obtained ETag value has to be placed in the If-Match or If-None-Match headers. Note that the quotes are part of the ETag value.

The If-Modified-Since and If-Unmodified-Since headers are not supported, because no Last-Modified response header is present, and they are less precise.

Using a conditional request for updating

In this example, we’re going to move the pay_before date one day earlier.

curl -i -X GET https://$APP_ID.$REGION.contentgrid.cloud/invoices/$INVOICE_ID   \
    -H "Authorization: Bearer $TOKEN"
GET /invoices/$INVOICE_ID HTTP/1.1
Authorization: Bearer $TOKEN
HTTP/1.1 200 OK
ETag: "1e0k4j9"
Content-Type: application/hal+json

{
  "id": "019be613-baa4-7644-9438-e252be3bbe84",
  "received": "2024-07-15",
  "document": {
    "filename": "invoice.txt",
    "length": 13,
    "mimetype": "text/plain"
  },
  "pay_before": "2024-08-14",
  "total_amount": 15.95,
  "_links": {
    [...]
  }
}

After retrieving the invoice response, we take note of the ETag header, and place if in the If-Match header for our next request. Then we perform a PATCH request to update only the pay_before attribute.

Between our requests…

To execute this example, you will need to change the If-Match header to the correct value yourself.

curl -i -X PATCH https://$APP_ID.$REGION.contentgrid.cloud/invoices/$INVOICE_ID   \
    --json "{\"pay_before\": \"2024-08-13\"}"   \
    -H 'If-Match: "1e0k4j9"'   \
    -H "Authorization: Bearer $TOKEN"
PATCH /invoices/$INVOICE_ID HTTP/1.1
Authorization: Bearer $TOKEN
If-Match: "1e0k4j9"
Content-Type: application/json

{"pay_before": "2024-08-13"}
HTTP/1.1 204 No Content
ETag: "i4xcj6"

The response to the modification also contains the new ETag value for the updated version of the object.

curl -i -X PATCH https://$APP_ID.$REGION.contentgrid.cloud/invoices/$INVOICE_ID   \
    --json "{\"pay_before\": \"2024-08-13\"}"   \
    -H 'If-Match: "1e0k4j9"'   \
    -H "Authorization: Bearer $TOKEN"
PATCH /invoices/$INVOICE_ID HTTP/1.1
Authorization: Bearer $TOKEN
If-Match: "1e0k4j9"
Content-Type: application/json

{"pay_before": "2024-08-13"}
HTTP/1.1 412 Precondition Failed
Content-Type: application/problem+json

{
  "type": "https://contentgrid.cloud/problems/unsatisfied-version",
  "title": "Object has changed",
  "detail": "Requested version constraint 'is any of [exactly '1e0k4j9']' can not be satisfied (actual version exactly '1r061qt')",
  "status": 412,
  "actual_version": "1r061qt"
}

More details about the error response format can be found in the error handling section.

Range requests

Range requests allow retrieving a partial representation of the entity-content resource. They are an implementation of RFC9110 Range Requests.

Range requests are only supported on the entity-content resource (/<entity-name-plural>/{id}/<attribute-name>), as indicated by the Accept-Ranges header.

You usually don’t use range requests directly, but some tools that work with potentially large files (like PDF viewers or download managers) can make use of them.

Warning

When composing a file together from multiple parts, it is critical to check that the ETags of all parts are identical.

Otherwise, a concurrent change of the content will result in the composed file being corrupted.

Using a range request to retrieve a part of a resource

In this example, the first 4 bytes are fetched separately.

curl -i https://$APP_ID.$REGION.contentgrid.cloud/invoices/$INVOICE_ID/document   \
    -H 'Range: bytes=0-3'   \
    -H "Authorization: Bearer $TOKEN"
GET /invoices/$INVOICE_ID/document HTTP/1.1
Authorization: Bearer $TOKEN
Range: bytes=0-3
HTTP/1.1 206 Partial Content
Accept-Ranges: bytes
Content-Range: bytes 0-3/13
ETag: "8xhtyfikrvnts4izydc95vfcn"
Content-Length: 4
Content-Type: text/plain

dumm

After the initial fetch, If-Match is used to ensure that the rest of the same file is fetched.

curl -i https://$APP_ID.$REGION.contentgrid.cloud/invoices/$INVOICE_ID/document   \
    -H 'If-Match: "8xhtyfikrvnts4izydc95vfcn"'   \
    -H 'Range: bytes=4-'   \
    -H "Authorization: Bearer $TOKEN"
GET /invoices/$INVOICE_ID/document HTTP/1.1
Authorization: Bearer $TOKEN
If-Match: "8xhtyfikrvnts4izydc95vfcn"
Range: bytes=4-
HTTP/1.1 206 Partial Content
Accept-Ranges: bytes
Content-Range: bytes 4-12/13
ETag: "8xhtyfikrvnts4izydc95vfcn"
Content-Length: 9
Content-Type: text/plain

y-invoice

HAL Usage

ContentGrid applications expose a REST API using JSON as a response format. All JSON responses use HAL as a hypertext format, linking to related resources. Actions that can be taken on resources are specified using the HAL-FORMS format.

Before using the API, you need to authenticate and obtain an access token.

This guide explains how HAL features make the API discoverable and how to leverage HAL features to build robust integrations.

Basic usage of the API, where no automatic adaptations to the data model are necessary is described in API Usage.

When to use HAL?

Using HAL is only recommended when building integrations with ContentGrid that have to dynamically discover the datamodel and adjust to it.

It is particularly useful for systems that integrate with multiple distinct ContentGrid applications, which will have different datamodels.

When building such a dynamic system, you want it to automatically discover the datamodel and automatically adjust to it when it changes.

HAL provides several benefits for these systems:

  • Discoverability: Navigate the API by following links instead of constructing URLs manually
  • Flexibility: Discover available operations dynamically instead of hardcoding them
  • Evolvability: Adapt to data model changes without breaking your integration
  • Self-documentation: The API tells you what actions are possible on each resource

HAL Basics

HAL and HAL-FORMS are built on top of JSON. They add 3 special fields to the resource objects:

  • _links: Contains links to related resources
  • _embedded: Directy embeds related resources
  • _templates: Describes operations that can be performed on this resource (HAL-FORMS)
Invoice resource with HAL and HAL-FORMS fields
{
  "id": "3211be1d-1ed1-4850-8ea6-3fa3218031f6",
  "received": "2024-07-15",
  "document": null,
  "pay_before": "2024-08-14",
  "total_amount": 15.95,
  "_links": {
    "self": {
      "href": "https://app.contentgrid.example/invoices/3211be1d-1ed1-4850-8ea6-3fa3218031f6"
    },
    "cg:relation": [
      {
        "href": "https://app.contentgrid.example/invoices/3211be1d-1ed1-4850-8ea6-3fa3218031f6/supplier",
        "title": "Supplier",
        "name": "supplier"
      }
    ],
    "cg:content": [
      {
        "href": "https://app.contentgrid.example/invoices/3211be1d-1ed1-4850-8ea6-3fa3218031f6/document",
        "title": "Document",
        "name": "document"
      }
    ],
    "curies": [
      {
        "href": "https://contentgrid.cloud/rels/contentgrid/{rel}",
        "name": "cg",
        "templated": true
      }
    ]
  },
  "_templates": {
    "default": {
      "method": "PUT",
      "contentType": "application/json",
      "properties": [
        {
          "name": "received",
          "prompt": "Received",
          "required": true,
          "type": "datetime"
        },
        {
          "name": "pay_before",
          "prompt": "Pay before",
          "required": true,
          "type": "datetime"
        },
        {
          "name": "total_amount",
          "prompt": "Total amount",
          "required": true,
          "type": "number"
        }
      ]
    },
    "delete": {
      "method": "DELETE",
      "properties": []
    },
    "set-supplier": {
      "method": "PUT",
      "contentType": "text/uri-list",
      "properties": [
        {
          "name": "supplier",
          "type": "url"
        }
      ],
      "target": "https://app.contentgrid.example/invoices/3211be1d-1ed1-4850-8ea6-3fa3218031f6/supplier"
    },
    "clear-supplier": {
      "method": "DELETE",
      "properties": [],
      "target": "https://app.contentgrid.example/invoices/3211be1d-1ed1-4850-8ea6-3fa3218031f6/supplier"
    }
  }
}

Links connect resources together. Instead of constructing URLs yourself, you use the provided link to go to the related resource.

Links express a relation between the resource they appear on (the link context), and the link target. The link relation type (RFC8288 Sec 3.3) determines the kind of relation between the link context and the link target.

Next to the standard link relation types, ContentGrid defines extension link relation types.

CURIEs

CURIEs (Compact URIs) are a shorthand notation for extension link relation types. They are used as defined in the HAL specification. Before comparing a link relation, a CURIE must be expanded into the full URI.

See the Link Relation Types reference for the formal definition of CURIE expansion and examples.

Working with Embedded Resources

Instead of linking to a related resource, it is also possible to directly embed it in the main resource. This is of only possible when the resource to be embedded is also a HAL resource.

In the ContentGrid application, this is mostly used for the entity-collection resource, where the page is the main resource, and all items on the page are embedded resources.

Link Relation Types work the same for embedded resources as they do for links.

An embedded resource should have a self link. The link attributes on the self-link of the embedded resource can be considered as the link attributes that would be present on a normal link to the embedded resource.

A HAL resource with embedded resources
{
  "_embedded": {
    "item": [
      {
        "id": "62282aba-13bc-11f1-9a7d-ebcebd0bcf11",
        [...]
        "_links": {
          "self": {
            "href": "/invoices/62282aba-13bc-11f1-9a7d-ebcebd0bcf11",
            "title": "Invoice 62282aba-13bc-11f1-9a7d-ebcebd0bcf11"
          },
          [...]
        }
      }
    ]
  },
  [...]
}

The equivalent using only links would be

{
  "_links": {
    "item": [
      {
        "href": "/invoices/62282aba-13bc-11f1-9a7d-ebcebd0bcf11",
        "title": "Invoice 62282aba-13bc-11f1-9a7d-ebcebd0bcf11"
      },
    ]
  },
  [...]
}

Working with HAL-FORMS Templates

Where HAL links describe related resources, HAL-FORMS templates describe the operations available on the resource.

HAL-FORMS are described in their own specification. ContentGrid adds additional functionality to the base specification, described in HAL-FORMS extensions.

The keys in _templates tell you what forms are available.

ContentGrid standardizes the following keys:

Resource Key Description
entity-item default Update the resource
entity-item delete Delete the resource
entity-item set-<relation> Update a to-one relation
entity-item add-<relation> Add items to a to-many relation
entity-item clear-<relation> Remove all items from a relation
entity-profile create-form Create a new item of the type described by the profile
entity-profile search Search within the collection of the type described by the profile

HAL-FORMS Extensions

ContentGrid extends the HAL-FORMS specification with support for nested JSON objects, additional body encodings (multipart/form-data and text/uri-list), and extended options element behavior.

See the HAL-FORMS Extensions reference for the full specification.

The entity-collection resource provides HAL links that can be used pagination.

Pagination uses the standard IANA-registered link relation types for their purpose:

  • first: First page of results (may be absent if you’re already on the first page)
  • prev: Previous page of results (absent when on the first page)
  • next: Next page of results (absent when on the last page)
  • self: Current page

To navigate through pages, follow the next link until it’s absent. The cursors in these links are opaque values managed by the server - don’t try to parse or construct them yourself.

Discovering the API Structure

Start at the entities-root resource to discover the entire API.

Example
curl -i https://$APP_ID.$REGION.contentgrid.cloud/   \
    -H "Authorization: Bearer $TOKEN"
GET / HTTP/1.1
Authorization: Bearer $TOKEN
HTTP/1.1 200 OK
Content-Type: application/prs.hal-forms+json

{
  "_links": {
    "self": {
      "href": "https://app.contentgrid.example/"
    },
    "profile": {
      "href": "https://app.contentgrid.example/profile"
    },
    "cg:entity": [
      {
        "href": "https://app.contentgrid.example/invoices",
        "title": "Invoices",
        "name": "invoice"
      },
      {
        "href": "https://app.contentgrid.example/suppliers",
        "title": "Suppliers",
        "name": "supplier"
      }
    ],
    "curies": [
      {
        "href": "https://contentgrid.cloud/rels/contentgrid/{rel}",
        "name": "cg",
        "templated": true
      }
    ]
  }
}

The root resource contains cg:entity links to all entity collections in your data model. Use these to discover what entities are available without hardcoding entity names.

Alternatively, follow the profile link to arrive at the profile-root, and follow the cg:entity links there to arrive at the entity-profiles for every entity in your data model.

Using Entity Profiles

Entity profiles provide machine-readable metadata about entity types in a ContentGrid application. They describe the attributes, relations, constraints, and available operations for each entity type.

The entity-profile supports content negotiation between two formats:

Accept Header Format
application/prs.hal-forms+json HAL-FORMS format with embedded resources and templates
application/schema+json JSON Schema format (JSON Schema 2020-12)

Profiles are linked from:

HAL-FORMS Format

The HAL-FORMS profile format is the richest representation. It describes the full model of an entity type, its attributes and relations, and provides search and create-form templates.

The full HAL-FORMS entity profile format is documented in the Entity Profiles: HAL-FORMS Format reference.

JSON Schema Format

Request a profile with Accept: application/schema+json to receive a JSON Schema (2020-12) representation of the entity type.

In this format:

  • Attributes are mapped to properties with their corresponding JSON Schema types
  • Relations are represented as properties with "format": "uri"
  • Content attributes use "$ref": "#/$defs/content", with sub-attribute properties defined in $defs.content
  • Required attributes are listed in the required array
Entity profile in JSON Schema format
curl -i https://$APP_ID.$REGION.contentgrid.cloud/profile/invoices   \
    -H 'Accept: application/schema+json'   \
    -H "Authorization: Bearer $TOKEN"
GET /profile/invoices HTTP/1.1
Authorization: Bearer $TOKEN
Accept: application/schema+json
HTTP/1.1 200 OK
Content-Type: application/schema+json

{
  "title": "Invoice",
  "$defs": {
    "content": {
      "type": "object",
      "properties": {
        "filename": {
          "title": "Filename",
          "readOnly": false,
          "type": "string"
        },
        "length": {
          "title": "Size",
          "description": "File size in bytes",
          "readOnly": true,
          "type": "integer"
        },
        "mimetype": {
          "title": "Mimetype",
          "description": "Technical indicator of the type of the file",
          "readOnly": false,
          "type": "string"
        }
      }
    }
  },
  "properties": {
    "pay_before": {
      "title": "Pay before",
      "readOnly": false,
      "type": "string",
      "format": "date"
    },
    "total_amount": {
      "title": "Total amount",
      "readOnly": false,
      "type": "number"
    },
    "document": {
      "title": "Document",
      "readOnly": false,
      "$ref": "#/$defs/content"
    },
    "supplier": {
      "title": "Supplier",
      "readOnly": false,
      "type": "string",
      "format": "uri"
    },
    "received": {
      "title": "Received",
      "readOnly": false,
      "type": "string",
      "format": "date"
    },
    "id": {
      "title": "id",
      "readOnly": true,
      "type": "string",
      "format": "uuid"
    }
  },
  "required": [
    "received",
    "pay_before",
    "total_amount"
  ],
  "type": "object",
  "$schema": "https://json-schema.org/draft/2020-12/schema"
}

Error handling

The ContentGrid REST API uses the standard HTTP status codes to signal errors.

RFC9457 Problem Details are used to provide additional information about the error condition.

HTTP Status Codes

The ContentGrid Application API uses standard HTTP status codes to indicate the success or failure of API requests.

These status codes provide high-level information about what happened with your request, and are generically described in RFC9110.

Successful Responses (2xx)

  • 200 OK: The request succeeded. The response body contains the requested resource or data.
  • 201 Created: A new resource was successfully created. The Location header contains the URL of the newly created resource.
  • 204 No Content: The request succeeded but there is no response body. This is typically returned after successful updates or deletions.

Redirection Responses (3xx)

  • 302 Found: The resource is temporarily available at a different URI. The Location header contains the URI where the resource can be found. This is commonly used when accessing relations.

Client Error Responses (4xx)

  • 400 Bad Request: The request is malformed or contains invalid data. The response body may contain a problem detail explaining what went wrong if the request is sufficiently syntactically valid to be able to be processed.
  • 401 Unauthorized: The request lacks valid authentication credentials.
  • 403 Forbidden: The server understood the request but refuses to authorize it. This occurs when you don’t have permission to perform the requested operation.
  • 404 Not Found: The requested resource does not exist. This could be an entity that doesn’t exist, an empty relation, or an unknown endpoint.
  • 409 Conflict: The request conflicts with the current state of the resource. This occurs with unique constraint violations or when trying to delete an entity that is still referenced by required relations.
  • 412 Precondition Failed: The condition specified in the request headers (If-Match or If-None-Match) was not met. This typically occurs when using optimistic locking and the entity has been modified by another request.
  • 415 Unsupported Media Type: The Content-Type header specifies a media type that the server doesn’t support for this endpoint.

Server Error Responses (5xx)

  • 500 Internal Server Error: An unexpected error occurred on the server. Contact support if this persists.

Problem Details (RFC 9457)

When an error occurs (4xx or 5xx status codes), the API returns a standardized error response using the RFC9457 Problem Details format. This provides a consistent, machine-readable way to communicate error information.

Problem Detail Structure

All error responses have Content-Type: application/problem+json and include these standard fields:

  • type (string): A URI identifying the problem type (e.g., https://contentgrid.cloud/problems/not-found/entity-item)
  • title (string): A short, human-readable summary of the problem type
  • detail (string): A human-readable explanation specific to this occurrence of the problem
  • status (number): The HTTP status code for this problem
  • Additional properties: Problem-specific fields that provide more context
Example
curl -i -X PATCH https://$APP_ID.$REGION.contentgrid.cloud/invoices/$INVOICE_ID   \
    --json "{\"pay_before\": \"2024-08-13\"}"   \
    -H 'If-Match: "outdated-version"'   \
    -H "Authorization: Bearer $TOKEN"
PATCH /invoices/$INVOICE_ID HTTP/1.1
Authorization: Bearer $TOKEN
If-Match: "outdated-version"
Content-Type: application/json

{"pay_before": "2024-08-13"}
HTTP/1.1 412 Precondition Failed
Content-Type: application/problem+json

{
  "type": "https://contentgrid.cloud/problems/unsatisfied-version",
  "title": "Object has changed",
  "detail": "Requested version constraint 'is any of [exactly 'outdated-version']' can not be satisfied (actual version exactly '1r061qt')",
  "status": 412,
  "actual_version": "1r061qt"
}

Problem Types Reference

For a complete catalog of all problem types returned by the ContentGrid API, see the Problem Types reference.

Reference Docs

Subsections of Reference Docs

How ContentGrid stores data

ContentGrid is designed to provide secure, scalable, and reliable data storage for your applications. This page explains the core principles and technologies behind how ContentGrid stores and manages your data.

Overview

ContentGrid separates your data into two main categories:

  • Metadata: Information about your content, such as file names, types and the other attributes you define in your data model.
  • Content: The actual files, documents, or binary data you upload and manage.

This separation allows ContentGrid to optimize for both performance and security.

Metadata Storage: PostgreSQL

All metadata is stored in a PostgreSQL database. PostgreSQL is a robust, open-source relational database system known for its reliability and advanced features. By using PostgreSQL, ContentGrid ensures:

  • Data Integrity: Strong consistency and transactional guarantees.
  • Query Performance: Fast and flexible querying of metadata.
  • Scalability: Ability to handle large volumes of metadata efficiently.

Content Storage: S3 Compatible Storage with Application-Side Encryption

ContentGrid implements application-side encryption for all content. Every document is encrypted individually using its own unique encryption key, which is securely stored in the PostgreSQL database alongside the metadata. This approach ensures that even if the storage backend is compromised, your data remains protected.

ContentGrid uses an encryption algorithm that supports range requests, allowing efficient access to parts of large files without decrypting the entire file. By default, the algorithm used is AES-CTR (Advanced Encryption Standard in Counter mode).

The actual content (files, documents, etc.) is stored in an S3 (Simple Storage Service) compatible service, a highly durable and scalable object storage service. To protect your data, ContentGrid applies encryption to all content before it is stored in S3.

  • Application-Side Encryption: Each document is encrypted with its own key before leaving your application environment.
  • Key Management: Encryption keys are stored securely in PostgreSQL, separate from the content itself.
  • Encryption Algorithm: AES-CTR is used by default, supporting range requests for efficient file access.
  • Durability: Object storage systems (like S3) provide very high durability, minimizing the risk of data loss.
  • Scalability: S3 can handle virtually unlimited amounts of data, making it suitable for projects of any size.

Security and Compliance

  • End-to-End Encryption: Data is encrypted both in transit and at rest, with application-side encryption ensuring content is protected before it reaches storage.
  • Per-Document Keys: Each document uses a unique encryption key, enhancing security and isolation.
  • Access Controls: Fine-grained access controls ensure that only authorized users and services can access your data.
  • Auditability: All access and changes to data are logged for compliance and auditing purposes.

Summary

  • Metadata is stored in PostgreSQL for reliability and performance.
  • Content is stored in encrypted form in S3 compatible storage for durability and security.
  • ContentGrid is designed to keep your data safe, compliant, and accessible.

Permissions Reference

Permissions Reference

Permissions in ContentGrid operate on the level of entities. By default, all permissions are denied and you cannot read or write any data. In order to grant a permission, you must define a policy. A policy is created for a particular entity and states that certain operations are allowed on that entity under certain conditions. For instance, a policy might say that the read operation is allowed on the invoice entity if the user has the bookkeeping attribute.

Operations

The currently supported operations are:

  • Create: The creation of an entity. Policies involving entity attributes are evaluated against the attributes that you send. This corresponds to http POST.
  • Read: Covers both reading a specific entity referred by id, as well as listing all visible entities. If you’re getting an entity by id but you don’t meet all the conditions, you will get a “Not Found” error. If you’re listing all entities, the list will be filtered down to the entities for which you meet the conditions. This corresponds to http GET and http HEAD.
  • Update: Updating the attributes and relations of an entity. If there are policies specifying conditions on the attributes, you must meet the conditions both before and after the update to the attributes. This corresponds to http PUT and http PATCH.
  • Delete: Deleting an entity. This corresponds to http DELETE.

Conditions

A policy has a list of conditions. If this list is empty, the operation in the policy is simply allowed. Otherwise, all the conditions have to match for the policy to allow access.

For instance, you can have two policies for the blog entity. The first policy has the operation READ and no conditions. The second policy has the operations CREATE and UPDATE, but also has a condition stating that the user must have the blogger attribute equal to true. The consequences of these policies are that anyone can read (and list) blog entities, but only people marked as bloggers can write them.

A condition consists of three parts:

  • A left value: This is the value that you want to check. It can be an entity attribute, a user attribute, or a constant.
  • A comparison operator: This is the symbol that defines the type of comparison you want to make. For example, if you want to check whether the left value is equal to the right value, you would use the equals operator. If you want to check whether the left value is a list that contains the right value, you would use the contains operator.
  • A right value: This is the value that you want to compare the left value to. It can be an entity attribute, a user attribute, or a constant.

Types of Values

There are three types of values that can be used in a condition:

  • Entity attribute: This refers to an attribute of the entity being accessed, or to an attribute of an entity that has a relation to the entity being accessed. For example, if you have a policy that controls access to a report, you could use the report’s confidentiality level as a value. If you have a policy that controls access to an employee’s information, you can use the employee’s relation to their department to ensure that only people in a particular department are visible.
  • User attribute: This refers to a characteristic of the user who is trying to access the resource. For example, if you have a policy that controls access to files private to a particular department, you can verify that the user is a member of the department to which the file belongs.
  • Constant: This is a fixed value that you define in the condition. It can be of subtype boolean, number, or string. It is useful as the right value side of a comparison where the left value is an entity attribute or user attribute. For example, if you have a policy that says admins can see everything, you could use a constant value of true as the right value in a condition where the left value is user.is_admin.
Some examples
  • Allowing access to users of certain department: user.department equals "Sales"
  • Allow access when attribute category has a certain value: entity.category equals "Published"
  • Allow access where attribute vat matches the user attribute: entity.vat equals user.vat
  • Following the relation belongs_to to party, using the attribute department on party: entity.belongs_to.department equals "Bookkeeping"

Types of comparison operators

These are the comparison operators that you can use in a condition:

  • equals (=)
    • Used for any type of value, to check that the left value is equal to the right value
  • not equals (≠)
    • Used for any type of value, to check that the left value is different from the right value
  • greater than (>)
    • Used for numeric types or date types, to check that the left value is greater (or later in time) than the right
  • greater or equals (≥)
    • Used for numeric types or date types, to check that the left value is greater or equal (or later in time) than the right
  • less than (<)
    • Used for numeric types or date types, to check that the left value is smaller (or earlier in time) than the right
  • less than or equals (≤)
    • Used for numeric types or date types, to check that the left value is smaller or equal (or earlier in time) than the right
  • contains (∋)
    • Used with a collection as the type of the left value, to check that the right value appears in the collection
  • in (∈)
    • Used with a collection as the type of the right value, to check that the left value appears in the collection

Application API

Reference documentation for the ContentGrid Application REST API.

Resource Types

This document describes the core resource types in the REST API.

Links express a relation between the resource they appear on (the link context), and the link target. Link Relation Types determine the kind of relation between the link context and the link target.

Problem Types

Problem Details are used to provide additional information about the error condition. This reference describes all Problem Detail types that can be returned by the API.

HAL-FORMS Extensions

ContentGrid extensions to the HAL-FORMS specification

Entity Profile: HAL format

The HAL profile format describes the model of an entity type, its attributes and relations.

Subsections of Application API

Resource Types

This document describes the core resource types in a ContentGrid application.

Overview

A ContentGrid application can be configured with kinds of entities that have attributes and relations.

A single instance of such an entity is generically referred to as an entity-item. Multiple entity-items of the same kind together form an entity-collection.

These two form the main resource types in a ContentGrid application. To support these resource types, a couple of additional resource types exist.

entity-item

An entity-item is a single, concrete instance of a certain entity.

Attributes are exposed as fields at the top level of the JSON object. Additionally, an id field is exposed, containing the unique id for the entity-item.

Some attribute types (like content) are exposed as a nested object, which contains the information stored in that attribute.

Like any HAL object, entity-item also has _links and _templates fields.

Example entity-item

This is an example of an invoice with a file in the document attribute.

{
  "id": "3211be1d-1ed1-4850-8ea6-3fa3218031f6",
  "received": "2024-07-15",
  "document": {
     "size": 123456,
     "mimetype": "application/pdf",
     "filename": "example-invoice.pdf"
  },
  "pay_before": "2024-08-14",
  "total_amount": 15.95,
  "_links": {
    [...]
  },
  "_templates": {
    [...]
  }
}

entity-collection

An entity-collection is an (ordered) set of entity-item resources.

Due to the usage of HAL, entity-items are available in a nested JSON object. The list of entity-items will always be located in _embedded.item.

All entity-items in a collection are always of the same type and expose the same attributes. The entity-items may be a reduced representation of a full entity-item resource, where some fields are omitted. The id field, which is used to address the individual entity will always be available.

Example entity-collection

This is an example of a collection of invoices

{
  "_embedded": {
    "item": [
      {
        "id": "3211be1d-1ed1-4850-8ea6-3fa3218031f6",
        "received": "2024-07-15",
        "document": {
            "size": 123456,
            "mimetype": "application/pdf",
            "filename": "example-invoice.pdf"
        },
        "pay_before": "2024-08-14",
        "total_amount": 15.95,
        "_links": {
            [...]
        },
        "_templates": {
          [...]
        }
      },
      {
        "id": "56fe1819-3f70-476c-937d-b400b660ee81",
        "received": "2020-01-01",
        "document": null,
        "pay_before": "2020-02-01",
        "total_amount": 123.4,
        "_links": {
            [...]
        },
        "_templates": {
          [...]
        }
      },
      [...]
    ]
  },
  "_links": {
    [...]
  },
  "page": {
    "size": 20,
    "next_cursor": "1wskufc1",
    "total_items_estimate": 38,
    "total_items_exact": 38
  }
}

Collection meta-information

The entity-collection contains meta-information about itself in the top-level page field:

Field Type Description
size integer, >= 1 Page size; the maximum number of entity-item resources returned on a single page
prev_cursor string or absent Cursor to retrieve the previous page. Absent when there is no previous page
next_cursor string or absent Cursor to retrieve the next page. Absent when there is no next page
total_items_estimate number, >= 0 Estimated total number of items in the collection. Also present when an exact count exists
total_items_exact number, >= 0 or absent Exact total number of items in the collection. Absent when calculation is too resource-intensive to execute

relation

The relation resource represents a relation between two entities. When reading, its format is dependent on whether it is a to-one or a to-many relation:

  • To-one relation (one-to-one, many-to-one): Reading returns a redirect to the linked entity-item
  • To-many relation (one-to-many, many-to-many): Reading returns a redirect to an entity-collection that contains all linked entity-items

relation-item

The relation-item resource represents a single item inside a to-many relation.

Reading returns a redirect to the entity-item, if it is linked in the relation.

entity-content

The entity-content resource represents the file contents that are stored on an entity attribute of type Content.

This resource contains arbitrary file contents, so no assumptions should be made about its format.

entities-root

The entities-root resource is the parent resource of all entity-collection resources.

It contains metadata about the application, in particular the list of all available entity types (in HAL format).

curl -i https://$APP_ID.$REGION.contentgrid.cloud/   \
    -H "Authorization: Bearer $TOKEN"
GET / HTTP/1.1
Authorization: Bearer $TOKEN
HTTP/1.1 OK
Content-Type: application/hal+json

{
  "_links": {
    "cg:entity": [
      {
        "href": "/invoices",
        "name": "invoice"
        [...]
      },
      {
        "href": "/suppliers",
        "name": "supplier"
        [...]
      }
    ],
    [...]
  }
}

entity-profile

The entity-profile resource is a computer-readable description of an entity type.

It describes the entity’s attributes, relations, constraints, and available operations. The full format is documented in the Entity Profiles reference.

profile-root

The profile-root resource is the parent resource of all entity-profile resources.

It contains metadata about the profiles, in particular the list of all available entity-profile types (in HAL format).

Link Relation Types

Links express a relation between the resource they appear on (the link context), and the link target. The link relation type (RFC8288 Sec 3.3) determines the kind of relation between the link context and the link target.

Next to the standard link relation types, which are to be interpreted with their standardized meaning, ContentGrid defines following extension link relation types:

Link relation type Description
https://contentgrid.cloud/rels/contentgrid/relation Refers to a ContentGrid relation of the entity-item. The link’s name attribute is the relation name
https://contentgrid.cloud/rels/contentgrid/content Refers to the binary content of a content attribute. The link’s name attribute is the content attribute name
https://contentgrid.cloud/rels/contentgrid/entity Refers to a ContentGrid entity (from a root resource entities-root and profile-root). The link’s name attribute is the entity name

CURIEs

CURIEs (Compact URIs) are a shorthand notation for extension link relation types.

They are used as defined in the HAL specification. The curies link relation provides expansion templates.

Before comparing a link relation, a CURIE must be expanded into the full URI.

Formal definition of CURIE expansion

Since the HAL specification does not formally specify how CURIEs should be expanded, a definition is given here.

  1. A safe_curie or a curie MAY be used as an extension link relation type. They MUST be expanded to a URI to obtain the formal extension link relation type and SHOULD NOT be used as-is.
  2. A CURIE consists of a prefix and a reference, separated by a colon (:). In the context of HAL; the CURIE MUST have a prefix, and the reference MUST also be an NCName.
    safe_curie := '[' curie ']'
    curie := prefix ':' reference
    prefix := NCName
    reference := NCName
  3. The curies links:
    1. MUST have a name link attribute, which is the CURIE prefix.
    2. MUST be a URI template, which MUST end in a variable named rel. (e.g.: http://example.com/relations/{rel}) The CURIE is expanded by expanding the URI template with the rel variable set to the CURIE reference.
  4. Expansion of a CURIE to a URI begins at the HAL Resource object that contains the particular link relation
    1. If a curies link relation is present with its name attribute equal the CURIE prefix, this link relation is used for expanding the CURIE to an URL
    2. Else, the parent HAL Resource object is consulted using the same algorithm.
    3. If the top-level HAL Resource object was reached without finding a link relation for the CURIE, expansion has failed. An error SHOULD be reported, and the link relation MUST be ignored for all purposes.
A HAL resource using CURIEs
{
  [...]
  "_links": {
    "self": {
      "href": "https://example.com/users/123"
    },
    "cg:content": [
      {
        "name": "profile_picture",
        "href": "https://example.com/users/123/picture"
      }
    ],
    "https://contentgrid.cloud/rels/contentgrid/relation": [
      {
        "name": "friends",
        "href": "https://example.com/users/123/friends"
      }
    ],
    "[example:curied-link]": {
      "href": "https://example.com/dahl-curry"
    },
    "unregistered:curie": {
      "href": "https://example.com/unregistered-curie"
    },
    "curies": [
      {
        "name": "cg",
        "href": "https://contentgrid.cloud/rels/contentgrid/{rel}",
        "template": true
      },
      {
        "name": "example",
        "href": "https://example.com/relations/{rel}",
        "template": true
      }
    ]
  }
}

This is equivalent to the following resource when CURIEs are expanded:

{
  [...]
  "_links": {
    "self": {
      "href": "https://example.com/users/123"
    },
    "https://contentgrid.cloud/rels/contentgrid/content": [
      {
        "name": "profile_picture",
        "href": "https://example.com/users/123/picture"
      }
    ],
    "https://contentgrid.cloud/rels/contentgrid/relation": [
      {
        "name": "friends",
        "href": "https://example.com/users/123/friends"
      }
    ],
    "https://example.com/relations/curied-link": {
      "href": "https://example.com/dahl-curry"
    }
  }
}
  1. IANA-registered link relation types and URI-based extension link relatinos are passed through as-is.
  2. CURIEs and Safe CURIEs are expanded based on the URI templates of the curies links
  3. CURIEs using unknown prefixes are completely ignored

Problem Types

The ContentGrid REST API uses the standard HTTP status codes to signal errors.

RFC9457 Problem Details are used to provide additional information about the error condition.

This reference describes all Problem Detail types that can be returned by the ContentGrid Application REST API.

Unless specified otherwise, the HTTP Status code for problems will be 400 Bad Request.

Input Validation Problems

These problems occur when the data you provide in the request body doesn’t meet the requirements.

Input validation problems are always represented by the a Problem Detail with type https://contentgrid.cloud/problems/input/validation.

input/validation

https://contentgrid.cloud/problems/input/validation

The ProblemDetail contains an additional property, errors, which contains a list of validation errors.

Each validation error has the same structure as a ProblemDetail, with as type one of the subtypes listed below. Each validation error also has a field property, which refers to the specific input field for which validation failed with a property path.

Example
{
  "type": "https://contentgrid.cloud/problems/input/validation",
  "title": "Validation error",
  "detail": "2 validation errors",
  "status": 400,
  "errors": [
    {
      "type": "https://contentgrid.cloud/problems/input/validation/required",
      "title": "Mandatory field",
      "detail": "A value must be present, but it is missing or empty",
      "field": "pay_before"
    },
    {
      "type": "https://contentgrid.cloud/problems/input/validation/type",
      "title": "Invalid data type",
      "detail": "Expected value of type date, but got decimal",
      "field": "received",
      "expected_type": "date",
      "actual_type": "decimal"
    }
  ]
}

input/validation/type

https://contentgrid.cloud/problems/input/validation/type

The value provided has the wrong data type.

A data type error can happen when handling a JSON or a multipart form body.

For example:

  • sending a decimal instead of a long
  • sending a JSON array instead of a single value
  • sending a JSON object instead of a single value
  • repeating the same multipart form field multiple times instead of once
  • sending a multipart file instead of a normal form field

Additional Properties:

  • expected_type (string): The technical name of the expected data type
  • actual_type (string): The technical name of the actual data type received

input/validation/type/format

https://contentgrid.cloud/problems/input/validation/type/format

The value is of the correct type, but it doesn’t match the expected format.

In this case the type is correct (usually a string), but the type (e.g. date, timestamp, relation link) requires a specific format, which was not fulfilled.

For example:

  • sending an arbitrary string instead of a date or timestamp in RFC3339 format
  • sending a link to an entity that is of the wrong type for this relation

Additional Properties:

  • expected_type (string): The expected data type
  • format_error (string): Description of the format error

input/validation/no-content

https://contentgrid.cloud/problems/input/validation/no-content

A content attribute can not be set when there is no content present.

This happens when writing to nested fields in a content attribute, but there is no content present.

It is only valid to change content fields (e.g. set filename or mimetype) when an actual file is present.

input/validation/required

https://contentgrid.cloud/problems/input/validation/required

A required field is set to null.

Note

When using PUT to update an entity, all attributes that are not present in the request body are regarded to be null (including attributes of type content). Use the PATCH method if you only want to update some fields.

input/validation/duplicate

https://contentgrid.cloud/problems/input/validation/duplicate

The value provided for an attribute must be unique, but an other entity item already uses this value.

Additional Properties:

  • conflicting_item (string): URL of the entity that already has this value

input/validation/allowed-values

https://contentgrid.cloud/problems/input/validation/allowed-values

The value is not one of the allowed values.

The possible values for an attribute can be restricted to a list of allowed values. It is then not allowed to use any other value.

Additional Properties:

  • allowed_values (list): A list of all the values that are allowed for the field

input/validation/missing-relation-target

https://contentgrid.cloud/problems/input/validation/missing-relation-target

An entity-item that is being linked to a relation does not exist.

Additional Properties:

  • missing_item (string): URL of the entity-item that does not exist

Query Parameter Problems

These problems occur when request query parameters don’t meet the requirements.

Contrary to input validation, only a single problem is reported at a time.

invalid-query-parameter/filter/format

https://contentgrid.cloud/problems/invalid-query-parameter/filter/format

Query parameter values are always strings. For some operations, they need to be converted to a certain type (e.g. to a number for a greater-than filter, or to a timestamp for a before filter).

If the value can not be converted, this problem is reported

Additional Properties:

  • query_parameter (string): The name of the query parameter
  • expected_type (string): The expected data type
  • format_error (string): Description of the format error
  • additional_errors (array): List of additional filter format errors, when multiple occur at the same time.

invalid-query-parameter/sort/format

https://contentgrid.cloud/problems/invalid-query-parameter/sort/format

The _sort query parameter is malformed and doesn’t match the expected format.

The expected format for sorting is <attribute_name>,asc or <attribute_name>,desc for ascending and descending sorting. Other formats are invalid.

Additional Properties:

  • query_parameter (string): The name of the query parameter (always _sort)
  • format_error (string): Description of the format error

invalid-query-parameter/sort/target

https://contentgrid.cloud/problems/invalid-query-parameter/sort/target

The _sort query parameter references a sort target that doesn’t exist, or on which sorting is not possible.

Sorting is only possible on attributes that are indexed in a way that supports sorting.

Additional Properties:

  • query_parameter (string): The name of the query parameter (always _sort)
  • target_name (string): The sort target name which couldn’t be sorted on

invalid-query-parameter/pagination

https://contentgrid.cloud/problems/invalid-query-parameter/pagination

A pagination-related query parameter (_cursor or _limit) is invalid.

  • The _limit field value must be 1-1000.
  • The _cursor field value should never be manually constructed; use the next_cursor and prev_cursor values from API responses (or follow the HAL links).

Additional Properties:

  • query_parameter (string): The name of the query parameter
  • format_error (string): Description of the format error

General request problems

These problems occur when there is something wrong with the request in general, in ways that are normally never encountered when integrating with ContentGrid.

They are still listed here for reference.

invalid-request/body

https://contentgrid.cloud/problems/invalid-request/body

The request body is malformed and can not be parsed.

Note that depending on the content-type, a more specific problem detail may be returned (e.g. for json or uri-list formats)

invalid-request/body/json

https://contentgrid.cloud/problems/invalid-request/body/json

The request body contains malformed JSON that can not be parsed.

invalid-request/body/uri-list

https://contentgrid.cloud/problems/invalid-request/body/uri-list

The request body contains a malformed URI list that can not be parsed.

https://contentgrid.cloud/problems/invalid-request/body/single-link

The request body does not have exactly one link.

This problem occurs when sending a request containing multiple (or zero) links to a to-one relation. A PUT on a to-one relation must contain exactly one link.

invalid-request/required-header

https://contentgrid.cloud/problems/invalid-request/required-header

A required HTTP header is missing from the request.

Additional Properties:

  • header (string): The name of the header that is missing

invalid-request/forbidden-header

https://contentgrid.cloud/problems/invalid-request/forbidden-header

A HTTP Header is present that is not allowed

Additional Properties:

  • header (string): The name of the header that is not allowed

invalid-request/invalid-header

https://contentgrid.cloud/problems/invalid-request/invalid-header

A HTTP Header has a value that is not syntatically valid

Additional Properties:

  • header (string): The name of the header that is not valid

Version Conflict Problems

unsatisfied-version

https://contentgrid.cloud/problems/unsatisfied-version

The entity version specified in the If-Match and/or If-None-Match headers doesn’t match the current version.

This error can happen when using conditional requests, and the entity is modified by another request between get and update.

HTTP Status: 412 Precondition Failed

Additional Properties:

  • actual_version (string): The current version of the entity

Not Found Problems

These problems have HTTP status code 404 Not Found.

Different problem types are used to help distinguish between typos in URLs and entity items that may not be present anymore.

not-found/endpoint

https://contentgrid.cloud/problems/not-found/endpoint

The requested API endpoint does not exist.

This problem type indicates an error in the fixed parts of the URL: in the entity, relation or attribute name.

Check the URL for typos. Ensure you’re using the correct entity type name and that the entity type exists in your data model. If the URL used to work, check that the entity/attribute/relation definition was not renamed or deleted in a newer version of the data model.

not-found/entity-item

https://contentgrid.cloud/problems/not-found/entity-item

No entity was found with the specified id. The entity may have been deleted, or may have never existed.

not-found/relation-item

https://contentgrid.cloud/problems/not-found/relation-item

The relation does not contain the requested entity-item.

This can happen for all relation types:

  • to-one relations: When nothing is linked, a GET or DELETE on the relation resource will result in this error
  • to-many relations: When a specific item is not linked, a GET or DELETE on the relation-item resource will result in this error

Integrity Problems

These problems occur when an operation would violate data integrity constraints.

Unless specified otherwise, integrity problems will use the 409 Conflict HTTP status code.

integrity/blind-relation-overwrite

https://contentgrid.cloud/problems/integrity/blind-relation-overwrite

In a to-one relation, the target entity is already referenced by another entity. The operation would silently overwrite that reference without ability to apply proper concurrency control.

Additional Properties:

  • new_item (string): URL of the entity you’re trying to link
  • new_relation (string): URL of the relation you’re trying to set
  • existing_item (string): URL of the entity currently linked
  • existing_relation (string): URL of the existing relation
  • target_item (string): URL of the target entity
  • target_relation (string): URL of the inverse relation (if available)
  • additional_errors (array): List of additional blind relation overwrite errors, when multiple occur at the same time.

How to Fix: This is a safety mechanism to prevent data loss. You have several options:

  1. Unlink the existing relation first (DELETE on existing_relation), then create the new link
  2. Use the target entity’s relation endpoint to manage the link, if available
  3. Review your data model - you may need a to-many relation instead of to-one

integrity/required-relation

https://contentgrid.cloud/problems/integrity/required-relation

This problem indicates that the entity-item is referenced in a required relation

It can happen when:

  1. trying to delete the entity-item, and it is referenced by any required relation
  2. trying to unlink the entity-item from a relation, and the other side of the relation is required.

Additional Properties:

  • affected_relation (string): URL of the relation that requires this entity

How to Fix: Before deleting or unlinking the entity, either:

  1. Delete the referencing entity first
  2. Update the referencing entity to link to a different entity
  3. If the relation should be optional in your data model, update your model definition

HAL-FORMS Extensions

The HAL-FORMS specification defines request body encodings for application/json and application/x-www-form-urlencoded.

ContentGrid extends the HAL-FORMS specification:

  • to allow construction of nested JSON objects for application/json request body encoding
  • to define request body encodings for multipart/form-data and text/uri-list
  • the options element is extended to define the behavior of a link with a mediatype of application/hal+json and application/prs.hal-forms+json

Body encoding for application/json

To encode a nested JSON object, the property name will be interpreted as a property path.

Properties for which no value is present are not taken into account for serialization purposes.

Nested JSON encoding

Given this HAL-FORMS template:

{
  "contentType": "application/json",
  "properties": [
    {
      "name": "received",
      "prompt": "Received",
      "required": true,
      "type": "datetime"
    },
    {
      "name": "document.mimetype",
      "prompt": "Document mimetype",
      "type": "text"
    },
    {
      "name": "document.filename",
      "prompt": "Document filename",
      "type": "text"
    },
    {
      "name": "pay_before",
      "prompt": "Pay before",
      "required": true,
      "type": "datetime"
    },
    {
      "name": "total_amount",
      "prompt": "Total amount",
      "required": true,
      "type": "number"
    }
  ]
}

The request body is serialized as:

{
  "received": "2024-05-08T14:58:23Z",
  "document": {
      "mimetype": "text/plain",
      "filename": "example.txt"
  },
  "pay_before": "2024-06-08T00:00:00Z",
  "total_amount": 14.58
}

Body encoding for multipart/form-data

When sending bodies encoded as multipart/form-data, bodies SHOULD be encoded according to the WhatWG HTML standard.

Body encoded for text/uri-list

When sending bodies encoded as text/uri-list, the HAL-FORMS template MUST only contain a single property of type url.

Only the value (or values if the property is multi-value) of the property is part of the request body, in accordance to the text/uri-list mime type.

Multi-valued properties are serialized by having each value on a separate line.

uri-list encoding

Given this HAL-FORMS template:

{
  "contentType": "text/uri-list",
  "properties": [
    {
      "name": "supplier",
      "type": "url"
    }
  ]
}

The request body is serialized as:

http://example.com/example-path?q=abc

Extension for options element

The options element contains an enumerated list of possible values for a property.

When the options element contains a link attribute, the possible values are to be retrieved from a remote HTTP resource.

The link attribute has following child-attributes, with the same meaning as the HAL Link Object:

  • href
  • type
  • templated
  • profile

The behavior of the options element promptField and valueField attributes are dependent on the content-type of the remote HTTP resource. The type attribute may be used as a hint for the content-type of the remote HTTP resource, but the actual content-type of the response is authoritative.

Content-Type of remote resource list of values taken from Interpretation of promptField / valueField
application/json items in the top-level array element name of the JSON field to use for prompt/value of the element (default prompt/value)
text/csv rows in the CSV document 0-indexed column number of the row to use for prompt/value of the element (default 0/1)
application/hal+json or application/prs.hal-forms+json embedded resources (_embedded) with link relation type item. Pagination using prev/next links may be used. JSON pointer to the JSON value to use for prompt/value of the element (default empty pointer//_links/self/href)
Example

Given a HAL-FORMS property:

{
  "name": "supplier",
  "options": {
    "link": {
      "href": "https://app.contentgrid.example/suppliers"
    }
  }
}

The following responses by the remote HTTP resources are possible, which all result in a very similar way of displaying.

Content-Type Simple Array Key/Value Array

application/json

["Fedex","UPS","DHL"]
[
  {"prompt" : "Federal Express", "value" : "FedEx"},
  {"prompt" : "United Parcel Service", "value" : "UPS"},
  {"prompt" : "DHL Express", "value" : "DHL"}
]

text/csv

Fedex
UPS
DHL
Federal Express,FedEx
United Parcel Service,UPS
DHL Express,DHL

application/hal+json or application/prs.hal-forms+json

N/A

{
  "_embedded": {
    "item": [
      {
        "name": "Federal Express",
        "_links": {
          "self": {
            "href": "https://app.contentgrid.example/suppliers/FedEx"
          }
        }
      },
      {
        "name": "United Parcel Service",
        "_links": {
          "self": {
            "href": "https://app.contentgrid.example/suppliers/UPS"
          }
        }
      }
    ]
  },
  "_links": {
    "next": {
      "href":  "https:/app.contentgrid.example/suppliers?page=2"
    }
  }
}

with on the next page:

{
  "_embedded": {
    "item": [
      {
        "name": "DHL Express",
        "_links": {
          "self": {
            "href": "https://app.contentgrid.example/suppliers/DHL"
          }
        }
      }
    ]
  },
  "_links": {
    "prev": {
      "href":  "https:/app.contentgrid.example/suppliers?page=1"
    }
  }
}
Note

For the HAL-based resource, the resolved prompt value would be the full item object. It is up to the implementation how to display this object. If this is unwanted, promptField can be set to a JSON pointer to select a specific field as a prompt. (e.g. "promptField": "/name")

Property path

A property path is used to reference fields in nested JSON objects.

The property path is built by separating JSON fields with .. e.g. document.filename will be serialized as a top-level field document which is assigned to an object with a filename field.

Entity Profile: HAL format

The HAL profile format describes the model of an entity type, its attributes and relations.

It also provides the search and create-form HAL-FORMS templates, which describe how to search and create new items of this type.

The entity profile uses standard link relation types and ContentGrid-specific extension link relation types. See the Link Relation Types reference for general information on link relation types and CURIEs.

The entity profile introduces the blueprint CURIE prefix (https://contentgrid.cloud/rels/blueprint/{rel}) for the following link relation types:

Link relation type Context Description
blueprint:attribute entity An attribute defined on the entity
blueprint:relation entity A relation defined on the entity
blueprint:attribute attribute An attribute that is nested inside an other attribute (nested object)
blueprint:constraint attribute A constraint on the attribute
blueprint:search-param attribute A search parameter for the attribute
blueprint:target-entity relation Points to the entity profile of the relation’s target entity type

Entity Profile Fields

The top-level entity profile resource contains:

Field Type Description
name string Internal entity name
title string Human-readable entity name (singular)
description string or null Description of the entity

The entity profile also contains embedded HAL resources:

  • blueprint:attribute: A list of the attributes that are present on the entity
  • blueprint:relation: A list of the relations that are present on the entity

The entity profile contains HAL links:

Link relation type name attribute Description
self N/A Links to this profile
describes collection Links to the entity collection URL
describes item URI template for linking to individual entity items
curies all See CURIEs
Entity profile overview
curl -i https://$APP_ID.$REGION.contentgrid.cloud/profile/invoices   \
    -H 'Accept: application/prs.hal-forms+json'   \
    -H "Authorization: Bearer $TOKEN"
GET /profile/invoices HTTP/1.1
Authorization: Bearer $TOKEN
Accept: application/prs.hal-forms+json
HTTP/1.1 200 OK
Content-Type: application/prs.hal-forms+json

{
  "name": "invoice",
  "title": "Invoice",
  "description": null,
  "_embedded": {
    "blueprint:attribute": [...],
    "blueprint:relation": [...]
  },
  "_links": {
    "self": {
      "href": "https://app.contentgrid.example/profile/invoices",
      "title": "Invoice"
    },
    "describes": [
      {
        "href": "https://app.contentgrid.example/invoices",
        "title": "Invoices",
        "profile": "https://app.contentgrid.example/profile/invoices",
        "name": "collection"
      },
      {
        "href": "https://app.contentgrid.example/invoices/{id}",
        "title": "Invoice",
        "name": "item",
        "templated": true
      }
    ],
    "curies": [
      {
        "href": "https://contentgrid.cloud/rels/contentgrid/{rel}",
        "name": "cg",
        "templated": true
      },
      {
        "href": "https://contentgrid.cloud/rels/blueprint/{rel}",
        "name": "blueprint",
        "templated": true
      }
    ]
  },
  "_templates": {
    "search": {...},
    "create-form": {...}
  }
}

Embedded Resources

The entity profile contains embedded resources that describe the entity’s attributes and relations.

Attribute (blueprint:attribute)

Entity attributes are embedded resources in the entity profile under the link relation blueprint:attribute.

Field Type Description
name string Attribute name
title string Human-readable title
type string Data type: string, long, double, boolean, date, datetime, object
description string or null Attribute description
readOnly boolean Whether the attribute is read-only
required boolean Whether the attribute is required

Each attribute itself contains embedded resources, describing details about the attribute:

Simple attribute
{
  "name": "received",
  "title": "Received",
  "type": "date",
  "description": null,
  "readOnly": false,
  "required": true,
  "_embedded": {
    "blueprint:constraint": [
      {
        "type": "required"
      }
    ],
  }
}

Sub-attributes

Attributes with type: "object" (such as content attributes) can contain nested blueprint:attribute sub-attributes. Sub-attributes have the same structure as top-level attributes, recursively.

Content attribute with sub-attributes
{
  "name": "document",
  "title": "Document",
  "type": "object",
  "description": null,
  "readOnly": false,
  "required": false,
  "_embedded": {
    "blueprint:attribute": [
      {
        "name": "filename",
        "title": "Filename",
        "type": "string",
        "description": null,
        "readOnly": false,
        "required": false
      },
      {
        "name": "mimetype",
        "title": "Mimetype",
        "type": "string",
        "description": "Technical indicator of the type of the file",
        "readOnly": false,
        "required": false
      },
      {
        "name": "length",
        "title": "Size",
        "type": "long",
        "description": "File size in bytes",
        "readOnly": true,
        "required": false
      }
    ]
  }
}

Constraint (blueprint:constraint)

Constraints are embedded in an attribute under the link relation blueprint:constraint.

Field Applies to Type Description
type all string Constraint type; see below
values allowed-values array List of allowed values for the attribute

Following constraint types are supported:

Constraint type Description
required The attribute must have a value and cannot be null.
unique The attribute value must be unique across all entities of this type.
allowed-values The attribute value must be one of a predefined set of allowed values.
created-date The attribute is managed by the system and contains the creation timestamp.
created-by The attribute is managed by the system and contains the creator’s identity.
modified-date The attribute is managed by the system and contains the last modified timestamp.
modified-by The attribute is managed by the system and contains the last modifier’s identity.

Search Parameter (blueprint:search-param)

Search parameters are embedded in an attribute under the link relation blueprint:search-param. They describe the query parameters that can be used to filter on this attribute in the search template.

Field Type Description
name string Query parameter name
title string Human-readable title
type string Search type; see below

Following search parameter types are supported:

Search parameter type Description
exact-match Match the attribute value exactly (case-sensitive string comparison).
prefix-match Match the attribute value by prefix.
greater-than Match values greater than the specified value.
less-than Match values less than the specified value.
greater-than-or-equal Match values greater than or equal to the specified value.
less-than-or-equal Match values less than or equal to the specified value.
full-text Perform a full-text search on the attribute value.
Attribute with search parameters
{
  "name": "pay_before",
  "title": "Pay before",
  "type": "date",
  "description": null,
  "readOnly": false,
  "required": true,
  "_embedded": {
    "blueprint:search-param": [
      {
        "name": "pay_before",
        "title": "Pay before",
        "type": "exact-match"
      },
      {
        "name": "pay_before~after",
        "title": "Pay before: After",
        "type": "greater-than"
      },
      [...]
    ]
}

Relation (blueprint:relation)

Relations are embedded in the entity profile under blueprint:relation.

Field Type Description
name string Relation name
title string Human-readable title
description string or null Relation description
many_source_per_target boolean Multiple source entities can reference the same target
many_target_per_source boolean Source can reference multiple targets
required boolean Whether the relation is required

Each relation has a blueprint:target-entity link, referencing to the profile of the target entity type.

Relation
{
  "name": "supplier",
  "title": "Supplier",
  "description": null,
  "many_source_per_target": true,
  "many_target_per_source": false,
  "required": false,
  "_links": {
    "blueprint:target-entity": {
      "href": "https://app.contentgrid.example/profile/suppliers",
      "title": "Supplier"
    }
  }
}

Templates

The entity profile provides HAL-FORMS templates that describe how to search and create entities of this type. See HAL-FORMS extensions for additional details on the extensions used.

search Template

The search template describes how to filter and sort the entity collection.

Search template properties are composed from 3 sources:

  • search parameters defined on attributes
  • Cross relation search parameters, where the search parameters from the relation target entity are prefixed by the relation name and a dot (e.g. supplier.name)
  • Special properties: _sort

The _sort property is a special property with a list of options. Next to the standard options as defined in HAL-FORMS, each option has property and direction fields. These additional fields can be used to programmatically select an option for a particular attribute and a particular sorting direction without having to resort to string manipulation.

search template
{
  "_templates": {
    "search": {
      "method": "GET",
      "target": "https://app.contentgrid.example/invoices",
      "properties": [
        {
          "name": "pay_before",
          "prompt": "Pay before",
          "type": "date"
        },
        [...]
        {
          "name": "supplier.name",
          "prompt": "Supplier: Name",
          "type": "text"
        },
        {
          "name": "_sort",
          "prompt": "Sort",
          "type": "text",
          "options": {
            "minItems": 0,
            "promptField": "prompt",
            "valueField": "value",
            "inline": [
              {
                "property": "pay_before",
                "direction": "asc",
                "prompt": "Pay before ascending",
                "value": "pay_before,asc"
              },
              {
                "property": "pay_before",
                "direction": "desc",
                "prompt": "Pay before descending",
                "value": "pay_before,desc"
              },
              {
                "property": "total_amount",
                "direction": "asc",
                "prompt": "Total amount 0\u21929",
                "value": "total_amount,asc"
              },
              {
                "property": "total_amount",
                "direction": "desc",
                "prompt": "Total amount 9\u21920",
                "value": "total_amount,desc"
              }
            ]
          }
        }
      ]
    }
  }
}

create-form Template

The create-form template describes how to create a new entity of this type.

Depending on whether content attributes are present, the form can use application/json or multipart/form-data content types.

The create form includes fields for both attributes and relations, following the HAL-FORMS extensions specification.

create-form template
{
  "_templates": {
    "create-form": {
      "method": "POST",
      "target": "https://app.contentgrid.example/invoices",
      "contentType": "multipart/form-data",
      "properties": [
        {
          "name": "received",
          "prompt": "Received",
          "required": true,
          "type": "date"
        },
        {
          "name": "document",
          "prompt": "Document",
          "type": "file"
        },
        {
          "name": "pay_before",
          "prompt": "Pay before",
          "required": true,
          "type": "date"
        },
        {
          "name": "total_amount",
          "prompt": "Total amount",
          "required": true,
          "type": "number"
        },
        {
          "name": "supplier",
          "prompt": "Supplier",
          "type": "url",
          "options": {
            "link": {
              "href": "https://app.contentgrid.example/suppliers",
              "title": "Suppliers"
            },
            "minItems": 0,
            "maxItems": 1,
            "valueField": "/_links/self/href"
          }
        }
      ]
    }
  }
}

Architecture

This section provides technical documentation about the ContentGrid Runtime platform architecture, designed for architects and technical decision-makers evaluating the platform.

What You’ll Learn

The ContentGrid platform is built on a foundation of well-considered architectural principles that prioritize developer experience, scalability, and extensibility. This section covers:

Key Architectural Characteristics

Model-First & API-First: ContentGrid generates complete REST APIs directly from your data model, exposing intuitive URL structures you’d expect from hand-crafted applications.

Small Core, Large Ecosystem: The platform provides essential ECM functionality in its core while enabling extensive customization through external automations that integrate seamlessly via standard protocols.

Standards-Based: Built on established standards (HAL, HAL-FORMS, RFC 9110) to reduce learning curves and leverage existing tooling.

Kubernetes-Native: Designed for cloud-native deployment with dynamic service discovery and modern DevOps practices.

Target Audience

This documentation is written for technical prospects who need to understand:

  • How ContentGrid applications are architected and deployed
  • The trade-offs and decisions behind the platform design
  • How the platform integrates with existing infrastructure
  • Security and data protection approaches

If you’re looking for implementation guides or API references, see the Guides and Reference sections.

Subsections of Architecture

System Overview

ContentGrid is a model- and API-first Enterprise Content Management (ECM) platform that generates complete, standards-based REST APIs directly from your data model. The platform consists of two primary environments: a Management Platform for application configuration and a Runtime Platform for execution.

What Makes ContentGrid Different

Traditional ECM systems often rely on folder hierarchies, generic database schemas, and monolithic architectures. ContentGrid takes a fundamentally different approach:

Model-First Design: Define your data model with entities, attributes, and relations. ContentGrid automatically generates intuitive REST APIs that mirror your domain model—URLs like /invoices and /invoices/{id} emerge naturally from your schema.

Relational Database Foundation: Unlike ECMs that use generic key-value storage, ContentGrid leverages PostgreSQL’s full capabilities by mapping entities directly to tables and columns. This enables efficient queries, proper indexes, and standard database tooling.

Attribute-Based Access Control: Instead of folder-based permissions, ContentGrid uses ABAC policies that evaluate entity and user attributes. This provides flexible, fine-grained access control independent of data organization.

Small Core, Extensible Ecosystem: The platform core provides only essential ECM functionality. Everything else is implemented as external automations that integrate seamlessly through standard protocols.

Architecture at a Glance

graph TB

    subgraph "Runtime Platform"
        Gateway[Gateway<br/>Routing & Authorization]
        Keycloak[Keycloak<br/>Authentication]
        OPA[OPA<br/>Policy Evaluation]
        Solon[Solon<br/>Policy Collection]
        Navigator[Navigator<br/>Web UI]

        subgraph "Application Instance"
            AppServer[Application Server<br/>Dynamic API]
        end

        subgraph "Persistent Storage"
            DB[(PostgreSQL<br/>Metadata & Structure)]
            S3[(S3-Compatible<br/>Content Storage)]
        end
    end

    Gateway --> OPA
    Gateway --> AppServer
    AppServer --> Solon
    Solon --> OPA
    AppServer --> DB
    AppServer --> S3
    Users[Clients] --> Keycloak
    Users --> Gateway
    Users --> Navigator

    MP[Management Platform]
    MP -->|Configure| Gateway
    MP -->|Configure| AppServer

Management Platform

The Management Platform is where you define, configure, and deploy ContentGrid applications. It consists of three primary components:

  • Architect: Source of truth for application models (entities, permissions, configurations)
  • Scribe: Transforms models into deployment artifacts (application model, database migrations, policies, OpenAPI specs)
  • Captain: Orchestrates infrastructure provisioning and Kubernetes deployment

The platform runs as a SaaS service—you don’t need to host or manage it yourself. For complete details on how these components work together to enable the full application lifecycle, see Management Platform.

Runtime Platform

The Runtime Platform provides the infrastructure and services to run ContentGrid applications securely and efficiently. Key components include:

  • Gateway: Entry point for all requests, handles routing, authentication, and policy evaluation
  • Keycloak: Identity and access management via OpenID Connect
  • OPA: Centralized policy engine for attribute-based access control
  • Navigator: Shared React frontend that dynamically adapts to any application model
  • Solon: Collects policies from all applications and bundles them for OPA.
  • Pathfinder: Automatic Ingress management and TLS certificate provisioning

All components run in Kubernetes and use dynamic service discovery to automatically detect and integrate new applications. For complete details on platform architecture, request flow, and operational characteristics, see Runtime Platform.

Application Server

Each ContentGrid application runs as an instance of the Application Server—a single shared container image configured with application-specific artifacts. The server uses a configuration-driven approach rather than code generation, enabling rapid deployment of model changes. For complete details on architecture, query construction, and performance characteristics, see Application Server.

Data Storage

ContentGrid separates structured metadata from binary content for optimal performance:

PostgreSQL: Each application has its own database with schema generated automatically from the data model. Entities map to tables, attributes to columns, and relations to foreign keys. Flyway manages migrations.

S3-Compatible Storage: Binary content (documents, images, videos) is stored in dedicated S3 buckets per application.

Encryption: Transparent application side encryption at rest. For complete details on encryption architecture, key management, and range request support, see Data Storage.

Deployment Pipeline

The deployment pipeline automates the complete path from application model to running service:

  1. Artifact Generation: Scribe generates a versioned ZIP artifact with model, migrations, and policies
  2. Infrastructure Provisioning: Captain provisions database, S3 bucket, and Keycloak realm
  3. Kubernetes Deployment: Captain creates all necessary Kubernetes resources
  4. Application Startup: Application Server fetches artifact, runs migrations, and begins serving

For complete details on the deployment see Deployment Pipeline.

Integration Points

The platform provides several mechanisms for external automations to extend functionality:

  • Service accounts: Automations authenticate via OIDC client credentials flow
  • Webhook Notifications: Automations subscribe to create/update/delete events

External automations integrate seamlessly, they can use the same HAL/HAL-FORMS patterns as core functionality, making them familiar to API consumers.

Key Architectural Benefits

Developer Experience: Standards-based APIs (HAL, HAL-FORMS, RFC 9110) reduce learning curves and enable existing tooling. Model-first approach means intuitive URLs that match your domain.

Scalability: Each application is independently scalable. Kubernetes-native architecture enables horizontal scaling of application instances. Database and storage scale independently.

Security: ABAC policies evaluated at query time prevent unauthorized data from being loaded. Content encryption ensures data confidentiality. Separate isolation per application.

Flexibility: Small core with automation extension points allows customization without forking. Replace or disable automations independently. Multiple implementations of the same functionality can coexist.

Operational Simplicity: Single shared container image for all applications. Configuration-driven rather than code changes. Standard Kubernetes operations for deployment and scaling.

Next Steps

For deeper understanding of specific subsystems:

Architecture Principles

This document explains the design philosophy driving ContentGrid’s architecture and the principles that guide technical decisions. Understanding these principles helps evaluate how the platform aligns with your requirements and technical values.

Foundational Principles

API-First

The application exposes a REST API as the only interaction point—for both end-users and external automations. There is no separate “admin API” or backend interface; everything flows through the same well-defined API.

The REST API uses the HAL response format. HAL’s hypermedia links allow referencing related endpoints and discovering capabilities dynamically. At the same time, the core data is available as plain JSON objects—developers who don’t need HAL’s hypermedia features can simply use the standard JSON payload.

For a single application with a known model, the API structure is regular and predictable. HAL becomes essential when building generic tools that need to work across multiple different models (e.g., different projects or organizations). These generic consumers use HAL to adapt automatically to any data model.

Model-First

The API is generated directly from your relational content model. Entities with attributes and relations map naturally to URLs you’d expect from a hand-crafted application. For example, an invoice entity generates /invoices and /invoices/{id} endpoints automatically.

The data APIs don’t expose model abstractions like “entity,” “attribute,” or “relation” in their payload structure. These concepts are internal implementation details. Your API consumers work with invoices, suppliers, and documents—domain concepts, not meta-model concepts.

For developers building generic integrations, a separate model API (/profile and /profile/{entity}) does expose the entity and attribute metadata. This information is static for a particular application version and enables automations to adapt to any model without hardcoded entity knowledge.

Small Core

The application core provides only essential ECM functionality that cannot be implemented externally. Everything else is pushed to external automations.

Why keep the core small?

  1. Scalability: Core functionality is critical for the functioning of ContentGrid. Keeping it small makes it easier to scale this part horizontally, making sure it can handle the requests with good response times.

  2. Reduced Security Surface: Fewer features in the core mean fewer places where permission checks can be misconfigured. A smaller codebase is easier to audit and secure.

  3. Easier Maintenance: A well-defined, focused core is easier to understand, test, and maintain. The dynamic nature of model-driven applications already introduces complexity—keeping the core small manages this complexity.

Extensibility

Extensions add capabilities beyond the core platform while remaining loosely coupled. They can be developed in-house, by customers, or by third parties.

Extensions are not implicitly trusted. Token-based authentication ensures users and extensions have appropriate access without exposing primary credentials. Extensions can act on behalf of users (delegated access) or under their own identity (system access). When extensions act on behalf of users, they receive at most the user’s privileges—never more.

This security model enables safe integration of third-party services while maintaining ContentGrid’s permission model. Extensions access data through the same REST API as any other client, ensuring consistent authorization enforcement.

Summary

ContentGrid’s architecture principles prioritize developer experience, operational flexibility, and data security. By keeping the core small and designing for extensibility, the platform enables rapid development while maintaining long-term maintainability.

The model-first, API-first approach generates intuitive, standards-compliant REST APIs from your domain model. The clear separation between core and automations provides flexibility without sacrificing integration quality.

Management Platform

The Management Platform is where you define, configure, and deploy ContentGrid applications. It runs as a SaaS service—you don’t need to host or manage it yourself. This page explains its internal architecture and how it transforms application models into deployable artifacts.

Platform Overview

The Management Platform consists of three primary components that work together to enable the full application lifecycle: from model definition through artifact generation to deployment orchestration.

graph TB
    subgraph "Management Platform"
        Architect[Architect<br/>Model Definition]
        Scribe[Scribe<br/>Artifact Generation]
        Captain[Captain<br/>Deployment Orchestration]
    end

    subgraph "Runtime Platform"
        K8sResources[Resources<br/>ConfigMaps, Secrets, Deployments]
    end

    subgraph "Persistent Storage"
        DB[(PostgreSQL<br/>Metadata & Structure)]
        S3[(S3-Compatible<br/>Content Storage)]
    end

    ArtifactStorage[(S3 Artifact Storage)]
    Architect --> Scribe
    Scribe --> Captain
    Captain -->|Upload Artifact| ArtifactStorage
    Captain -->|Deploy| K8sResources
    Captain -->|Provision database| DB
    Captain -->|Provision bucket| S3
    Captain -->|Provision realm| Keycloak[Keycloak<br/>Authentication]

Core Components

Architect

Architect serves as the source of truth for application models. It stores and manages all configuration that defines how your ContentGrid application behaves. The concepts Architect manages are described in the concepts section.

What Architect Stores:

  • Domain Model: Entity definitions with attributes and relations
  • Permission Model: Access control policies and rules
  • Automation Configurations: Settings for external automation integrations

Versioning and History: Architect maintains a complete history of model changes, enabling audit trails and the ability to understand how the application evolved over time.

API Access: Architect provides an API for retrieving application models. When you make changes through the management interface, Architect persists them and makes them available to other components.

Scribe

Scribe transforms application models from Architect into deployable artifacts. It’s the compilation step that bridges the gap between high-level model definitions and concrete implementation artifacts.

Artifact Generation Process:

  1. Fetch Model: Scribe retrieves the complete application model from Architect
  2. Generate Migrations: Analyzes model changes and generates SQL migration scripts (Flyway format)
    • For new applications, generates CREATE TABLE statements
    • For updates, compares current model to previous version and generates ALTER TABLE statements
    • Ensures database schema stays synchronized with the model
  3. Compile Policies: Converts permission definitions into Rego policies for OPA
    • Translates user-friendly permission rules into efficient policy evaluation code
    • Generates policy packages with proper namespacing per application
  4. Generate OpenAPI Specification: Creates OpenAPI documentation for the REST API based on the model
  5. Package Artifact: Creates a ZIP file containing:
    • application-model.json: Complete model definition
    • migrations/: SQL scripts for database schema changes
    • policies.rego: Access control policies
    • openapi.yaml: API specification
    • manifest.json: Metadata (organization, project, version, changeset, timestamps)

Artifact Metadata: The manifest includes traceability information—which changeset triggered the build, what version of Scribe generated it, and when. This enables audit trails and troubleshooting.

Reproducibility: Given the same input model and version of Scribe, the output artifact is deterministic. This ensures consistent behavior across environments and deployments.

Captain

Captain orchestrates the entire deployment process. It coordinates between Scribe, infrastructure provisioning, and Kubernetes deployment, handling all the complexity of getting an application from model to running service.

Responsibilities:

  • Application Configuration: Storing application settings
  • IAM: Creating Realms, managing users, groups and attributes
  • Artifact Management: Request artifact generation from Scribe and upload to shared artifact storage
  • Infrastructure Provisioning: Provision databases, S3 buckets, and Keycloak realms for new applications
  • Kubernetes Orchestration: Create and update Kubernetes resources (Deployments, Services, ConfigMaps, Secrets)
  • Lifecycle Management: Handle application creation, updates, deletion
  • Credential Management: Generate and distribute database credentials, S3 access keys, and authentication secrets
  • Environment Configuration: Inject environment-specific configuration (database URLs, domain names, etc.)

Zero-Configuration Deployment: Captain abstracts away the complexity of Kubernetes and cloud infrastructure, providing a simple “deploy this application” interface to the management platform. When you deploy an application, Captain handles all the details automatically.

Deployment Flow

When you deploy an application, the Management Platform components work together to transform application models into running services. The high-level flow involves:

  1. Deploy Request: User initiates deployment through the management interface
  2. Artifact Generation: Captain requests Scribe to generate an artifact from Architect’s model
  3. Infrastructure Deployment: Captain stores artifact, provisions infrastructure and coordinates Kubernetes deployment
  4. Application Startup: Application Server fetches artifacts and begins serving

For detailed information about the deployment pipeline and lifecycle, see Deployment Pipeline.

Summary

The Management Platform provides a complete application lifecycle management system:

  • Architect: Source of truth for application models
  • Scribe: Transforms models into deployable artifacts with migrations and policies
  • Captain: Orchestrates infrastructure provisioning and Kubernetes deployment

The platform abstracts away the complexity of cloud infrastructure and deployment orchestration, providing a simple interface for defining and deploying applications. Its artifact-based approach ensures reproducible, auditable deployments with full version history.

For details on how applications run after deployment, see Runtime Platform.

Runtime Platform

The Runtime Platform provides the infrastructure and services required to run, secure, and manage ContentGrid applications. It integrates several components to handle authentication, authorization, policy enforcement, routing, and frontend delivery.

Platform Overview

The Runtime Platform is designed as a Kubernetes-native system. Components dynamically discover resources like ConfigMaps, Secrets, and Services to route requests, enforce policies, and configure applications. This dynamic discovery enables zero-configuration deployment of new applications—the platform automatically detects and integrates them.

graph TB
    subgraph "Runtime Platform"
        Gateway[Gateway<br/>Entry Point & Routing]
        Keycloak[Keycloak<br/>Authentication]
        OPA[OPA<br/>Policy Evaluation]
        Solon[Solon<br/>Policy Collection]
        Navigator[Navigator<br/>Shared Frontend]
        Liaison[Liaison<br/>Config Service]
        Pathfinder[Pathfinder<br/>Ingress Management]
        Ingress[Kubernetes Ingress]

        subgraph "Application Instance"
            AppServer[Application Server]
        end
    end

    Client --> Gateway
    Client --> Keycloak
    Gateway --> OPA
    Gateway --> AppServer
    Client --> Navigator
    Navigator --> |webbrowser request| Liaison
    AppServer --> Solon
    Solon --> OPA
    Pathfinder -.->|Creates| Ingress
    Ingress -.->|Routes to| Gateway

Core Components

Gateway

The Gateway serves as the entry point for all ContentGrid applications. It handles routing and coordinates with authentication and authorization services.

Primary Responsibilities:

  • Route requests to the appropriate application based on the request’s domain
  • Enforce CORS policies configured per application
  • Coordinate user authentication with Keycloak
  • Communicate with Open Policy Agent (OPA) for policy evaluation

Dynamic Routing: The Gateway maintains a mapping from domains to application IDs by reading Kubernetes ConfigMaps. When a request arrives for a specific domain name, the Gateway uses this mapping to determine the corresponding application and routes to the appropriate Service for that application.

CORS Configuration: Each application’s CORS origins are configured in a ConfigMap. The Gateway reads these configurations and merges CORS settings for both the API backend and the Navigator frontend, ensuring cross-origin requests are properly handled.

Application Server

The Application Server serves dynamic REST APIs generated from application models. Each ContentGrid application runs as an instance of the same Application Server container, with behavior determined by the application artifact loaded at startup.

The Application Server follows a configuration-driven approach where a single container image serves all applications, enabling consistent operations and rapid iteration without code generation.

For complete details on the Application Server architecture and components, see Application Server.

Keycloak

Keycloak provides authentication and identity management for the platform. Each application has a corresponding realm in Keycloak, though applications can share a realm (typically one realm per organization).

Key Functions:

  • Authenticate users via OpenID Connect (OIDC)
  • Store user attributes used in authorization policies
  • Issue JWT tokens containing user identity and attributes
  • Manage OAuth clients for both API access and frontend applications

User attributes stored in Keycloak (such as department, role, or clearance level) are included in JWT tokens and used by applications when evaluating attribute-based access control policies.

Keycloak is an open source project. More information about Keycloak can be found on keycloak.org.

Open Policy Agent

OPA is a centralized policy engine that evaluates attribute-based access control (ABAC) policies for all applications in the platform.

Key Functions:

  • Evaluates Rego policies to determine authorization decisions
  • Performs partial evaluation to return residual expressions when complete evaluation isn’t possible
  • Receives policy bundles from Solon containing all application policies
  • Queried by the Gateway before requests reach applications

When the Gateway receives a request, it queries OPA with user attributes and request context. OPA evaluates the relevant policy and returns either a decision (allow/deny) or a residual expression that the Gateway encodes in a JWT for the application to apply at the database level.

Open Policy Agent is an open source project. More information about Open Policy Agent can be found on openpolicyagent.org.

Solon

Solon collects Rego policy files from all applications and makes them available to OPA for policy evaluation.

Policy Collection:

  • Discovers applications by querying Kubernetes Services with policy annotations
  • Fetches policy files from application management endpoints via HTTP
  • Bundles all policies together for OPA consumption
  • Keeps OPA’s policy bundle up to date as applications are deployed or updated

Solon acts as the bridge between individual applications (which serve their own policy files) and the centralized OPA instance (which needs all policies to evaluate authorization requests).

Navigator is a shared React frontend application used by all ContentGrid applications. Rather than deploying separate frontends per application, a single Navigator instance dynamically adapts to each application’s data model.

Adaptive Behavior:

  • Discovers entities and available operations through HAL links
  • Renders forms dynamically using HAL-FORMS templates
  • Adapts to user permissions automatically (forms only show permitted actions)
  • No application-specific code required—purely hypermedia-driven

Deployment Model: Pathfinder creates a separate Ingress resource for Navigator for each application, routing based on domain. The Navigator instance then loads application-specific configuration from Liaison based on the request’s Host header.

Liaison

Liaison serves configuration for Navigator on a per-application basis. It acts as a configuration service that provides the necessary settings for Navigator to connect to the correct application and authentication realm.

Configuration Delivery:

  • Serves Navigator configuration based on the domainname of the request
  • Provides OIDC client ID and issuer URL for authentication
  • Enables a single Navigator instance to serve multiple applications

Pathfinder

Pathfinder automatically creates and manages Kubernetes Ingress resources for applications. It watches ConfigMaps and translates them into Ingress configurations, enabling external access to application services.

Two Deployment Variants:

  • Pathfinder: Creates Ingress resources for application API backends
  • Pathfinder for webapp: Creates Ingress resources for Navigator frontend

Resource Management:

  • Reads ConfigMaps with domain routing configuration
  • Creates Ingress resources with appropriate routing rules
  • Coordinates with cert-manager for TLS certificate provisioning

Certificate Management: When Pathfinder creates an Ingress, cert-manager automatically provisions TLS certificates. Pathfinder adds annotations to ConfigMaps indicating which cluster issuer to use, and cert-manager handles the certificate lifecycle.

Application Deployment

When an application is deployed to the Runtime Platform, several Kubernetes resources are created to integrate it with the platform services.

Application Service

A Kubernetes Service makes the application accessible to the Gateway and other platform components. The Service is labeled with the application ID and service type, enabling dynamic discovery.

Key Labels:

  • app.contentgrid.com/application-id: Unique identifier for the application
  • app.contentgrid.com/deployment-id: Unique identifier for the deployment
  • app.contentgrid.com/service-type: Type of service (e.g., api, webapp)

Service Discovery: The Gateway uses these labels to discover Services. When routing a request, the Gateway queries for Services matching the application ID determined from the domain mapping.

OPA Integration: The Service also includes an annotation (authz.contentgrid.com/policy-package) indicating the OPA policy package location. This enables the platform to collect policies from applications.

Request Flow

Understanding how a request flows through the platform illustrates how these components work together.

sequenceDiagram
    autonumber
    participant Client
    participant Keycloak
    participant Gateway
    participant OPA as Centralized OPA
    participant App as Application Server
    participant DB as PostgreSQL
    Note over Client, Keycloak: Authentication Flow
    Client ->> Keycloak: Login (if no valid token)
    Keycloak -->> Client: JWT with user attributes
    Note over Client, DB: API Request Flow
    Client ->> Gateway: HTTPS Request + JWT
    Gateway ->> Gateway: Validate JWT signature
    Gateway ->> Gateway: Look up application by domain
    Gateway ->> OPA: Authorization query (JWT claims)
    OPA -->> Gateway: Allow/Deny + Residual expression
    Gateway ->> Gateway: Encode residual in new JWT
    Gateway ->> App: Forward request + JWT with residual
    App ->> App: Decode residual from JWT
    App ->> DB: Query with authorization filter
    DB -->> App: Filtered results
    App -->> Client: HAL JSON response

Step-by-Step:

  1. Authentication: Client authenticates directly with Keycloak and receives a JWT with user attributes
  2. Request with Token: Client makes HTTPS request to Ingress with JWT in Authorization header
  3. JWT Validation: Gateway validates JWT signature using Keycloak’s public keys
  4. Gateway Routing: Gateway maps domain to application ID
  5. Policy Evaluation: Gateway queries centralized OPA with user attributes from JWT
  6. Residual Encoding: OPA returns residual expression that Gateway encodes in a new JWT
  7. Application Processing: Gateway forwards request with JWT containing residual to Application Server
  8. Data Access: Application decodes residual, translates to SQL filter, and queries database
  9. Response: Application formats response as HAL JSON and returns through Gateway

Scaling and High Availability

The Runtime Platform is designed for horizontal scaling and high availability:

Application Servers: Scale horizontally by increasing replica count. Each replica is stateless (except for database connections) and can handle requests independently. Kubernetes Services load balance across replicas.

Gateway: Runs as a highly available Deployment with multiple replicas. All replicas share the same configuration ( from ConfigMaps), and the Ingress load balances across them.

Keycloak: Can be deployed in clustered mode for high availability. Database-backed session storage enables failover between instances.

Navigator and Liaison: Stateless services that scale horizontally. Liaison reads configuration from Kubernetes API on each request (with caching), so all replicas have consistent configuration.

Database and Storage: PostgreSQL and S3 are external to the platform and have their own high-availability mechanisms (e.g., PostgreSQL replication, S3 redundancy).

Operational Characteristics

Zero-Configuration Deployment: Adding a new application requires only creating the standard Kubernetes resources ( Deployment, Service, ConfigMaps, Secrets). The platform automatically discovers and integrates the application.

Independent Scaling: Each application scales independently. Heavy workloads on one application don’t affect others.

Resource Isolation: Each application has its own database, S3 bucket, and Keycloak realm (or shared by organization). Resource limits prevent one application from affecting others.

Observability: Standard Kubernetes observability tools work out of the box. Platform components expose Prometheus metrics, health check endpoints, and structured logs.

Summary

The ContentGrid Runtime Platform provides a Kubernetes-native infrastructure that:

  • Automatically discovers and integrates applications through labels and dynamic service discovery
  • Scales applications independently with horizontal scaling and load balancing
  • Manages authentication and authorization through Keycloak and OPA
  • Provides a shared Navigator frontend that adapts to any data model
  • Handles TLS, routing, and CORS through Gateway and Ingress management

The platform’s design enables operational simplicity—deploying applications requires no platform configuration changes, and standard Kubernetes operations handle scaling, updates, and failover.

Application Server

The ContentGrid Application Server is the core runtime component that serves dynamic REST APIs based on application models. Each ContentGrid application runs as an instance of the same Application Server container image, configured with application-specific artifacts.

The ContentGrid Application Server is developed in open source. The sources can be found on https://github.com/xenit-eu/contentgrid-appserver.

Design Philosophy

The Application Server is built with a configuration-driven approach. A single shared container image serves all applications. The only difference between different deployments is the application artifact ( model JSON, database migrations, and policies) loaded at startup.

This approach provides several advantages:

  • Deployment speed: No need to build new Docker images on model changes
  • Consistent behavior: All applications run identical server code
  • Simplified operations: One image to build, test, and deploy

Architecture Overview

graph TD
;
    subgraph "Application Server"
        REST[REST Layer] --> DOMAIN[Domain Layer];
        DOMAIN --> QUERY[Query Engine];
        DOMAIN --> CONTENT[Content Store];
        DOMAIN --> APPLICATION[Application Resolver];
    end

    subgraph "External Persistence"
        POSTGRES[PostgreSQL];
        STORAGE[Content Storage];
        ARTIFACT[Application Artifact];
    end

    QUERY --> POSTGRES;
    CONTENT --> STORAGE;
    APPLICATION --> ARTIFACT;

Components

The Application Server consists of several layered components, each with clear responsibilities and boundaries.

REST Layer

The REST Layer serves dynamic data endpoints based on the application model loaded from the artifact.

Responsibilities:

  • Parse incoming HTTP requests and extract parameters
  • Look up the application model and resolve entity definitions
  • Follow relations to navigate between entities
  • Parse authorization expressions from request context
  • Format responses in HAL/HAL-FORMS format
  • Serve HAL profile endpoints for entity metadata
  • Handle upload and download of content
  • Serve OpenAPI specifications
  • Serve Rego policies for ABAC

Dynamic Endpoint Generation: When a request arrives for /invoices, the REST Layer looks up the “invoices” entity definition. It then uses this definition to understand which attributes and relations exist, generating the appropriate response structure on the fly.

Content Negotiation: The REST Layer supports multiple response formats based on the Accept header:

  • application/hal+json: Standard HAL responses with hypermedia links
  • application/prs.hal-forms+json: HAL-FORMS responses with action templates
  • application/schema+json: JSON Schema descriptions for entity profiles

Domain Layer

The Domain Layer provides business logic and abstracts data access patterns. It sits between the REST Layer and the Query Engine/Content Store, enforcing business rules and model constraints.

Responsibilities:

  • Expose logical operations: find, create, update, partial update, delete (for data); set/clear for to-one relations; add/remove for to-many relations; find, update, delete for content
  • Convert search filters and authorization rules into query expressions
  • Enforce data constraints configured in the model (required fields, validation rules)
  • Maintain audit information (created/modified timestamps and users)
  • Coordinate transactions across multiple operations

Authorization Translation: When the REST Layer provides authorization rules from OPA, the Domain Layer translates these into query expressions that the Query Engine can push down to the database. This ensures unauthorized data never leaves the database—filtering happens at the SQL level.

Constraint Enforcement: The Domain Layer validates all data modifications against the model’s constraint definitions before allowing changes to persist. This ensures data integrity independent of client validation.

Query Engine

The Query Engine translates the application model and query expressions into efficient database queries. It’s responsible for all interactions with PostgreSQL.

Responsibilities:

  • Dynamically construct SQL queries based on model definitions
  • Apply pagination and sorting parameters
  • Implement counting strategies (exact and estimated counts)
  • Handle optimistic locking using row versions
  • Push authorization filters down to SQL WHERE clauses

The query engine is implemented using JOOQ, a type-safe SQL query construction library for Java that provides direct control over SQL while maintaining compile-time validation and type safety.

Dynamic Query Construction: Unlike traditional ORMs that use fixed entity classes, the Query Engine builds queries at runtime based on the model. When the Domain Layer requests “all invoices where department=‘sales’”, the Query Engine knows the invoices table structure from the model and constructs the appropriate SQL.

Counting Strategies: For large collections, exact counts can be expensive. The Query Engine implements fallback strategies:

  1. Attempt an exact count with a timeout
  2. If timeout occurs, use PostgreSQL’s query planner statistics for an estimate
  3. Indicate in the response whether the count is exact or estimated
graph TD
    subgraph "REST Layer"
        REQUEST[Request Parameters]
        ABAC[ABAC Rules]
    end

    subgraph "Domain Layer"
        THUNX[Generate Query Expression]
        APP[Application Model]
    end
    

    REQUEST --> THUNX
    ABAC --> THUNX
    APP --> THUNX

    subgraph "Query Engine"
        SQL["Translate to SQL"]
    end

    THUNX -- Expression --> SQL
    THUNX -- Application --> SQL
    THUNX -- Entity --> SQL
    SQL --> POSTGRES[Execute Query on PostgreSQL]

Application Resolver

The Application Resolver provides model lookup for other components. It loads the application model from the artifact and makes it available throughout the application lifecycle.

Responsibilities:

  • Load application model from JSON artifact at startup
  • Provide entity definitions to REST and Domain layers
  • Validate model consistency and completeness
  • Cache model in memory for fast access

Current Implementation: The Application Server currently uses a single-tenant model—one container runs one application. The Application Resolver loads one model and always returns it. This design could support multi-tenancy in the future by loading multiple models and routing between them, but the current focus is on simplicity and isolation.

Model Format: The application model is defined in JSON following a published schema. The model includes:

  • Entity definitions with attributes and types
  • Relation definitions (one-to-one, one-to-many, many-to-one)
  • Constraint rules and validation

Content Store

The Content Store provides persistence and access for binary content objects (documents, images, PDFs, etc.). It abstracts storage implementation, allowing different backends.

Capabilities:

  • Read content by reference with support for HTTP Range requests
  • Store new content objects and return references
  • Remove content by reference
  • Support transparent content encryption

Content References: The Content Store uses opaque references to identify content. These references are stored in the database as part of entity attributes. The actual content bytes are stored in one of the implementations, referenced by these identifiers.

Range Request Support: For large files (videos, large PDFs), clients can request only specific byte ranges. The Content Store translates these HTTP Range requests to the appropriate backend format, minimizing data transfer. Range request support depends on the backend implementation - for example, S3-compatible storage supports native range requests.

Implementations:

  • S3ContentStore: Stores content in S3-compatible object storage (AWS S3, MinIO, etc.). This is the default option, used in our own Runtime Platform.
  • FileSystemContentStore: Stores content on local filesystem (useful for development)
  • EncryptedContentStore: Wraps another ContentStore to add transparent encryption/decryption

Database Migrations

Database migrations are used to create the tables needed for a ContentGrid application, but also to migrate the database schema from one version of the application to the next. Database schema changes are managed using Flyway, a database migration tool. Migration scripts are included in the application artifact and executed automatically during startup.

Migration Process:

  1. Application Server starts and loads the artifact
  2. Before serving requests, it runs Flyway migrations
  3. Flyway tracks which migrations have already been applied
  4. Only new migrations execute, enabling incremental schema changes
  5. Once migrations complete, the application begins serving traffic

Migration scripts are generated by Scribe based on model changes. When you add an entity or attribute, Scribe generates a Flyway migration that creates or alters the corresponding table and columns.

ABAC Policy Integration

The Application Server integrates with Open Policy Agent (OPA) for attribute-based access control. Policies are written in Rego and included in the application artifact.

Policy Lifecycle:

  1. Application Server loads Rego policies from artifact at startup
  2. Serves policies to Solon over HTTP
  3. For each request, the application extracts the residual expressions
  4. Application Server translates residual expressions to SQL filters

Residual Expressions: OPA’s partial evaluation feature is crucial for efficiency. When evaluating “user can see invoices from their department,” OPA knows the user’s department but doesn’t know which invoices exist. It returns a residual expression: invoice.department == "user's department". The Query Engine translates this to a SQL WHERE clause, filtering at the database level.

This architecture means the application never loads unauthorized data—authorization filters are applied before data leaves the database.

A more detailed description can be found in the section about access control.

Technology Stack

The Application Server leverages several proven technologies:

Query Construction: JOOQ provides type-safe SQL query construction in Java. Unlike traditional ORMs, JOOQ gives direct control over SQL while providing type safety and compile-time validation.

Web Framework: Spring Boot and Spring MVC handle HTTP, dependency injection, and application lifecycle. This provides production-ready features like health checks, metrics, and configuration management.

Access Control: Custom query expression language (Thunx) translates Open Policy Agent residual expressions into SQL. This enables pushing authorization filters down to the database level.

JSON Processing: Jackson handles JSON serialization/deserialization, including HAL and HAL-FORMS formatting.

Request Flow Example

Here’s how a request flows through the Application Server components:

Request: GET /invoices?department=sales&_sort=date,desc

  1. REST Layer:

    • Parses request: collection=invoices, filter={department:sales}, sort=[date,desc]
    • Extracts JWT from Authorization header
    • Queries for “invoice” entity definition (matching the invoices path)
    • Parses authorization expressions from JWT
  2. Domain Layer:

    • Receives: entity=invoice, filter={department:sales}, authorization={}
    • Combines search filter with authorization expressions
    • Requests paginated query from Query Engine
  3. Query Engine:

    • Generates SQL: SELECT * FROM invoices WHERE department='sales' AND <auth filter> ORDER BY date DESC LIMIT 20
    • Executes against PostgreSQL
    • Returns result set
  4. Domain Layer:

    • Converts database rows to domain objects
    • Returns to REST Layer
  5. REST Layer:

    • Formats as HAL JSON with _links and _embedded
    • Adds HAL-FORMS templates for available actions
    • Adds pagination links (next, prev, first)
    • Returns HTTP response

Throughout this process, the only application-specific information is the model definition—the code executing these steps is identical across all applications.

Performance Characteristics

Query Performance: Database queries are the primary performance bottleneck. The Query Engine generates efficient SQL that leverages indexes. Authorization filters are pushed down to SQL WHERE clauses, minimizing data transfer.

Memory Usage: The application model is cached in memory, but it’s typically small (kilobytes to low megabytes). Content is streamed instead of entirely loaded in memory.

Horizontal Scaling: Application Servers are stateless (except database connections). Adding replicas linearly increases throughput. Database connection pooling prevents connection exhaustion.

Summary

The ContentGrid Application Server provides a sophisticated runtime that generates complete REST APIs from application models. Its layered architecture cleanly separates concerns:

  • REST Layer handles HTTP and hypermedia formatting
  • Domain Layer enforces constraints
  • Query Engine translates models to efficient SQL
  • Content Store abstracts binary content storage
  • Application Resolver provides model definitions

The configuration-driven approach, combined with dynamic query construction and OPA integration, enables rapid development and consistent operations while maintaining performance and security.

Access Control

ContentGrid uses Attribute-Based Access Control (ABAC) to enforce fine-grained permissions on data access. Instead of folder-based permissions or role hierarchies, policies evaluate entity attributes and user attributes to make authorization decisions.

Why ABAC?

Traditional ECM systems often tie permissions to folder structures. If a document is in a folder, you need permissions on that folder. This approach has significant limitations:

  • Artificial Organization: Data must be organized into folder hierarchies even when that doesn’t match the domain model
  • Rigid Permissions: Changing access requirements often requires reorganizing folders
  • No Multi-Dimensional Access: Hard to express “users can see invoices from their own department AND invoices they created”
  • Weak Scalability: Permission checks traverse folder hierarchies, becoming expensive with deep nesting

ContentGrid replaces this with ABAC, where permissions are based on attributes:

  • User Attributes: Department, role, clearance level, location, etc.
  • Entity Attributes: Status, owner, creation date, sensitivity level, etc.
  • Request Context: Operation (read/write/delete)

Example Policy: “Users can view invoices where invoice.department == user.department OR invoice.status == 'published'

This policy doesn’t care about folder structures, as it evaluates attributes directly. Changing the invoice’s attribute change the permissions, without having to explicitly change permissions.

Architecture Components

Open Policy Agent (OPA)

ContentGrid uses Open Policy Agent, an open-source policy engine, to evaluate access control policies. Policies are written in Rego, OPA’s policy language.

Why OPA?

  • Industry Standard: Widely adopted in cloud-native environments
  • Declarative Policies: Rego is declarative—you specify what should be allowed, not how to check it
  • Partial Evaluation: OPA can return residual expressions when it cannot fully evaluate a policy (crucial for efficiency)
  • Decoupled from Application: Policies live in Rego files, not scattered through application code

Rego Policies

Rego policies define authorization rules. Here’s a simplified example:

package contentgrid.invoices

import future.keywords

default can_read_invoice := false

# Allow reading invoices from the user's own department
can_read_invoice if {
    input.entity.department == input.auth.principal.department
}

# Allow reading published invoices regardless of department
can_read_invoice if {
    input.entity.status == "published"
}

allow if {
    input.request.method == "GET"
    # Path /invoices
    count(input.request.path) == 1
    input.request.path[0] == "invoices"
    can_read_invoice == true
}

When the application queries OPA, it provides:

  • input.request.method: The HTTP method (GET, POST, PUT, DELETE)
  • input.request.path: The path being accessed
  • input.auth.principal: User attributes from the JWT
  • input.entity: Entity object with attributes and relations (when checking a specific record)

OPA evaluates all allow rules. If any rule evaluates to true, access is granted.

Partial Evaluation

The key to ABAC efficiency in ContentGrid is OPA’s partial evaluation feature.

The Problem: When a user requests /invoices, the application needs to return only invoices the user can access. Naively, you might:

  1. Load all invoices from the database
  2. For each invoice, query OPA: “Can this user access this invoice?”
  3. Filter out invoices where OPA says “no”

This is catastrophically inefficient. For 10,000 invoices, you’d make 10,000 OPA queries and transfer 10,000 invoices from the database just to filter most of them out.

The Solution: Partial evaluation allows OPA to evaluate policies with incomplete information.

The application queries OPA: “The user wants to see invoices. What filter should I apply?”

OPA evaluates the policy but doesn’t know which invoices exist. It returns a residual expression, a simplified policy that only contains the parts that could not be fully evaluated:

input.entity.department == "sales" OR input.entity.status == "published"

The application translates this residual expression to a SQL WHERE clause:

SELECT *
FROM invoices
WHERE department = 'sales'
   OR status = 'published'

Now only authorized invoices leave the database. One OPA query, efficient SQL filtering and minimal data transfer.

Architecture

Centralized OPA

ContentGrid currently uses a shared OPA instance for all applications deployed in the Runtime Platform.

Components

  • Gateway: Entry point for all requests, responsible for policy enforcement
  • Centralized OPA: Shared OPA instance that evaluates policies for all applications
  • Solon: Service that collects Rego policy files from all applications and bundles them for OPA
  • Application: Serves its policy file and receives residual expressions from the Gateway

Request Flow

sequenceDiagram
    autonumber
    participant Client
    participant Gateway
    participant OPA as Centralized OPA
    participant Solon
    participant App as Application
    participant DB as Database
    Note over Solon, App: Policy Distribution Phase
    App ->> Solon: Serve policy file (HTTP endpoint)
    Solon ->> Solon: Bundle all policies
    OPA ->> Solon: Download policy bundle
    Note over Client, DB: Request Processing Phase
    Client ->> Gateway: HTTP Request
    Gateway ->> OPA: Authorization query
    OPA -->> Gateway: Allow/Deny/Residual expression
    Gateway ->> Gateway: Encode residual in JWT
    Gateway ->> App: Forward request + JWT with residual
    App ->> App: Decode residual from JWT
    App ->> DB: Query with residual as filter
    DB -->> App: Filtered results
    App -->> Client: Response

How It Works:

  1. Policy Distribution: Solon collects Rego policies from all applications via HTTP endpoints and bundles them for the centralized OPA
  2. Request Processing: When a request arrives, the Gateway queries the centralized OPA for authorization
  3. Residual Encoding: OPA returns a residual expression that the Gateway encodes in a JWT
  4. Application Processing: The application decodes the residual from the JWT and applies it as a SQL filter
  5. Data Retrieval: Only authorized data is retrieved from the database

Policy Evaluation in Practice

The examples below illustrate key concepts for readers. While payloads may differ from real implementations, the underlying principles remain the same.

For collection queries (e.g., “What invoices can this user see?”), the application provides partial information:

{
  "input": {
    "method": "GET",
    "entity": "invoices",
    "user": {
      "id": "user-123",
      "department": "sales",
      "role": "employee"
    }
  }
}

Notice entity is missing—the application doesn’t know which invoices exist yet. OPA performs partial evaluation and returns a residual expression:

{
  "result": {
    "or": [
      {
        "eq": [
          {
            "ref": [
              "entity",
              "department"
            ]
          },
          "sales"
        ]
      },
      {
        "eq": [
          {
            "ref": [
              "entity",
              "status"
            ]
          },
          "published"
        ]
      }
    ]
  }
}

The application translates this to SQL. The exact translation mechanism uses an internal query expression language called Thunx that maps to SQL WHERE clauses.

Policy Development Workflow

Policies are defined in the Management Platform alongside the data model:

  1. Define Permissions: In the Management Platform, you define permission rules as part of the application model
  2. Generate Rego: Scribe generates Rego policies from the permission definitions
  3. Bundle Policies: Rego policies are included in the application artifact
  4. Deploy: When the application starts, it makes them available for OPA (via Solon)
  5. Enforce: The Gateway queries OPA for every request, enforcing the policies

Policy changes follow the same deployment pipeline as model changes, and are deployed in tandem. Update permissions, regenerate artifact, redeploy application.

Performance Considerations

Query-Level Filtering: Pushing authorization filters to SQL ensures only authorized data leaves the database. This is dramatically more efficient than application-level filtering.

OPA Response Time: OPA policy evaluation is fast (microseconds to low milliseconds).

Database Indexes: Authorization filters often involve specific columns (e.g., department, owner). Proper indexes on these columns ensure efficient query execution.

Security Benefits

Defense in Depth: Authorization happens at multiple layers:

  1. OPA evaluates policies before queries execute
  2. Database enforces SQL filters (data never loaded without authorization)

Principle of Least Privilege: ABAC naturally supports least-privilege models. Users only see data matching their attributes, and policies can be as granular as needed.

Dynamic Permissions: Attribute changes immediately affect permissions—no cache invalidation needed. If a user changes departments, their access automatically reflects the new department.

Summary

ContentGrid’s Attribute-Based Access Control provides fine-grained, efficient authorization:

  • Flexible Policies: Based on entity and user attributes, not artificial folder hierarchies
  • Efficient Enforcement: Partial evaluation pushes authorization filters to SQL
  • Centralized Architecture: Shared OPA instance with policy collection via Solon
  • Declarative Rego: Policies are readable, maintainable, and separate from application code

ABAC enables expressing complex authorization requirements naturally while maintaining query performance through intelligent filter pushdown.

Data Storage & Encryption

ContentGrid separates structured metadata from binary content, storing each optimally. Metadata lives in PostgreSQL, while content (documents, images, videos) is stored in S3-compatible object storage.

Storage Architecture

PostgreSQL for Metadata

Each ContentGrid application has its own PostgreSQL database. The schema is generated automatically from the application model:

  • Entities → Tables: Each entity in your model becomes a table
  • Attributes → Columns: Entity attributes map to columns with appropriate types
  • One-to-x Relations → Foreign Keys: Relations between entities use foreign key constraints
  • Many-to-many Relations → Join Tables: Many-to-many relations between entities use join tables

This direct mapping enables leveraging PostgreSQL’s full capabilities:

  • Indexes: Standard B-tree and other indexes for efficient queries
  • Constraints: Check constraints, unique constraints, and foreign keys enforce data integrity
  • Transactions: Full ACID guarantees for all operations

Migration Management: Flyway manages schema migrations. When the model changes, Scribe generates SQL migration scripts that execute automatically on deployment.

S3-Compatible Storage for Content

Binary content is stored in S3-compatible object storage (AWS S3, MinIO, Ceph, etc.). Each application has dedicated buckets. This ensures:

  • Isolation: Applications cannot access each other’s buckets
  • Scalability: Object storage scales independently of compute and database

Content References: The database stores only references (unique identifiers) to content, not the content itself. When the application needs content, it retrieves it from S3 using the reference.

Immutability: Content objects are never overwritten. Updating content creates a new object with a new reference. The old content remains until explicitly deleted, enabling:

  • Safe Backups: Backup S3 buckets without worrying about in-flight modifications
  • Recoverability: Old content versions can be retained for recovery or audit
  • Atomic Updates: Database transactions can commit content reference changes without coordinating with S3

Content Encryption

ContentGrid provides transparent encryption at rest for content stored in S3. Encryption and decryption happen automatically—applications and users don’t need to manage keys or modify their workflows.

Content is encrypted with AES-128 in CTR mode.

Encryption Goals

The encryption architecture is designed to meet several requirements:

  1. Strong Security: Content encrypted using standard cryptographic primitives
  2. Key Protection: Encryption keys managed securely with database access controls
  3. Enable After Deployment: Encryption can be enabled for applications with existing unencrypted content
  4. Key Rotation: Encryption keys can be rotated on an individual basis without re-encrypting all content
  5. Application Isolation: Each application uses different encryption keys
  6. Range Request Support: Clients can request parts of files (HTTP Range) without decrypting the entire file

Data Encryption Keys

ContentGrid encrypts content at rest using Data Encryption Keys (DEKs). Each content object gets its own unique symmetric key, ensuring strong isolation between content objects.

flowchart LR
    DEK[Data Encryption Key<br/>Per Content Object]
    Content[Content<br/>Binary Data]
    DEK -->|Encrypts| Content

How It Works:

  • Unique Keys: Each content object has its own 128-bit AES symmetric key (DEK)
  • Strong Isolation: Compromising one DEK does not affect other content objects
  • Local Encryption: Content encryption and decryption happen in the application using the DEK (no external service calls)
  • Database Storage: DEKs are stored in the database alongside content metadata

Note: Future enhancements will add Key Encryption Keys (KEKs) stored in Hardware Security Modules (HSMs) or cloud Key Management Services (KMS) to provide an additional layer of protection for DEKs. See the Future Enhancements section below.

Encryption Process

When content is uploaded:

flowchart TD
    Content[Content Upload]
    GenDEK[Generate DEK]
    EncryptContent[Encrypt Content with DEK]
    StoreContent[Store Encrypted Content in S3]
    StoreKey[Store DEK in Database]
    Content --> GenDEK
    GenDEK --> EncryptContent
    EncryptContent --> StoreContent
    GenDEK --> StoreKey
  1. Generate DEK: Application generates a random symmetric key (AES-128)
  2. Encrypt Content: Application encrypts content using the DEK
  3. Store Content: Encrypted content is stored in S3
  4. Store DEK: DEK is stored in the database alongside content metadata

Decryption Process

When content is downloaded:

flowchart TD
    FetchKey[Fetch DEK from Database]
    FetchContent[Fetch Encrypted Content from S3]
    DecryptContent[Decrypt Content using DEK]
    Return[Return Content to Client]
    FetchKey --> DecryptContent
    FetchContent --> DecryptContent
    DecryptContent --> Return
  1. Fetch DEK: Application retrieves DEK from database
  2. Fetch Encrypted Content: Application retrieves encrypted content from S3
  3. Decrypt Content: Application decrypts content locally using the DEK
  4. Return to Client: Decrypted content is sent to the client

Key Storage and Management

Data Encryption Keys (DEKs):

  • Stored in the database in a dedicated table
  • Each DEK is a 128-bit AES symmetric key
  • Each content object has its own unique DEK
  • Access controlled through database permissions and connection authentication
  • DEKs are associated with their corresponding content references

Key Rotation

To rotate a DEK (e.g., upgrading encryption algorithm or if a DEK is compromised):

  1. Fetch and decrypt the old content using the old DEK
  2. Generate a new DEK
  3. Encrypt the content with the new DEK
  4. Store the new encrypted content in S3
  5. Update the database with the new DEK and content reference

DEK rotation is performed on a per-object basis and is typically only needed when upgrading cryptographic algorithms or responding to a security incident.

Range Request Support

HTTP Range requests allow clients to request specific byte ranges of a file (e.g., “bytes 1000-2000”). This is essential for:

  • Video Seeking: Jump to a timestamp without downloading the entire video
  • Large PDFs: Load only visible pages
  • Parallel Downloads: Split large files across multiple connections

Encryption Challenge: Not all encryption modes support decrypting arbitrary byte ranges—some require decrypting from the beginning.

Solution: ContentGrid uses block cipher modes that support random access (e.g., AES-CTR). The encryption implementation:

  1. Calculates which encrypted blocks contain the requested byte range, and adjust the counter for that
  2. Fetch only the exact amount of requested data from S3 (using S3 range requests)
  3. Pads the downloaded data to align to the correct block size
  4. Decrypts the blocks
  5. Trims to the exact requested range
  6. Returns to the client

No additional data needs to be fetched. There is no need to decrypt the entire file, only a small amount of extra decryption is performed to align with block boundaries.

Security Considerations

Data Confidentiality:

  • Content is encrypted using strong symmetric algorithms (AES-128)
  • Each content object has a unique encryption key (DEK)
  • DEKs are stored in the database with access controlled through database permissions
  • Encrypted content in S3 is protected from unauthorized access at the storage layer
  • Database connections are authenticated and encrypted

Data Integrity:

  • Immutability prevents accidental overwrites

Access Control:

  • Database access controls restrict which services and users can access DEKs
  • Application servers authenticate to the database using service credentials

Defense in Depth:

  • Content encrypted at rest (this architecture)
  • Data encrypted in transit (TLS)
  • Access control enforced at query level (ABAC)
  • Database connections authenticated and encrypted
  • Database encryption at rest can provide an additional protection layer

Performance Impact

Encryption Overhead:

Modern CPUs have AES hardware acceleration (AES-NI), making symmetric encryption very fast. The overhead for encrypting/decrypting content is minimal—typically less than 100 MB/s of throughput impact.

Range Requests:

Range requests with encryption decrypt slightly more data (to align with block boundaries), but the overhead is small. For a 1 KB range request, you might decrypt 1-2 KB. This is negligible compared to fetching and decrypting the entire file.

Future Enhancements

Envelope Encryption with Key Encryption Keys

ContentGrid’s encryption architecture is designed to support envelope encryption (also called two-level encryption), a standard technique used by AWS KMS, Google Cloud KMS, HashiCorp Vault, and other enterprise systems.

In envelope encryption, a Key Encryption Key (KEK) stored in a Hardware Security Module (HSM) or cloud Key Management Service (KMS) is used to encrypt the DEKs before storing them in the database. This provides additional security benefits:

  • Enhanced Key Protection: DEKs are encrypted before storage, with KEKs never leaving the HSM/KMS
  • Audit Logging: All key operations logged in the KMS for compliance and security monitoring
  • Efficient Key Rotation: Rotating the KEK only requires re-encrypting small DEKs, not the entire content

The implementation will be backward compatible with existing encrypted content. When enabled, new content will use envelope encryption, and existing DEKs can be migrated in the background without service interruption.

Summary

ContentGrid’s storage architecture separates structured metadata (PostgreSQL) from binary content (S3), optimizing each for its purpose:

  • PostgreSQL: Provides ACID transactions, relational integrity, and efficient querying for metadata
  • S3: Provides scalable, durable storage for large binary content
  • Content Encryption: Protects content at rest using unique Data Encryption Keys (DEKs) for each object
  • Range Request Support: Enables efficient access to large files without sacrificing encryption

Encryption is transparent to applications and users—no code changes or workflow modifications required. The architecture balances strong security with operational simplicity and performance.

Deployment Pipeline

ContentGrid applications are deployed through a fully automated pipeline that transforms high-level application models into running services. The deployment process is managed by the Management Platform and executed in the Runtime Platform.

Deployment Overview

The deployment pipeline bridges two environments:

Management Platform: Where you define application models, configure settings, and initiate deployments. This is the control plane.

Runtime Platform: Where applications actually run, serving APIs to end-users. This is the data plane.

The pipeline ensures consistent, repeatable deployments with zero manual intervention required for the deployment mechanics.

Management Platform Components

The deployment pipeline is orchestrated by three Management Platform components:

  • Architect: Source of truth for application models (entities, permissions, configurations)
  • Scribe: Transforms models into deployable artifacts (migrations, policies, OpenAPI specs)
  • Captain: Orchestrates infrastructure provisioning and Kubernetes deployment

For complete details on how these components work together, their responsibilities, and operational characteristics, see Management Platform.

Deployment Architecture

flowchart TD
    S3Artifact[S3 Artifact Bucket]

    subgraph Management Platform
        Arch[Architect<br/>Model Definition]
        Scr[Scribe<br/>Artifact Generator]
        Cap[Captain<br/>Orchestrator]
        Arch --> Scr
        Cap -->|Request Artifact| Scr
        Scr -->|Generated ZIP| Cap
    end

    subgraph Runtime Platform
        K8s[Kubernetes API]
        AS[Application Server]
    end

    CB[(S3 Content Bucket)]
    DB[(PostgreSQL Database)]
    Cap -->|Upload Artifact| S3Artifact
    Cap -->|Create/Update Resources| K8s
    K8s -->|Deploy| AS
    AS -->|Fetch Artifact| S3Artifact
    AS -->|Metadata| DB
    AS -->|Content| CB
    AS -->|Serve API| Users[Clients]

Deployment Lifecycle

The end-to-end deployment process follows these steps:

sequenceDiagram
    autonumber
    participant Architect
    participant Scribe
    participant Captain
    participant S3 as S3 Artifact Storage
    participant CB as S3 Content Bucket
    participant K8s as Kubernetes
    participant App as Application Server
    Captain ->> Scribe: Request artifact for application X
    Scribe ->> Architect: Fetch application model
    Architect -->> Scribe: Model JSON
    Scribe -->> Captain: ZIP (model.json, migrations/, policies.rego)
    Captain ->> S3: Upload artifact ZIP
    Captain ->> K8s: Create/Update Deployment/Service/ConfigMaps/Secrets
    K8s -->> App: Start application server pod
    App ->> S3: Fetch artifact ZIP
    App ->> App: Unpack artifact (model, migrations, policies)
    App ->> App: Run DB migrations, Serve Rego files
    App -->> Users: Serve API for application X

Step-by-Step Breakdown

1. Artifact Request

Captain initiates deployment by requesting Scribe to generate an artifact for a specific application. This happens when the user triggers a deployment via the Console.

2. Model Retrieval

Scribe fetches the current application model from Architect. The model includes all entity definitions, permissions, constraints, and configuration.

3. Artifact Generation

Scribe processes the model:

  • Migration Generation: Compares the current model to the previous version (if it exists) and generates SQL DDL statements to migrate the schema.
  • Policy Compilation: Converts permission rules (defined in a user-friendly format in Architect) into Rego policies that OPA understands.
  • Model Serialization: Serializes the model to JSON in the format expected by the Application Server.
  • Manifest File: Containing information about the artifact.
  • Packaging: Bundles everything into a ZIP with a consistent structure.

4. Artifact Upload

Captain uploads the artifact to a shared artifact storage bucket (S3). The artifact is stored with a path including the application ID and version, enabling:

  • Rollback: Previous artifacts remain available for reverting deployments
  • Audit: Complete history of what was deployed when
  • Distribution: All Application Server replicas fetch from the same location

5. Infrastructure Provisioning

For new applications, Captain provisions:

  • PostgreSQL Database: Creates a new database with credentials stored in a Kubernetes Secret
  • S3 Content Bucket: Creates a bucket with appropriate access policies
  • Keycloak Realm: Creates or configures a realm for authentication, sets up OIDC clients

6. Kubernetes Resource Creation

Captain creates or updates Kubernetes resources:

Deployment:

  • Specifies the Application Server container image (same for all applications)
  • Configures environment variables pointing to the artifact location
  • Sets resource limits (CPU, memory)
  • Configures health check endpoints

Service:

  • Exposes the Application Server pods to the Gateway
  • Labels with application ID and service type for discovery

ConfigMaps:

  • Gateway configuration (domains, CORS settings)
  • Webapp configuration (OIDC settings)

Secrets:

  • Database credentials
  • S3 bucket access keys
  • Gateway authentication credentials for Keycloak

7. Application Server Startup

Kubernetes starts the Application Server pod(s):

Init Phase:

  • Application Server downloads the artifact from S3
  • Unpacks the artifact to the local filesystem
  • Runs Flyway migrations against the database

Runtime Phase:

  • Loads the application model into memory
  • Starts the HTTP server
  • Registers health check endpoints
  • Begins serving API requests

The init phase ensures the database schema matches the model before serving traffic. If migrations fail, the pod won’t become ready, and Kubernetes won’t route traffic to it.

8. Service Readiness

Once health checks pass, Kubernetes marks the pod as ready and begins routing traffic.

Artifact Structure

A typical artifact has this structure:

application-artifact.zip
├── manifest.json                    # Metadata
├── application-model.json           # Model definition
├── policies.rego                    # Access control policies
└── migrations/                      # Flyway migrations
    ├── V1__initial_schema.sql
    ├── V2__add_invoices.sql
    └── V3__add_status_column.sql

manifest.json:

{
  "organizationId": "org-123",
  "organizationName": "Acme Corp",
  "projectId": "proj-456",
  "projectName": "Invoice Management",
  "version": "1.2.3",
  "changeset": "abc123def456",
  "timestamp": "2026-01-28T10:30:00Z",
  "scribeVersion": "2.1.0"
}

application-model.json:

The model JSON follows a published schema and includes complete entity definitions, attributes, relations, and constraints. This is the single source of truth for the Application Server’s runtime behavior.

policies.rego:

Rego policies define authorization rules. These are loaded by OPA and queried by the Application Server for every request.

migrations:

Flyway migration scripts are numbered sequentially. Flyway tracks which migrations have executed in a special database table (flyway_schema_history), ensuring each migration runs exactly once.

Update Strategy

When updating an existing application:

Update:

  • Captain deletes the old Deployment
  • Captain creates a new Deployment with the updated artifact
  • Kubernetes starts new pods with the updated artifact
  • New pods run migrations (only new migrations execute)
  • Health checks pass, new pods become ready

Database Migrations:

  • Flyway migrations are forward-only—there are no automatic rollbacks
  • Migrations should be backward-compatible when possible (e.g., adding nullable columns)
  • Breaking changes require coordination between schema and code deployments

Rollback:

  • Captain can redeploy a previous artifact version
  • Even if the database migrations ran, the old schema is still available, and the old version of the model will still run

Deployment Observability

Health Checks:

Application Server exposes health endpoints:

  • /actuator/health/liveness: Is the process alive?
  • /actuator/health/readiness: Is the application ready to serve traffic?

Kubernetes uses these for liveness probes (restart if unhealthy) and readiness probes (route traffic only when ready).

Logs:

All components log to stdout/stderr, collected by Kubernetes:

  • Captain logs deployment activities and decisions
  • Application Server logs startup, migrations, and request handling
  • OPA logs policy evaluation (if enabled)

Centralized logging (e.g., Elasticsearch, Loki) aggregates logs for analysis.

Metrics:

Application Server exposes Prometheus metrics:

  • Request counts and latencies
  • Database query performance
  • OPA policy evaluation times
  • Content storage access patterns

Monitoring dashboards provide visibility into application behavior.

Deployment Security

Principle of Least Privilege:

  • Captain has credentials to provision infrastructure but not to access application data
  • Application Server has credentials to access its own database and bucket, but not others
  • Keycloak realms isolate authentication between organizations

Secret Management:

  • Kubernetes Secrets store sensitive credentials
  • Secrets are injected as mounted files
  • Secrets are not included in artifacts or logged

Artifact Integrity:

  • Artifacts are versioned and immutable once created
  • Only Captain can upload artifacts to the shared bucket

Summary

The ContentGrid deployment pipeline provides a fully automated path from application model to running service:

  • Artifact Generation: Scribe transforms models into deployable artifacts with migrations and policies
  • Orchestration: Captain provisions infrastructure and coordinates Kubernetes deployments
  • Observability: Health checks, logs, and metrics provide visibility into deployment status and application health

The pipeline’s design enables rapid iteration—model changes deploy quickly and consistently. The use of a shared container image simplifies operations while maintaining application isolation through configuration and infrastructure separation.