Application Server

The ContentGrid Application Server is the core runtime component that serves dynamic REST APIs based on application models. Each ContentGrid application runs as an instance of the same Application Server container image, configured with application-specific artifacts.

The ContentGrid Application Server is developed in open source. The sources can be found on https://github.com/xenit-eu/contentgrid-appserver.

Design Philosophy

The Application Server is built with a configuration-driven approach. A single shared container image serves all applications. The only difference between different deployments is the application artifact ( model JSON, database migrations, and policies) loaded at startup.

This approach provides several advantages:

  • Deployment speed: No need to build new Docker images on model changes
  • Consistent behavior: All applications run identical server code
  • Simplified operations: One image to build, test, and deploy

Architecture Overview

graph TD
;
    subgraph "Application Server"
        REST[REST Layer] --> DOMAIN[Domain Layer];
        DOMAIN --> QUERY[Query Engine];
        DOMAIN --> CONTENT[Content Store];
        DOMAIN --> APPLICATION[Application Resolver];
    end

    subgraph "External Persistence"
        POSTGRES[PostgreSQL];
        STORAGE[Content Storage];
        ARTIFACT[Application Artifact];
    end

    QUERY --> POSTGRES;
    CONTENT --> STORAGE;
    APPLICATION --> ARTIFACT;

Components

The Application Server consists of several layered components, each with clear responsibilities and boundaries.

REST Layer

The REST Layer serves dynamic data endpoints based on the application model loaded from the artifact.

Responsibilities:

  • Parse incoming HTTP requests and extract parameters
  • Look up the application model and resolve entity definitions
  • Follow relations to navigate between entities
  • Parse authorization expressions from request context
  • Format responses in HAL/HAL-FORMS format
  • Serve HAL profile endpoints for entity metadata
  • Handle upload and download of content
  • Serve OpenAPI specifications
  • Serve Rego policies for ABAC

Dynamic Endpoint Generation: When a request arrives for /invoices, the REST Layer looks up the “invoices” entity definition. It then uses this definition to understand which attributes and relations exist, generating the appropriate response structure on the fly.

Content Negotiation: The REST Layer supports multiple response formats based on the Accept header:

  • application/hal+json: Standard HAL responses with hypermedia links
  • application/prs.hal-forms+json: HAL-FORMS responses with action templates
  • application/schema+json: JSON Schema descriptions for entity profiles

Domain Layer

The Domain Layer provides business logic and abstracts data access patterns. It sits between the REST Layer and the Query Engine/Content Store, enforcing business rules and model constraints.

Responsibilities:

  • Expose logical operations: find, create, update, partial update, delete (for data); set/clear for to-one relations; add/remove for to-many relations; find, update, delete for content
  • Convert search filters and authorization rules into query expressions
  • Enforce data constraints configured in the model (required fields, validation rules)
  • Maintain audit information (created/modified timestamps and users)
  • Coordinate transactions across multiple operations

Authorization Translation: When the REST Layer provides authorization rules from OPA, the Domain Layer translates these into query expressions that the Query Engine can push down to the database. This ensures unauthorized data never leaves the databaseā€”filtering happens at the SQL level.

Constraint Enforcement: The Domain Layer validates all data modifications against the model’s constraint definitions before allowing changes to persist. This ensures data integrity independent of client validation.

Query Engine

The Query Engine translates the application model and query expressions into efficient database queries. It’s responsible for all interactions with PostgreSQL.

Responsibilities:

  • Dynamically construct SQL queries based on model definitions
  • Apply pagination and sorting parameters
  • Implement counting strategies (exact and estimated counts)
  • Handle optimistic locking using row versions
  • Push authorization filters down to SQL WHERE clauses

The query engine is implemented using JOOQ, a type-safe SQL query construction library for Java that provides direct control over SQL while maintaining compile-time validation and type safety.

Dynamic Query Construction: Unlike traditional ORMs that use fixed entity classes, the Query Engine builds queries at runtime based on the model. When the Domain Layer requests “all invoices where department=‘sales’”, the Query Engine knows the invoices table structure from the model and constructs the appropriate SQL.

Counting Strategies: For large collections, exact counts can be expensive. The Query Engine implements fallback strategies:

  1. Attempt an exact count with a timeout
  2. If timeout occurs, use PostgreSQL’s query planner statistics for an estimate
  3. Indicate in the response whether the count is exact or estimated
graph TD
    subgraph "REST Layer"
        REQUEST[Request Parameters]
        ABAC[ABAC Rules]
    end

    subgraph "Domain Layer"
        THUNX[Generate Query Expression]
        APP[Application Model]
    end
    

    REQUEST --> THUNX
    ABAC --> THUNX
    APP --> THUNX

    subgraph "Query Engine"
        SQL["Translate to SQL"]
    end

    THUNX -- Expression --> SQL
    THUNX -- Application --> SQL
    THUNX -- Entity --> SQL
    SQL --> POSTGRES[Execute Query on PostgreSQL]

Application Resolver

The Application Resolver provides model lookup for other components. It loads the application model from the artifact and makes it available throughout the application lifecycle.

Responsibilities:

  • Load application model from JSON artifact at startup
  • Provide entity definitions to REST and Domain layers
  • Validate model consistency and completeness
  • Cache model in memory for fast access

Current Implementation: The Application Server currently uses a single-tenant modelā€”one container runs one application. The Application Resolver loads one model and always returns it. This design could support multi-tenancy in the future by loading multiple models and routing between them, but the current focus is on simplicity and isolation.

Model Format: The application model is defined in JSON following a published schema. The model includes:

  • Entity definitions with attributes and types
  • Relation definitions (one-to-one, one-to-many, many-to-one)
  • Constraint rules and validation

Content Store

The Content Store provides persistence and access for binary content objects (documents, images, PDFs, etc.). It abstracts storage implementation, allowing different backends.

Capabilities:

  • Read content by reference with support for HTTP Range requests
  • Store new content objects and return references
  • Remove content by reference
  • Support transparent content encryption

Content References: The Content Store uses opaque references to identify content. These references are stored in the database as part of entity attributes. The actual content bytes are stored in one of the implementations, referenced by these identifiers.

Range Request Support: For large files (videos, large PDFs), clients can request only specific byte ranges. The Content Store translates these HTTP Range requests to the appropriate backend format, minimizing data transfer. Range request support depends on the backend implementation - for example, S3-compatible storage supports native range requests.

Implementations:

  • S3ContentStore: Stores content in S3-compatible object storage (AWS S3, MinIO, etc.). This is the default option, used in our own Runtime Platform.
  • FileSystemContentStore: Stores content on local filesystem (useful for development)
  • EncryptedContentStore: Wraps another ContentStore to add transparent encryption/decryption

Database Migrations

Database migrations are used to create the tables needed for a ContentGrid application, but also to migrate the database schema from one version of the application to the next. Database schema changes are managed using Flyway, a database migration tool. Migration scripts are included in the application artifact and executed automatically during startup.

Migration Process:

  1. Application Server starts and loads the artifact
  2. Before serving requests, it runs Flyway migrations
  3. Flyway tracks which migrations have already been applied
  4. Only new migrations execute, enabling incremental schema changes
  5. Once migrations complete, the application begins serving traffic

Migration scripts are generated by Scribe based on model changes. When you add an entity or attribute, Scribe generates a Flyway migration that creates or alters the corresponding table and columns.

ABAC Policy Integration

The Application Server integrates with Open Policy Agent (OPA) for attribute-based access control. Policies are written in Rego and included in the application artifact.

Policy Lifecycle:

  1. Application Server loads Rego policies from artifact at startup
  2. Serves policies to Solon over HTTP
  3. For each request, the application extracts the residual expressions
  4. Application Server translates residual expressions to SQL filters

Residual Expressions: OPA’s partial evaluation feature is crucial for efficiency. When evaluating “user can see invoices from their department,” OPA knows the user’s department but doesn’t know which invoices exist. It returns a residual expression: invoice.department == "user's department". The Query Engine translates this to a SQL WHERE clause, filtering at the database level.

This architecture means the application never loads unauthorized dataā€”authorization filters are applied before data leaves the database.

A more detailed description can be found in the section about access control.

Technology Stack

The Application Server leverages several proven technologies:

Query Construction: JOOQ provides type-safe SQL query construction in Java. Unlike traditional ORMs, JOOQ gives direct control over SQL while providing type safety and compile-time validation.

Web Framework: Spring Boot and Spring MVC handle HTTP, dependency injection, and application lifecycle. This provides production-ready features like health checks, metrics, and configuration management.

Access Control: Custom query expression language (Thunx) translates Open Policy Agent residual expressions into SQL. This enables pushing authorization filters down to the database level.

JSON Processing: Jackson handles JSON serialization/deserialization, including HAL and HAL-FORMS formatting.

Request Flow Example

Here’s how a request flows through the Application Server components:

Request: GET /invoices?department=sales&_sort=date,desc

  1. REST Layer:

    • Parses request: collection=invoices, filter={department:sales}, sort=[date,desc]
    • Extracts JWT from Authorization header
    • Queries for “invoice” entity definition (matching the invoices path)
    • Parses authorization expressions from JWT

  2. Domain Layer:

    • Receives: entity=invoice, filter={department:sales}, authorization={}
    • Combines search filter with authorization expressions
    • Requests paginated query from Query Engine

  3. Query Engine:

    • Generates SQL: SELECT * FROM invoices WHERE department='sales' AND <auth filter> ORDER BY date DESC LIMIT 20
    • Executes against PostgreSQL
    • Returns result set

  4. Domain Layer:

    • Converts database rows to domain objects
    • Returns to REST Layer

  5. REST Layer:

    • Formats as HAL JSON with _links and _embedded
    • Adds HAL-FORMS templates for available actions
    • Adds pagination links (next, prev, first)
    • Returns HTTP response

Throughout this process, the only application-specific information is the model definitionā€”the code executing these steps is identical across all applications.

Performance Characteristics

Query Performance: Database queries are the primary performance bottleneck. The Query Engine generates efficient SQL that leverages indexes. Authorization filters are pushed down to SQL WHERE clauses, minimizing data transfer.

Memory Usage: The application model is cached in memory, but it’s typically small (kilobytes to low megabytes). Content is streamed instead of entirely loaded in memory.

Horizontal Scaling: Application Servers are stateless (except database connections). Adding replicas linearly increases throughput. Database connection pooling prevents connection exhaustion.

Summary

The ContentGrid Application Server provides a sophisticated runtime that generates complete REST APIs from application models. Its layered architecture cleanly separates concerns:

  • REST Layer handles HTTP and hypermedia formatting
  • Domain Layer enforces constraints
  • Query Engine translates models to efficient SQL
  • Content Store abstracts binary content storage
  • Application Resolver provides model definitions

The configuration-driven approach, combined with dynamic query construction and OPA integration, enables rapid development and consistent operations while maintaining performance and security.