DynamoDB

The DynamoDB data source uses the Amazon DynamoDB service to store and retrieve information.

Table management

Curity Identity Server is shipped with JSON files containing the schema information required to create the necessary DynamoDB tables. These files are located in the installation, under <IDSVR>/etc, and use a format compatible with the create-table AWS CLI command. There is one such file per table, containing:

  • The table name.
  • The definition of the attributes used on the primary key and on the indexes.
  • The primary key definition.
  • The indexes definitions, when applicable.

Each file also includes ProvisionedThroughput fields defining the table and index provisioned capacity. The values present in these fields are illustrative, and must be adapted to the operational requirements of each deployment environment. The table names present in these files can also be changed to reflect an optional table name prefix, which can be defined in the data source configuration. This allows for different data sources using distinct tables under the same AWS account. The remaining fields in the JSON files, such indexes names and attribute names, cannot be changed.

Database maintenance

Curity Identity Server relies on external maintenance. Therefore it is important to setup janitor procedures before going into production. The following tables need to be maintained and cleaned up.

Table Data Lifetime
curity-delegations Long lived. Each entry is set to expire when Refresh Token expires. See configuration.
curity-tokens Mixed. Refresh tokens may live long but access tokens shorter.
curity-nonces Short. Usually valid in the range of minutes.
curity-sessions Short. Usually valid in the range of hours.
curity-devices Long lived. Device attributes can be created with an expiry date.

Each item in these tables includes a deletableAt attribute containing the timestamp after which the item can be deleted. Its value is set to the item’s expiration timestamp, added with a retain duration. This retain duration is configurable per table and defines the amount of time, in seconds, that an item should be kept in the database after it is considered expired (e.g. for audit purposes). The deletableAt timestamp is stored using the Unix epoch time format, in seconds. Therefore, it can be used with the DynamoDB TTL feature, which automatically deletes items that are no longer needed without consuming any write throughput. Note that this automatic removal mechanism is not automatically enabled by Curity Identity Server: the items will have the required attribute deletableAt however automatic deletion needs to be explicitly enabled per table.

User Management Service

The Curity Identity Server User Management Service provides a SCIM interface that delegates its requests to the configured data sources, including SCIM queries. The shape and frequency of these queries depends on the usage scenarios, which aren’t known in advance. This ad-hoc querying characteristic doesn’t fit well into a database like DynamoDB, which is typically designed for a well-known set of access patterns. Due to this, the DynamoDB data source presents some limitations when used with the User Management Service.

The DynamoDB data source translates each query into a query plan, which can take one of two forms:

  • One to eight table query operations, taking advantage of the table’s primary key and indexes.
  • Or a single scan operation, which can imply reading the complete table.

Table scans are rather costly, both in time and in consumed read capacity units, specially for large tables. Therefore scans are disabled by default, meaning that any query that requires a table scan will fail. It is however possible to allow scans to be performed via configuration.

The DynamoDB data source will try to map a resource query into a set of DynamoDB table queries, before falling back to a table scan. However, this depends on the set of available indexes, which cannot be changed in the current version.

If a resource query is mapped into N DynamoDB table queries, then there will be at least N distinct requests to the Amazon DynamoDB API. In addition, these requests will not be performed in a single transaction, i.e., will not be isolated from table mutations occurring concurrently.

The index-based pagination used by SCIM also doesn’t fit well in the DynamoDB model, meaning that paginated SCIM queries incur an overhead when using DynamoDB.

Configuration

The DynamoDB data source configuration model defines the following operation aspects:

  • Connectivity:

    • AWS region to use.

    • Credentials to use, which can be defined as:

      • Access key identifier and secret;

        • or, a configuration profile;
        • or, use Amazon EC2 Instance Metadata Service.
      • Override the endpoint that Curity Identity Server uses to communicate with DynamoDB (e.g. useful when using the DynamoDB Local development server).

      • Timeouts:

        • Amount of time in seconds to wait for the execution of an DynamoDB API call to complete, including retries.
        • Amount of time in seconds to wait for each individual request to complete.
  • Query support:

    • Whether table scans are allowed or not.
  • TTL support:

    • Per table additional item retain duration.