The DynamoDB data source uses the Amazon DynamoDB service to store and retrieve information.
Curity Identity Server is shipped with JSON files containing the schema information required to create the necessary DynamoDB tables. These files are located in the installation, under <IDSVR>/etc, and use a format compatible with the create-table AWS CLI command.
There is one such file per table, containing:
Each file also includes ProvisionedThroughput fields defining the table and index provisioned capacity.
The values present in these fields are illustrative, and must be adapted to the operational requirements of each deployment environment.
The table names present in these files can also be changed to reflect an optional table name prefix, which can be defined in the data source configuration.
This allows for different data sources using distinct tables under the same AWS account.
The remaining fields in the JSON files, such indexes names and attribute names, cannot be changed.
Curity Identity Server relies on external maintenance. Therefore it is important to setup janitor procedures before going into production. The following tables need to be maintained and cleaned up.
Each item in these tables includes a deletableAt attribute containing the timestamp after which the item can be deleted. Its value is set to the item’s expiration timestamp, added with a retain duration. This retain duration is configurable per table and defines the amount of time, in seconds, that an item should be kept in the database after it is considered expired (e.g. for audit purposes).
The deletableAt timestamp is stored using the Unix epoch time format, in seconds.
Therefore, it can be used with the DynamoDB TTL feature, which automatically deletes items that are no longer needed without consuming any write throughput.
Note that this automatic removal mechanism is not automatically enabled by Curity Identity Server: the items will have the required attribute deletableAt however automatic deletion needs to be explicitly enabled per table.
The Curity Identity Server User Management Service provides a SCIM interface that delegates its requests to the configured data sources, including SCIM queries. The shape and frequency of these queries depends on the usage scenarios, which aren’t known in advance. This ad-hoc querying characteristic doesn’t fit well into a database like DynamoDB, which is typically designed for a well-known set of access patterns. Due to this, the DynamoDB data source presents some limitations when used with the User Management Service.
The DynamoDB data source translates each query into a query plan, which can take one of two forms:
Table scans are rather costly, both in time and in consumed read capacity units, specially for large tables. Therefore scans are disabled by default, meaning that any query that requires a table scan will fail.
It is however possible to allow scans to be performed via configuration.
The DynamoDB data source will try to map a resource query into a set of DynamoDB table queries, before falling back to a table scan.
However, this depends on the set of available indexes, which cannot be changed in the current version.
If a resource query is mapped into N DynamoDB table queries, then there will be at least N distinct requests to the Amazon DynamoDB API. In addition, these requests will not be performed in a single transaction, i.e., will not be isolated from table mutations occurring concurrently.
The index-based pagination used by SCIM also doesn’t fit well in the DynamoDB model, meaning that paginated SCIM queries incur an overhead when using DynamoDB.
The DynamoDB data source configuration model defines the following operation aspects: