Clustering with Docker Compose

Clustering with Docker Compose

operate

Clustering Curity using Docker Compose

Introduction

In most setups, it is required to have multiple runtime nodes. It is often desired to use containers instead of virtual machines or hardware, and this is a guide how to achieve that using Docker and Docker Compose. In this example we use docker-compose to create a cluster with 2 nodes. One admin node that is only used as a configuration service and one runtime node that is used to serve requests for end users.

Docker Compose

Preparing for clustering

Since we are building a clustered environment, we need to have cluster configuration in our docker-compose setup. There are several ways to create a xml file that contains the cluster configuration.

  1. Run Curity locally and download one from the Admin UI
  2. Download and unpack Curity and run the <idsvr_home>/bin/genClust -c <ADMIN_HOSTNAME_OR_IP command
  3. Run this docker command docker run -e CONFIG_SERVICE_HOST=<ADMIN_HOSTNAME_OR_IP> --entrypoint="/opt/idsvr/bin/genclust" curity/idsvr > cluster-conf.xml

In this example, the hostname of the admin node will be admin.

Admin node

Start by creating a docker-compose.yml file in your working directory. Add the admin service to the file.

version: '3.2'
services:
  admin:
    image: curity/idsvr
    ports:
      - 6749:6749

Here, we have created a service and exposed the port we need (used by admin UI and configuration API). Since the communication between runtime and admin will be all internal, there is no need for exposing the communication port (default: 6789).

It will still not be possible to start this admin node however, since we need to add a couple of environment variables. The most important is the PASSWORD which is the admin user's password for access to the configuration admin interfaces. Adding the PASSWORD environment variable also instructs the container to run the unattendedinstaller during its first boot. This will setup the admin user related configuration, enable the admin UI and generate the SSL cert used in the Admin UI.

We also change the start command of the server, to tell it to run with the service-role admin because we do not want this container to act as a runtime node. Update the service definition in docker-compose.yml with the relevant environment variables

version: '3.2'
services:
  admin:
    image: curity/idsvr
    command: ["sh", "-c", "idsvr -s admin"]
    ...
    environment:
    - PASSWORD=<ADMIN_USER_PASSWORD>
  ...

We can also add the CONFIG_ENCRYPTION_KEY environment variable, if we want the sensitive parts of the configuration to be encrypted.

Finally, we need to include the cluster configuration xml we created previously. Assuming this file is in the same folder as your docker-compose.yml, add the following:

version: '3.2'
services:
  admin:
    image: curity/idsvr
    ...
    volumes:
      - ./cluster-conf.xml:/opt/idsvr/etc/init/cluster-conf.xml

Now, you should be able to start the admin service!

docker-compose up admin

Then point your browser to https://localhost:6749/admin to configure your system.

Runtime nodes

In this section we ll add a service definition for the runtime node.

version: '3.2'
services:
  ...
    runtime:
      image: curity/idsvr
      volumes:
        - ./cluster-conf.xml:/opt/idsvr/etc/init/cluster-conf.xml
      environment:
        - SERVICE_ROLE=default
        - CONFIG_ENCRYPTION_KEY="" #(optional)
      ports:
        - 8443:8443
      depends_on:
        - admin

As you can see, there are some differences.

  • Exposed port is 8443, which is the default port for a runtime service.
  • No PASSWORD set. That is important because it instructs the container that this node doesn't require installation.
  • SERVICE_ROLE=default instructs the service that its role will be default.
  • depends_on: admin makes this service start after the admin.

Now start the runtime node

docker-compose up -d runtime

When the service has started, you should be able to access it on https://localhost:8443.

Full docker-compose.yml

version: '3.2'
services:
  admin:
    image: curity/idsvr
    command: ["sh", "-c", "idsvr -s admin"]
    environment:
    - PASSWORD=<ADMIN_USER_PASSWORD>
    - CONFIG_ENCRYPTION_KEY="" # optional
    ports:
      - 6749:6749
    volumes:
      - ./cluster-conf.xml:/opt/idsvr/etc/init/cluster-conf.xml

  runtime:
    image: curity/idsvr
    volumes:
      - ./cluster-conf.xml:/opt/idsvr/etc/init/cluster-conf.xml
    environment:
      - SERVICE_ROLE=default
      - CONFIG_ENCRYPTION_KEY="" # optional
    ports:
      - 8443:8443
    depends_on:
      - admin

Note that the cluster.xml file is mounted here in both services, as it is needed by admin in order to enable cluster mode. Normally, that would be part of the full configuration.

Managing configuration

Exporting configuration

docker-compose exec admin idsvr -d > backup-conf.xml

The full configuration will be in the backup-conf.xml.

Reloading from backup

Put the backup in admin/etc/init, and issue this command:

docker-compose exec admin idsvr -f

This method can also be used to reconfigure using the xml config file.

Useful volumes

While the above example is all that is needed for the system to run, it might helpful to add some volumes to be able to change the system from the host filesystem. For instance, these volumes would get you the log settings, and logs on the host OS.

volumes:
  - ./log4j2.xml:/opt/idsvr/etc/log4j2.xml
  - ./logs/admin-log:/opt/idsvr/var/log

To be able to mount your branding onto all the nodes, this might be useful

volumes:
  - ./template-overrides:/opt/idsvr/usr/share/templates/overrides
  - ./message-overrides:/opt/idsvr/usr/share/messages/overrides
  - ./custom-webroot:/opt/idsvr/usr/share/webroot/custom

Conclusion

We we have seen the process of adding nodes to the Curity Identity Server cluster. This example has a fixed number of nodes, but you should be able to take this further and automate the process of adding new nodes. As for the configuration part, it isn't necessary for it to be dynamically updated with new nodes.You can just as well add a big number of nodes, and start/stop them as wished. Same goes for the startup properties, they don't have to be created on the fly.

Resources

For more details docker see the Docker documentation

For more details about running Curity in a container see Curity Documentation

Was this page helpful?