Upgrading from 4.5.X to 5.0.0

Version 5.0.0 contains a number of changes to the configuration model. In case you want to upgrade existing XML configuration, read the paragraph Upgrading the XML Configuration

Upgrading the XML Configuration

Description Applicability Change Method
Administration service configuration Administration Web UI and RESTCONF Automatic using XSLT, or manual
Simple API configuration Authentication profile Automatic using XSLT, or manual
Uptime Uptime via REST(CONF) API Manual
OpenID Connect Authenticator Authentication profile Manual

As with previous releases, the XML configuration can be converted using a provided XSLT and its associated auto-upgrade.sh script. To run the XSLT on a configuration data set, run the auto-upgrade.sh script, like this:

Listing 84 Upgrading from 4.5.X to 5.0.0 using the provided XSLT and auto-upgrade.sh
$ $INSTALL_DIR/misc/upgrade/4.5-to-5.0/auto-upgrade.sh $IDSVR_HOME/etc/init/some-config.xml

Administration service configuration

The administration service configuration model, which includes both the administration Web UI interface and the RESTCONF interface, was changed to support added functionality, namely to permit the individual enable/disable of these interfaces.

  • The /environments/environment/admin-web-ui element is replaced by the /environments/environment/admin-service element.
  • The HTTP configuration elements listening-host, listening-port, and ssl-server-keystore, were moved inside the new /environments/environment/admin-service/http sub-element.
  • The /environments/environment/admin-service/http contains two new sub-elements, web-ui and restconf, to individually enable the administration Web UI and the RESTCONF interface, respectively. At least one of these elements needs to be present. To disable both administration Web UI and RESTCONF, then the /environments/environment/admin-service/http element needs to be absent. The old (deprecated) REST interface is also enabled using the restconf element.
Listing 85 Example of a previous configuration for admin-web-ui.
<admin-web-ui>
    <listening-host>0.0.0.0</listening-host>
    <listening-port>6749</listening-port>
    <ssl-server-keystore>admin-server-ssl-key-store</ssl-server-keystore>
</admin-web-ui>
Listing 86 Equivalent configuration using the new model
<admin-service>
    <http>
        <listening-host>0.0.0.0</listening-host>
        <listening-port>6749</listening-port>
        <ssl-server-keystore>admin-server-ssl-key-store</ssl-server-keystore>
        <web-ui />
        <restconf />
    </http>
</admin-service>

Notice that the previous configuration model is not directly compatible with the new 5.0 model. Due to this, a configuration containing the /environments/environment/admin-web-ui needs to be migrated, manually or using the auto-upgrade.sh script, before being used on a system running 5.0.

Simple API configuration

The Simple API authentication protocol configuration model was modified due to changes on the login token representation. Before login tokens were JWT-based and now are reference-based. Due to this, the signature-algorithm and signing-key elements are no longer applicable. The element jwt-expiration-time was also removed and is now a constant. As a result, the simple-api is now an empty element. If present, the Simple API protocol will be enabled, otherwise it will be disabled.

Listing 87 Example of a previous configuration for simple-api.
<authentication-service xmlns="https://curity.se/ns/conf/profile/authentication">
  <protocols>
    <protocol>
      <id>simple1</id>
      <simple-api>
        <jwt-expiration-time>300</jwt-expiration-time>
        <signature-algorithm>RS256</signature-algorithm>
        <signing-key>default-signing-key</signing-key>
      </simple-api>
    </protocol>
  </protocols>
</authentication-service>
Listing 88 Equivalent configuration using the new model.
<authentication-service xmlns="https://curity.se/ns/conf/profile/authentication">
  <protocols>
    <protocol>
      <id>simple1</id>
      <simple-api/>
    </protocol>
  </protocols>
</authentication-service>

Notice that the previous configuration model is not directly compatible with the new 5.0 model. Due to this, a configuration containing the simple-api needs to be migrated, manually or using the auto-upgrade.sh script, before being used on a system running 5.0.

Uptime State

Previously, when using the RESTCONF API or CLI, the uptime of a down node was reported as down. This has changed because the node may be up but not connected to the admin node. For this reason, the uptime of a down node is now reported as unavailable. Version 4.5.3 introduced a new field in the response called status. This can be used to see if the node is connected or not. When connected, the up time will reported as before. An example of the changes in the CLI is shown in the following listings:

Listing 89 Example of uptime response in CLI before 5.0
admin@spruce> show environments environment services runtime-service
ID                    NAME         ROLE         BOOT TIME   STATUS        UPTIME
--------------------------------------------------------------------------------------------------------
TestServer1-laE7kpp5  TestServer1  TestServer1  1578480460  connected     0 days, 0 hrs, 7 min, 42 sec
second-role-7ncZFdoV  second-node  other        1578480588  disconnected  down
Listing 90 Example of uptime response in CLI in 5.0
admin@spruce> show environments environment services runtime-service
ID                    NAME         ROLE         BOOT TIME   STATUS        UPTIME
--------------------------------------------------------------------------------------------------------
TestServer1-laE7kpp5  TestServer1  TestServer1  1578480460  connected     0 days, 0 hrs, 7 min, 42 sec
second-node-7ncZFdoV  second-node  other        1578480588  disconnected  unavailable

An example of the change in the RESTCONF API is shown in the following listings:

Listing 91 Example of uptime response in RESTCONF API before 5.0
{
    "base:runtime-service": [
        {
            "id": "TestServer1-laE7kpp5",
            "name": "TestServer1",
            "role": "TestServer1",
            "boot-time": 1578480460,
            "status": "connected",
            "uptime": "0 days, 0 hrs, 7 min, 32 sec"
        },
        {
            "id": "second-node-7ncZFdoV",
            "name": "second-node",
            "role": "other",
            "boot-time": 1578480588,
            "status": "disconnected",
            "uptime": "down"
        }
    ]
}
Listing 92 Example of uptime response in RESTCONF API in 5.0
{
    "base:runtime-service": [
        {
            "id": "TestServer1-laE7kpp5",
            "name": "TestServer1",
            "role": "TestServer1",
            "boot-time": 1578480460,
            "status": "connected",
            "uptime": "0 days, 0 hrs, 7 min, 32 sec"
        },
        {
            "id": "second-node-7ncZFdoV",
            "name": "second-node",
            "role": "other",
            "boot-time": 1578480588,
            "status": "disconnected",
            "uptime": "unavailable"
        }
    ]
}

OpenID Connect Authenticator

The configuration model for the key to use when decrypting an encrypted id token has been updated to match the configuration for fetch-userinfo. The node called `id-token-encryption-key has been renamed to encryption-key.

Listing 93 Example of a previous configuration for oidc-authenticator.
<authenticator>
    <id>oidc-encrypted-id</id>
    <authentication-context-class-reference>oidc-plain-userinfo</authentication-context-class-reference>
    <oidc xmlns="https://curity.se/ns/conf/authenticators/oidc">
        <encrypted-id-token>
            <id-token-decryption-key>testDecryptionKeyJwe</id-token-decryption-key>
        </encrypted-id-token>
        <configuration-url>https://localhost:7777/.well-known/openid-configuration</configuration-url>
        <client-id>virt-client</client-id>
        <client-secret>Password1</client-secret>
        <http-client>trustStoreHttpClient</http-client>
        <fetch-userinfo>
            <plain />
        </fetch-userinfo>
    </oidc>
</authenticator>
Listing 94 Equivalent configuration using the new model.
<authenticator>
    <id>oidc-encrypted-id</id>
    <authentication-context-class-reference>oidc-plain-userinfo</authentication-context-class-reference>
    <oidc xmlns="https://curity.se/ns/conf/authenticators/oidc">
        <encrypted-id-token>
            <decryption-key>testDecryptionKeyJwe</decryption-key>
        </encrypted-id-token>
        <configuration-url>https://localhost:7777/.well-known/openid-configuration</configuration-url>
        <client-id>virt-client</client-id>
        <client-secret>Password1</client-secret>
        <http-client>trustStoreHttpClient</http-client>
        <fetch-userinfo>
            <plain />
        </fetch-userinfo>
    </oidc>
</authenticator>

SDK

The URL of the Maven repository where the SDK is downloaded from should be changed to https://nexus.curity.se/repository/customer-release-repo. The old URL will continue to work, but is planned to be decommissioned in the future. To prepare for this, customers obtaining the SDK JAR file via the Maven package management system should use the new URL above.

The following note only affects custom plugins that directly use the ``SessionDataAccessProvider`` service:

Several methods of the se.curity.identityserver.sdk.datasource.SessionDataAccessProvider class that had been deprecated since version 3.x have been finally removed. Plugins still using these deprecated methods may need to be re-compiled against the new SDK version.

Attribute container classes, namely MapAttributeValue, Attributes, and derived classes, can now contain multiple attributes with the same name, as long as they have distinct authorities. As a consequence, the behavior of the with and append methods is changed, since the authority is also taken into consideration to decide if an attribute already exists in the container or not. For instance, appending an attribute with name n and authority a1 to a container with an attribute with the same name n and a different authority a0 will result in a container with two attributes with name n. If all manipulated attributes have the same authority or don’t have authority (i.e. Attribute.NO_AUTHORITY), then the behavior remains the same.

Updating Databases

A change to the devices table is introduced in 5.0 adding a new attributes column which will be used to store any additional attributes that are specific to certain type of devices. Migration scripts for each supported database is provided with the release in the $IDSVR_INSTALL/examples/upgrade/4.5-to-5.0/ directory. A change has been made in the postgres database. The attributes column in the accounts, dynamically_registered_clients and buckets tables has been changed to type JSONB and indexes have been created for these columns in order to improve performance. A migration script is provided with the release in the $IDSVR_INSTALL/system-admin-guide/upgrade/4.5-to-5.0/ directory.

The following change may only affect users of Microsoft SQL Server, Azure SQL and other databases in the same family:

A modification was made to the query used to update sessions in the TSQL dialect, which affects users of Microsoft’s SQL Server and Azure SQL, for example. This was necessary in order to avoid deadlock issues (which caused one of the would-be-deadlocked connections to get killed) in the database negatively impacting end users.

Besides the query being modified to reduce the probability of deadlocks, the Curity database client was also modified to retry failed session updates, as per Microsoft recommendations, in case a pre-emptive deadlock error is issued by the database.

To check if your database is having deadlock issues and causing the Curity database client to keep retrying session updates, enable DEBUG-level logging for the io.curity.identityserver.plugin.data.access.jdbc.JdbcSessionDataAccessProvider class and look for messages like Error when performing operation. If this proves to be occurring frequently, it may be possible to change default database settings to make such problems less likely. Visit the Microsoft documentation for details about this issue.

Optional, in case the following error is observed (this only affects PostgresDB): org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "idx_sessions_id_expires"

This error doesn’t cause any of the existing sessions to be lost or get corrupt. It will however cause a failure to update the session with the new session data. It is caused in cases of high levels of concurrency when a session with the same id and expires values is updated and postgres fails to update the index due to the UNIQUE constraint. The IDX_SESSIONS_ID_EXPIRES index doesn’t need to have the UNIQUE constraint in the sessions table because the id is a primary key and is already unique. There is an optional part in the migration script provided with the release in the $IDSVR_INSTALL/upgrade/4.5-to-5.0 directory which contains the SQL commands to remove the existing index and recreate it without the UNIQUE constraint. Note that it is advised to purge the sessions table of the expired entries so that the index is not rebuilt with unnecessary data.

Optional to boost performance in CockroachDB during high levels of concurrency:

The IDX_SESSIONS_ID_EXPIRES doesn’t need to have the UNIQUE constraint since it is created from both the id and the expires column value and since the id is the primary key of the sessions table it is unique and thus there wouldn’t be two duplicate sessions. The migration script specific to CockroachDB in the $IDSVR_INSTALL/upgrade/4.5-to-5.0 directory has an optional section with the SQL commands to remove the existing index and recreate it without the UNIQUE constraint. Note that it is advised to purge the sessions table of the expired entries so that the index is not rebuilt with unnecessary data.

Other

The Curity Identity Server depends on an external, third-party dependency called libcrypto. This comes with libSSL, OpenSSL, SSH, etc. The version required has changed from version 1.0.X to 1.1.X. This version of the library needs to be present on the host machine of all run-time and admin nodes. It is also needed in macOS test computers.

The genpkeys command has been removed in this release. It has been obsolete since version 4.1.0. If it is still being used at this point, it probably does not need to be. When upgrading, any custom invocation of this script should be removed. If that is not possible, for some reason, it is possible to add a script that does nothing to the path of the user invoking the script.

Cookie handling was changed to support newer Chrome browser versions, where cookies have SameSite behavior set to Lax by default (see The Chromium Projects: SameSite Updates). As a consequences some cookies may now be set with the attribute SameSite explicitly defined as None, to allow their use in cross site scenarios, namely with framing. In order to support other browsers that mishandle the SameSite attribute, Curity Identity Server may also set additional cookies (e.g. _sessionid). The changes introduced by newer Chrome browser versions also mean that cross site scenarios on those browsers are only possible if using HTTPS.

To use the Prometheus-compatible monitoring endpoint, the port (4466 by default) must be accessible from the remote polling device. This port is exposed by each run-time and admin node.