Blog
Upgrading Your IAM System the Right Way
The modern way to implement software security is to use the OAuth family of specifications. This provides the best architecture capabilities and also many advanced design patterns for managing security-related use cases. This helps to enable a state-of-the-art architecture, with efficient and simple application code, and the difficult security work outsourced to your Identity and Access Management (IAM) system.
You then need to manage the IAM system and ensure that it remains reliable over time. It must be highly available, since apps will connect to it frequently, whenever a user authenticates or the app needs to refresh a user’s access token. In the event of outages or misconfiguration, user-facing apps can experience problems. This is a source of concern for many software companies. In this article, I will describe some design patterns and best practices for maximizing reliability.
Software Pipelines
At most companies, there are multiple stages to a software pipeline, and typically different people are accountable for the reliability of different stages. A typical company setup might look like this, where there are six main stages to the pipeline:
The IAM settings then need to be promoted down a pipeline when changes occur. For example, this might involve registering new OAuth clients, adding an Identity Provider (IDP) connection to a business partner, adding a new token issuing policy, or upgrading the entire IAM system to a new version.
Over time the IAM system will contain many intricate security settings, and understanding of them will vary among technical staff. Developers often have a solid understanding of OAuth and OpenID Connect messages, since they are responsible for integrating them. Meanwhile, DevOps may have better knowledge of infrastructure areas such as clustering, certificates and SAML integrations.
The main source of problems is the potential for IAM settings to become mismatched between stages. This is a data management problem, and to solve it we first need to identify the cause, starting with classifying the different areas of identity data.
Identity Data
Many Identity and Access Management systems give you components similar to this. Each stage of the pipeline has the main Identity system that provides OAuth endpoints to applications. There is also a database, an Admin UI and an Administration API:
There are various types of data in an IAM system. The first main area is information that belongs in a database. This includes user accounts, token and session-related information and audit data. Each stage of the pipeline has different database records.
IAM data also contains a second main area, for configuration. This includes information about OAuth clients, such as the client ID, client secret, scopes, claims, redirect URI, and token lifetimes. More complex clients, such as those using B2B or financial-grade flows, will include or reference crypto keys. A token issuing policy will include various properties such as algorithms and signing keys. In larger organizations, the identity configuration will also grow across department boundaries and may involve multiple deployments of OAuth endpoints.
When the configuration is stored in a database it leads to problems. In the past, I have seen production upgrade processes requiring a DevOps person to run scripts or programs to call the Admin API to update configuration. In places, API automation was not supported, so the DevOps person also needed to make manual updates via the Admin UI. These configurations could be non-trivial, in which case the production upgrade process was stressful.
Usually, almost all configuration is the same for every stage of the pipeline. A few values are environment-specific, such as redirect URIs and client secrets. To ensure that the settings are deployed reliably, you must avoid configuring all of the main settings multiple times. The modern option is to follow an infrastructure-as-code approach.
IAM Configuration as Code
The configuration for the Identity and Access Management system should use a plaintext readable format and be stored in source control. You will then have an audit trail of how the configuration has changed over time. This also enables GitOps processes, where all changes to IAM configuration undergo people reviews:
Your IAM system should enable you to split the configuration into multiple files, as in this screenshot, and also to parameterize the configuration, to avoid duplication. In the Curity Identity Server, an XML plaintext format is used, with placeholders for parameters:
Builds and Deployment
Modern continuous delivery processes build binaries only once, then combine them with the environment-specific configuration at deployment time. When the parameterized configuration is used, it can be included in the binary representing the IAM system, such as a customized Docker image:
Environment-specific differences should be managed in plaintext files, and can also be stored alongside the parameterized configuration in source control. One option might be to store the values for a staging environment in a ‘staging.env’ file, containing key-value pairs:
When deploying to production, you simply combine the last tested Docker image with the environment-specific settings, which will typically be deployed as environment variables. Note also the use of tags to ensure that production does not include any settings under test:
Since the vast majority of settings are not duplicated, and there is no need to perform manual actions, the likelihood of misconfiguration is significantly reduced. The main thing that could go wrong is a typo in environment-specific data, which can be quickly resolved by correcting it in source control, then re-triggering the deployment.
Finalizing Technical Behaviour
There is some additional technical behavior to consider. Firstly, some environment-specific data contains secrets, which should be managed in a secure vault, rather than stored in source control. The deployment process must therefore also interact with the vault, to ensure that secrets are protected before being used as environment variables, to prevent the disclosure of sensitive information.
APP_CLIENT_SECRET=...
COOKIE_ENCRYPTION_KEY=...
Finally, you need to have a process for managing IAM configuration changes. These are usually made using the Admin UI, where the meaning of security settings is visually clearest. When configuration changes, the IAM system should give you a mechanism to update the source of truth. One option is to create an automated pull request to enable the following end-to-end flow: Details on how to implement these techniques for the Curity Identity Server are provided in the links at the end of this article.
Finalise People Processes
An IAM system is also an operational product, so technical solutions, such as automated configuration backups, will not solve all of your use cases. It is common for application-level changes to be initiated by engineers, as part of a development task. Meanwhile, DevOps are also likely to need to make their own changes, such as integrating an external identity provider:
It is recommended to think through your use cases and plan for this type of scenario at a people level. The diagram suggests one option, based on multiple source control branches. This should provide visibility to all interested parties, so that they know which IAM configuration changes are coming soon.
Finally, aim to keep the structure of the IAM configuration the same at all stages of the pipeline if possible. If there are five partner IDPs configured in production, and three internal engineering apps, aim to deploy equivalent configuration data everywhere, even if some entries are disabled in some stages of the pipeline.
Conclusion
Promoting an IAM system down a pipeline is not a trivial requirement, since some intricate security configurations must be managed. Your IAM provider must implement a solid architectural design, with a clean separation of concerns. This will minimize the scope for data mismatches that could impact your user-facing applications.
Aim to keep configuration equivalent at each stage. This will ensure that both changes to configuration settings, and upgrades to the IAM system itself, are all practiced frequently, and stabilized. This will take the pain out of production upgrades.
For further information on this topic, and to run an end-to-end solution on your local computer, see these resources on promoting the Curity Identity Server down a pipeline:
