Integrate Identity with Business Data
On this page
Introduction to Identity and Business Data Integration
The Identity and Access Management (IAM) Primer article explains how to get started in terms of technical components. The security complexity is then externalized from your applications to an Identity Server:
Once this integration is done, you are on track to realize a modern architecture with state-of-the-art security features:
|Flexible Authentication||End users can authenticate and verify their identity in multiple ways, including use of third party providers and multiple factors.|
|Advanced Flows||Advanced OpenID Connect flows such as CIBA or integrating with an external SAML identity provider can be easily enabled.|
|Secure Data Access||APIs receive digitally verifiable JWT access tokens that provide a useful user identity and claims needed for enforcing business rules.|
|Simple Apps||The complex security plumbing is externalized from your APIs, web and mobile apps, which require only simple OAuth code.|
How to perform Identity Mapping
Most real-world systems store some kind of transaction records against users, and in many systems, this data may have existed for a decade or more. Usually a key part of the IAM integration is to ensure that you can maintain the user data history for both existing and future transactions:
This article suggests some useful approaches for identifying requirements, designing data migration, and integrating in a risk-free manner. There are multiple possible solutions, and the options you choose will depend on your use cases and how you want to manage data for users.
There are multiple existing IAM architectures, many of which are not based on OAuth, or only use it partially, without realizing its full potential:
|Website Security||Applications implement their own logins and password management but have limited authentication capabilities.|
|Social Identity Providers||Applications use different sign-in methods but lack support for more advanced flows.|
|OAuth and OpenID Connect||An Authorization Server is used in a basic manner, but with some pain points and workarounds.|
|Fully Integrated||The system uses OAuth as a toolbox to fully integrate the business and identity domains.|
With the correct steps, any type of system can update to the fully integrated architecture. However, it requires a design where the business domain and identity domain work together to ensure business continuity. This article will describe some common patterns to enable you to better answer the following questions:
- What is the integration behavior my organization needs from the IAM system?
- What are the implementation steps to migrate users and apps in a risk-free manner?
Design User Account Data
In an OAuth architecture, you first need to decide which user fields you will store in the IAM system and which to keep in your business data. There are multiple ways this could be designed, and some fields could exist in both locations, but each field should have a single source of truth. Some example fields are listed in the following sections.
Online Retail System
In a relatively simple architecture, you may decide that all of the main user data should be migrated to the identity data:
|User Field||Source of Truth||Description|
|User ID||Domain-Specific Data||Most commonly an existing database primary key|
|Subject||Identity Data||A unique identifier for each user, generated by the IAM system|
|Username||Identity Data||The value a user types in a login screen|
|Given + Family Name||Identity Data||Used in login screens or emails to the user, but may also need to be stored in business data|
|Identity Data||Used to send email verification messages, but may also need to be stored in business data|
|Country Code||Identity Data||In some scenarios, the user's country may be important later when dealing with data sovereignty|
|Subscription Level||Identity Data||An application-specific value assigned to the user|
|Consent Flag(s)||Identity Data||Flags related to privacy or legal terms the user must consent to, such as GDPR in Europe|
|Consent Time(s)||Identity Data||When the user last consented to the legal terms|
Corporate software platforms sometimes consist of multiple mature products with many extra fields used for authorization. You may prefer to manage some user fields within your business data, either to reduce the impact of the IAM migration, or because you feel that future changes will be easier to manage there:
|User Field||Source of Truth||Description|
|User ID (Product A)||Domain-Specific Data||A database primary key in this product|
|Roles (Product A)||Domain-Specific Data||An array of roles that a user can be assigned for this product|
|Permissions (Product A)||Domain-Specific Data||A collection of permissions that can potentially be calculated in complex ways|
|User ID (Product B)||Domain-Specific Data||A database primary key in this product|
|Tenant ID (Product B)||Domain-Specific Data||Some products may partition their data storage by partner or data owner|
|User Group (Product B)||Domain-Specific Data||The group to which a user belongs in this product|
|Permissions (Product B)||Domain-Specific Data||A collection of permissions that can potentially be calculated in complex ways|
A medical system may have its own particular user information, some of which is used during authentication workflows. You may prefer to keep these extra fields in the domain-specific data:
|User Field||Source of Truth||Description|
|Patient Identification Number||Domain-Specific Data||A unique identifier for the patient|
|Date of Birth||Domain-Specific Data||Used as an additional authentication factor, and also stored against medical records|
|Social Security Number||Domain-Specific Data||Used as an additional authentication factor, and also stored against medical records|
The Subject Claim
When a user is created in the IAM system, an Account ID is assigned, which is usually a generated identifier such as a UUID. This will remain constant after creation, even if a user changes their name. It can then be used in access tokens as the subject claim.
It is also possible to use Pairwise Pseudonymous Identifiers (PPID), which is considered the best option from a privacy viewpoint, so use that option if you can. In many business systems, however, the Account ID may be preferred to reliably map the user to the business data.
When data is stored at rest, ensure that an ETL process can continue to form a complete picture of user data by joining the two data sources together. This can be done by either adding the business identifier to the IAM data or vice versa. The three example scenarios might be handled as follows:
The subject claim should not be used as a direct primary or foreign key in your business data. This allows the option to switch to a new IAM vendor in the future, which will assign its own generated UUIDs to users.
API User Identification
User account modeling also needs to account for how APIs will manage requests from applications, since APIs will need to read, write and extend business data stored against users:
|Online Retail||An internet retailer typically only shows the logged-in user their own history of purchases.|
|Corporate||A corporate system may record a history of which employees changed which corporate assets.|
|Medical||A logged-in patient must only be able to access their own medical history.|
By default, an OAuth system identifies a user only by the subject claim:
However, this is often insufficient for an API in business terms, so you will likely need to provide more useful business identifiers to APIs in access tokens:
APIs will then digitally verify the JWT in the standard way and can trust the claims received, after which business data can be updated naturally.
Existing User Migration
The user account data is just a blueprint at this stage, and the information is likely to be stored differently in the starting architecture. To migrate to the preferred structure, you need to get a full picture of the existing data for each user, then populate data sources via these steps:
|Select existing data||Select the users from the source database(s) into a list, which will be iterated over.|
|Insert each user into the IAM data||The IAM system should have a SCIM API to enable you to insert a user, then receive back the account identifier.|
|Update each user in the business data||If required, insert the account ID into the business data and possibly make other updates.|
This type of migration process often only makes additive changes to business data. It can also be run multiple times and retried in the event of errors. Once complete, you can continue to get a full data picture of all fields at rest if you need to do further IAM migrations in the future.
Credentials and authentication methods are managed separately to the core user account data. When moving to an IAM system, it is often possible to leave credentials such as hashed passwords or LDAP stores in the existing location and for the IAM system to use that as a data source. Another option is to import hashed passwords into a database managed by the IAM system.
Either of these approaches will ensure that end-users who sign in via passwords do not experience problems when the updated IAM system is rolled out. If neither of these are possible, then users can be configured to reset their password on the first login to the IAM system, which only impacts each user once.
Existing User Flows
Once the data separation is understood, you need to design the end-to-end flow for existing users to get a clear idea of how user experience and data integration will be updated. This may vary depending on your use cases, so some examples are presented below.
After the rollout, the end-to-end flow for an internet retail system might look like this, where the user continues to sign in with their existing password:
Social Identity Providers
If the starting architecture used social identity providers directly, the user could continue to sign in with the same experience, e.g., via Google or Facebook.
For this migration logic to work, the source architecture needs to contain an existing unique identifier for the social user. This can be an email or the social provider's subject claim, but it must be imported into the identity data.
Corporate systems often use an existing company identity solution, which might not be OAuth-based and is perhaps provided by a cloud vendor. This can continue to be used after the migration since the IAM system's main role is that of an OAuth Authorization Server, which should be able to integrate with any standards-based identity provider:
An interesting point shown here is that with the corporate user account design, the Identity Server needs to be able to reach out to domain-specific APIs or data sources at the time of token issuance. This is a crucial capability to look for since you will often want to add business fields to access tokens in more complex systems.
Medical or Government Logins
This type of system is a little more interesting since multi-factor authentication is used, and some of these factors may require additional custom screens to collect patient identifiers or government eIDs and then verify them against domain-specific data:
Therefore, the IAM system should have extensibility features needed to enable this so that companies can meet their industry-specific strong authentication requirements. It would also be critical in this type of system for the login user experience to use medical or government branding so that authentication screens have the same look and feel as application screens.
Once running a modern OAuth-based architecture, you will also want to reliably onboard future users. This is likely to require data updates in both the identity data and the business data. The technical goal is to perform the difficult work of proving that the user is new in the identity server, then provide simple choices to APIs.
Online Retail System
In this type of system, each user is typically allowed to register and create their own credential. They are then asked to confirm it via email or SMS verification. This creates a basic identity, but you may not really know who the user is.
Once the basic verification is completed, you may need to create the user in both data sources. One option is to reach out to your business data during the registration process. This ensures that the token returned to APIs contains the claims needed to identify the user in business terms:
You will want to ensure that all business transactions for the basic identity remain stored together, including cases where the end-user forgets they are registered and repeats the sign-up process. A custom action could look up a verified identity in the account data, e.g., by finding the email, then ensure that the user gets linked back to their existing data history.
In corporate systems, it is common for new users to be provisioned by an administrator, which might involve using a script or Admin UI that calls back-end services. The logic for creating a user is likely to be similar to that described earlier for user migrations:
|Save to the IAM system||Use SCIM endpoints to insert a user with custom attributes, then receive back the account identifier, used as a subject claim.|
|Save to the business system||Create the business user, with assigned roles and permissions, and optionally include the subject claim to link to IAM data.|
In this type of setup, a user may have a valid IAM account and access to one product but not another. You will want to avoid a UX where a user signs in successfully but then experiences an error upon return to the application, and this can be achieved cleanly via an authentication action:
Medical or Government System
Internet users may also be able to sign themselves up to this type of system, but they will need to use strong authentication factors, such as providing eIDs or scanning a passport. This registration process will again require the ability to use custom screens and actions.
The below workflow shows an alternative way to create new users in business data after authentication. The API receives empty domain-specific claims and can trust that the hard work of identifying the user as new has been done in the IAM system. User creation and onboarding can then occur within the system, perhaps by taking the user through some welcome screens.
Legal and Regulatory
These days, a user account data design also needs to include legal and regulatory requirements fields. This is another area where the IAM system provides useful features that can enable you to implement them with the best user experience without adding complexity to your apps:
|Legal Terms and Conditions||The company's terms and conditions can be integrated into registration forms when users complete a registration process.|
|User Privacy||Users typically need to consent to regulations such as GDPR in Europe, which can be managed via a post-authentication action that presents a screen, then denies access unless accepted.|
|User Consent||OAuth consent forms should be customizable via actions, and are commonly used in payment scenarios to ask a user for access to their bank account, to meet regulations such as PSD2.|
|Data Sovereignty||Keeping a user's data in their home country may be a legal requirement. In this case, it is recommended to include a country code in IAM data, then use that as the basis for Dynamic User Routing of API requests.|
User Data Updates
Occasionally users need to change their personal information, and in an OAuth architecture, this may include updating fields in both the identity data and business data. This might be managed by displaying user information on a single screen, then saving data down to both data sources. This is straightforward since the primary technical identifiers for the user remain unchanged so that you can handle scenarios such as name changes due to marriage.
In a modern IAM system, the user can potentially sign in via different authentication methods, which changes the first factor in the authentication workflow. If a user signs in via the default Username Password option, then at some later point changes their primary factor to WebAuthn, this could create two distinct user accounts, with different subject claims:
This is a common problem when social identity providers are coded directly into applications, and the standard solution is to use the Account Linking features of the IAM system. This requires custom actions to work with attributes and match credentials to accounts, using two high-level techniques:
|Deterministic Logic||Use either a built-in action or custom script if it is possible to reliably match users to existing accounts based on data. This might work when users have to use the same email as an identifier for two different authentication methods.|
|Involving the User||Otherwise, run an action that involves the user, who will be indirectly asked to provide their existing subject claim. In the above diagram, on the first WebAuthn login, the user may be prompted to sign in with a username and password to link the WebAuthn public key data to the user account.|
With the correct IAM capabilities, along with some scripting to manage attribute matching, you will be able to ensure that all forms of authentication link to the same IAM identity, which in turn links to the same records in your business data:
This article has identified several areas where an IAM system should enable customization. It is recommended to avoid committing to a vendor solution until you are sure it will give you the business continuity you need. In particular, the following capabilities are worth verifying early on:
|Data Storage||The IAM system must be able to store custom attributes against user accounts|
|Registration||Ensure that users can provide custom data during onboarding for self-sign-up scenarios|
|Authentication||The ability to customize behavior via scripted actions, custom screens, and calls to business APIs|
|Login Branding||A professional login user experience so that IAM screens look like an integrated part of your apps|
|Consent||Look for a system with good options for managing user consent after authentication|
|Account Linking||Logic and user actions to ensure that the subject claim in tokens remains consistent, regardless of the authentication method used|
|Token Issuing||The ability to include domain-specific claims in access tokens at the time of issuance, to enable reliable data updates in APIs|
The Curity Identity Server
At Curity, we are passionate about supporting the most up-to-date standards and flows. Still, we also understand that customers need extensibility features in important places when integrating IAM into their business. The following building blocks enable the behaviors described in this article:
|Authenticator||A number of built-in authenticators exist for popular social providers, OpenID Connect and SAML.|
|UX Branding||All screens used for authentication, consent and registration can be branded separately per client application.|
|Credential Manager||Credentials can be read, written and verified using one or more credential managers, each of which maps to a data source.|
|Account Manager||Account data can be read from one or more account managers, each of which maps to a data source.|
|Consentor||The process around capturing consent is fully customizable via custom screens and actions.|
Each type of component is also available in the Java SDK, which enables you to build completely bespoke implementations if ever needed. There are many custom examples in our GitHub repositories that can be used as a guide for your own solutions.
Identity, privacy, and meeting regulations are difficult areas, but software companies need to deal with them somehow. Rather than doing so in your application code, a good IAM system can provide years of effort at a low cost. Your applications will then have simpler code, richer security flows, and access to some powerful new design patterns.
A good OAuth solution requires a toolbox that you can adapt to meet your business requirements in a clean way. Understand your preferred end-to-end flows and then ensure that the extensibility options you need are possible with your chosen provider. Your migration to the new IAM system might then begin with these risk-free steps:
- Design user account data and access token claims
- Populate IAM user account data in the new system
- Customize authentication workflows to reliably identify users
- Customize token issuance to enable APIs to map identities to business data
- Implement OAuth flows in proof-of-concept (POC) applications before updating flagship apps