Integrate Identity with Business Data

Integrate Identity with Business Data

Overview

The Identity and Access Management (IAM) Primer article explains how to get started in terms of technical components. The security complexity is then externalized from your applications to an Identity Server:

Components

Once this integration is done, you are on track to realize a modern architecture with state-of-the-art security features:

Business BenefitDescription
Flexible AuthenticationEnd users can authenticate and verify their identity in multiple ways, including use of third party providers and multiple factors.
Advanced FlowsAdvanced OpenID Connect flows such as CIBA or integrating with an external SAML identity provider can be easily enabled.
Secure Data AccessAPIs receive digitally verifiable JWT access tokens that provide a useful user identity and claims needed for enforcing business rules.
Simple AppsThe complex security plumbing is externalized from your APIs, web and mobile apps, which require only simple OAuth code.

Identity Mapping

Most real-world systems store some kind of transaction records against users, and in many systems, this data may have existed for a decade or more. Usually a key part of the IAM integration is to ensure that you can maintain the user data history for both existing and future transactions:

Business Data

This article suggests some useful approaches for identifying requirements, designing data migration, and integrating in a risk-free manner. There are multiple possible solutions, and the options you choose will depend on your use cases and how you want to manage data for users.

Starting Architecture

There are multiple existing IAM architectures, many of which are not based on OAuth, or only use it partially, without realizing its full potential:

ArchitectureDescription
Website SecurityApplications implement their own logins and password management but have limited authentication capabilities.
Social Identity ProvidersApplications use different sign-in methods but lack support for more advanced flows.
OAuth and OpenID ConnectAn Authorization Server is used in a basic manner, but with some pain points and workarounds.
Fully IntegratedThe system uses OAuth as a toolbox to fully integrate the business and identity domains.

With the correct steps, any type of system can update to the fully integrated architecture. However, it requires a design where the business domain and identity domain work together to ensure business continuity. This article will describe some common patterns to enable you to better answer the following questions:

  • What is the integration behavior my organization needs from the IAM system?
  • What are the implementation steps to migrate users and apps in a risk-free manner?

Design User Account Data

In an OAuth architecture, you first need to decide which user fields you will store in the IAM system and which to keep in your business data. There are multiple ways this could be designed, and some fields could exist in both locations, but each field should have a single source of truth. Some example fields are listed in the following sections.

Online Retail System

In a relatively simple architecture, you may decide that all of the main user data should be migrated to the identity data:

User FieldSource of TruthDescription
User IDDomain-Specific DataMost commonly an existing database primary key
SubjectIdentity DataA unique identifier for each user, generated by the IAM system
UsernameIdentity DataThe value a user types in a Username/Password screen
Given + Family NameIdentity DataUsed in login screens or emails to the user, but may also need to be stored in business data
EmailIdentity DataUsed to send email verification messages, but may also need to be stored in business data
Country CodeIdentity DataIn some scenarios, the user’s country may be important later when dealing with data sovereignty
Subscription LevelIdentity DataAn application-specific value assigned to the user
Consent Flag(s)Identity DataFlags related to privacy or legal terms the user must consent to, such as GDPR in Europe
Consent Time(s)Identity DataWhen the user last consented to the legal terms

Corporate Platform

Corporate software platforms sometimes consist of multiple mature products with many extra fields used for authorization. You may prefer to manage some user fields within your business data, either to reduce the impact of the IAM migration, or because you feel that future changes will be easier to manage there:

User FieldSource of TruthDescription
User ID (Product A)Domain-Specific DataA database primary key in this product
Roles (Product A)Domain-Specific DataAn array of roles that a user can be assigned for this product
Permissions (Product A)Domain-Specific DataA collection of permissions that can potentially be calculated in complex ways
User ID (Product B)Domain-Specific DataA database primary key in this product
Tenant ID (Product B)Domain-Specific DataSome products may partition their data storage by partner or data owner
User Group (Product B)Domain-Specific DataThe group to which a user belongs in this product
Permissions (Product B)Domain-Specific DataA collection of permissions that can potentially be calculated in complex ways

Medical System

A medical system may have its own particular user information, some of which is used during authentication workflows. You may prefer to keep these extra fields in the domain-specific data:

User FieldSource of TruthDescription
Patient Identification NumberDomain-Specific DataA unique identifier for the patient
Date of BirthDomain-Specific DataUsed as an additional authentication factor, and also stored against medical records
Social Security NumberDomain-Specific DataUsed as an additional authentication factor, and also stored against medical records

The Subject Claim

When a user is created in the IAM system, an Account ID is assigned, which is usually a generated identifier such as a UUID. This will remain constant after creation, even if a user changes their name. It can then be used in access tokens as the subject claim.

It is also possible to use Pairwise Pseudonymous Identifiers (PPID), which is considered the best option from a privacy viewpoint, so use that option if you can. In many business systems, however, the Account ID may be preferred to reliably map the user to the business data.

Joining Data

When data is stored at rest, ensure that an ETL process can continue to form a complete picture of user data by joining the two data sources together. This can be done by either adding the business identifier to the IAM data or vice versa. The three example scenarios might be handled as follows:

User Mappings

The subject claim should not be used as a direct primary or foreign key in your business data. This allows the option to switch to a new IAM vendor in the future, which will assign its own generated UUIDs to users.

API User Identification

User account modeling also needs to account for how APIs will manage requests from applications, since APIs will need to read, write and extend business data stored against users:

System TypeBehavior
Online RetailAn internet retailer typically only shows the logged-in user their own history of purchases.
CorporateA corporate system may record a history of which employees changed which corporate assets.
MedicalA logged-in patient must only be able to access their own medical history.

By default, an OAuth system identifies a user only by the subject claim:

Subject Claim

However, this is often insufficient for an API in business terms, so you will likely need to provide more useful business identifiers to APIs in access tokens:

API Identifiers

APIs will then digitally verify the JWT in the standard way and can trust the claims received, after which business data can be updated naturally.

Existing User Migration

The user account data is just a blueprint at this stage, and the information is likely to be stored differently in the starting architecture. To migrate to the preferred structure, you need to get a full picture of the existing data for each user, then populate data sources via these steps:

StepDescription
Select existing dataSelect the users from the source database(s) into a list, which will be iterated over.
Insert each user into the IAM dataThe IAM system should have a SCIM API to enable you to insert a user, then receive back the account identifier.
Update each user in the business dataIf required, insert the account ID into the business data and possibly make other updates.

This type of migration process often only makes additive changes to business data. It can also be run multiple times and retried in the event of errors. Once complete, you can continue to get a full data picture of all fields at rest if you need to do further IAM migrations in the future.

Credential Migration

Credentials and authentication methods are managed separately to the core user account data. When moving to an IAM system, it is often possible to leave credentials such as hashed passwords or LDAP stores in the existing location and for the IAM system to use that as a data source. Another option is to import hashed passwords into a database managed by the IAM system.

Either of these approaches will ensure that end-users who sign in via passwords do not experience problems when the updated IAM system is rolled out. If neither of these are possible, then users can be configured to reset their password on the first login to the IAM system, which only impacts each user once.

Existing User Flows

Once the data separation is understood, you need to design the end-to-end flow for existing users to get a clear idea of how user experience and data integration will be updated. This may vary depending on your use cases, so some examples are presented below.

Passwords

After the rollout, the end-to-end flow for an internet retail system might look like this, where the user continues to sign in with their existing password:

Password Flow

Social Identity Providers

If the starting architecture used social identity providers directly, the user could continue to sign in with the same experience, e.g., via Google or Facebook.

Social Flow

For this migration logic to work, the source architecture needs to contain an existing unique identifier for the social user. This can be an email or the social provider’s subject claim, but it must be imported into the identity data.

Corporate Logins

Corporate systems often use an existing company identity solution, which might not be OAuth-based and is perhaps provided by a cloud vendor. This can continue to be used after the migration since the IAM system’s main role is that of an OAuth Authorization Server, which should be able to integrate with any standards-based identity provider:

Corporate Flow

An interesting point shown here is that with the corporate user account design, the Identity Server needs to be able to reach out to domain-specific APIs or data sources at the time of token issuance. This is a crucial capability to look for since you will often want to add business fields to access tokens in more complex systems.

Medical or Government Logins

This type of system is a little more interesting since multi-factor authentication is used, and some of these factors may require additional custom screens to collect patient identifiers or government eIDs and then verify them against domain-specific data:

Medical Flow

Therefore, the IAM system should have extensibility features needed to enable this so that companies can meet their industry-specific strong authentication requirements. It would also be critical in this type of system for the login user experience to use medical or government branding so that authentication screens have the same look and feel as application screens.

Future Users

Once running a modern OAuth-based architecture, you will also want to reliably onboard future users. This is likely to require data updates in both the identity data and the business data. The technical goal is to perform the difficult work of proving that the user is new in the identity server, then provide simple choices to APIs.

Online Retail System

In this type of system, each user is typically allowed to register and create their own credential. They are then asked to confirm it via email or SMS verification. This creates a basic identity, but you may not really know who the user is.

Once the basic verification is completed, you may need to create the user in both data sources. One option is to reach out to your business data during the registration process. This ensures that the token returned to APIs contains the claims needed to identify the user in business terms:

New Retail User

You will want to ensure that all business transactions for the basic identity remain stored together, including cases where the end-user forgets they are registered and repeats the sign-up process. A custom action could look up a verified identity in the account data, e.g., by finding the email, then ensure that the user gets linked back to their existing data history.

Corporate System

In corporate systems, it is common for new users to be provisioned by an administrator, which might involve using a script or Admin UI that calls back-end services. The logic for creating a user is likely to be similar to that described earlier for user migrations:

ActionBehavior
Save to the IAM systemUse SCIM endpoints to insert a user with custom attributes, then receive back the account identifier, used as a subject claim.
Save to the business systemCreate the business user, with assigned roles and permissions, and optionally include the subject claim to link to IAM data.

In this type of setup, a user may have a valid IAM account and access to one product but not another. You will want to avoid a UX where a user signs in successfully but then experiences an error upon return to the application, and this can be achieved cleanly via an authentication action:

New Corporate User

Medical or Government System

Internet users may also be able to sign themselves up to this type of system, but they will need to use strong authentication factors, such as providing eIDs or scanning a passport. This registration process will again require the ability to use custom screens and actions.

The below workflow shows an alternative way to create new users in business data after authentication. The API receives empty domain-specific claims and can trust that the hard work of identifying the user as new has been done in the IAM system. User creation and onboarding can then occur within the system, perhaps by taking the user through some welcome screens.

New Medical User

These days, a user account data design also needs to include legal and regulatory requirements fields. This is another area where the IAM system provides useful features that can enable you to implement them with the best user experience without adding complexity to your apps:

DataCharacteristics
Legal Terms and ConditionsThe company’s terms and conditions can be integrated into registration forms when users complete a registration process.
User PrivacyUsers typically need to consent to regulations such as GDPR in Europe, which can be managed via a post-authentication action that presents a screen, then denies access unless accepted.
User ConsentOAuth consent forms should be customizable via actions, and are commonly used in payment scenarios to ask a user for access to their bank account, to meet regulations such as PSD2.
Data SovereigntyKeeping a user’s data in their home country may be a legal requirement. In this case, it is recommended to include a country code in IAM data, then use that as the basis for Dynamic User Routing of API requests.

User Data Updates

Occasionally users need to change their personal information, and in an OAuth architecture, this may include updating fields in both the identity data and business data. This might be managed by displaying user information on a single screen, then saving data down to both data sources. This is straightforward since the primary technical identifiers for the user remain unchanged so that you can handle scenarios such as name changes due to marriage.

User Data Updates

Evolving Authentication

In a modern IAM system, the user can potentially sign in via different authentication methods, which changes the first factor in the authentication workflow. If a user signs in via the default Username Password option, then at some later point changes their primary factor to WebAuthn, this could create two distinct user accounts, with different subject claims:

Duplicate User

This is a common problem when social identity providers are coded directly into applications, and the standard solution is to use the Account Linking features of the IAM system. This requires custom actions to work with attributes and match credentials to accounts, using two high-level techniques:

OptionBehavior
Deterministic LogicUse either a built-in action or custom script if it is possible to reliably match users to existing accounts based on data. This might work when users have to use the same email as an identifier for two different authentication methods.
Involving the UserOtherwise, run an action that involves the user, who will be indirectly asked to provide their existing subject claim. In the above diagram, on the first WebAuthn login, the user may be prompted to sign in with a username and password to link the WebAuthn public key data to the user account.

With the correct IAM capabilities, along with some scripting to manage attribute matching, you will be able to ensure that all forms of authentication link to the same IAM identity, which in turn links to the same records in your business data:

Linked Accounts

Clarified Requirements

This article has identified several areas where an IAM system should enable customization. It is recommended to avoid committing to a vendor solution until you are sure it will give you the business continuity you need. In particular, the following capabilities are worth verifying early on:

AreaCapability
Data StorageThe IAM system must be able to store custom attributes against user accounts
RegistrationEnsure that users can provide custom data during onboarding for self-sign-up scenarios
AuthenticationThe ability to customize behavior via scripted actions, custom screens, and calls to business APIs
Login BrandingA professional login user experience so that IAM screens look like an integrated part of your apps
ConsentLook for a system with good options for managing user consent after authentication
Account LinkingLogic and user actions to ensure that the subject claim in tokens remains consistent, regardless of the authentication method used
Token IssuingThe ability to include domain-specific claims in access tokens at the time of issuance, to enable reliable data updates in APIs

The Curity Identity Server

At Curity, we are passionate about supporting the most up-to-date standards and flows. Still, we also understand that customers need extensibility features in important places when integrating IAM into their business. The following building blocks enable the behaviors described in this article:

ComponentDescription
Authentication ActionAuthentication can be extended via a number of built-in actions or via Javascript code.
AuthenticatorA number of built-in authenticators exist for popular social providers, OpenID Connect and SAML.
UX BrandingAll screens used for authentication, consent and registration can be branded separately per client application.
Claims Value ProvidersA claims value provider can run Javascript code to add extra claims to tokens, e.g., retrieved from a business API.
Credential ManagerCredentials can be read, written and verified using one or more credential managers, each of which maps to a data source.
Account ManagerAccount data can be read from one or more account managers, each of which maps to a data source.
ConsentorThe process around capturing consent is fully customizable via custom screens and actions.
Data SourceJavaScript actions can use data sources directly to access data from various types of repositories.
HTTP ClientJavaScript actions can use HTTP clients to make secure remote connections to business APIs.

Each type of component is also available in the Java SDK, which enables you to build completely bespoke implementations if ever needed. There are many custom examples in our GitHub repositories that can be used as a guide for your own solutions.

Conclusion

Identity, privacy, and meeting regulations are difficult areas, but software companies need to deal with them somehow. Rather than doing so in your application code, a good IAM system can provide years of effort at a low cost. Your applications will then have simpler code, richer security flows, and access to some powerful new design patterns.

A good OAuth solution requires a toolbox that you can adapt to meet your business requirements in a clean way. Understand your preferred end-to-end flows and then ensure that the extensibility options you need are possible with your chosen provider. Your migration to the new IAM system might then begin with these risk-free steps:

  • Design user account data and access token claims
  • Populate IAM user account data in the new system
  • Customize authentication workflows to reliably identify users
  • Customize token issuance to enable APIs to map identities to business data
  • Implement OAuth flows in proof-of-concept (POC) applications before updating flagship apps