OAuth Introspection and Phantom Tokens

In most deployments it's desired to use reference tokens (opaque tokens) outside the internal network, and then Json Web Tokens (JWTs) on the internal network. To achieve this the phantom token approach can be used. This tutorial describes how to setup the Curity Identity Server for Phantom Tokens.

The flow is based on a system that has a Reverse Proxy (RP) or an API Gateway (API Firewall) as the first entrypoint into the network. The task of the RP is to block any incoming requests that do not contain an Access Token. If you are using Nginx, see the also the Nginx phantom token module on GitHub.

This is done by calling the Introspect endpoint. The normal response from that endpoint is a Json document with the contents of the token. However, it lends itself nicely to include more data, specifically a new Access Token in JWT format, with the exact same properties as the incoming token. This can then be used to pass on downstream to the APIs.

Setup in Curity

There are two ways to setup this flow in Curity. Which one is best for your system depends on what data the RP needs. Many RPs don't require any information about the actual token other than if it's valid or not. If that's the case, then the first approach application/jwt is recommended. If the Gateway requires more information, i.e., needs to act on the contents of the token, then the second approach using token procedures is recommended.

Configure Introspection Endpoint

For a client to be able to introspect tokens, the introspection concepts need to be available and published on the runtimes. We assume that there is a Token profile called oauth in the system and that there are two nodes in the system called node1 and node2.

The following steps are needed

1. Add the introspection endpoint to the token profile if necessary
2. Allow the introspection capability on the profile
3. Publish the introspection endpoint on the running services

If you're using the example configuration the introspection endpoint is already configured, and a client called gateway_client exists with the correct capabilities.

Using the Command Line Interface

On the admin node, start the CLI (or ssh to it directly).

/opt/idsvr/bin/idsh
configure
set profiles profile oauth oauth-service endpoints endpoint introspect endpoint-kind oauth-introspect uri /introspection
set profiles profile oauth oauth-service settings authorization-server client-capabilities introspection
set environments environment services service node1 endpoints introspect
set environments environment services service node2 endpoints introspect
commit

Now the endpoint is set and active on both node1 and node2.

Using the Web UI

1 - Go to Token Service -> Your Profile -> Endpoints

If the endpoint with the type introspection doesn't exist, click New Endpoint

2 - Go to Token Service -> Your Profile -> General

Enable the capability "Introspection"

3 - Go to System -> Deployments -> Node1 (The nodes may have other names)

Do this for every node that you want the concepts published on. Usually all nodes, or all internal nodes.

4 - Click Changes in the top menu bar and Commit

Configure Client for Introspection (Gateway client)

Introspection clients are rarely the same as clients that can request tokens. A gateway client should be setup to only be allowed to do introspection and no other OAuth flow.

Create Introspection Client Using the CLI

/opt/idsvr/bin/idsh
configure
edit profiles profile oauth oauth-service settings authorization-server client-store config-backed
set client gateway_client secret Secr3t!
set client gateway_client capabilities introspection
commit

Application/JWT Approach

The application/jwt approach is a shorthand version of phantom token that is incredibly effective when using smaller RPs. A good example is when using NGINX as a reverse proxy with the Nginx phantom token module.

The flow does not require any additional setup. Instead, it switches mode based on the Accept header of the introspection request. By adding Accept: application/jwt in the request, Curity will respond with the JWT version of the incoming token.

curl -X POST \
  https://localhost:8443/introspection \
  -H 'accept: application/jwt' \
  -H 'cache-control: no-cache' \
  -H 'content-type: application/x-www-form-urlencoded' \
  -d 'token=ba06557a-c2b6-439e-85a0-759c8e953e14&client_id=gateway_client&client_secret=Secr3t!'

Instead of the full json response, Curity will instead respond with the JWT directly in the body.

eyJraWQiOiItMzgwNzQ4MTIiLCJ4NXQiOiJNUi1wR1RhODY2UmRaTGpONlZ3cmZheT....LJHlj1Og

This can then be used to pass on to the API in the Authorization header as usual.

Token Procedure Approach

If the RP needs more information to make its own decisions, it's possible to combine the regular introspection response with the phantom token flow.

This requires the endpoint to have a token procedure configured. There are several ways to configure a token procedure, but using the Web User Interface is the easiest.

1. Open the OAuth profile that is being used.
2. Click Endpoints on the left-hand side.
3. Search for the introspection endpoint that is being used.
4. In the Procedures dropdown associated with that endpoint, select introspect-procedure. An Edit button will appear. Click this button.
5. Another editor will open. In this, replace the existing script with the following:

function result(context) {
    var responseData = {
        active: context.presentedToken.active
    };

    if (context.presentedToken.active) {
        appendObjectTo(context.presentedToken.data, responseData);
        responseData.token_type = context.presentedToken.type;
        responseData.client_id = context.presentedToken.delegation.clientId;
        responseData.expired_scope = context.presentedToken.expiredScopes;

        var defaultAtJwtIssuer = context.getDefaultAccessTokenJwtIssuer();
        responseData.phantom_token = defaultAtJwtIssuer.issue(context.presentedToken.data,
            context.delegation);
    }

    return responseData;
}

The only change from the original procedure is the step which gets the defaultAtJwtIssuer and uses that to create a new access token based on the presentedToken.

Testing the flow

Send a normal introspection request (this time don't send application/jwt)

curl -X POST \
  https://localhost:8443/introspection \
  -H 'cache-control: no-cache' \
  -H 'content-type: application/x-www-form-urlencoded' \
  -d 'token=6685ad41-bb3c-43ee-952c-fc70e91189a7&client_id=gateway_client&client_secret=Secr3t!'

The response is a json response with the phantom token as a parameter.

{
    "sub": "server-client",
    "purpose": "access_token",
    "iss": "https://localhost:8443/~",
    "active": true,
    "token_type": "bearer",
    "client_id": "server-client",
    "aud": "server-client",
    "nbf": 1512642625,
    "phantom_token": "eyJraWQiOiIxNTQzMTE2MDE1I...",
    "scope": "read",
    "expired_scope": [],
    "exp": 1512642925,
    "delegationId": "aca9436d-fefc-4aaa-8385-fbcb6a7847be",
    "iat": 1512642677
}

Conclusion

The phantom token is a very powerful pattern when building a microservice based architecture. It allows all APIs to rely on the by-value JWT token, without exposing internal data on the Internet.

It is also possible to add more data to the internal token if desired using the token procedures if needed.

Resources

• For more details about configuring procedures in the Curity Identity Server see The scripting guide in the Documentation

• For more details about microservices and API Security see this Nordic APIs blogpost

• Using the command line interface, see Curity Command Line Interface