OAuth Introspection and Phantom Tokens

In most deployments it's desired to use reference tokens (opaque tokens) outside the internal network, and then JSON Web Tokens (JWTs) on the internal network. To achieve this the phantom token approach can be used. This tutorial describes how to setup the Curity Identity Server for Phantom Tokens.

The flow is based on a system that has a Reverse Proxy (RP) or an API Gateway (API Firewall) as the first entrypoint into the network. The task of the RP is to block any incoming requests that do not contain an Access Token. See the Phantom Token page for example integrations with various gateways/proxies.

This is done by calling the Introspect endpoint. The normal response from that endpoint is a JSON document with the contents of the token. However, it lends itself nicely to include more data, specifically a new Access Token in JWT format, with the exact same properties as the incoming token. This can then be used to pass on downstream to the APIs.

Setup in Curity

There are two ways to setup this flow in Curity. Which one is best for your system depends on what data the RP needs. Many RPs don't require any information about the actual token other than if it's valid or not. If that's the case, then the first approach application/jwt is recommended. If the Gateway requires more information, i.e., needs to act on the contents of the token, then the second approach using token procedures is recommended.

Configure Introspection Endpoint

For a client to be able to introspect tokens, the introspection concepts need to be available and published on the runtimes. We assume that there is a Token profile called token-service in the system and that there is a service role in the system called default.

The following steps are needed

1. Allow the introspection capability on the profile
2. Add the introspection endpoint to the token profile if necessary
3. Publish the introspection endpoint on the service profile

If you're using the example configuration the introspection endpoint is already configured, and a client called gateway_client exists with the correct capabilities.

Using the Admin UIUsing the Command Line Interface

1. Go to Token Service → Your Profile → General → Client Settings

Enable the Introspection capability

2. Go to Token Service → Your Profile → Endpoints

If the endpoint with the type introspection doesn't exist, click + New Endpoint to create a new endpoint.

3. With the endpoint added, click the Not Deployed button in the Running On column and select the service role(s) to deploy the endpoint to.

4. Click the green checkmark in the Action column and then go to the Changes menu in the top menu bar and Commit the changes.

Configure Client for Introspection (Gateway client)

Introspection clients are rarely the same as clients that can request tokens. A gateway client should be setup to only be allowed to do introspection and no other OAuth flow.

Create Introspection Client Using the CLI

/opt/idsvr/bin/idsh
configure
edit profiles profile token-service oauth-service settings authorization-server client-store config-backed
set client gateway_client secret Secr3t!
set client gateway_client capabilities introspection
commit

Application/JWT Approach

The application/jwt approach is a shorthand version of phantom token that is incredibly effective when using smaller RPs. A good example is when using NGINX as a reverse proxy with the Nginx phantom token module.

The flow does not require any additional setup. Instead, it switches mode based on the Accept header of the introspection request. By adding Accept: application/jwt in the request, Curity will respond with the JWT version of the incoming token.

curl -X POST \
  https://localhost:8443/introspection \
  -H 'accept: application/jwt' \
  -H 'cache-control: no-cache' \
  -H 'content-type: application/x-www-form-urlencoded' \
  -d 'token=ba06557a-c2b6-439e-85a0-759c8e953e14&client_id=gateway_client&client_secret=Secr3t!'

Instead of the full JSON response, Curity will respond with the JWT directly in the body.

eyJraWQiOiItMzgwNzQ4MTIiLCJ4NXQiOiJNUi1wR1RhODY2UmRaTGpONlZ3cmZheT....LJHlj1Og

This can then be used to pass on to the API in the Authorization header as usual.

Token Procedure Approach

If the RP needs more information to make its own decisions, it's possible to combine the regular introspection response with the phantom token flow.

This requires the endpoint to have a token procedure configured. There are several ways to configure a token procedure, but using the Web User Interface is the easiest.

1. Open the Token Service profile that is being used.
2. Click Endpoints on the left-hand side.
3. Search for the introspection endpoint that is being used.
4. In the Flows column, expand the drop-down
5. In the Introspection drop-down, select + New procedure, give it a name (phantom-token-procedure) and click Save.

6. An editor will open, replace the existing script with the following:

/**
 * @param {se.curity.identityserver.procedures.context.IntrospectionTokenProcedureContext} context
 */
function result(context) {
    var responseData = {
        active: context.presentedToken.active
    };

    if (context.presentedToken.active) {
        appendObjectTo(context.presentedToken.data, responseData);
        responseData.token_type = context.presentedToken.type;
        responseData.client_id = context.presentedToken.delegation.clientId;
        responseData.expired_scope = context.presentedToken.expiredScopes;

        var defaultAtJwtIssuer = context.getDefaultAccessTokenJwtIssuer();
        responseData.phantom_token = defaultAtJwtIssuer.issue(context.presentedToken.data,
            context.delegation);
    }

    return responseData;
}

The only change from the original procedure is the step which gets the defaultAtJwtIssuer and uses that to create a new access token based on the presentedToken.

Testing the flow

Send a normal introspection request (this time don't send application/jwt)

curl -X POST \
  https://localhost:8443/introspection \
  -H 'cache-control: no-cache' \
  -H 'content-type: application/x-www-form-urlencoded' \
  -d 'token=6685ad41-bb3c-43ee-952c-fc70e91189a7&client_id=gateway_client&client_secret=Secr3t!'

The response is a JSON response with the phantom token as a parameter.

{
    "sub": "server-client",
    "purpose": "access_token",
    "iss": "https://localhost:8443/~",
    "active": true,
    "token_type": "bearer",
    "client_id": "server-client",
    "aud": "server-client",
    "nbf": 1512642625,
    "phantom_token": "eyJraWQiOiIxNTQzMTE2MDE1I...",
    "scope": "read",
    "expired_scope": [],
    "exp": 1512642925,
    "delegationId": "aca9436d-fefc-4aaa-8385-fbcb6a7847be",
    "iat": 1512642677
}

Conclusion

The phantom token is a very powerful pattern when building a microservice based architecture. It allows all APIs to rely on the by-value JWT token, without exposing internal data on the Internet.

It is also possible to add more data to the internal token if desired using the token procedures if needed.

Resources

• For more details about configuring procedures in the Curity Identity Server see The scripting guide in the Documentation

• For more details about microservices and API Security see this Nordic APIs blogpost

• Using the command line interface, see Curity Command Line Interface