Dynamic User Routing with NGINX

Intro

This tutorial shows how to implement dynamic routing of OAuth requests using an NGINX reverse proxy. The code resources are available in the GitHub Repository link.

This routing capability enables companies to deploy the Curity Identity Server to multiple regions, while forwarding OAuth requests for users to their home region. The overall solution is described in the Implementing Dynamic User Routing walkthrough.

Reverse Proxy Custom Logic

The NGINX reverse proxy will need to implement some logic to inspect incoming OAuth requests and route them according to the logic in the below pseudo code. So an easy way to extend NGINX with custom code is needed.

zone = read_zone_from_http_only_cookie
if not zone then
  zone = read_zone_from_wrapped_jwt
end
if not zone then
  zone = default_zone
end

OpenResty

The example uses OpenResty, which is an enhanced and productive version of NGINX, with good extensibility via standard and custom plugins, using the high level LUA Programming Language.

Docker Image

The default docker image for OpenResty has been customized to copy in the following two plugins:

FROM openresty/openresty:1.19.3.1-8-bionic
RUN luarocks install lua-resty-jwt
COPY reverse-proxy/nginx/zonetransfer.lua /usr/local/openresty/lualib

Proxy Configuration

The configuration of the NGINX proxy is managed by an nginx.conf file that would be different for each stage of a company's deployment pipeline. The important parts of the configuration are shown below, where the call to proxy_pass routes requests using a host name calculated at runtime:

http {
    map $zone_value $zone_host_name {
        default  'internal-curity-eu:8443';
        'eu'     'internal-curity-eu:8443';
        'us'     'internal-curity-us:8443';
    }

    server {
        listen 80 default_server;
        location ~ ^/ {

            set $zone_value '';
            rewrite_by_lua_block {

                local config = {
                    cookie_name = 'zone',
                    claim_name = 'zone'
                }

                local zonetransfer = require 'zonetransfer'
                ngx.var.zone_value = zonetransfer.get_zone_value(config)
            }

            proxy_pass http://$zone_host_name$uri$is_args$args;
        }
    }
}

There is a map of zone values to corresponding host names, including a default host name when no zone is found in the incoming request. NGINX invokes the plugin to inspect every request and look for a zone in a cookie or wrapped token. If found, the corresponding host name is then used when calling NGINX's proxy_pass command.

Plugin Code

The code for the plugin is easy to follow and quite similar to the pseudo code provided earlier. It simply involves reading HTTP request values, and these responsibilities are broken down into functions.

--
-- Try to read the zone value from an OAuth request
--
function _M.get_zone_value(config)

  if not verify_options(config) then
    return nil
  end

  local method = string.lower(ngx.var.request_method)
  if method == 'options' or method == 'head' then
    return nil
  end

  -- First try to find a value in the zone cookie
  local zone = get_zone_from_cookie(config.cookie_name)

  -- Otherwise, for POST messages look in the form body
  if zone == nil and method == 'post' then
    zone = get_zone_from_form(config.claim_name)
  end

  return zone
end

Docker Compose

In the repository's Docker Compose file, the overall deployment of the NGINX reverse proxy is defined, which includes copying in the environment specific nginx.conf file. In the example setup NGINX is exposed to the host PC using port 80:

nginx:
    image: custom_openresty:1.19.3.1-8-bionic
    hostname: internal-nginx
    ports:
      - 80:80
    volumes:
      - ./reverse-proxy/nginx/nginx.conf:/usr/local/openresty/nginx/conf/nginx.conf

Working Plugin

When the reverse proxy is run as part of an overall solution, NGINX outputs log messages to show zones received in OAuth requests, so that the plugin's behavior can be visualized:

*** Found zone 'eu' in cookie, client: 172.19.0.1, server: , request: "POST /authn/authentication/UserName-Password HTTP/1.1"
*** Found zone 'eu' in wrapped token, client: 172.19.0.1, server: , request: "POST /oauth/v2/oauth-token HTTP/1.1"
*** Found zone 'eu' in cookie, client: 172.19.0.1, server: , request: "POST /authn/authentication/UserName HTTP/1.1"
*** Found zone 'us' in cookie, client: 172.19.0.1, server: , request: "POST /authn/authentication/UserName-Password HTTP/1.1"
*** Found zone 'us' in wrapped token, client: 172.19.0.1, server: , request: "POST /oauth/v2/oauth-token HTTP/1.1"

Conclusion

It is a standard job to implement dynamic routing in API gateways, and NGINX OpenResty provides some extensibility features to help with this. When combined with the Curity Identity Server's multi zone features, a company can deploy a global IAM system with user data separated by region, and with good reliability.