Strengthen OAuth Client Credentials in a Service Mesh

In Kubernetes deployments it is common to let the cloud native platform deal with mutual TLS, so that client certificate usage and renewal is transparent for applications. A service mesh can deploy sidecar containers to application pods, where the sidecars deal with the details of sending and receiving client certificates.

Each workload receives a SPIFFE Verifiable Identity Document (SVID) with a unique identifier, the SPIFFE ID, such as spiffe://prod.acme.com/billing/api. In most cases the identity document uses an X509 format, which serves as a client certificate. X509 SVIDs can also be used as OAuth client credentials. Doing so is more secure and easier to manage than using a client secret.

The following sections summarize the main steps for getting mutual TLS client credentials working in a SPIFFE and sidecar based setup. This tutorial uses Istio, though you can follow the same techniques to provide a working setup with any other service mesh.

Deploy the Service Mesh

When the service mesh is installed it will either use a default root CA for mutual TLS, or you can provide your own root certificate, as described in the certificate documentation. Retrieve the root CA an save it to a file such as root-cert.pem, using a command of this form:

kubectl -n istio-system get secret/istio-ca-secret -o jsonpath='{.data.ca-cert\.pem}' | base64 --decode

Deploy the Curity Identity Server

The Istio demo installation provides an example deployment of the Curity Identity Server to a service mesh. Sidecars are injected into the pods of the Curity Identity Server, so that mutual TLS is used for all OAuth requests. Doing so also ensures that all OAuth requests inside the cluster are kept confidential, without the need to manage any internal SSL certificates.

Deploy the SPIFFE CA to the Curity Identity Server as a client trust store. For example, use Parameterized Configuration, as shown in the following XML snippet.

export SPIFFE_ROOT_CERT=$(openssl base64 -in ca-cert.pem | tr -d '\n')

<facilities xmlns="https://curity.se/ns/conf/base">
  <crypto>
    <ssl>
      <client-truststore>
        <client-certificate>
          <id>SPIFFE_CA</id>
          <size>2048</size>
          <keystore>#{SPIFFE_ROOT_CERT}</keystore>
          <type>rsa</type>
        </client-certificate>
      </client-truststore>      
    </ssl>
</facilities>

Alternatively, use the Admin UI and navigate to Facilities → Keys and Cryptography → Trust Anchors → Client Trust Stores, then select the + option and import the certificate file.

Configure Mutual TLS

Next, in the Admin UI, navigate to Profiles → Token Service → Client Settings and activate Mutual TLS. Select the Terminated by a Proxy option and supply the name of an HTTP header, such as x-client-certificate below. This header will contain the full client certificate value during OAuth grant requests.

Assign Service Accounts to Clients

When deploying OAuth client applications to the cluster, include a service account, as in the following example YAML for a website, to avoid a default identity being assigned. In this way, each client gets a unique SPIFFE ID, such as spiffe://cluster.local/ns/applications/sa/mywebsite, generated from the namespace and service account name:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: mywebsite
  namespace: applications
---
apiVersion: v1
kind: Service
metadata:
  name: mywebsite
  namespace: applications
  labels:
    app: mywebsite
    service: mywebsite
spec:
  ports:
  - port: 80
    name: http
  selector:
    app: mywebsite
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mywebsite
  namespace: applications
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mywebsite
  template:
    metadata:
      labels:
        app: mywebsite
    spec:
      serviceAccountName: mywebsite
      containers:
      - name: mywebsite
        image: mywebsite:v1

Configure the Client

In the Curity Identity Server, configure the client with the mutual-tls-by-proxy option. Select the client trust store so that only mutual TLS client credentials issued by the SPIFFE authority are accepted. The client's SPIFFE ID is a URI in the subject alternative name extension. So, select client-uri as the subject alternative name and enter the SPIFFE ID:

Forward the Client Certificate

By default, the service mesh terminates mutual TLS at the sidecar and may not forward the full client certificate to the Curity Identity Server. You may therefore need to manually populate the x-client-certificate HTTP header configured earlier. Istio provides a custom Envoy Filter resource that enables this. A YAML file for an example filter is provided below:

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: client-certificate-filter
  namespace: curity
spec:
  workloadSelector:
    labels:
      role: curity-idsvr-runtime
  configPatches:
    
  - applyTo: HTTP_FILTER
    match:
      context: SIDECAR_INBOUND
      listener:
        portNumber: 8443
        filterChain:
          filter:
            name: "envoy.filters.network.http_connection_manager"
            subFilter:
              name: "envoy.filters.http.router"

    patch:
      operation: INSERT_BEFORE
      value:
       name: envoy.filters.http.lua
       typed_config:
          "@type": "type.googleapis.com/envoy.extensions.filters.http.lua.v3.Lua"
          inlineCode: |
            function envoy_on_request(request_handle)

              local streamInfo = request_handle:streamInfo()
              if streamInfo ~= nil then
                local downstreamSsl = streamInfo:downstreamSslConnection()
                if downstreamSsl ~= nil then
                  request_handle:headers():add("x-client-certificate", downstreamSsl:urlEncodedPemEncodedPeerCertificate())
                end
              end
            end

Client Token Requests

When the client application wants to get tokens it sends its client_id in the token request in accordance with the RFC8705 standard, which describes the flow for OAuth 2.0 Mutual-TLS Client Authentication and Certificate-Bound Access Tokens. There is no need to add a client_secret parameter. Although the client makes a plain HTTP request, sidecars upgrade the connection between the client pod and the Curity Identity Server to use mutual TLS:

curl -X POST http://curity-idsvr-runtime-svc.curity:8443/oauth/v2/oauth-token \
     -H 'Content-Type: application/x-www-form-urlencoded' \
     -d 'grant_type=authorization_code' \
     -d 'code=I9xL9DY9jAYHPuHSiW2OpWUaNRW4otei' \
     -d 'client_id=website-client' \
     -d 'redirect_uri=http://www.example.com' \
     -d 'scope=openid example' \
     -d 'code_verifier=HlfffYlGy7SIX3pYHOMJfhnO5AhUW1eOIKfjR42ue28'

Token Refresh

In deployments where token refresh is used, such as for a web client, some SPIFFE implementations may not comply with RFC8705, and issue distinct certificates for each of the client's sidecars. This can lead to the Curity Identity Server rejecting the token refresh request, if the certificate sent during token refresh is different to that used during the initial authorization code grant.

If required, this problem can be worked around by circumventing certificate-bound access tokens. This reduces the security strength of access tokens, since they are no longer sender constrained. To do so, use token procedures to remove the cnf claim that binds the access token to a certificate, as defined in the RFC8705 standard.

An example authorization code grant procedure is shown here, which uses a client property specific to web clients that use SPIFFE based client credentials. See the Custom Token Issuer tutorial for further details on this technique.

function result(context) {
  var delegationData = context.getDefaultDelegationData();

  var clientType = context.client.properties["client-type"];
  if (clientType === "spiffe-web") {
    delete delegationData["mtlsClientCertificate"];
    delete delegationData["mtlsClientCertificateThumbprintS256"];
  }

  var issuedDelegation = context.delegationIssuer.issue(delegationData);
  var accessTokenData = context.getDefaultAccessTokenData();

  var clientType = context.client.properties["client-type"];
  if (clientType === "spiffe-web") {
    delete accessTokenData["cnf"];
  }

  ...
}

Similarly, an example refresh token grant procedure is shown here:

function result(context) {
  var accessTokenData = context.getDefaultAccessTokenData(context.delegation);

  var clientType = context.client.properties["client-type"];
  if (clientType && clientType === "spiffe-web") {
    delete accessTokenData["cnf"];
  }

  ...
}

Alternative Deployments

In some setups you may prefer to issue internal SSL certificates to the Curity Identity Server and avoid the use of sidecars. In such cases you would then simply use the direct Mutual TLS option, and omit the extra Mutual TLS by Proxy configuration, since there would be no need to forward a client certificate header.

Conclusion

Using a service mesh can strengthen security within a Kubernetes cluster. A service mesh encrypts requests between components transparently to ensure confidentiality, without the need to manage SSL certificates at the application level. In addition, X509 SVIDs can serve as OAuth client credentials, which have better protections against misuse than client secrets. The crypto is also easy to manage, since the cloud native platform takes care of keeping such credentials short lived and automatically renewing them.