OpenTelemetry Tracing

Starting from version 10.0 of the Curity Identity Server you can activate OpenTelemetry tracing. OpenTelemetry is a standards-based observability framework that components can use, regardless of their technology. With OpenTelemetry you own the observability data and can visualize or interrogate it using standard tools.

OpenTelemetry in OAuth Flows

OAuth flows often trigger calls to multiple backend components. When all of these components implement OpenTelemetry tracing, they follow the same pattern to enable correlation of related requests. The client can generate a unique TraceId and send it in a standardized traceparent header that upstream components consume.

All components in the trace should use Context Propagation, to add data as a span under the TraceId. A span is a child unit of work, like a child HTTP request. Each span has a unique SpanId within the trace and contains Resource Attributes. All backend components use an OpenTelemetry SDK that reports this data to the OpenTelemetry Collector.

The OpenTelemetry collector can distribute trace data to tools that specialize in visualizing it, such as the Jaeger Tracing Platform. For example, a visualization tool could accept a particular client TraceId and then render the entire trace from the OpenTelemetry data.

Open Telemetry in the Curity Identity Server

The System Admin Guide explains the Curity Identity Server's current support for OpenTelemetry, which includes the following behaviors:

• Each incoming HTTP request creates a span under the parent trace.
• Any outgoing HTTP requests create additional spans under the parent trace.
• Log output includes the TraceId and SpanId fields, to enable log lookup.

Activate OpenTelemetry Tracing

To activate OpenTelemetry in the Curity Identity Server, navigate to System → General and toggle on the feature.

The Curity Identity Server uses the OpenTelemetry SDK with the Zero-code SDK autoconfigure module. You can provide your preferred OpenTelemetry SDK options. For example, the following settings enable exporting of trace data, with OpemTelemetry defaults for all other options:

• otel.traces.exporter=otlp
• otel.exporter.otlp.endpoint=http://otel-collector:4317

In the Admin UI, use Changes → Commit to save and then Changes → Download to get the configuration of the Curity Identity Server in XML format. Inspect the XML file to see that the environment/telemetry section contains the OpenTelemetry settings.

<config xmlns="http://tail-f.com/ns/config/1.0">
  <environments xmlns="https://curity.se/ns/conf/base">
    <environment>
      ...
      <telemetry>
        <telemetry-provider>
          <opentelemetry xmlns="https://curity.se/ns/ext-conf/opentelemetry">
            <property>
              <name>otel.traces.exporter</name>
              <value>otlp</value>
            </property>
            <property>
              <name>otel.exporter.otlp.endpoint</name>
              <value>http://otel-collector:4317</value>
            </property>
          </opentelemetry>
        </telemetry-provider>
      </telemetry>
    </environment>
  </environments>
...
</config>

Prepare Curity Identity Server Logs

The Curity Identity Server uses the log4j2 logging framework. Read about the concepts in Logging Best Practices. When the data for a log entry originates from an HTTP request it can include OpenTelemetry TraceId and SpanId fields.

• Each logger represents a category of data, like system logs, request logs and audit logs.
• Each logger has one or more appenders that write to stdout, files or even databases.
• Each appender has a layout that determines the output format and fields to log.
• The layout can include the OpenTelemetry TraceId and SpanId in its output.

The default layout uses a pattern based layout with columnar data. Use the %X prefix and the TraceId and SpanId field names to include them in output.

<PatternLayout pattern="%date{yyyy-MM-dd'T'HH:mm:ss:SSSZ} %-5level {%thread} %logger %X{TraceId} %X{SpanId} %msg%n"/>

For JSON-based structured logs, the contextMap contains the TraceId and SpanId fields so you do not need additional field-level configuration. The following example shows a request log entry that includes the TraceId and SpanId fields.

{
    "loggerFqcn" : "org.apache.logging.log4j.spi.AbstractLogger",
    "level" : "INFO",
    "thread" : "req-106",
    "message" : "",
    "threadPriority" : 5,
    "threadId" : 12,
    "hostname" : "curity-idsvr-runtime-7454859bbc-9r5js",
    "http" : {
    "duration" : "403",
    "protocol" : "HTTP/1.1",
    "method" : "GET",
    "size" : "383",
    "content-type" : "application/vnd.auth+json",
    "params" : "",
    "secure" : "true",
    "uri" : "/oauth/v2/oauth-authorize",
    "accept" : "application/vnd.auth+json",
    "status" : "200"
    "contextMap" : {
      "RequestId" : "Lo0wiUp3",
      "SpanId":"073a05b1c33cacc3",
      "TraceId":"bbaadc1c1749c60237ae3cd8f3148101"
    },
    "loggerName" : "se.curity.identityserver.app.RequestReceiver",
    "timestamp" : "2025-02-07T16:40:25.032+0000"
}

OpenTelemetry Tracing Demo

If you are new to OpenTelemetry, the OpenTelemetry Collector Demo provides a Docker Compose example that you can run on a development computer, to understand the moving parts.

Conclusion

OpenTelemetry tracing improves observability for complex HTTP requests that consist of a number of related units of work. It is therefore a good fit for OAuth environments. The Curity Identity Server supports the essential OpenTelemetry tracing behaviors as part of its modern troubleshooting capabilities.