OpenTelemetry

OpenTelemetry is an observability framework and toolkit designed to create and manage telemetry data, such as traces, metrics, and logs. It focuses on how applications/systems can be instrumented in order to generate, collect, and export such data. The Curity Identity Server implements some aspects of OpenTelemetry, namely around traces and logs. Metrics are only exposed as Prometheus-compliant metrics, as described in Monitoring.

The following features are available:

  • Context propagation from inbound requests.

    • Context is extracted from HTTP request headers.
    • By default, the headers defined by the W3C TraceContext specification are used (namely traceparent).

  • A span is created for the processing of each HTTP request.

    • It follows the semantic conventions for HTTP server spans.
    • Its trace and parent span IDs are set using the context extracted from the request, if any, i.e. the server contributes to an ongoing distributed trace.
    • Note that this span is always created, even if there isn’t any context in the request. In that case, the span will be the root span of the trace (new trace ID).

  • A span is created for outbound requests sent via Http Clients.

    • It follows the semantic conventions for HTTP client spans.
    • Its parent is the Curity Identity Server main request span (inbound).
    • The context is injected in the outbound request headers (namely the current trace and span IDs). By default, the headers defined by the W3C TraceContext specification are used.

  • The current trace and span IDs are available for logging in the context of a request.

    • The TraceId and SpanId context keys can be used when configuring log appenders (e.g. %X{TraceId}).

  • Data for the created spans is exported to an observability backend.

    • Logs are not exported via OpenTelemetry.

The Resource attributes associated with the different spans include information about the Curity Identity Server instance - namely the version and node ID - using service.* attributes, as well as basic information about the host/container running the system.

The features described above are only active if the OpenTelemetry support is enabled. In addition, the details of how the telemetry data is exported depend on the actual configuration, described in the next section.

Configuration

OpenTelemetry can be enabled and configured using the Telemetry configuration section. In the Admin UI, these options are available in the System > General section.

The Curity Identity Server uses the official OpenTelemetry Java SDK, specifically the support for zero-code autoconfiguration. However, the configuration properties and their values are taken from the server configuration model, and then supplied to the OpenTelemetry SDK.

By default, when OpenTelemetry is enabled, the server extracts and propagates trace context, making the trace and span IDs available for logging, but doesn’t export trace data. The following table lists the properties that are set by default. Anything not listed here assumes the defaults specified by OpenTelemetry.

Property Value Observations
otel.java.enabled.resource.providers io.opentelemetry.instrumentation.resources.HostResourceProvider,io.opentelemetry.instrumentation.resources.ContainerResourceProvider Container and host resource attributes.
otel.propagators tracecontext Use the headers defined by the W3C TraceContext specification for context propagation.
otel.traces.exporter none Don’t export span data by default. Refer the next sections for an example on how to enable this.
otel.metrics.exporter none The server doesn’t produce metrics via OpenTelemetry. Shouldn’t be enabled.
otel.logs.exporter none The server doesn’t produce logs via OpenTelemetry. Shouldn’t be enabled.

The following components are included in the distribution but not enabled by default:

Refer to the OpenTelemetry zero-code autoconfiguration documentation for detailed information on how to enable these components.

Examples

The following is an example of a configuration excerpt enabling export of trace data.

Listing 84 Example configuration enabling export of trace data
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
<telemetry>
    <telemetry-provider>
        <opentelemetry xmlns="https://curity.se/ns/ext-conf/opentelemetry">
            <property>
                <name>otel.traces.exporter</name>
                <value>otlp</value>
            </property>
            <property>
                <name>otel.exporter.otlp.endpoint</name>
                <value>http://observability-backend:4317</value>
            </property>
        </opentelemetry>
    </telemetry-provider>
</telemetry>

The following is an example of a configuration excerpt enabling the AWS X-Ray context propagators.

Listing 85 Example configuration enabling the AWS X-Ray context propagators
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
<telemetry>
    <telemetry-provider>
        <opentelemetry xmlns="https://curity.se/ns/ext-conf/opentelemetry">
            <property>
                <name>otel.propagators</name>
                <value>tracecontext,xray</value>
            </property>
        </opentelemetry>
    </telemetry-provider>
</telemetry>

Extending

The OpenTelemetry SDK has different extensibility points, namely via the use of service provider interfaces (SPIs) to extend autoconfiguration beyond the built-in components. If such extensibility components (other than the ones listed above) need to be used, the corresponding artifacts should be added to the lib/plugins/telemetry.opentelemetry directory in the server distribution.

Note about unstable components

While the main OpenTelemetry Java SDK is stable, the Curity Identity Server also relies on unstable components provided by OpenTelemetry, namely resource provider implementations and semantic conventions for span attributes. Some of these components may change in future releases, impacting the shape of the server telemetry data. Nevertheless, their usage makes the resulting telemetry data more useful, so they were included.