OpenTelemetry#

OpenTelemetry is an observability framework and toolkit designed to create and manage telemetry data, such as traces, metrics, and logs. It focuses on how applications/systems can be instrumented in order to generate, collect, and export such data. The Curity Identity Server implements some aspects of OpenTelemetry, namely around traces and logs. Metrics are only exposed as Prometheus-compliant metrics, as described in Monitoring .

The following features are available:

• Context propagation from inbound requests.

  • Context is extracted from HTTP request headers.
  • By default, the headers defined by the W3C TraceContext specification are used (namely traceparent).

• A span is created for the processing of each HTTP request.

  • It follows the semantic conventions for HTTP server spans.
  • Its trace and parent span IDs are set using the context extracted from the request, if any, i.e. the server contributes to an ongoing distributed trace.
  • Note that this span is always created, even if there isn’t any context in the request. In that case, the span will be the root span of the trace (new trace ID).

• A span is created for outbound requests sent via Http Clients .

  • It follows the semantic conventions for HTTP client spans.
  • Its parent is the Curity Identity Server main request span (inbound).
  • The context is injected in the outbound request headers (namely the current trace and span IDs). By default, the headers defined by the W3C TraceContext specification are used.

• The current trace and span IDs are available for logging in the context of a request.

  • The TraceId and SpanId context keys can be used when configuring log appenders (e.g. %X{TraceId}).

• Data for the created spans is exported to an observability backend.

  • Logs are not exported via OpenTelemetry.

The Resource attributes associated with the different spans include information about the Curity Identity Server instance - namely the version and node ID - using service.* attributes, as well as basic information about the host/container running the system.

The features described above are only active if the OpenTelemetry support is enabled. In addition, the details of how the telemetry data is exported depend on the actual configuration, described in the next section.

Configuration#

OpenTelemetry can be enabled and configured using the Telemetry configuration section. In the Admin UI, these options are available in the System > General section.

The Curity Identity Server uses the official OpenTelemetry Java SDK, specifically the support for zero-code autoconfiguration. However, the configuration properties and their values are taken from the server configuration model, and then supplied to the OpenTelemetry SDK.

By default, when OpenTelemetry is enabled, the server extracts and propagates trace context, making the trace and span IDs available for logging, but doesn’t export trace data. The following table lists the properties that are set by default. Anything not listed here assumes the defaults specified by OpenTelemetry.

The following components are included in the distribution but not enabled by default:

• Any resource provider in the io.opentelemetry.instrumentation:opentelemetry-resources artifact.
• Amazon Web Services (AWS) resource provider.
• AWS X-Ray context propagators.
• Google Cloud Platform (GCP) resource providers.

Refer to the OpenTelemetry zero-code autoconfiguration documentation for detailed information on how to enable these components.

Examples#

The following is an example of a configuration excerpt enabling export of trace data.

Example configuration enabling export of trace data

<telemetry>
    <telemetry-provider>
        <opentelemetry xmlns="https://curity.se/ns/ext-conf/opentelemetry">
            <property>
                <name>otel.traces.exporter</name>
                <value>otlp</value>
            </property>
            <property>
                <name>otel.exporter.otlp.endpoint</name>
                <value>http://observability-backend:4317</value>
            </property>
        </opentelemetry>
    </telemetry-provider>
</telemetry>

The following is an example of a configuration excerpt enabling the AWS X-Ray context propagators.

Example configuration enabling the AWS X-Ray context propagators

<telemetry>
    <telemetry-provider>
        <opentelemetry xmlns="https://curity.se/ns/ext-conf/opentelemetry">
            <property>
                <name>otel.propagators</name>
                <value>tracecontext,xray</value>
            </property>
        </opentelemetry>
    </telemetry-provider>
</telemetry>

Extending#

The OpenTelemetry SDK has different extensibility points, namely via the use of service provider interfaces (SPIs) to extend autoconfiguration beyond the built-in components. If such extensibility components (other than the ones listed above) need to be used, the corresponding artifacts should be added to the lib/plugins/telemetry.opentelemetry directory in the server distribution.

Note about unstable components#

While the main OpenTelemetry Java SDK is stable, the Curity Identity Server also relies on unstable components provided by OpenTelemetry, namely resource provider implementations and semantic conventions for span attributes. Some of these components may change in future releases, impacting the shape of the server telemetry data. Nevertheless, their usage makes the resulting telemetry data more useful, so they were included.

Other Approaches - Service ID Header#

Curity has support for adding a HTTP response header containing the node’s service-id, which is a unique string based on the service-name (see cluster for more information), to every HTTP response.

To enable this header, set the system property called se.curity:identity-server:http:service-id-header when starting up Curity. The value of this property should be the name of the header you want to contain the service-id.

$ JAVA_OPTS="-Dse.curity:identity-server:http:service-id-header=X-Service-Id" idsvr

When that’s done, every Curity response will contain a header like X-Service-Id: the-service-id, making it easier to trace requests to particular Curity nodes.