Audit#

This chapter describes audit logging in Curity Identity Server. Audit logging is an always-on feature, facilitated through Apache Log4j 2.

Audit messages are logged at the INFO level to the audit-events logger. An audit message is formatted as a Syslog (RFC 5424) Structured Data Message, where the structured data parameters of the message are populated by the audit data parameters of the event. Most individual parameters can be accessed through the Map Conversion Pattern provided by the Log4j Pattern Layout; e.g. %K{key}. The id, type and message parameters are exceptions and must be accessed through the structured_data_id, structured_data_type and structured_data_message Conversion Patterns, respectively.

Configuration#

Audit logging is configured through the Log4j configuration file located at ${IDSVR_HOME}/etc/log4j2.xml. For in-depth details on how to configure Log4j, please consult the Apache Log4j 2 Manual.

The audit log configuration consists of a logger and two appenders.

Logger#

The default audit logger configuration as it appears in ${IDSVR_HOME}/etc/log4j2.xml:

<AsyncLogger name="audit-events" level="INFO" additivity="false">
  <AppenderRef ref="audit-log"/>
  <AppenderRef ref="audit-db"/>
</AsyncLogger>

The default audit logger configuration has the following properties:

Messages logged to audit-events are collected.
The logger is asynchronous; for performance, as many audit events occur on a request thread.
The logger is not additive; to prevent the root logger from also collecting audit messages.
The audit-log and audit-db appenders are utilized to persist the messages.

File Appender#

The default audit to file appender configuration as it appears in ${IDSVR_HOME}/etc/log4j2.xml:

<RollingFile name="audit-log" fileName="${env:IDSVR_HOME}/var/log/audit.log"
             filePattern="${env:IDSVR_HOME}/var/log/audit.log.%i.gz">
  <Policies>
    <SizeBasedTriggeringPolicy size="10MB"/>
  </Policies>
  <Rfc5424Layout appName="Curity" facility="AUDIT" newLine="true" includeMDC="false" />
</RollingFile>

The default audit to file appender configuration has the following properties:

Audit messages are appended to the file ${IDSVR_HOME}/var/log/audit.log.
The log file rolls when it reaches 10MB in size. Rolled files are stored as ${IDSVR_HOME}/var/log/audit.log.<index>.gz; where <index> is an integer counter.
Audit messages are formatted as Syslog (RFC 5424) compliant structured data messages.

Database Appender#

The default audit to database appender configuration as it appears in ${IDSVR_HOME}/etc/log4j2.xml:

<JDBC name="audit-db" tableName="audit">
  <ConnectionFactory class="se.curity.identityserver.logging.AuditDatabaseConnectionFactory" method="getDatabaseConnection" />
  <Column name="id" pattern="%structured_data_id" />
  <Column name="event_type" pattern="%structured_data_type" />
  <Column name="message" pattern="%structured_data_message" />
  <Column name="instant" isEventTimestamp="true" />
  <Column name="event_instant" pattern="%K{instant}" />
  <Column name="server" pattern="%K{server}" />
  <Column name="subject" pattern="%K{subject}" />
  <Column name="client" pattern="%K{client}" />
  <Column name="resource" pattern="%K{resource}" />
  <Column name="authenticated_subject" pattern="%K{authenticatedSubject}" />
  <Column name="authenticated_client" pattern="%K{authenticatedClient}" />
  <Column name="acr" pattern="%K{acr}" />
  <Column name="endpoint" pattern="%K{endpoint}" />
  <Column name="session" pattern="%K{session}" />
</JDBC>

The default audit to database appender configuration has the following properties:

Audit events are posted to the audit table.
The proprietary se.curity.identityserver.logging.AuditDatabaseConnectionFactory Connection Factory is used; which provides connectivity through a configured JDBC Data Access Provider . Only one such provider can be configured to collect audit messages at a time.
Each column in the target database table is explicitly mapped using a Column element.

If you are using PostgreSQL datasource to log audit events, please note, that in the current version of Log4J (2.7) all columns (except if isEventTimestamp=“true”) are treated as unicode strings and their values are inserted via the PreparedStatement.setNString method, that is not supported by PostgreSQL JDBC driver. Log entries containing Feature is not implemented: PreparedStatement.setNString will indicate this problem in the server log.

To work around this issue you should explicitly declare all string columns as non-Unicode in your appender configuration. This is done by adding isUnicode="false" property to column definitions.

Batching Log Messages for performance#

By default, every message logged by the Database Appender will need to obtain a database connection from the JDBC connection pool. This can become a bottleneck if the server handles a very large event load.

To optimise for performance (at the cost of reliability), it is possible to batch messages to minimize the impact of each individual message.

To do that, use the bufferSize attribute as shown below:

Setting the bufferSize attribute:

<JDBC name="audit-db" tableName="audit" bufferSize="8">

Curity includes a healthcheck for the Database Appender, but to enable it, you must set the log4j2.formatMsgAsync system property to true. That will cause a message similar to the one below to be logged every few seconds:

Logger healthcheck message:

se.curity.identityserver.app.Log4jMonitor - Current logging system health level is [OK]: PT0.000449917S

If the delay between an event occurring and the message being flushed out is within a reasonable limit, the above message will contain [OK].

If it takes longer than expected, you may also see one of [SLOW] (over 250ms), [VERY_SLOW] (over 1sec) and [CRITICAL] (over 2sec).

A system that is working within reasonable conditions should only very rarely report anything other than [OK]. Long delays are normally caused by a logger appender running too slowly. Batching messages as explained above may help if that’s a problem.

For more information about configuring the JDBC appender, please consult the Apache Log4j 2 Manual.

Audit Data#

Audit events carry a common set of parameters; some present in all events, others not.

Mandatory#

These parameters are present in all audit events.

type : the type of the audit event.
id : the unique ID of the audit event.
instant : the instant the audit event occurred.
message : a human readable message detailing the audit event.

Optional#

These parameters are present in some audit events; depending on the context of the event.

subject : the subject affected by the audit event.
client : the client affected by the audit event.
resource : the resource affected by the audit event.
authenticatedSubject : the authenticated subject that triggered the audit event. For userinfo and token related events, contains the subject value provided by authentication, before pseudo-anonymization is applied.
authenticatedClient : the authenticated client that triggered the audit event.
acr : the ACR of the authenticator involved in the audit event.
endpoint : the endpoint called to trigger the audit event.
session : the session on which the audit event was triggered.