/images/resources/tutorials/configuration/tutorials-datadog.png

Log Aggregation to Datadog

On this page

Datadog is a popular monitoring service. This article focuses on how to send logs from Curity Identity Server to Datadog.

Prerequisites

This tutorial covers how to configure Curity Identity Server with Datadog. If you need specific help with Datadog, please refer to their documentation.

The documentation of the Curity Identity Server also outline general information on how to configure logging.

Configuring Curity Identity Server

Logs must be in JSON format to be consumed by Datadog.

Curity Identity Server offers much flexibility when it comes to logging. You can specify different log settings for various parts of the product. For example, you can enable low-level logging for a specific element. You can have some logs written to files and others sent to Datadog.

Curity Identity Server uses Log4j 2, a logging framework. You can change the log settings by editing $IDSVR_HOME/etc/log4j2.xml.

Using Docker

If you are using Docker, you can either mount your modified log4j2.xml file or create your own image based on ours but with updated log settings.

In this file, we will create a new Appender that logs in JSON format. Keep in mind; the strategy is slightly different if you are running in a container or have Curity Identity Server binaries directly.

If installed locally, we will send the logs to a file. The Datadog agent will then tail this file.

xml
123456
<RollingFile name="datadog-log" fileName="${env:IDSVR_HOME}/var/log/datadog.log" filePattern="${env:IDSVR_HOME}/var/log/datadog.log.%i.gz">
<Policies>
<SizeBasedTriggeringPolicy size="10MB"/>
</Policies>
<JSONLayout compact="true" eventEol="true" properties="true" stacktraceAsString="true"/>
</RollingFile>

Using the Appender

Once we have an appender, we must select which Loggers will use it.

Since Curity Identity Server offers logging flexibility, there are multiple Loggers in the log4j2.xml file. You configure which appender to use by setting the AppenderRef property.

To set the main server log to use the new Datadog appender, it would look something like this:

xml
123
<AsyncLogger name="se.curity" level="INFO">
<AppenderRef ref="datadog-log"/>
</AsyncLogger>

You can use the same Appender to many (or all) Loggers.

You can also have multiple Appenders attached to a Logger. Just add an extra AppenderRef.

Setting up the Datadog Agent

The Datadog agent is responsible for transporting logs to Datadog. This agent is configured a bit differently, depending on your environment.

In the case of a local install of Curity Identity Server, you install the agent in the same way.

Start by installing the agent:

bash
1
DD_AGENT_MAJOR_VERSION=7 DD_API_KEY=[YOUR_KEY] DD_SITE="datadoghq.[eu|com]" bash -c "$(curl -L https://s3.amazonaws.com/dd-agent/scripts/install_script.sh)"

Then, you must configure it. First, enable log management. This is done by editing /etc/datadog-agent/datadog.yaml.

Set the following:

yml
1
logs_enabled: true

You must also set up where to find your logs. Create the file /etc/datadog-agent/conf.d/java.yaml with the following content:

yml
123456
logs:
- type: file
path: "<IDSVR_HOME>/var/log/datadog.log"
service: java
source: java
sourcecategory: sourcecode

Make sure to replace <IDSVR_HOME> with your installation directory of Curity Identity Server.

Lastly, restart the agent:

bash
12
sudo systemctl stop datadog-agent
sudo systemctl start datadog-agent

Conclusion

The setup above will send both logs and system metrics to Datadog. You can also connect Datadog to the metrics provided by Curity Identity Server.

Also, by utilizing multiple Loggers, you can choose which logs to send to Datadog. For example, you can log sensitive data to a file instead of sending it to Datadog.

Join our Newsletter

Get the latest on identity management, API Security and authentication straight to your inbox.

Start Free Trial

Try the Curity Identity Server for Free. Get up and running in 10 minutes.

Start Free Trial