Log Aggegation to Splunk
Splunk is a popular tool for log management. There are various ways to get the logs from an application to Splunk. Once the logs are there they can be categorized into fields to make it easy for the operator to find the things they are interested in. The Curity Identity Server already divides its logs into fields with e.g. severity and where it happened. In this tutorial we will look into how to setup Curity to transfer its logs to Splunk together with our fields.
This tutorial covers how to configure Curity Identity Server. If you need help on configuring Splunk please use their documentation.
Splunk is expected to be setup and working.
General information of how to configure logging is available in the documentation of the Curity Identity Server.
We will use the http appender of log4j to send the logs to Splunk.
First, create a
HTTP Event Collector in Splunk. Set the type to
Structured -> _json. Make a note of the token you get.
Then we will update the Curity Identity Server logging configuration by editing the
Using DockerIf you are using Docker you can either mount your modified log4j2.xml file or create your own image based on ours but with updated log settings.
Add an appender looking like this:
<Http name="http" url="http://mysplunkserver:8088/services/collector/raw">; <Property name="Authorization" value="Splunk d5512c63-4ead-4f5d-a802-944a9c796d2a"/> <JsonLayout properties="true" /> </Http>
Make sure it points towards your Splunk instance. Use the endpoint
/services/collector/raw and set the
Authorization attribute to
JsonLayout makes it possible for Splunk to automatically find which fields we have.
You can add additional metadata to the appender which can then be used in Splunk.
<Http name="splunk-server-log" url="http://mysplunkserver:8088/services/collector/raw">; <Property name="Authorization" value="Splunk d5512c63-4ead-4f5d-a802-944a9c796d2a"/> <JsonLayout properties="true"> <KeyValuePair key="environment" value="prod"/> </JsonLayout> </Http>
In this case Splunk will create an
environment field and set all log events from this appender to
This example shows how to configure plain http. If you want to use TLS or set other properties to the appender you refer to the full documentation of Log4j.
Once we have an appender we have to select which
Loggers that should use it.
Curity Identity Server offers a lot of flexibility regarding logging. This is why you see multiple
Loggers in the
log4j2.xml file. Each
Logger allows you to have different log settings. It could be to have a certain log level for parts of the product or that audit logs should be treated differently. You configure which appender to use by setting the
If you want to set the main server log to use the new Splunk appender it would look something like this:
<AsyncLogger name="se.curity" level="INFO"> <AppenderRef ref="splunk-server-log"/> </AsyncLogger>
You can use the same
Appender to many (or all)
You can also have multiple
Appenders attached to a
Logger. Just add an extra
Splunk is a convenient way to manage your logs. An important thing is to configure fields in order to be able to quickly find what you are looking for. By using the fields provided by Curity Identity Server the configuration of Splunk is minimal.
By having multiple Loggers you have the possibility to send only the logs you want to Splunk. Sensitive data can be logged to file instead for example.