JSON Data Source

The JSON datasource is represented by a RESTful service implementation. Its purpose is to offer a simple extensibility point to integrate data providers and credential verification services with the Curity Identity Server. This chapter describes how the server will interact with an external JSON data source.

Credential verification

The service that verifies credentials using a JSON data source, supports both verification as well as updating credentials for a given subject.

Protocol

Verification of user credentials can be performed through different strategies, either by providing credentials to the datasource to be verified by the datasource, or by returning the actual stored credential to the Curity Identity Server so it can be processed and verified inside the server. While this strategy is configurable in the server, it reflects on the attributes that have to be returned by the Credential verification endpoint of the JSON data source.

Credential verification supports the following HTTP request methods:

  1. POST-request, with request parameters in the request-body formatted as either url-encoded formdata or json
  2. GET-request, with request parameters in the querystring

Warning

The use of GET is strongly discouraged, as the resulting querystring can include the secret password, which may be exposed unexpectedly or even written to log-files.

While the request types are different, the response for either of these requests is of the same type. Success is reported by responding with a HTTP/200 OK status, with optional account-related attributes in the response body. Errors are reported by responding with a non-success (non-2xx) HTTP status code. Depending on the error, either HTTP/401, HTTP/400 or HTTP/500 should be returned as status code of the response. The body should be a JSON document. When it is, the error member will be used as the error that is returned to the client if detailed error reporting is enabled in the applicable profile.

Note that all non-success status codes are treated as error. In case an error occurs, the whole response body will be written to the server’s log-file as debug message.

Note

Under normal circumstances, a failed-authentication alarm will be raised when a web server responds with 401 Unauthorized, as this often indicates that the client, not the user, failed to authenticate. Since the JSON datasource allows the remote server to respond with 401 Unauthorized when the user’s credentials are invalid, failed-authentication alarms are disabled for requests that verify the user’s credentials. This overrides the failed-authentication alarm settings on the HTTP Client configured on the JSON datasource. All alarm settings on the HTTP Client are applied for all other requests.

Message format

Request message

request parameter description
username required, the username, for example the value a user has submitted in a login form
password the password. If the data source backend does not verify the credential, it is optional to provide the password.

Success response message

response attributes description
password the password from the backend. If the data source backend does not verify the credential, it is mandatory to return this.
* any other attribute that is returned, is added to the subject attributes that represent the authenticated subject internally.

Note that if the response attributes contain an attribute with name subject, it is replaced with the value of the username from the request when representing the authenticated subject internally.

Error response message

response attributes description
error a human readable description of what went wrong. This message will end up in the server’s log file and may be reported to the user if detailed error reporting is enabled.

Examples

Some practical examples follow. Note that lines of the HTTP request- and response bodies may be split for readability purpose only.

POST with url-encoded formdata

The following example is of a request using the POST-method, to a JSON data source that verifies the password. The request parameters are passed as url-encoded formdata. Note that the response does not contain a password-attribute.

Request:

POST /credverif HTTP/1.1
Host: json-ds.example.com
Content-Type: application/x-www-form-urlencoded
Accept: application/json

username=teddie&password=Secret%231

Response:

HTTP/1.1 200 OK
Content-Type: application/json

{"username":"teddie","phonenr":"+1234567890","email":["teddie+1@example.com","teddie+2@example.com"],
"address":{"street":"Main Street","country":"SE"},"externalIds":[{"type":"internal","value":"internal:teddie"},
{"type":"external","value":"external:teddie"}],"firstName":"teddie","lastName":"User"}

POST with JSON-encoded formdata

The following example is the same as above, but it uses the JSON-format to encode the request parameters.

Request:

POST /credverif HTTP/1.1
Host: json-ds.example.com
Content-Type: application/json
Accept: application/json

{"username":"teddie","password":"Secret#1"}

Response:

HTTP/1.1 200 OK
Content-Type: application/json

{"username":"teddie","phonenr":"+1234567890","email":["teddie+1@example.com","teddie+2@example.com"],
"address":{"street":"Main Street","country":"SE"},"externalIds":[{"type":"internal","value":"internal:teddie"},
{"type":"external","value":"external:teddie"}],"firstName":"teddie","lastName":"User"}

GET with querystring parameters

Again, this example is the same as before, but it uses the querystring to present the request parameters to the service.

Request:

GET /credverif?username=teddie&password=Secret%231 HTTP/1.1
Host: json-ds.example.com
Accept: application/json

Response:

HTTP/1.1 200 OK
Content-Type: application/json

{"username":"teddie","phonenr":"+1234567890","email":["teddie+1@example.com","teddie+2@example.com"],
"address":{"street":"Main Street","country":"SE"},"externalIds":[{"type":"internal","value":"internal:teddie"},
{"type":"external","value":"external:teddie"}],"firstName":"teddie","lastName":"User"}

GET with querystring parameters to backend that does not verify

This is an example of a GET-request to a backend that does not verify a credential, instead it returns it in its response for the receiver to be able to perform the verification.

Request:

GET /credverif?username=teddie HTTP/1.1
Host: json-ds.example.com
Accept: application/json

Response:

HTTP/1.1 200 OK
Content-Type: application/json

{"username":"teddie","phonenr":"+1234567890","firstName":"teddie","lastName":"User",
"password":"$6$/1PH/dP/$A0jYR22X.4HeE21FXpw0fam1ANry4w3cb1HXbRwZJMu.IhguY1z8.0isTSd2ZXHuFaI3R8ci/ZMnQTlFXGOe0."}

Notice that a password credential is returned, which is an encrypted hash. The receiver can use this to verify the password that the user provided.

POST with JSON-encoded formdata with invalid password

The following example shows a POST-request with invalid user credentials.

Request:

POST /credverif HTTP/1.1
Host: json-ds.example.com
Content-Type: application/json
Accept: application/json

{"username":"teddie","password":"invalid"}

Response:

HTTP/1.1 401 Unauthorized
Content-Type: application/json

{"error":"invalid or unknown username and password provided."}

POST with invalid JSON-encoded formdata

The following example shows a POST-request that doesn’t include the required request username-parameter.

Request:

POST /credverif HTTP/1.1
Host: json-ds.example.com
Content-Type: application/json
Accept: application/json

{"firstname":"teddie"}

Response:

HTTP/1.1 400 Bad Request
Content-Type: application/json

{"error":"invalid or unknown username and password provided."}

Attribute Provider

The attribute provider is responsible for returning attributes for a given subject.

Protocol

Getting attributes is a matter of making a GET-request to the attributes-endpoint that includes the subject, and getting back a response-message with the attributes. A success is also indicated by the HTTP/200 OK HTTP status in the response. If an error occurred because the subject for whom attributes are requested could not be derived, the error response is being returned by an error HTTP response code together with a JSON-formatted error response message. If the error is caused by missing parameters or a malformed request, the HTTP status should be HTTP/400 Bad Request. On the other hand, when a valid request was made and the error occurred while processing this request, the response should be HTTP/500 Internal Server Error. If the subject did not exist in the backend, the backend should still return successfully but with no attributes in the response-message.

Message format

Request message

Regarding how the value for subject is transferred in a request, the data source can behave in RESTful way or it can take an actual parameter to do so. The RESTful way means that the subject’s attributes are considered a resource, and the URI to the resource includes the subject. For example, the attributes of user teddie are to be found by making a GET-request to https://json-ds.example.com/users/teddie. Note that the subject-part of this URI (.../users/:subject) is URL-encoded, so user teddie the man is located by .../users/teddie%20the%20man.

The alternative way to point to the subject’s attributes, is to include the subject as a parameter. This can be either a parameter from the HTTP request header, or a parameter on the querystring. When the subject is included as header-parameter, its value is considered as UTF8-encoded String, and Base64-encoded before it gets set in the HTTP request. When it is in a querystring, it is URL-encoded.

request parameter description
subject required, the subject that attributes are requested for. If the service has a RESTful API, the subject is included in the URL, otherwise it is transferred either as header parameter, or querystring parameter.

Note that the Curity Identity Server allows you to configure the name of the parameter.

Success response message

response attributes description
* all attributes in a JSON-structure. The attributes can be nested.

The attributes are returned in JSON-format.

Error response message

response attributes description
error a human readable description of what went wrong. This message will end up in the server’s log file

Parameter Mappings

In addition to the subject parameter, the Curity Identity Server allows you to send additional parameters. The parameters can either be looked up from the attributes belonging to the subject, or static values from configuration.

  • When using query parameters, the value will be URL encoded.
  • When using header parameters, the value will be Base64 encoded.
configuration parameters description
parameter-name The parameter to send to the JSON data source. If no value mapping is configured, the value of the attribute with the same name in the subject attributes will be used.
use-value-of-attribute The name of the attribute to get the value from. If the attribute is not found, the parameter is omitted. Optional
static-value A static string to send as the value for the parameter-name. Optional

The following configuration would result in a GET request for the data source, with the query parameters subject, display-name and organization.

Listing 211 parameter mappings example :linenos:
        <attributes>
            <parameter>
                <provide-as>query-parameter</provide-as>
                <username-parameter>subject</username-parameter>
            </parameter>
            <parameter-mappings>
                <parameter-mapping>
                    <parameter-name>display-name</parameter-name>
                    <use-value-of-attribute>displayName</attribute-value>
                </parameter-mapping>
                <parameter-mapping>
                    <parameter-name>organization</parameter-name>
                    <static-value>Development</static-value>
                </parameter-mapping>
                <parameter-mapping>
                    <parameter-name>role</parameter-name>
                </parameter-mapping>
            </parameter-mappings>
        </attributes>
  • subject will get its value from the authenticated subject.

  • display-name will be filled in from the authenticated subject attribute displayName. If displayName is not found, it will be ignored.

  • organization will always have the value Development

  • role will be filled in from the authenticated subject attribute role. If role is not found, it will be ignored.

    GET /users?subject=teddie&display-name=Teddie+Bear&organization=Development&role=developer HTTP/1.1
    Host: json-ds.example.com
    Accept: application/json
    

Examples

Some practical examples follow. Note that lines of the HTTP request- and response bodies may be split for readability purpose only.

GET the RESTful way

The following example is a RESTful GET-request to a URL for teddie’s attributes. The subject exists in the remote backend.

Request:

GET /users/teddie HTTP/1.1
Host: json-ds.example.com
Accept: application/json

Response:

HTTP/1.1 200 OK
Content-Type: application/json

{"username":"teddie","phonenr":"+1234567890","email":["teddie+1@example.com","teddie+2@example.com"],
"address":{"street":"Main Street","country":"SE"},"externalIds":[{"type":"internal","value":"internal:teddie"},
{"type":"external","value":"external:teddie"}],"firstName":"teddie","lastName":"User"}

GET with header parameter

The following example is a GET-request to a URL, with teddie as subject in the HTTP request header. The subject exists in the remote backend.

Request:

GET /users HTTP/1.1
Host: json-ds.example.com
Accept: application/json
subject: dGVkZGll

Response:

HTTP/1.1 200 OK
Content-Type: application/json

{"username":"teddie","phonenr":"+1234567890","email":["teddie+1@example.com","teddie+2@example.com"],
"address":{"street":"Main Street","country":"SE"},"externalIds":[{"type":"internal","value":"internal:teddie"},
{"type":"external","value":"external:teddie"}],"firstName":"teddie","lastName":"User"}

GET with query parameter and unknown subject

The following example is a GET-request to a URL, with teddie the man as subject in the query string. The subject does not exist in the remote backend.

Request:

GET /users?subject=teddie+the+man HTTP/1.1
Host: json-ds.example.com
Accept: application/json

Response:

HTTP/1.1 200 OK
Content-Type: application/json

{}

Note the HTTP/200 OK response.

GET without providing subject

The following example is a GET-request to a URL without providing the subject.

Request:

GET /users HTTP/1.1
Host: json-ds.example.com
Accept: application/json

Response:

HTTP/1.1 400 Bad Request
Content-Type: application/json

{"error": "No or invalid subject provided."}

Bucket Access

The HTTP service backing the data source should support the three bucket operations: fetch, store and clear. All operations are parameterized by the bucket’s subject and purpose, which should be treated as a composite key.

The data source allows individual configuration of the HTTP method and URL for each operation, as illustrated in the following configuration snippet.

Listing 212 Buckets configuration example (default values shown)
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
<buckets>
    <fetch>
        <method>get</method>
        <url>/buckets?subject=:subject&amp;purpose=:purpose</url>
    </fetch>
    <store>
        <method>put</method>
        <url>/buckets?subject=:subject&amp;purpose=:purpose</url>
    </store>
    <clear>
        <method>delete</method>
        <url>/buckets?subject=:subject&amp;purpose=:purpose</url>
    </clear>
</buckets>

For each operation, the method configuration setting defines the HTTP method used on the corresponding requests to the backing service. The url setting defines a template for the URL path and query to be used on those requests. The :subject and :purpose placeholders must be used to include the bucket’s subject and purpose in the request.

The following sections include additional details on the protocol to be implemented by the backing service for each operation. When the response status code or content are not as expected, the ongoing DAP operation fails with an exception and a failed communication alarm is raised for the data source.

Protocol - Fetch

By default, a GET request is made to /buckets, including the subject and purpose query parameters. The request body is empty.

The response must match one of the following:

  • HTTP/200 OK HTTP status with a JSON payload containing an object with the bucket content (may be an empty object).
  • HTTP/204 No Content or HTTP/404 Not Found HTTP status if no content was found for the bucket.

Example - Default configuration

Request:

GET /buckets?subject=47690376&purpose=test HTTP/1.1
Host: json-ds.example.com
Accept: application/json

Response:

HTTP/1.1 200 OK
Content-Type: application/json

{ "key": "value" }

Example - Custom URL configuration

The requests to the backing service depend on the data source configuration. For example, if the URL template is configured as follows:

Listing 213 Configuration of a custom URL for fetch operations
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
<data-source>
    <id>JsonDS</id>
    <json xmlns="https://curity.se/ns/conf/data-access/json">
        <buckets>
            <fetch>
                <method>get</method>
                <url>/buckets/:subject?purpose=:purpose</url>
            </fetch>
        </buckets>
    </json>
</data-source>

Then, the fetch request will be:

Request:

GET /buckets/47690376?purpose=test HTTP/1.1
Host: json-ds.example.com
Accept: application/json

Response:

HTTP/1.1 200 OK
Content-Type: application/json

{ "key": "value" }

Protocol - Store

By default, a PUT request is made to /buckets, including the subject and purpose query parameters. The request body is a JSON payload with the content to be stored.

The response must have an HTTP status in the 200-299 range. The response body is never considered.

Example

Request:

PUT /buckets?subject=47690376&purpose=test HTTP/1.1
Host: json-ds.example.com
Content-Type: application/json

{ "key": "value" }

Response:

HTTP/1.1 204 No Content

Protocol - Clear

By default, a DELETE request is made to /buckets, including the subject and purpose query parameters. The request body is empty.

The response must match one of the following:

  • 2XX HTTP status, indicating that the bucket was cleared.
  • HTTP/404 Not Found HTTP status if no content was found for the bucket.

The response body is not considered.

Example

Request:

DELETE /buckets?subject=47690376&purpose=test HTTP/1.1
Host: json-ds.example.com

Response:

HTTP/1.1 204 No Content

Authentication

The backend server can authenticate using any of the following schemes:

  1. Basic Authentication
  2. OAuth Client Credentials Authentication
  3. Client Certificate Authentication

These are all configured in the Curity Identity Server in the http client configuration.