Persistent Jobs

Persistent Jobs are a facility for running background work asynchronously and reliably. A persistent job is stored in a data source, so it survives restarts, and is not tied to the node that created it.

This feature requires a data source that implements Persistent Jobs data access. All built-in data sources support this, but the database structure must be up-to-date (initially support was added in version 11.3). Currently, all server features that rely on persistent jobs are opt-in, so database upgrade is only needed if such features are used or if a custom plugin wants to use persistent jobs.

A persistent job encompasses a task (the actual code to be executed) and associated state to track its execution. A job can be submitted to run as soon as possible or to run at certain instant in the future (“not before” semantics), and it may be claimed for execution by any node in the cluster. The server makes no guarantee about exactly when a job runs; this depends on load and the configured polling interval (see below).

Persistent jobs are used by built-in server features but can also be used by plugins (see below).

Configuration#

Persistent jobs can be configured in the System page:

Admin UI → System → General → Persistent Jobs

.

The following settings are available:

  • Data Source: data source used to persist jobs. Must support Persistent Jobs data access.
    • If unset, the caching services data source is used, if it supports persistent jobs.
    • If no data source is configured or available, persistent jobs are not available and any attempt to submit jobs will fail.
  • Polling Interval: how often a node polls for jobs for execution.
  • Max Retries: maximum retries for tasks that fail with a retriable error. A task will be retried up to the configured number of times before being marked as failed.

Usage from plugins#

Plugins can submit jobs for execution using the PersistentJobManager SDK service. Tasks are types that implement the PersistentJobTask interface, and must be registered in the plugin descriptor by overriding the getPersistentJobTasks method.

Tasks must be implemented as idempotent, because they may be retried by the server. Specifically, tasks can throw RetriableFailureException to signal that a retriable error occurred. In this case, the server will reset the job’s status so that it’s picked up again for execution.

Refer to the plugins SDK documentation for more details.

Was this helpful?