Data purger

The data purger (purger) service maintains the size of the database by purging old data from the database.

About the data purger

The relational database needs to be kept to a reasonable size to maintain optimal efficiency. The data purger manages the database size by purging old data from the database at regular intervals, which consists of dropping partitions that are older than the calculated data retention date.

Logging levels

There are logging levels that determine the detail of messages that the data loaders record in the log files. In decreasing level of detail, these levels are ALL (all messages), TRACE, DEBUG, INFO, WARN, ERROR, FATAL, and OFF (no messages).

By default, the data purger logs messages of ERROR level or higher (that is, all ERROR and FATAL messages) to the data purger log file, which is located in the Platform Analytics server log directory (ANALYTICS_TOP/log in the Platform Analytics server host).

Default behavior

The data purger runs as the following scheduled tasks on the Platform Analytics server:

  • PartitionMaintenanceGroup1

  • PartitionMaintenanceGroup2

  • PartitionMaintenanceGroup3

Each scheduled tasks is responsible for purging different tables according to different schedules. This allows the workload to be split among different times.

Each scheduled task calculates the data retention date according to the data purger configuration, examines the tables (and their corresponding partitions) for which it is configured and drops any partitions that are older than the calculated data retention date.

Data purger interactions

The data purger drops database partitions from the data tables through the server data sources.

The following diagram illustrates the interaction between the data purger and other components.

Data purger actions

The data purger is installed as scheduled tasks. Change the schedules of the data purger services as you would for scheduled tasks (see Scheduled tasks).