Data compression

Data compression enables a client to improve data throughput in the network by reducing the size of data it sends to Symphony. The data size threshold can be specified by the application at runtime.

Scope


Applicability

Details

Operating system

  • Windows

  • Linux

  • Solaris

Limitations

N/A


About data compression

By default, data compression is not enabled. Once a session is created with data compression enabled, it is preserved for the lifetime of the session and cannot be changed by opening or updating the session.

Data compression can be used in conjunction with the direct data transfer feature. In this case, compressed data will be sent to and from the service.

During the compression/decompression step, the client and service will consume additional memory, as required, to complete the compression/decompression operation. Once compression/decompression has completed, the additional memory will be released.

When to use data compression

If your data is highly compressible and over 1Kb in size, you can experience an improvement in data throughput by using data compression. Highly compressible data is characterized as mostly text or a mixture of text and some binary. Tests have shown that a compression ratio between 85% and 95% can be achieved for highly compressible data. Since these ratios depend heavily on the makeup of the data, data analysis and trial runs of different messages within the application can be performed to get a better idea of the compression ratio for your specific application’s data.

The compression ratio of application data is determined by the following formula:

compression ratio = (1- compressed data size/uncompressed data size) x 100

Enabling data compression for sessions

When data compression is enabled, all task inputs and outputs, as well as all common data and common data updates can be compressed. Whether the data is compressed depends on the threshold setting.

Here is the sequence for compressing data at the session level and submitting it to Symphony.

  1. The client creates a connection to Symphony.

  2. The client creates a session and sets the session attributes so that data compression is enabled.

  3. The client submits tasks using the session it has created. Data is sent to Symphony in compressed format if the data size exceeds the threshold setting.

Client API

The data compression feature can only be enabled through the client API.

Enabling data compression for sessions

You can enable data compression for all input/output tasks associated with a session, or for sending common data or common data updates. To enable data compression at the session level, the client application must do the following:

  1. Create a session using the appropriate session attribute to inform the API of the client’s intention to send data to Symphony in a compressed format. The session attribute is a member of the SessionCreationAttributes class.

  2. Send the task input messages to Symphony.

The following sample code shows how a SessionCreationAttributes object is set for data compression in each supported language. For more information about the SessionCreationAttributes and SessionOpenAttributes classes, refer to the API reference documentation.

C++

SesssionCreationAttributes attributes;
attributes.enableDataCompression(true);

Java

SessionCreationAttributes attributes = new SessionCreationAttributes();
attributes.enableDataCompression(true);

C# (.NET)

SessionCreationAttributes attributes = new SessionCreationAttributes();
attributes.EnableDataCompression = true;

Setting the data compression threshold

When data compression is enabled, the threshold specifies the message size in kilobytes that triggers compression. As long as the byte size of the message is below the threshold, the data will be sent according to the default Symphony model. If the byte size is greater than the threshold, the data will be sent in a compressed format. The default threshold is 1 kilobyte.

Note:

In general, empirical data has shown that a threshold of less than 1K does not yield reasonable compression ratios. In fact, if your threshold is too low, you can experience negative compression ratios, depending on the type of data.

To set the data compression threshold, the client application must do the following:

  1. Create a session using the appropriate session attribute to set the data compression threshold. The session attribute is a member of the SessionCreationAttributes class.

  2. Send the task input message with the task attributes.

The following sample code shows how to set the data compression threshold with a SessionCreationAttributes object in each supported language. For more information about the SessionCreationAttributes class, refer to the API reference documentation.

C++

SessionCreationAttributes attributes;
attributes.setDataCompressionThreshold(5);

Java

SessionCreationAttributes attributes = new SessionCreationAttributes();
attributes.setDataCompressionThreshold(5);

C# (.NET)

SessionCreationAttributes attributes = new SessionCreationAttributes();
attributes.DataCompressionThreshold = 5;

Setting the data compression flag

The following flags offer the ability to further modify data compression behavior when compression is enabled. The flags are mutually exclusive.

  • BetterSize: will give good compression in a reasonable time.

  • BestSpeed: performs compression but considering time as a factor. This will yield a smaller compression ratio but will give the best compression time.

The default flag setting is BestSpeed.

Here is a summary of compression characteristics based on the analysis of empirical data:

  • The time to decompress data will always be a fraction of the time it takes to compress the data. Depending on the compression flag selected, the time to decompress can be between 5% and 50% of the time to compress.

  • For data containing mostly text (such as XML), once the data size is greater than 20KB, a compression ratio above 85% is expected. As the data size increases, the compression ratio approaches 95%.

  • For data containing a mixture of text and binary formats, the average compression ratio, once the data size is greater than 1KB, is between 55% and 60%.

  • The option to choose BestSpeed compression (default) or not can make a difference for highly compressible data, i.e., mostly text. In general, BestSpeed provides ratios that are about 4% lower than BetterSize in cases where data is highly compressible, but consumes noticeably less time (between 0.5 and 0.75 of the time it takes to compress with BetterSize).

The following sample code shows how to set the compression flags with a SessionCreationAttributes object in each supported language. For more information about the SessionCreationAttributes, refer to the API reference documentation.

C++

SesssionCreationAttributes attributes;
attributes.setDataCompressionFlags(Session::BestSpeed);

Java

SessionCreationAttributes attributes = new SessionCreationAttributes();
attributes.setDataCompressionFlags(DataCompressionFlags.
BEST_SPEED);

C# (.NET)

SessionCreationAttributes attributes = new SessionCreationAttributes();
attributes.DataCompressionFlags = DataCompressionFlags.BestSpeed;