Optimize the Platform Analytics node

Optional. Modify the Platform Analytics node to enhance performance.

What you need to do

  1. Increase JVM memory
  2. Disperse the Platform Analytics node workload
  3. Optimize specific data loaders

Increase JVM memory

Increase the Java Virtual Machine (JVM) memory to at least 2 GB for the Platform Analytics node instead of using the default value to prevent the JVM from running out of memory.

Note:

Java cannot use more than 1638 MB of memory on 32-bit platforms. If you are using 32-bit Windows, you can only increase the JVM memory to 1638 MB.

  1. Log into the Platform Analytics ndoe host.
  2. Edit the ANALYTICS_TOP/conf/wsm/wsm_plc.conf file.
  3. Change the Java starting options to increase the JVM memory.

    Navigate to JAVA_OPTS and increase the JVM memory to at least 2 GB.

    For example,

    JAVA_OPTS=‑Xms64m ‑Xmx2048m

Disperse the Platform Analytics node workload

You can reduce performance issues by dispersing the Platform Analytics node workload among multiple hosts or multiple loader controllers within a host so that each host or loader controller is responsible for a specific type (or types) of data loading.

Based on the type of data, you can categorize all the data loaders into different types, and enable each node or loader controller to be responsible for only one type of data loader. The following table describes the different data loader categories and their corresponding loader controller configuration files:


Data loader category

Loader controller configuration file

Host-related

  • plc_ego.xml

  • plc_coreutil.xml

Job-related

  • plc_lsf.xml

  • plc_bjobs-sp012.xml

Advanced job-related

  • plc_lsf_advanced.xml

License-related

  • plc_license.xml


Disperse the Platform Analytics node workload to multiple hosts

A single node might experience performance issues when handling the high workload. To resolve this issue, you can disperse the node workload to multiple hosts so that each host is responsible for a specific type of data loading.

  1. Install and configure the Platform Analytics node to other hosts in the same LSF cluster.

    For example, if you plan to have one node for each data loader category, you should install and configure three additional Platform Analytics node hosts. Therefore, your original node would handle host-related data loaders, while each of the other three nodes would handle each of the other three categories.

    Alternatively, you can have fewer nodes and still disperse workload to some degree. For example, you could have two nodes: your original node could handle host-related and job-related data loaders while another node could handle advanced job-related and license-related data loaders.

  2. For each Platform Analytics node, disable the data loading categories that the node will not be handling.
    1. Navigate to the ANALYTICS_TOP/conf/plc directory.
    2. Move or rename the configuration files for the data loader categories that the Platform Analytics node will no longer handle.

      For example, if you want the Platform Analytics node to only handle host-related data loaders, move or rename all configuration files except plc_ego.xml and plc_coreutil.xml.

Disperse the Platform Analytics node workload to multiple loader controllers in one node

If your node host is powerful enough to handle the workload (for example, the host has at least four cores and at least 6 GB of memory), you can reduce the performance bottlenecks by creating multiple loader controllers and having each loader controller be responsible for a specific type of data loading. This enables each of the multiple cores in your host to control a single loader controller and be more efficient in sharing the workload among multiple cores.

  1. For each data loader category beyond the first, create a new loader controller on the same host.

    Therefore, since there are four data loader categories, create three additional loader controllers.

    In this section, PLC_NAME represents the name of the new loader controller that you will create.

    For example, you can create plc_job to handle job-related data loaders, plc_advanced_job to handle advanced job-related data loaders, and plc_license to handle license-related data loaders, while the original loader controller (plc) handles host-related data loaders.

    1. From the ANALYTICS_TOP/conf/wsm directory, copy the wsm_plc.conf file to a new file, one for each new loader controller.

      For each new loader controller, name the new file wsm_PLC_NAME.conf

      For example, copy wsm_plc.conf to wsm_plc_job.conf, wsm_plc_advanced_job.conf, and wsm_plc_license.conf.

    2. Edit each new wsm_PLC_NAME.conf file and specify the new loader controller name.

      Navigate to the SERVICE_NAME, SERVICE_COMMAND, and LOG_PREFIX parameters to use the new loader controller name as follows:

      SERVICE_NAME=PLC_NAME
      SERVICE_COMMAND=com.platform.perf.dataloader.Main –f PLC_NAME.xml
      LOG_PREFIX=PLC_NAME

      For example, for the wsm_plc_job.conf file:

      SERVICE_NAME=plc_job
      SERVICE_COMMAND=com.platform.perf.dataloader.Main –f plc_job.xml
      LOG_PREFIX=plc_job

      Make similar edits to the wsm_plc_advanced_job.conf and wsm_plc_license.conf files.

    3. From the ANALYTICS_TOP/conf directory, create a new subdirectory for each new loader controller.

      Name each directory PLC_NAME.

      For example, create three new directories named plc_job, plc_advanced_job, and plc_license.

    4. Move the loader controller configuration files, corresponding to the data loader categories, from ANALYTICS_TOP/conf/plc to the subdirectory corresponding to the new loader controller that will handle the category data loaders.

      For example,

      1. Move plc_lsf.xml and plc_bjobs-sp012.xml from ANALYTICS_TOP/conf/plc to ANALYTICS_TOP/conf/plc_job.

      2. Move plc_lsf_advanced.xml from ANALYTICS_TOP/conf/plc to ANALYTICS_TOP/conf/plc_advanced_job.

      3. Move plc_license.xml from ANALYTICS_TOP/conf/plc to ANALYTICS_TOP/conf/plc_license.

    5. From the ANALYTICS_TOP/conf directory, copy the the loader controller file (plc.xml) to a new file, one for each new loader controller.

      For each new loader controller, name the new file PLC_NAME.xml.

      For example, copy plc.xml to plc_job.xml, plc_advanced_job.xml, and plc_license.xml.

    6. Edit each new loader controller file and change the Port parameter to a new port and change the PLCDir parameter to the new loader controller directory.

      For example,

      1. In plc_job.xml, change Port to from 4046 to 4047 and change PLCDir from plc to plc_job.

      2. In plc_advanced_job.xml, change Port to from 4046 to 4048 and change PLCDir from plc to plc_advanced_job.

      3. In plc_license.xml, change Port to from 4046 to 4049 and change PLCDir from plc to plc_license.

  2. Restart the new loader controllers.

    perfadmin stop all

    perfadmin start all

    Note:

    To stop or start an individual data loader, use perfadmin stop PLC_NAME and perfadmin start PLC_NAME

    For example, to stop the loader controller that handles license-related data loaders, run perfadmin stop plc_license.

Optimize specific data loaders

If you are encountering specific problems or are working under specific environments, you can optimize certain data loaders to enhance the working performance of these individual data loaders.

  1. Optimize the FLEXnet usage data loader (flexlicusageloader) to improve data loading time if it cannot finish loading the data within one sampling interval.

    If the FLEXnet usage data loader cannot finish the data loading of one sampling interval within the time of one sampling interval (typically five minutes), the data loader will be behind. You need to enable the multi-threads setting to catch up with the multi-servers and daemons workload scale.

    1. Log into the Platform Analytics node host that is running the FLEXnet usage data loader.
    2. Edit the ANALYTICS_TOP/conf/dataloader/flexlicusage.properties file.
    3. Edit the ThreadNumber parameter to increase the number of threads for collecting data concurrently.

      For example,

      ThreadNumber=10

  2. Optimize the FLEXnet Manager data loader (fnmloader) to achieve a larger capacity if you need to handle more than 1 million events per day.

    Using the default configuration means that you do not have to modify the data schema of the FLEXnet Manager reporting database, but the capacity of this data loader is limited to 1 million events per day.

    If you need to achieve a larger capacity, you need to modify the data schema of the FLEXnet Manager reporting database to increase performance. For more details, refer to the ANALYTICS_TOP/conf/dataloader/FNMLOADER.readme file.

  3. Optimize the LSF events data loader (lsfeventsloader) to prevent data loss if you are using LSF versions 7.0.3 or earlier.

    The LSF events data loader reads data from the lsb.stream file. If you are using LSF versions 7.0.3 or earlier, you should increase the size of the lsb.stream file to 2 GB or more to prevent data loss.

    1. Log into the Platform Analytics node host that is running the LSF events data loader.
    2. Edit the LSF_ENVDIR/lsbatch/cluster_name/configdir/lsb.params file.
    3. In the Parameters section, define a new parameter named MAX_EVENT_STREAM_SIZE.

      For example,

      Begin Parameters
      ...
      MAX_EVENT_STREAM_SIZE = 2048
      ...
      End Parameters
    4. Reconfigure the LSF cluster for your changes to take effect.

      lsadmin reconfig