Configuration overview of guaranteed resource pools

Guaranteed resource pools are defined in lsb.resources and used by consumers defined within SLAs in lsb.serviceclasses.

Note:

Hosts can only be included in one guaranteed resource pool.

  1. Choose the type of resource pool to configure. The three possible types are hosts, slots, or resources (such as licenses).
  2. Configure a GuaranteedResourcePool section in lsb.resources. Begin with the line Begin GuaranteedResourcePool and end with the line End GuaranteedResourcePool. Specify the following:
    1. NAME; the name of the guaranteed resource pool.
    2. TYPE; the guarantee type (slots, hosts, or resources).
    3. DISTRIBUTION; share assignments for all SLAs using the resource pool. Can be percent or absolute numbers.
    4. Optional parameters for the GuaranteedResourcePool section are HOSTS, RES_SELECT, LOAN_POLICIES, DESCRIPTION, and SLOTS_PER_HOST.

    You can configure as many resource pools as you need. One resource pool can be used by several SLAs, and one SLA can access multiple resource pools.

    For example:

    Begin GuaranteedResourcePool
    NAME = GRP1A
    TYPE = hosts
    HOSTS = host1 host2 hgB
    DISTRIBUTION = [sla1,3] [sla2,25%] [sla3,3]
    DESCRIPTION = A resource pool used by sla1, sla2, and sla3.
    End GuaranteedResourcePool
  3. Configure a ServiceClass section in lsb.serviceclasses for each SLA. Begin with the line Begin ServiceClass and end with the line End ServiceClass. For each SLA you must specify:
    1. NAME; the name of the SLA.
    2. GOALS = [GUARANTEE]
    3. Optional parameters for the ServiceClass section are ACCESS_CONTROL, AUTO_ATTACH, and DESCRIPTION.

    You can configure as many service class sections as you need.

    Important:

    The name you use for your SLA cannot be the same as an existing host partition or user group name.

    For example:

    Begin ServiceClass
    NAME = sla1
    GOALS = [GUARANTEE]
    ACCESS_CONTROL=USERS[ user1 user2 user3 ]
    AUTO_ATTACH=Y
    DESCRIPTION = A guarantee SLA with access restricted to specific users. Jobs from these users will be attached to the SLA automatically and run on guaranteed resources if possible.
    End ServiceClass

Configure SLOTS_PER_HOST in a slot pool

Guaranteed resource pools with TYPE=slots can be used to spread jobs over hosts within a cluster by limiting the number of slots available on each host. This is useful for jobs requiring large amounts of memory.

SLOTS_PER_HOST limits the number of slots available on each host in the pool for all SLA jobs with a guarantee in the pool, even when jobs are running outside of the guarantee.

Configure a guaranteed resource pool in lsb.resources with TYPE = slots and SLOTS_PER_HOST defined.
Begin GuaranteedResourcePool
...
TYPE = slots
SLOTS_PER_HOST = 1
...
End GuaranteedResourcePool

SLOTS_PER_HOST limits the number of slots each host contributes to a slot-type resource pool.

Configure loans to specific queues

Loans allow queues to access unused resources from guaranteed resource pools.

  1. Configure a guaranteed resource pool in lsb.resources with the required NAME, TYPE, and DISTRIBUTION parameters.
  2. Add a loan policy to the guaranteed resource pool.

    LOAN_POLICIES= QUEUES[queue_name] allows you to specify which queues can access loaned resources. Use the keyword all to loan to jobs from any queue.

    For example, to allow loans to jobs from the queue my_queue:

    Begin GuaranteedResourcePool
    ...
    LOAN_POLICIES = QUEUES[my_queue]
    ...
    End GuaranteedResourcePool

Configure loans to short jobs

Loans can allow queues to access unused resources from guaranteed resource pools.

  1. Configure a guaranteed resource pool in lsb.resources with the required NAME, TYPE, and DISTRIBUTION parameters and a loan policy such as QUEUES[queue1].
  2. Add the policy DURATION[minutes] to the guaranteed resource pool configuration, where minutes is an integer.

    DURATION allows you to set a maximum job runtime limit (or estimated runtime, whichever is shorter) for jobs to borrow resources. Omit DURATION completely to allow jobs with any runtime to borrow from the gurantee.

    For example, to allow loans to jobs from any queue with a runtime of 10 minutes or less:

    Begin GuaranteedResourcePool
    ...
    LOAN_POLICIES = QUEUES[all] DURATION[10]
    ...
    End GuaranteedResourcePool

Configure loans to stop when jobs are waiting for guaranteed resources

Loans can be restricted so that jobs have access to the loaned resources only when consumers with unused guaranteed resources do not have pending loads.

Restricting loans is useful when running parallel jobs that require several slots, or jobs that require large blocks of memory. With restricted loans enabled, small borrowing jobs will not delay jobs waiting for resources such as slots or memory to accumulate.

  1. Configure a guaranteed resource pool in lsb.resources with a loan policy such as QUEUES[queue1].
  2. Add the policy CLOSE_ON_DEMAND to the guaranteed resource pool configuration. For example:
    Begin GuaranteedResourcePool
    ...
    LOAN_POLICIES = QUEUES[queue1] CLOSE_ON_DEMAND
    ...
    End GuaranteedResourcePool

Configure a queue with access to all guaranteed resources

Queues with very high priority (such as administrator test queues) can be configured with access to all guaranteed resources, regardless of SLA demand.

Configure a queue in lsb.queues with SLA_GUARANTEES_IGNORE = Y.
Note:

Using SLA_GUARANTEES_IGNORE=Y defeats the purpose of guaranteeing resources. This should be used sparingly for low traffic queues only.

Configure preemptable loans

Guarantee SLA jobs can use preemption to reclaim loaned resources when preemption is configured.

Configuring preemption with guarantees returns loaned resources quickly even when loans are made to jobs of any length (without specifying maximum runtime using DURATION).

  1. Configure a guaranteed resource pool in lsb.resources with a loan policy. For example:
    Begin GuaranteedResourcePool
    NAME = my_pool
    TYPE = hosts
    HOSTS = host1 host2 hgB
    DISTRIBUTION = [sla1,50%]
    LOAN_POLICIES = QUEUES[loanQ]
    DESCRIPTION = A resource pool used by sla1.
    End GuaranteedResourcePool
  2. Configure a ServiceClass section in lsb.serviceclasses for each SLA. For example:
    Begin ServiceClass
    NAME = SLA1
    GOALS = [GUARANTEE]
    AUTO_ATTACH = Y
    ACCESS_CONTROL = QUEUES[SLA1_queue]
    DESCRIPTION = Service class SLA1 uses GRP my_pool.
    End ServiceClass

    In this example, unused resources from the guaranteed resource pool are loaned to jobs of any length submitted to loanQ. All jobs submitted to SLA1_queue are automatically attached to SLA1, and are guaranteed 50% of the resources in my_pool.

  3. Configure the queue using loaned resources in lsb.queues as preemptable and low priority. For example:
    Begin Queue
    QUEUE_NAME=loanQ
    PREEMPTION=PREEMPTABLE [SLA1_queue]
    PRIORITY=1
    End Queue

    Jobs submitted to the queue loanQ run on unused hosts in the guarantee for SLA1.

  4. Configure the SLA job queue in lsb.queues as preemptive and higher priority:
    Begin Queue
    QUEUE_NAME=SLA1_queue
    PREEMPTION=PREEMPTIVE [loanQ]
    Priority=10
    End Queue

    Jobs submitted to SLA1 through the queue SLA1_queue can preempt jobs from loanQ in order to reclaim the guaranteed resources as quickly as possible.