Understanding how the lim daemon detects cores, threads, and processors

Traditionally, the value of ncpus has been equal to the number of physical CPUs. However, most CPUs consist of multiple cores and threads, so the traditional 1:1 mapping is no longer useful. A more useful approach is to set ncpus to equal one of the following:
  • The number of processors (this is the ncpus default setting)

  • Cores: the number of cores (per processor) * the number of processors

  • Threads: the number of threads (per core) * the number of cores (per processor) * the number of processors

An EGO cluster administrator globally defines how ncpus is computed using the EGO_DEFINE_NCPUS parameter (instead of EGO_ENABLE_DUALCORE) in ego.conf (shared directory).

The lim daemons detect and store the number of processors, cores, and threads for all supported architectures. The following diagram illustrates the flow of information between daemons, CPUs, and other components.

Although the ncpus computation is applied globally, it can be overridden on a per-host basis.

To correctly detect processors, cores, and threads, lim daemons/services assume that all physical processors on a single machine are of the same type.

In cases where CPU architectures and operating system combinations may not support accurate processor, core, thread detection, lim uses the defaults of 1 processor, 1 core per physical processor, and 1 thread per core. If lim detects that is it is running in a virtual environment (for example, VMware®), each detected processor is similarly reported (as a single-core, single-threaded, physical processor).

Lim detection uses processor- or OS-specific techniques (for example, Intel’s CPUID instruction, or Solaris’ kstat()/core_id). Note that if the operating system doesn't recognize a CPU or core (for example, if an older OS does not recognize a quad-core processor and instead detects it as dual-core), then the lim won't recognize it either; the lim only detects hardware that is recognized by the operating system.

Note:

RQL normalization never considers threads. Consider a hyper-thread enabled Pentium: Threads are not full-fledged CPUs, so considering them as CPUs would artificially lower the system load.

On a machine running AIX, detection of ncpus is different. Under AIX, the number of detected physical processors is always 1, whereas the number of detected cores is always the number of cores across all physical processors. Thread detection is the same as other operating systems (the number of threads per core).