Multiple repository servers

In a cluster that spans different geographic regions, the network connection between hosts in the different regions is typically a WAN. When a service in one region needs to download a large package from the repository server (RS), which happens to be in the other region, the connection can become a bottleneck and adversely affect other network traffic. To mitigate this issue, you can deploy a local RS. With this arrangement, a master RS retains global control of package deployment while each local RS serves an individual region. The local RS registers with the master RS when the local RS starts up. Each region can only have one local RS.

Cluster regions

The master RS maintains a table of which local RS should serve the package download request from a compute host. This table is defined in $EGO_CONFDIR/../../eservice/rs/conf/region.xml. There are two ways to configure the definition of a region:

  1. You can define the region by IP range. The master RS checks the IP address of the compute host and redirects the request to the local RS.

  2. You can also define the region by resource groups. The master RS checks the resource group the compute host belongs to and redirects the request appropriately.

The use of either IP ranges or resource groups must be consistent for all regions. i.e., you cannot mix the definition.

If the RS master is unable to find the region associated with a download request in the table, the master RS itself will service the request. Therefore, you do not need to configure the region where the master RS is running since the master RS will serve that region by default.

Region configuration

Here are some key details to be aware of when defining a region using IP ranges:

  • Only IPv4 is supported

  • Each field of the IP address can be:

    • a number (as 192)

    • a range (as 1-10)

    • a wildcard character ( *)

  • If one range is a subset of another range, or one range is overlapped by another range, the master RS selects the first region defined in region.xml.

Sample configuration of region.xml using IP range:

<Region name="region1">
<IPrange>172.168.1.1</IPrange>
<IPrange>192.168.2.1-100</IPrange>
<IPrange>192.168.4-10.*</IPrange>
</Region>
<Region name="region2">
<IPrange>10.10.*.* </IPrange>
</Region>

Sample configuration of region.xml using resource groups:

<Region name="region3">
<resourceGroup>rg1</resourceGroup>
<resourceGroup>rg2</resourceGroup>
</Region>
<Region name="region4">
<resourceGroup>rg3</resourceGroup>
</Region>

Local RS and master RS interaction

The local RS registers with the master RS when the local RS starts up. During registration, the local RS passes its region name to the master RS. If the master RS is down or the connection between the local RS and master RS is broken, the local RS will log an error message and try to re-establish a connection.

The master RS maintains a region/resource group table and a region/IP table. When a client needs to upload a package, it connects to the master RS. After the client uploads the package to the master RS, the master RS pushes the package to each local RS. If the local RS is down when the client uploads the package, the package in the local RS will not be in sync with the latest package. Therefore, the local RS always checks with the master RS for the latest version each time the package needs to be downloaded to a compute host.

When a compute host needs to download a package, it sends a redirection request to the master RS. (A redirection request basically asks the master RS for the URL of the local RS.) If the regions are defined using resource groups, the redirection request includes the compute host’s resource group name. The master RS searches the region.xml file for the compute host’s resource group or IP address, whichever is applicable, and redirects the request to the proper local RS.

When the client removes a package from the master RS, the master RS propagates the request to every local RS.

Local RS host requirements

The local RS host must meet the following requirements:

  • It is accessible by all compute hosts in the local region.

  • It has enough disk space (either local or shared) to store all packages.

  • The OS currently supports RS.