Tutorial 6: Create an EGO Service
This tutorial describes how to create and run an EGO service.
Using this tutorial, you will ...
- Open a connection to Platform EGO
- Print out cluster information
- Check if there are any registered clients connected to Platform EGO
- Log on to Platform EGO
- Register the client with Platform EGO
- Print out allocation and container reply information from a previous connection
- Print out host group information
- Create and run an EGO service
- Use mutex objects to synchronize service query requests between the client (main thread) and the service thread
- Query for the IP address of the host where the service is running
- Update the Service Director with a new entry for service instance location
- Disable and remove an EGO service
Underlying principles
This sample uses two threads, main and service, and two sets of mutex objects and condition variables to synchronize the functions of the service thread. One of the tasks of the service thread is to create a service. The service thread can also be asked to query a service by a client; this only happens when the client requests it. This synchronization is achieved by using the client and service mutexes and condition variables. The service thread waits on the service condition variable, and when the client wants to query the service, it signals the service condition variable. Once the service thread obtains the information, it tells the client that the information is ready by signaling the client's condition variable.
Step 1: Preprocessor directives and declarations
The first step is to include a reference to the system and API header files, followed by the declaration of global structures that are implemented in the sample.
#include <stdlib.h> #include <stdio.h> #include <unistd.h> #include <string.h> #include <errno.h> #include <pthread.h> #include "vem.api.h" #include "esc.api.h" #include "samples.h" static int addResourceCB(vem_allocreply_t *areply); static int reclaimForceCB(vem_allocreclaim_t *areclaim); static int containerStateChgCB(vem_containerstatechg_t *cschange); static int hostStateChangeCB(vem_hoststatechange_t *hschange); static char *get_service_def_xml(); service_state_t *service_stateP; esc_service_info_reply_t *service_info_reply;Step 2: Implement the principal method
Lines 4-7: define and initialize a data structure that is used to request a connection with the EGO host cluster. The data structure contains a reference to a configuration file where the master host name and port numbers are stored.
Line 8: pass the data structure as an argument to the vem_open () method, which opens a connection to the master host. If the connection attempt is successful, a handle is returned; otherwise the method returns NULL. The handle acts as a communication channel to the master host and all subsequent communication occurs through this handle.
Lines 14-15: the vem_name_t structure (defined as clusterName) is initialized with NULL. This structure holds the cluster name, system name, and version. The vem_uname () method is passed the communication handle and, if successful, returns a valid vem_name_t structure ; otherwise the method returns NULL
Line 23: the cluster info is printed out to the screen.
Lines 26-42: locate all the registered clients and print out the client info (name, description, and location). Define the client info structure. Use vem_locate() to get all registered clients. Since NULL is provided as the client name, all registered clients will be located and the method returns the number of registered clients. Note that Platform EGO is equipped with a number of default clients (services) such as the Service Controller, so as a minimum, the info relevant to these clients is printed out and the associated memory is released.
1 int 2 sample6() 3 { 4 vem_openreq_t orequest; 5 vem_handle_t *vhandle = NULL; 6 orequest.file = "ego.conf"; 7 orequest.flags=0; 8 vhandle = vem_open(&orequest); 9 if (vhandle == NULL) { 10 // error opening 11 fprintf(stderr, "Error opening cluster: %s\n", vem_strerror(vemerrno)); 12 return -1; 13 } 14 vem_name_t *clusterName = NULL; 15 clusterName = vem_uname(vhandle); 16 if (clusterName == NULL) { 17 // error connecting 18 fprintf(stderr, "Error connecting to cluster: %s\n", 19 vem_strerror(vemerrno)); 20 return -2; 21 } 22 23 fprintf(stdout, " Connected... %s %s %4.2f\n", clusterName->clustername, 24 clusterName->sysname, clusterName->version); 25 vem_clientinfo_t *clients; 26 int rc = vem_locate(vhandle, NULL, &clients); 27 if (rc >=0) { 28 if (rc == 0) { 29 printf("No registered clients exist\n"); 30 } else { 31 int i=0; 32 for (i=0; i<rc; i++) { 33 printf("%s %s %s\n", clients[i].name, clients[i].description, 34 clients[i].location); 35 } 36 // free 37 vem_clear_clientinfo(clients); 38 } 39 } else { 40 // error connecting 41 fprintf(stderr, "Error geting clients: %s\n", vem_strerror(vemerrno)); 42 }Lines 43-47: authenticate the user to Platform EGO.
Lines 48-52: define and initialize a structure for callback methods. These callback methods are invoked by Platform EGO when resources are added or reclaimed, or when a change occurs to host status or a container. When Platform EGO wants to communicate about these events, it invokes these methods thereby calling back to the client.
Lines 54-68: define the vem_allocation_info_reply_t and vem_container_info_reply_t structures. If a client gets disconnected and then re-registers, its existing allocations and containers are returned to these structures. If the client had never registered before, the structures would be empty. Define and initialize a structure (rreq) that holds client info for registration purposes. (This includes assigning the client callback structure (cbf) to the callback member of the rreq structure.) Register with Platform EGO via the open connection using vem_register().
43 if (login(vhandle, username, password)<0) { 44 fprintf(stderr, "Error logon: %s\n", 45 vem_strerror(vemerrno)); 46 goto leave; 47 } 48 vem_clientcallback_t cbf; 49 cbf.addResource = addResourceCB; 50 cbf.reclaimForce = reclaimForceCB; 51 cbf.containerStateChg = containerStateChgCB; 52 cbf.hostStateChange = hostStateChangeCB; 53 54 vem_allocation_info_reply_t aireply; 55 vem_container_info_reply_t cireply; 56 vem_registerreq_t rreq; 57 58 rreq.name = "sample6_client"; 59 rreq.description = "Sample6 Client"; 60 rreq.flags = VEM_REGISTER_TTL; 61 rreq.ttl = 3; 62 rreq.cb = &cbf; 63 rc = vem_register(vhandle, &rreq, &aireply, &cireply); 64 if (rc < 0) { 65 fprintf(stderr, "Error registering: %s\n", 66 vem_strerror(vemerrno)); 67 goto leave; 68 }Lines 69-72: print out information related to the allocation requests and containers. Once the info is printed out, the memory for the allocations is freed.
Lines 77-82: the vem_gethostgroupinfo() method collects the information for the requested hostgroup. In this case, the requested hostgroup in the input argument is set to NULL, which means that information about all hostgroups is requested. If the method call is successful, hostgroup information is printed out to the screen.
69 print_vem_allocation_info_reply(&aireply); 70 print_vem_container_info_reply(&cireply); 71 // freeup any previous allocations 72 release_vem_allocation(vhandle, &aireply); 73 74 vem_hostgroupreq_t hgroupreq; 75 hgroupreq.grouplist = NULL; 76 vem_hostgroup_t *hgroup; 77 rc = vem_gethostgroupinfo(vhandle, &hgroupreq, &hgroup); 78 if (rc < 0) { 79 fprintf(stderr, "Error getting hostgroup: %s\n", vem_strerror(vemerrno)); 80 } else { 81 printf("%s %s %d %d\n", hgroup->groupName, hgroup->members, hgroup->free, hgroup->allocated); 82 }Lines 83-86: initialize the service_stateP and client_stateP structures. These structures contain the respective mutex objects and condition variables.
Lines 87-88: create a service definition in XML format. Each service is described by an XML file that contains information about the service such as the type of resources required to run the service instances and how to start and monitor them. Store the service definition in the service_stateP structure.
Lines 89-93: create and run the service_thread. The service thread is responsible for defining and creating the EGO service.
Lines 94-109: use vem_locate() to get all registered clients. Since NULL is provided as the client name, all registered clients will be located and the method returns the number of registered clients. If successful, print out the client info (name, description, and location) and free the associated memory.
83 service_stateP = calloc(1, sizeof(service_state_t)); 84 initialize_service(service_stateP); 85 client_state_t *client_stateP = calloc(1, sizeof(client_state_t)); 86 initialize_client(client_stateP); 87 char *xml = get_service_def_xml(); 88 service_stateP->xml = xml; 89 pthread_t service_thread; 90 if (pthread_create(&service_thread, NULL, service_thread_fn, 91 service_stateP)) { 92 perror("Error creating service thread: "); 93 } 94 rc = vem_locate(vhandle, NULL, &clients); 95 if (rc >=0) { 96 if (rc == 0) { 97 printf("No registered clients exist\n"); 98 } else { 99 int i=0; 100 for (i=0; i<rc; i++) { 101 printf("%s %s %s\n", clients[i].name, clients[i].description, 102 clients[i].location); 103 } 104 vem_clear_clientinfo(clients); 105 } 106 } else { 107 // error connecting 108 fprintf(stderr, "Error geting clients: %s\n", vem_strerror(vemerrno)); 109 }Lines 114-120: lock the service_mutex object and wait until the service is running. Set the service condition variable to unblock the service thread, which causes the service thread to query the service.
Lines 124-127: lock the client_mutex object and block the main thread with the client condition variable. When the service info is available, the service thread signals the main thread to resume execution using the client condition variable. The main thread prints out the service info reply.
Line 130: set the shutdown flag, which causes the service thread to disable and remove the service.
Line 134: block execution of the main thread until the service thread has terminated.
110 sleep(60); 111 // get service info 112 service_info_reply = calloc(1, sizeof(esc_service_info_reply_t)); 113 int succeed=0; 114 while(!succeed) { 115 pthread_mutex_lock(&service_stateP->service_mutex); 116 if (service_stateP->ready && service_stateP->client == NULL) { 117 service_stateP->client = client_stateP; 118 pthread_cond_signal(&service_stateP->service_cond); 119 succeed = 1; 120 } 121 pthread_mutex_unlock(&service_stateP->service_mutex); 122 } 123 fprintf(stderr, "Sent Request to Service:\n"); 124 pthread_mutex_lock(&client_stateP->client_mutex); 125 pthread_cond_wait(&client_stateP->client_cond, 126 &client_stateP->client_mutex); 127 print_service_info_reply(service_info_reply); 128 pthread_mutex_unlock(&client_stateP->client_mutex); 129 sleep(60); 130 shutdown = 1; 131 pthread_mutex_lock(&service_stateP->service_mutex); 132 pthread_cond_signal(&service_stateP->service_cond); 133 pthread_mutex_unlock(&service_stateP->service_mutex); 134 pthread_join(service_thread, NULL);Step 3: Create the service
In this sample, the service definition and creation is handled by the service thread. The service simply executes the sleep command for 60 milliseconds and then ends.
Lines 137-147: set the EGO_CONFDIR environment variable to the current working directory so that the Service Controller, which is just another EGO client, can find the EGO configuration. The configuration is stored in the ego.conf file.
Lines 148-151: define and initialize a security structure. The username and password variables have been initialized with "egoadmin" from sample # 2.
Lines 153-154: print out the service definition.
Lines 155-163: pass the security structure and the service definition to the esc_createservice() method. This API call creates a new service object. If startType is automatic in the service definition, the service is enabled automatically. In this case, the startType is set to manual so the service must be enabled by the esc_enableservice() method. This API call starts the service. Once the service is started, the Service Controller allocates resources and starts service instances. A service can be started by esc_enableservice() or by starting a service that depends on it.
135 void *service_thread_fn(service_state_t *service_stateP) 136 { 137 int size=4096; 138 char *buf = calloc(size, sizeof(char)); 139 sprintf(buf, "EGO_CONFDIR="); 140 char *cwd = getcwd((buf+12), size-12); 141 int errn = errno; 142 if(cwd != NULL) { 143 putenv(buf); 144 } else { 145 fprintf(stderr, "Error getting CWD: %s\n", 146 strerror(errn)); 147 } 148 esc_security_def_t security; 149 security.username = username; 150 security.password = password; 151 security.credential = NULL; 152 char *sname = "sample6_service"; 153 char *xml = service_stateP->xml; 154 fprintf(stderr, "%s\n", xml); 155 if(esc_createservice(xml, &security)) { 156 fprintf(stderr, "Error creating service: %s\n", 157 esc_strerror(escerrno)); 158 //goto bailout; 159 } 160 if(esc_enableservice(sname, &security)) { 161 fprintf(stderr, "Error enabling service: %s\n", 162 esc_strerror(escerrno)); 163 }Step 4: Query the service
Lines 165-175: lock the service_mutex object and wait until the service condition variable is set by the main thread, which is acting like a client requesting service information. (In order to wait on a condition variable, the associated mutex has to be locked first. Inside the wait function, the mutex will be automatically unlocked so anyone can then acquire it. On returning from wait, the mutex is automatically acquired by the service thread.) When the service thread resumes execution, pass the name of the service to the esc_queryservice()API method, which gets the service instance information from the EGO Service Controller.
Lines 176-177: lock the client_mutex object and print out the service information.
Line 178: since the service info has been retrieved, notify the client by setting the client condition variable. The main thread resumes execution.
Lines 179-181: unlock the client and service mutex objects and initialize the client structure.
164 while(!shutdown) { 165 pthread_mutex_lock(&service_stateP->service_mutex); 166 service_stateP->ready=1; 167 pthread_cond_wait(&service_stateP->service_cond, 168 &service_stateP->service_mutex); 169 170 fprintf(stderr, "Got Request:\n"); 171 if (service_stateP->client == NULL) continue; 172 if(esc_queryservice(sname, service_info_reply)) { 173 fprintf(stderr, "Error geting service info: %s\n", 174 esc_strerror(escerrno)); 175 } 176 pthread_mutex_lock(&service_stateP->client->client_mutex); 177 print_service_info_reply(service_info_reply); 178 pthread_cond_signal(&service_stateP->client->client_cond); 179 pthread_mutex_unlock(&service_stateP->client->client_mutex); 180 service_stateP->client = NULL; 181 pthread_mutex_unlock(&service_stateP->service_mutex);Step 5: Disable and remove the service
Lines 182-190: disable the service by calling the esc_disableservice() API method. The Service Controller stops all service instances and de-allocates resources. Remove the service by calling the esc_removeservice() API method. The Service Controller destroys the service object and removes the service definition from the configuration.
182 if(esc_disableservice(sname, &security)) { 183 fprintf(stderr, "Error disabling service: %s\n", 184 esc_strerror(escerrno)); 185 } 186 187 if(esc_removeservice(sname, &security)) { 188 fprintf(stderr, "Error removing service: %s\n", 189 esc_strerror(escerrno)); 190 }Run the client application
- Select Run > Run.
The Run dialog appears.
- In the Configurations list, either select an EGO C Client Application or click New for a new configuration.
For a new configuration, enter the configuration name.
- Enter the project name and C/C++ Application name.
- Click Apply and then Run.
[ Top ]
[ Platform Documentation ]
Date Modified: July 12, 2006
Platform Computing: www.platform.com
Platform Support: support@platform.com
Platform Information Development: doc@platform.com
Copyright © 1994-2006 Platform Computing Corporation. All rights reserved.