Making use of Univa Grid Engine involves little more than launching an instance and running a few commands once the instance is running. A video of this entire process documented below is available at 'Creating a HPC cluster with Univa Grid Engine'
From the AWS Marketplace after you have subscribed to the Univa Grid Engine offering there are three choices for launching Univa Grid Engine. Either '1-Click Launch' or 'Manual Launch' can be chosen. The instructions for either of them are similar but not exactly the same. For both ways of launching the high level steps are described in the next sections, they are:
Note: the IAM role must be added to the Univa Grid Engine Master Instance after the instance is launched and before any other commands or compute nodes are added to the cluster. Select the instance in the Amazon AWS console and right click on it to select 'Instance Settings->Attach/Replace IAM Role' and attach the role to the instance.
This new instance will host the Univa Grid Engine qmaster. Should the plan be to deploy Univa Grid Engine at a scale of 100s to 1,000s of nodes, to execute 100,000s of jobs simultaneously, a more capable instance type may be warranted. Please consult the documentation regarding Univa Grid Engine for additional details.
The instance type m4.large will be used by default for compute nodes. Consult the documentation for Navops Launch should multiple, different types of instances be required or the instance type is not supported in the selected region.
Please continue with the sections below to configure the security group and Univa Grid Engine for Workload-Managed Computing.
Before proceeding to login to the new Univa Grid Engine master instance the Security group attached to the instance needs to be configured to allow all traffic from hosts in the same security group. Each security group name is unique so this rule must be added manually. The last rule in the image below allows traffic for all members of the security group, so any node in the group can communicate with other nodes in the group.
Once the instance requested is running, connect to it via "ssh -i your_ssh_key.pem centos@
Upon login as user "centos", the following "message of the day" (i.e., the contents of "/etc/motd") will be presented:
As indicated in this message, become root via "sudo -i", and then issue the command "setup-aws". If use of an IAM role was made in the previous steps for '1-Click Launch' or 'Manual Launch' then output along the following lines will be presented:
If the option to make use of an IAM role was not selected, AWS access and secret access keys will be prompted for manually via the command line.
At this point, AWS has been configured for use with Univa Grid Engine. In fact, as noted above, a single compute node can be added manually via the command "add-nodes --count 1 --software-profile execd --hardware-profile aws". (Note that the integer value after "--count" can be adjusted to add additional compute nodes.) Details regarding the "add-nodes" command, as well as the referenced profiles, can be found in the documentation for Navops Launch.
Compute nodes will be automatically enabled to execute workload in a managed fashion via Univa Grid Engine. To ensure their availability from a workload-management perspective, the "qhost" command allows for a rapid check of suitably enabled compute nodes. Because various metrics have been successfully obtained from the 4 compute nodes identified below, these nodes are ready to execute workload via Univa Grid Engine. Details on use of Univa Grid Engine from the perspective of an end user is available via the Users' Guide; additionally man pages are available online via the running instance, and the interested user is directed to those pages relevant to job submission ("man qsub") plus monitoring and accounting ("man qstat" and "man qacct", respectively).
For those who seek to fine tune the configuration of their Univa Grid Engine cluster, detailed information can be found in the Administrator's Guide; additionally, online man pages are provided, and the interested administrator is directed to "man qconf" as a starting point. Univa offers training courses throughout the year at various locations internationally; please consult Training & Courses for additional details such as the course syllabus and current schedule.
Finally, manually deleting nodes is also noted in the Message Of The Day (MOTD) output.
Univa Unisight is the monitoring and reporting solution designed exclusively for use with Univa Grid Engine. This offering from AWS Marketplace has been designed to significantly automate the installation and integration of a fully functional deployment of Unisight.
To proceed with the default deployment for Unisight, running "start-unisight" as the root user is the only command that needs to be issued. (Note, however, that this assumes the default choice of the "m5.xlarge" instance is available in the AWS region and your cluster is moderately sized) "start-unisight" facilitates a number of steps that include:
Upon completion, the script exits with the URL of the Unisight WebUI - for example, "http://
Cut and paste the URL displayed on the console into a safe place, it will be needed later to login to Unisight.
At this point, a fully functional deployment of Univa Grid Engine enabled with monitoring and reporting via Univa Unisight, is ready for immediate use.
Logging into the Unisight console is simple. Paste the saved URL that was displayed on the ssh console screen into a web browser. If Unisight is up and the security group is configured properly to allow port '3003' then in a few moments you should see the Unisight login screen. The default credentials for Unisight are:
If you cannot login to Unisight or Unisight does not launch properly please contact email@example.com for assistance.
The "m5.xlarge" instance type should be sufficient for Unisight to manage moderately sized clusters (~100 compute nodes). Should there exist a need for a different type of instance, the above process (via "start-unisight") needs to be replaced with the following multistep process:
For additional details regarding Univa Unisight please consult the Administration Guide, and/or the related resources (e.g., our webinars) that are available online at http://www.univa.com/resources/.