Home »Resources

Usage Instructions:

Making use of Univa Grid Engine involves little more than launching an instance and running a few commands once the instance is running. A video of this entire process documented below is available at 'Creating a HPC cluster with Univa Grid Engine'

Launching an Instance of Univa Grid Engine

From the AWS Marketplace after you have subscribed to the Univa Grid Engine offering there are three choices for launching Univa Grid Engine. Either '1-Click Launch' or 'Manual Launch' can be chosen. The instructions for either of them are similar but not exactly the same. For both ways of launching the high level steps are described in the next sections, they are:

Steps for '1-click Launch'

  • 1. Select the Univa Grid Engine version
  • 2. Choose the region
  • 3. Choose instance type. Avoid small instances because Univa Grid Engine will run very slow. The m4.xlarge instance type is the default for Univa Grid Engine.
  • 4. Select a Amazon Virtual Private Cloud (VPC). Many Amazon EC2 instance types are only supported with a VPC and running Univa Grid Engine in a VPC ensures there are no additional network issues with the cluster once you start adding nodes. It is highly recommended you use a VPC.
  • 5. Choose a security group. Normally a security group is created for the VPC. The security group is very similar to a firewall and using a Security Group attached to the VPC provides fine grained network access control for your VPC. After the Univa Grid Engine instance is launched the Security group will need to be modified to ensure that the correct ports are open for Univa Grid Engine to communicate and access to Unisight from the Internet is configured. For more information on Amazon AWS Security Groups please go to aws security groups
  • 6. Choose the Keypair that will be used with this instance.
  • 7. At this point if you are using the '1-click Launch' you can 'Launch' the Univa Grid Engine Instance, however there is one additional step that cannot be chosen from the '1-click Launch' that will simplify the usage of Univa Grid Engine on Amazon Marketplace. An IAM Role should be created for Univa Grid Engine that allows Full Access to Amazon EC2 and Full Access to Amazon S3. The IAM (Identity and Access Management) Role allows Univa Grid Engine to perform actions on your behalf in Amazon EC2 without the need of your Access Key or Secret Access Key. For further information on Amazon EC2 IAM please go to IAM on Amazon
  • 8. After the Univa Grid Engine instance is launched please go to the sections 'Configuring the Security Group' then 'Configuring Univa Grid Engine for Workload-Managed Computing'

Note: the IAM role must be added to the Univa Grid Engine Master Instance after the instance is launched and before any other commands or compute nodes are added to the cluster. Select the instance in the Amazon AWS console and right click on it to select 'Instance Settings->Attach/Replace IAM Role' and attach the role to the instance.

Steps for 'Manual Launch'

  • 1. Select the Univa Grid Engine Version
  • 2. Choose the AMI and Region and 'Launch with EC2 Console'
  • 3. From the EC2 Console choose the instance type. Avoid small instances because Univa Grid Engine will run very slow. The m4.xlarge instance type is the default for Univa Grid Engine.
  • 4. Select the Network. Univa highly recommends you create or choose a Amazon VPC. Many Amazon EC2 instance types are only supported with a VPC and running Univa Grid Engine in a VPC ensures there are no additional network issues with the cluster once you start adding nodes.
  • 5. Choose the Subnet within the Amazon VPC. There may only be one subnet in that case it is an easy choice.
  • 6. Do not choose a Placement group. If you need a cluster in a placement group contact Univa support at support@univa.com
  • 7. Select the IAM Role for this instance. A IAM Role should be created for Univa Grid Engine that allows Full Access to Amazon EC2 and Full Access to Amazon S3. The IAM (Identity and Access Management) Role allows Univa Grid Engine to perform actions on your behalf in Amazon EC2 without the need of your Access Key or Secret Access Key. For further information on Amazon EC2 IAM please go to IAM on Amazon EC2
  • 8. Move to Add Storage step and update the storage size. The default is 10 GiB. Increase the storage size if necessary. For larger clusters more storage is needed and 10 GiB is not sufficient.
  • 9. Next, optionally add tags to the instance.
  • 10. Choose the Security Group. If a VPC was selected in step 4 above then typically the VPC has a security group associated with it. The security group is very similar to a firewall and using a Security Group attached to the VPC provides fine grained network access control for your VPC. After the Univa Grid Engine instance is launched the Security group will need to be modified to ensure that the correct ports are open for Univa Grid Engine to communicate and access to Unisight from the Internet is configured. For more information on Amazon AWS Security Groups please go to Amazon EC2 Security Groups
  • 11.Select Launch and choose the Keypair to be used with this instance.

This new instance will host the Univa Grid Engine qmaster. Should the plan be to deploy Univa Grid Engine at a scale of 100s to 1,000s of nodes, to execute 100,000s of jobs simultaneously, a more capable instance type may be warranted. Please consult the documentation regarding Univa Grid Engine for additional details.

The instance type m4.large will be used by default for compute nodes. Consult the documentation for Navops Launch should multiple, different types of instances be required or the instance type is not supported in the selected region.

Please continue with the sections below to configure the security group and Univa Grid Engine for Workload-Managed Computing.

Configuring the Security Group

Before proceeding to login to the new Univa Grid Engine master instance the Security group attached to the instance needs to be configured to allow all traffic from hosts in the same security group. Each security group name is unique so this rule must be added manually. The last rule in the image below allows traffic for all members of the security group, so any node in the group can communicate with other nodes in the group.

security

Configuring the Instance for Workload-Managed Computing

Once the instance requested is running, connect to it via "ssh -i your_ssh_key.pem centos@.compute-<##>.amazonaws.com". (This assumes an instance with a publicly accessible IP name has been requested and, of course, that access to the local ssh key is available.)

Upon login as user "centos", the following "message of the day" (i.e., the contents of "/etc/motd") will be presented:

output1

As indicated in this message, become root via "sudo -i", and then issue the command "setup-aws". If use of an IAM role was made in the previous steps for '1-Click Launch' or 'Manual Launch' then output along the following lines will be presented:

output2

If the option to make use of an IAM role was not selected, AWS access and secret access keys will be prompted for manually via the command line.

At this point, AWS has been configured for use with Univa Grid Engine. In fact, as noted above, a single compute node can be added manually via the command "add-nodes --count 1 --software-profile execd --hardware-profile aws". (Note that the integer value after "--count" can be adjusted to add additional compute nodes.) Details regarding the "add-nodes" command, as well as the referenced profiles, can be found in the documentation for Navops Launch.

Compute nodes will be automatically enabled to execute workload in a managed fashion via Univa Grid Engine. To ensure their availability from a workload-management perspective, the "qhost" command allows for a rapid check of suitably enabled compute nodes. Because various metrics have been successfully obtained from the 4 compute nodes identified below, these nodes are ready to execute workload via Univa Grid Engine. Details on use of Univa Grid Engine from the perspective of an end user is available via the Users' Guide; additionally man pages are available online via the running instance, and the interested user is directed to those pages relevant to job submission ("man qsub") plus monitoring and accounting ("man qstat" and "man qacct", respectively).

output3

For those who seek to fine tune the configuration of their Univa Grid Engine cluster, detailed information can be found in the Administrator's Guide; additionally, online man pages are provided, and the interested administrator is directed to "man qconf" as a starting point. Univa offers training courses throughout the year at various locations internationally; please consult Training & Courses for additional details such as the course syllabus and current schedule.

Finally, manually deleting nodes is also noted in the Message Of The Day (MOTD) output.

Configuring the Instance for Monitoring and Reporting (OPTIONAL)

Univa Unisight is the monitoring and reporting solution designed exclusively for use with Univa Grid Engine. This offering from AWS Marketplace has been designed to significantly automate the installation and integration of a fully functional deployment of Unisight.

To proceed with the default deployment for Unisight, running "start-unisight" as the root user is the only command that needs to be issued. (Note, however, that this assumes the default choice of the "m5.xlarge" instance is available in the AWS region and your cluster is moderately sized) "start-unisight" facilitates a number of steps that include:

  • • Obtaining the required software tarballs from an AWS S3 bucket for use local to the instance
  • • Installation and integration of Unisight with Univa Grid Engine

Upon completion, the script exits with the URL of the Unisight WebUI - for example, "http://.compute-1.amazonaws.com:3003".

Cut and paste the URL displayed on the console into a safe place, it will be needed later to login to Unisight.

At this point, a fully functional deployment of Univa Grid Engine enabled with monitoring and reporting via Univa Unisight, is ready for immediate use.

Logging into the Unisight console is simple. Paste the saved URL that was displayed on the ssh console screen into a web browser. If Unisight is up and the security group is configured properly to allow port '3003' then in a few moments you should see the Unisight login screen. The default credentials for Unisight are:

  • • demo@example.com
  • • demo123

If you cannot login to Unisight or Unisight does not launch properly please contact support@univa.com for assistance.

Choosing a larger instance type for Unisight

The "m5.xlarge" instance type should be sufficient for Unisight to manage moderately sized clusters (~100 compute nodes). Should there exist a need for a different type of instance, the above process (via "start-unisight") needs to be replaced with the following multistep process:

  • 1. Enable Univa Unisight as a component via the command "enable-component --software-profile unisight unisight". Note that invoking this command will have the effect of downloading software tarballs from AWS S3, and subsequently making them available for use locally. This command also has the effect of enabling reporting for workload managed via Univa Grid Engine.
  • 2. To employ a different instance type, issue the command "adapter-mgmt update -r aws -p unisight -s instancetype=", where "" is a placeholder for the type preferred over the "m5.xlarge" default.
  • 3. Set the password for the Univa Grid Engine web service to the default value (of "password") using the command "echo sge:password | chpasswd". (Note: This command must be executed by the root user on the UGE qmaster node.)
  • 4.Finally to deploy a standalone node for the monitoring and reporting service, and subsequently install and integrate Unisight with the existing deployment of Univa Grid Engine, it is necessary to issue the command "add-nodes --count 1 --software-profile unisight --hardware-profile aws -A unisight". Once issued, the instance will appear via the EC2 Dashboard with "Unisight" as its name, and the user will be directed to the URL of the Unisight WebUI.

For additional details regarding Univa Unisight please consult the Administration Guide, and/or the related resources (e.g., our webinars) that are available online at http://www.univa.com/resources/.