GSoC project

[Douglas Chau <dc...@binghamton.edu>]

Hello,

For my GSoC project I want to tackle the issue of cluster recommendation for airavata users; I will be working with John, another GSoC student, on this component. The current PGA interface forces a user to pick a specific resource, but of course, as an end user, they may likely have no idea as to which resource to choose. The end goal is to provide a tip/hint in the user interface that points to the cluster that will be most “optimal” for the job; however, users will still be allowed to select a resource of their choice regardless of recommendation.

The cluster recommendation component will consist of two metrics (these categories can be further fleshed out): availability and runtime benchmarks. Availability will be determined by the cluster health, queue prediction times, total allocations (SU’s), filesystem availability, etc. Most of our clusters now are under XSEDE, so this information can be queried from their API. For other clusters outside XSEDE we will have to use other methods to send back information (i.e. manually ssh-ing into a cluster to grab information). But, for now I am going to focus on the grabbing information from the XSEDE clusters. Runtime benchmarking falls under John’s GSoC project, but the idea is beyond availability of a cluster, we also need to consider which cluster is historically known to run specific jobs faster; this is a more challenging problem. In the end, these two metrics will be evaluated to give a recommended cluster for a specific job.

All this cluster information will be stored in registry. Then we can add a new airavata API method to query and crunch this cluster information from the registry. After a recommendation is determined, the information will return to the PGA client and populate the page with the recommendation information.

This is a quick overview of what I want to do and there are probably more details that I may have left out, so if there are any questions please ask! Also, tips or advice on how to tackle this problem would be appreciated.

Doug