You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airavata.apache.org by sm...@apache.org on 2016/10/28 14:51:05 UTC
[1/3] airavata-sandbox git commit: Added Readme file which explains
the design changes which were made
Repository: airavata-sandbox
Updated Branches:
refs/heads/master 39613cd5a -> cc4fcf050
Added Readme file which explains the design changes which were made
Project: http://git-wip-us.apache.org/repos/asf/airavata-sandbox/repo
Commit: http://git-wip-us.apache.org/repos/asf/airavata-sandbox/commit/cc4fcf05
Tree: http://git-wip-us.apache.org/repos/asf/airavata-sandbox/tree/cc4fcf05
Diff: http://git-wip-us.apache.org/repos/asf/airavata-sandbox/diff/cc4fcf05
Branch: refs/heads/master
Commit: cc4fcf050d46707cd68aafcc521aac9d75146196
Parents: 1932e86
Author: Abhishek Jain <aj...@binghamton.edu>
Authored: Thu Oct 27 18:00:30 2016 -0400
Committer: Suresh Marru <sm...@apache.org>
Committed: Fri Oct 28 10:50:48 2016 -0400
----------------------------------------------------------------------
gsoc2016/Design Changes in Airavata/README.md | 36 ++++++++++++++++++++++
1 file changed, 36 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/airavata-sandbox/blob/cc4fcf05/gsoc2016/Design Changes in Airavata/README.md
----------------------------------------------------------------------
diff --git a/gsoc2016/Design Changes in Airavata/README.md b/gsoc2016/Design Changes in Airavata/README.md
new file mode 100644
index 0000000..d6e5857
--- /dev/null
+++ b/gsoc2016/Design Changes in Airavata/README.md
@@ -0,0 +1,36 @@
+# Re-Factoring Aurora Marathon Integration Module and gfac-orchestrator core files to incorporate design changes.
+
+##AuroraMarathonIntegration
+
+* Apache style packaging : Organized the AuroraMarathonIntegration code to follow the Apache guidelines.
+* Maven integration : Integrated Maven with the module as Airavata is build using Maven.
+* Dependency injection pattern
+ * Dependency Injection makes it simple for to manage dependencies between objects. In turn, that makes it easier to break coherent functionality off into it's own contract (interface). As a result, the code has been far more modularized.
+ * Another result of this is that I can much more easily see my way through to a design that supports the Open-Closed Principle.[software entities (classes, modules, functions, etc.) should be open for extension, but closed for modification]. This is one of the most confidence inspiring techniques (second only to automated testing).
+ * Major advantage : Two important ones are the ability to control functionality from a central place (the Main() function) instead of spreading it throughout your program, and the ability to more easily test each class in isolation (because you can pass mocks or other faked objects into its constructor instead of a real value).
+* Extract out the redundant code and create utilities.
+* Kept updating code and made the above design changes to be in sync with his changes.
+
+
+##Orchestrator
+* Path to file : orchestrator/orchestrator-core/src/main/java/org/apache/airavata/orchestrator/core/utils/OrchestratorConstants.java
+ * Change the class containing constants to enums.
+ * Type safety
+ * You get free compile time checking of valid values.
+ * Catches typos
+
+* Path to file : orchestrator/orchestrator-core/src/main/java/org/apache/airavata/orchestrator/core/utils/OrchestratorUtils.java
+ * Use enums instead of directly accessing the static final constants. [Changed the usage]
+
+* Path to file : /orchestrator/orchestrator-core/src/main/java/org/apache/airavata/orchestrator/cpi/impl/SimpleOrchestratorImpl.java
+ * Captured in a multiple catch block, instead of 3 different catch clauses.
+ * In some functions, the function argument and the instance variable have the same name. This may create ambiguity. Hence the function arguments names were changed to remove the ambiguity. The following are the functions in SimpleOrchestratorImpl.java which had this issue :
+
+* Path to file : airavata/modules/orchestrator/orchestrator-core/src/main/java/org/apache/airavata/orchestrator/core/OrchestratorConfiguration.jav
+
+
+##GFAC
+* Path to files :
+ * gfac/gfac-core/src/main/java/org/apache/airavata/gfac/core/authentication/GSIAuthenticationInfo.java
+ * gfac/gfac-core/src/main/java/org/apache/airavata/gfac/core/authentication/SSHKeyAuthentication.java
+ * gfac/gfac-core/src/main/java/org/apache/airavata/gfac/core/authentication/SSHPasswordAuthentication.java
[3/3] airavata-sandbox git commit: Added files with the design changes
Posted by sm...@apache.org.
Added files with the design changes
Project: http://git-wip-us.apache.org/repos/asf/airavata-sandbox/repo
Commit: http://git-wip-us.apache.org/repos/asf/airavata-sandbox/commit/1932e865
Tree: http://git-wip-us.apache.org/repos/asf/airavata-sandbox/tree/1932e865
Diff: http://git-wip-us.apache.org/repos/asf/airavata-sandbox/diff/1932e865
Branch: refs/heads/master
Commit: 1932e865684d2f57b20eb5ca15fe38ba7185a56f
Parents: 39613cd
Author: Abhishek Jain <aj...@binghamton.edu>
Authored: Thu Oct 27 17:46:01 2016 -0400
Committer: Suresh Marru <sm...@apache.org>
Committed: Fri Oct 28 10:50:48 2016 -0400
----------------------------------------------------------------------
.../AuroraMarathonIntegration/README.md | 67 ++
.../AuroraMarathonIntegration/pom.xml | 44 ++
.../auroraClient/AuroraJobSchedulerI.java | 45 ++
.../auroraClient/AuroraJobSchedulerImpl.java | 239 +++++++
.../marathonClient/MarathonJobSchedulerI.java | 51 ++
.../MarathonJobSchedulerImpl.java | 318 ++++++++++
.../bigDataInjections/AuroraInjectorImpl.java | 101 +++
.../bigDataInjections/BigDataInjectorI.java | 32 +
.../bigDataInjections/MarathonInjectorImpl.java | 72 +++
.../drivers/auroraDriver/AuroraAdminDriver.java | 98 +++
.../auroraExceptions/AuroraException.java | 36 ++
.../marathonExceptions/MarathonException.java | 36 ++
.../cloud/util/auroraUtilities/AuroraUtilI.java | 31 +
.../util/auroraUtilities/AuroraUtilImpl.java | 63 ++
.../util/marathonUtilities/MarathonUtilI.java | 33 +
.../marathonUtilities/MarathonUtilImpl.java | 64 ++
.../gfac/gfac-core/pom.xml | 190 ++++++
.../authentication/GSIAuthenticationInfo.java | 43 ++
.../authentication/SSHKeyAuthentication.java | 90 +++
.../SSHPasswordAuthentication.java | 42 ++
.../Design Changes in Airavata/gfac/pom.xml | 47 ++
.../orchestrator/NULL | 1 +
.../orchestrator/orchestrator-core/pom.xml | 103 +++
.../core/OrchestratorConfiguration.java | 122 ++++
.../core/utils/OrchestratorConstants.java | 77 +++
.../core/utils/OrchestratorUtils.java | 327 ++++++++++
.../cpi/impl/SimpleOrchestratorImpl.java | 625 +++++++++++++++++++
.../orchestrator/pom.xml | 45 ++
28 files changed, 3042 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/airavata-sandbox/blob/1932e865/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/README.md
----------------------------------------------------------------------
diff --git a/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/README.md b/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/README.md
new file mode 100644
index 0000000..e940146
--- /dev/null
+++ b/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/README.md
@@ -0,0 +1,67 @@
+#AURORA MARATHON INTEGRATION
+
+#Requirements:
+1. Apache Aurora
+2. Apache Mesos
+3. Marathon
+
+#Installation:
+##Please follow the link to set up Apache Aurora and Apache Mesos
+1. http://mesos.apache.org/gettingstarted/ OR https://open.mesosphere.com/getting-started/install/
+2. http://aurora.apache.org/documentation/latest/operations/installation/
+
+#Usage:
+
+```Options:
+-o = create/kill
+-n = name of the job
+-r = amount of RAM
+-c = CPU count
+-d = disk space
+-k = name of the task to be killed
+-i = executable/image
+```
+#Example:
+To create and launch:
+```
+java -jar AuroraMarathonIntegration-0.17-SNAPSHOT.jar -o create -r 1024 -n batik -c 2.0 -d 1000 -i gouravr/dacapo:tag9
+```
+To kill:
+```
+java -jar AuroraMarathonIntegration-0.17-SNAPSHOT.jar -o kill -n batik
+```
+
+To update:
+```
+java -jar AuroraMarathonIntegration-0.17-SNAPSHOT.jar -o update -n batik
+```
+
+To retrieve update information:
+```
+java -jar AuroraMarathonIntegration-0.17-SNAPSHOT.jar -o update-info -n batik
+```
+
+To pause an update:
+```
+java -jar AuroraMarathonIntegration-0.17-SNAPSHOT.jar -o update-pause -n batik
+```
+
+To list the update progress:
+```
+java -jar AuroraMarathonIntegration-0.17-SNAPSHOT.jar -o update-list -n batik
+```
+
+To abort an update:
+```
+java -jar AuroraMarathonIntegration-0.17-SNAPSHOT.jar -o update-abort -n batik
+```
+
+To resume an update:
+```
+java -jar AuroraMarathonIntegration-0.17-SNAPSHOT.jar -o update-resume -n batik
+```
+
+To restart a job:
+```
+java -jar AuroraMarathonIntegration-0.17-SNAPSHOT.jar -o restart -n batik
+```
http://git-wip-us.apache.org/repos/asf/airavata-sandbox/blob/1932e865/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/pom.xml
----------------------------------------------------------------------
diff --git a/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/pom.xml b/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/pom.xml
new file mode 100644
index 0000000..04b0a67
--- /dev/null
+++ b/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/pom.xml
@@ -0,0 +1,44 @@
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+ <parent>
+ <groupId>org.apache.airavata</groupId>
+ <artifactId>airavata-cloud</artifactId>
+ <version>0.17-SNAPSHOT</version>
+ <relativePath>../pom.xml</relativePath>
+ </parent>
+
+ <artifactId>AuroraMarathonIntegration</artifactId>
+ <groupId>org.apache.airavata.cloud.drivers.auroraDriver</groupId>
+ <packaging>jar</packaging>
+ <version>0.17-SNAPSHOT</version>
+ <name>Airavata Aurora Marathon Integration</name>
+ <url>http://airavata.apache.org/</url>
+
+
+ <build>
+ <directory>target</directory>
+ <outputDirectory>target/classes</outputDirectory>
+ <finalName>${project.artifactId}-${project.version}</finalName>
+ <sourceDirectory>src/main/java</sourceDirectory>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-jar-plugin</artifactId>
+ <configuration>
+ <archive>
+ <manifest>
+ <mainClass>org.apache.airavata.cloud.drivers.auroraDriver.AuroraAdminDriver</mainClass>
+ </manifest>
+ </archive>
+ </configuration>
+ </plugin>
+ </plugins>
+ </build>
+
+ <properties>
+ <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
+ <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
+ </properties>
+
+</project>
http://git-wip-us.apache.org/repos/asf/airavata-sandbox/blob/1932e865/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/bigDataClientSideServices/aurora/auroraClient/AuroraJobSchedulerI.java
----------------------------------------------------------------------
diff --git a/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/bigDataClientSideServices/aurora/auroraClient/AuroraJobSchedulerI.java b/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/bigDataClientSideServices/aurora/auroraClient/AuroraJobSchedulerI.java
new file mode 100644
index 0000000..092d0aa
--- /dev/null
+++ b/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/bigDataClientSideServices/aurora/auroraClient/AuroraJobSchedulerI.java
@@ -0,0 +1,45 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ *
+ */
+
+package org.apache.airavata.cloud.bigDataClientSideServices.aurora.auroraClient;
+
+import org.apache.airavata.cloud.exceptions.auroraExceptions.AuroraException;
+
+public interface AuroraJobSchedulerI {
+
+
+ public void auroraJobCommand(String info, String command) throws AuroraException;
+ public void jobUpdateList(String info) throws AuroraException;
+ public void jobUpdateAbort(String info) throws AuroraException;
+ public void jobUpdateResume(String info) throws AuroraException;
+ public void jobUpdatePause(String info) throws AuroraException;
+ public void jobUpdateInfo(String info) throws AuroraException;
+ public void jobUpdate(String update) throws AuroraException;
+ public void jobRestart(String restart) throws AuroraException;
+ public void jobKill(String kill) throws AuroraException;
+ public void jobLaunch(String name) throws AuroraException;
+ public void configCreate(String name, String ram, String cpu, String disk, String image) throws AuroraException;
+ public void jobDiff(String key, String config) throws AuroraException;
+ public void configList(String config) throws AuroraException;
+ public void jobInspect(String key, String config) throws AuroraException;
+ public void clusterQuota(String key) throws AuroraException;
+ public void openWebUI(String key) throws AuroraException;
+}
http://git-wip-us.apache.org/repos/asf/airavata-sandbox/blob/1932e865/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/bigDataClientSideServices/aurora/auroraClient/AuroraJobSchedulerImpl.java
----------------------------------------------------------------------
diff --git a/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/bigDataClientSideServices/aurora/auroraClient/AuroraJobSchedulerImpl.java b/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/bigDataClientSideServices/aurora/auroraClient/AuroraJobSchedulerImpl.java
new file mode 100644
index 0000000..b905ccb
--- /dev/null
+++ b/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/bigDataClientSideServices/aurora/auroraClient/AuroraJobSchedulerImpl.java
@@ -0,0 +1,239 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ *
+ */
+
+
+package org.apache.airavata.cloud.bigDataClientSideServices.aurora.auroraClient;
+
+import java.io.BufferedReader;
+import java.io.BufferedWriter;
+import java.io.File;
+import java.io.FileWriter;
+import java.io.IOException;
+import java.io.InputStreamReader;
+
+import org.apache.airavata.cloud.exceptions.auroraExceptions.AuroraException;
+import org.apache.airavata.cloud.util.auroraUtilities.AuroraUtilImpl;
+import org.apache.airavata.cloud.util.auroraUtilities.AuroraUtilI;
+
+public class AuroraJobSchedulerImpl implements AuroraJobSchedulerI {
+ AuroraUtilI util = new AuroraUtilImpl();
+
+
+ public void auroraJobCommand(String info, String command) throws AuroraException{
+ try{
+ String completeCommandToRunProcess = "aurora task run example/benchmarks/devel/"+info+" "+command;
+ BufferedReader stdout = util.executeProcess(completeCommandToRunProcess);
+ util.printLog(stdout);
+ }
+
+ catch (Exception ex) {
+ throw new AuroraException("Exception occured while passing the command.\n"+ex.toString());
+ }
+ }
+ public void jobUpdateList(String info) throws AuroraException{
+ try{
+ String completeCommandToRunProcess = "aurora update list example/benchmarks/devel/"+info;
+ BufferedReader stdout = util.executeProcess(completeCommandToRunProcess);
+ util.printLog(stdout);
+ }
+
+ catch (Exception ex) {
+
+ throw new AuroraException("Exception occured while listing the update.\n"+ex.toString());
+ }
+ }
+ public void jobUpdateAbort(String info) throws AuroraException{
+ try{
+ String completeCommandToRunProcess = "aurora abort pause example/benchmarks/devel/"+info;
+ BufferedReader stdout = util.executeProcess(completeCommandToRunProcess);
+ util.printLog(stdout);
+ }
+
+ catch (Exception ex) {
+
+ throw new AuroraException("Exception occured while aborting the update.\n"+ex.toString());
+ }
+ }
+ public void jobUpdateResume(String info) throws AuroraException{
+ try{
+ String completeCommandToRunProcess ="aurora update resume example/benchmarks/devel/"+info;
+ BufferedReader stdout = util.executeProcess(completeCommandToRunProcess);
+ util.printLog(stdout);
+ }
+
+ catch (Exception ex) {
+
+ throw new AuroraException("Exception occured while resuming the update.\n"+ex.toString());
+ }
+ }
+ public void jobUpdatePause(String info) throws AuroraException{
+ try{
+ String completeCommandToRunProcess = "aurora update pause example/benchmarks/devel/"+info;
+ BufferedReader stdout = util.executeProcess(completeCommandToRunProcess);
+ util.printLog(stdout);
+ }
+
+ catch (Exception ex) {
+
+ throw new AuroraException("Exception occured while pausing the update.\n"+ex.toString());
+ }
+ }
+ public void jobUpdateInfo(String info) throws AuroraException{
+ try{
+ String completeCommandToRunProcess = "aurora update info example/benchmarks/devel/"+info;
+ BufferedReader stdout = util.executeProcess(completeCommandToRunProcess);
+ util.printLog(stdout);
+ }
+
+ catch (Exception ex) {
+
+ throw new AuroraException("Exception occured while retrieving the update info."+ex.toString());
+ }
+ }
+
+ public void jobUpdate(String update) throws AuroraException{
+ try{
+ String completeCommandToRunProcess = "aurora update start example/benchmarks/devel/"+update+" "+update+".aurora";
+ BufferedReader stdout = util.executeProcess(completeCommandToRunProcess);
+ util.printLog(stdout);
+ }
+
+ catch (Exception ex) {
+
+ throw new AuroraException("Exception occured while updating the job.\n"+ex.toString());
+ }
+ }
+ public void jobRestart(String restart) throws AuroraException{
+ try{
+ String completeCommandToRunProcess = "aurora job restart example/benchmarks/devel/"+restart;
+ BufferedReader stdout = util.executeProcess(completeCommandToRunProcess);
+ util.printLog(stdout);
+ }
+
+ catch (Exception ex) {
+ throw new AuroraException("Exception occured while restarting the job.\n"+ex.toString());
+ }
+ }
+
+ public void jobKill(String kill) throws AuroraException{
+ try{
+ String completeCommandToRunProcess = "aurora job killall example/benchmarks/devel/"+kill;
+ BufferedReader stdout = util.executeProcess(completeCommandToRunProcess);
+ util.printLog(stdout);
+ }
+
+ catch (Exception ex) {
+ throw new AuroraException("Exception occured while killing the job.\n"+ex.toString());
+ }
+ }
+ public void jobLaunch(String name) throws AuroraException{
+ try{
+ String completeCommandToRunProcess = "aurora job create example/benchmarks/devel/"+name+" "+name+".aurora";
+ BufferedReader stdout = util.executeProcess(completeCommandToRunProcess);
+ util.printLog(stdout);
+ }
+
+ catch (Exception ex) {
+ throw new AuroraException("Exception occured while launching the job.\n"+ex.toString());
+ }
+ }
+ public void configCreate(String name, String ram, String cpu, String disk, String image) throws AuroraException{
+ try {
+ String config = "import hashlib\n"+name+"= Process(name = '"+name+"', cmdline = 'java -jar /dacapo-9.12-bach.jar "+name+" -s small')\n"+name+"_task = SequentialTask(processes = [ "+name+"], resources = Resources(cpu = "+cpu+", ram = "+ram+"*MB, disk="+disk+"*MB))\njobs = [ Job(cluster = 'example', environment = 'devel', role = 'benchmarks', name = '"+name+"', task = "+name+"_task, instances =1 , container = Container(docker = Docker(image = '"+image+"')))]\n";
+ File file = new File(name+".aurora");
+
+ if (!file.exists()) {
+ file.createNewFile();
+ }
+
+ FileWriter fw = new FileWriter(file.getAbsoluteFile());
+ BufferedWriter bw = new BufferedWriter(fw);
+ bw.write(config);
+ bw.close();
+
+ }catch (IOException ex) {
+ throw new AuroraException("IO Exception occured while creating the configuration file.\n"+ex.toString());
+ }catch (Exception ex) {
+ throw new AuroraException("Exception occured while creating the configuration file.\n"+ex.toString());
+ }
+ }
+
+
+ public void jobDiff(String key, String config) throws AuroraException{
+ try{
+ String completeCommandToRunProcess = "aurora job diff "+key+" "+config;
+ BufferedReader stdout = util.executeProcess(completeCommandToRunProcess);
+ util.printLog(stdout);
+ }
+
+ catch (Exception ex) {
+ throw new AuroraException("Exception occured while retrieving the job difference.\n"+ex.toString());
+ }
+ }
+
+ public void configList(String config) throws AuroraException{
+ try{
+ String completeCommandToRunProcess = "aurora config list "+config;
+ BufferedReader stdout = util.executeProcess(completeCommandToRunProcess);
+ util.printLog(stdout);
+ }
+
+ catch (Exception ex) {
+ throw new AuroraException("Exception occured while retrieving the list of jobs.\n"+ex.toString());
+ }
+ }
+
+ public void jobInspect(String key, String config) throws AuroraException{
+ try{
+ String completeCommandToRunProcess = "aurora job inspect "+key+" "+config;
+ BufferedReader stdout = util.executeProcess(completeCommandToRunProcess);
+ util.printLog(stdout);
+ }
+
+ catch (Exception ex) {
+ throw new AuroraException("Exception occured while inspecting the job.\n"+ex.toString());
+ }
+ }
+
+ public void clusterQuota(String key) throws AuroraException{
+ try{
+ String completeCommandToRunProcess = "aurora quota get "+key;
+ BufferedReader stdout = util.executeProcess(completeCommandToRunProcess);
+ util.printLog(stdout);
+ }
+
+ catch (Exception ex) {
+ throw new AuroraException("Exception occured while retrieving the production quota of the cluster.\n"+ex.toString());
+ }
+ }
+
+ public void openWebUI(String key) throws AuroraException{
+ try{
+ String completeCommandToRunProcess = "aurora job open "+key;
+ BufferedReader stdout = util.executeProcess(completeCommandToRunProcess);
+ util.printLog(stdout);
+ }
+
+ catch (Exception ex) {
+ throw new AuroraException("Exception occured while opening the browser.\n"+ex.toString());
+ }
+ }
+}
http://git-wip-us.apache.org/repos/asf/airavata-sandbox/blob/1932e865/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/bigDataClientSideServices/marathon/marathonClient/MarathonJobSchedulerI.java
----------------------------------------------------------------------
diff --git a/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/bigDataClientSideServices/marathon/marathonClient/MarathonJobSchedulerI.java b/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/bigDataClientSideServices/marathon/marathonClient/MarathonJobSchedulerI.java
new file mode 100644
index 0000000..dc90582
--- /dev/null
+++ b/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/bigDataClientSideServices/marathon/marathonClient/MarathonJobSchedulerI.java
@@ -0,0 +1,51 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ *
+ */
+
+package org.apache.airavata.cloud.bigDataClientSideServices.marathon.marathonClient;
+
+import org.apache.airavata.cloud.exceptions.marathonExceptions.MarathonException;
+
+public interface MarathonJobSchedulerI {
+
+ public void jobKill(String kill, String address) throws MarathonException;
+ public void jobLaunch(String name, String address) throws MarathonException;
+ public void configCreate(String name, String ram, String cpu, String disk, String image, String command) throws MarathonException;
+ public void jobList(String address) throws MarathonException;
+ public void jobListById(String address, String id) throws MarathonException;
+ public void jobListByName(String address, String name) throws MarathonException;
+ public void jobDelete(String address, String appid) throws MarathonException;
+ public void runningJobs(String address, String appid) throws MarathonException;
+ public void createGroups(String address, String json) throws MarathonException;
+ public void groups(String address) throws MarathonException;
+ public void groupsId(String address, String groupid) throws MarathonException;
+ public void jobDeleteId(String address, String appid, String taskid) throws MarathonException;
+ public void deleteDeployment(String address, String id) throws MarathonException;
+ public void deploymentList(String address) throws MarathonException;
+ public void deleteGroups(String address, String id) throws MarathonException;
+ public void eventsList(String address) throws MarathonException;
+ public void eventSubscriptionList(String address) throws MarathonException;
+ public void launchQueue(String address) throws MarathonException;
+ public void deleteMarathonLeader(String address) throws MarathonException;
+ public void marathonLeader(String address) throws MarathonException;
+ public void marathonInfo(String address) throws MarathonException;
+
+
+}
http://git-wip-us.apache.org/repos/asf/airavata-sandbox/blob/1932e865/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/bigDataClientSideServices/marathon/marathonClient/MarathonJobSchedulerImpl.java
----------------------------------------------------------------------
diff --git a/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/bigDataClientSideServices/marathon/marathonClient/MarathonJobSchedulerImpl.java b/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/bigDataClientSideServices/marathon/marathonClient/MarathonJobSchedulerImpl.java
new file mode 100644
index 0000000..db166b7
--- /dev/null
+++ b/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/bigDataClientSideServices/marathon/marathonClient/MarathonJobSchedulerImpl.java
@@ -0,0 +1,318 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ *
+ */
+
+package org.apache.airavata.cloud.bigDataClientSideServices.marathon.marathonClient;
+
+import java.io.BufferedReader;
+import java.io.BufferedWriter;
+import java.io.File;
+import java.io.FileWriter;
+import java.io.IOException;
+import java.io.InputStreamReader;
+
+import org.apache.airavata.cloud.exceptions.marathonExceptions.MarathonException;
+import org.apache.airavata.cloud.util.marathonUtilities.MarathonUtilImpl;
+import org.apache.airavata.cloud.util.marathonUtilities.MarathonUtilI;
+
+public class MarathonJobSchedulerImpl implements MarathonJobSchedulerI {
+ MarathonUtilI util = new MarathonUtilImpl();
+
+
+ public void jobKill(String name, String address) throws MarathonException{
+ try{
+
+ String completeCommandToRunProcess = "curl -X DELETE "+address+""+name;
+ BufferedReader stdout = util.executeProcess(completeCommandToRunProcess);
+ util.printLog(stdout);
+ }
+
+ catch (Exception ex) {
+ throw new MarathonException("Exception occured while launching the job.\n"+ex.toString());
+ }
+ }
+ public void jobLaunch(String name, String address) throws MarathonException{
+ try{
+
+ String completeCommandToRunProcess = "curl -X POST "+address+"/v2/apps -d @"+name+" -H Content-type: application/json";
+ BufferedReader stdout = util.executeProcess(completeCommandToRunProcess);
+ util.printLog(stdout);
+ }
+
+ catch (Exception ex) {
+ throw new MarathonException("Exception occured while launching the job.\n"+ex.toString());
+ }
+ }
+ public void configCreate(String name, String ram, String cpu, String disk, String image, String command) throws MarathonException{
+ try {
+ String config = "'id': "+name+",'cmd': \""+command+"\", \"container\": {\"type\": \"DOCKER\", \"docker\": {\"image\": \"danielpan/dacapo\", \"forcePullImage\": bool(1)}},\"constraints\":[[\"hostname\",\"UNIQUE\"]],\"cpus\": float("+cpu+"), \"mem\": "+ram+"), \"disk\": "+disk+", \"instances\": 1";
+ File file = new File(name+".json");
+
+ if (!file.exists()) {
+ file.createNewFile();
+ }
+
+ FileWriter fw = new FileWriter(file.getAbsoluteFile());
+ BufferedWriter bw = new BufferedWriter(fw);
+ bw.write(config);
+ bw.close();
+
+ }catch (IOException ex) {
+ throw new MarathonException("IO Exception occured while creating the configuration file.\n"+ex.toString());
+ }catch (Exception ex) {
+ throw new MarathonException("Exception occured while creating the configuration file.\n"+ex.toString());
+ }
+ }
+
+ public void jobList(String address) throws MarathonException{
+ try{
+
+ String completeCommandToRunProcess = "curl GET "+address+"/v2/apps";
+ BufferedReader stdout = util.executeProcess(completeCommandToRunProcess);
+ util.printLog(stdout);
+ }
+
+ catch (Exception ex) {
+ throw new MarathonException("Exception occured while retrieving the list jobs.\n"+ex.toString());
+ }
+ }
+
+ public void jobListById(String address, String id) throws MarathonException{
+ try{
+
+ String completeCommandToRunProcess ="curl GET "+address+"/v2/apps/"+id;
+ BufferedReader stdout = util.executeProcess(completeCommandToRunProcess);
+ util.printLog(stdout);
+ }
+
+ catch (Exception ex) {
+ throw new MarathonException("Exception occured while retrieving the list jobs.\n"+ex.toString());
+ }
+ }
+ public void jobListByName(String address, String name) throws MarathonException{
+ try{
+
+ String completeCommandToRunProcess = "curl GET "+address+"/v2/apps/"+name;
+ BufferedReader stdout = util.executeProcess(completeCommandToRunProcess);
+ util.printLog(stdout);
+ }
+
+ catch (Exception ex) {
+ throw new MarathonException("Exception occured while retrieving the list jobs.\n"+ex.toString());
+ }
+ }
+
+ public void jobDelete(String address, String appid) throws MarathonException{
+ try{
+
+ String completeCommandToRunProcess = "curl DELETE "+address+"/v2/apps/"+appid+"/tasks";
+ BufferedReader stdout = util.executeProcess(completeCommandToRunProcess);
+ util.printLog(stdout);
+ }
+
+ catch (Exception ex) {
+ throw new MarathonException("Exception occured while retrieving the list jobs.\n"+ex.toString());
+ }
+ }
+
+ public void runningJobs(String address, String appid) throws MarathonException{
+ try{
+
+ String completeCommandToRunProcess = "curl GET "+address+"/v2/apps/"+appid+"/tasks";
+ BufferedReader stdout = util.executeProcess(completeCommandToRunProcess);
+ util.printLog(stdout);
+ }
+
+ catch (Exception ex) {
+ throw new MarathonException("Exception occured while retrieving the list jobs.\n"+ex.toString());
+ }
+ }
+
+ public void createGroups(String address, String json) throws MarathonException{
+ try{
+
+ String completeCommandToRunProcess = "curl -X POST -H \"Content-type: application/json\" "+address+"/v2/groups/"+json;
+ BufferedReader stdout = util.executeProcess(completeCommandToRunProcess);
+ util.printLog(stdout);
+ }
+
+ catch (Exception ex) {
+ throw new MarathonException("Exception occured while creating the group.\n"+ex.toString());
+ }
+ }
+
+ public void groups(String address) throws MarathonException{
+ try{
+
+ String completeCommandToRunProcess = "curl GET "+address+"/v2/groups/";
+ BufferedReader stdout = util.executeProcess(completeCommandToRunProcess);
+ util.printLog(stdout);
+ }
+
+ catch (Exception ex) {
+ throw new MarathonException("Exception occured while retrieving the list of groups.\n"+ex.toString());
+ }
+ }
+
+ public void groupsId(String address, String groupid) throws MarathonException{
+ try{
+
+ String completeCommandToRunProcess = "curl GET "+address+"/v2/groups/"+groupid;
+ BufferedReader stdout = util.executeProcess(completeCommandToRunProcess);
+ util.printLog(stdout);
+ }
+
+ catch (Exception ex) {
+ throw new MarathonException("Exception occured while retrieving the list of groups.\n"+ex.toString());
+ }
+ }
+ public void jobDeleteId(String address, String appid, String taskid) throws MarathonException{
+ try{
+
+ String completeCommandToRunProcess = "curl DELETE "+address+"/v2/apps/"+appid+"/"+taskid;
+ BufferedReader stdout = util.executeProcess(completeCommandToRunProcess);
+ util.printLog(stdout);
+ }
+
+ catch (Exception ex) {
+ throw new MarathonException("Exception occured while retrieving the list jobs.\n"+ex.toString());
+ }
+ }
+
+ public void deleteDeployment(String address, String id) throws MarathonException{
+ try{
+
+ String completeCommandToRunProcess = "curl -X DELETE "+address+"/v2/deployments/"+id;
+ BufferedReader stdout = util.executeProcess(completeCommandToRunProcess);
+ util.printLog(stdout);
+ }
+
+ catch (Exception ex) {
+ throw new MarathonException("Exception occured while deleting the deployment.\n"+ex.toString());
+ }
+ }
+ public void deploymentList(String address) throws MarathonException{
+ try{
+
+ String completeCommandToRunProcess = "curl GET "+address+"/v2/deployments/";
+ BufferedReader stdout = util.executeProcess(completeCommandToRunProcess);
+ util.printLog(stdout);
+ }
+
+ catch (Exception ex) {
+ throw new MarathonException("Exception occured while retrieving the list of deployment.\n"+ex.toString());
+ }
+ }
+
+ public void deleteGroups(String address, String id) throws MarathonException{
+ try{
+
+ String completeCommandToRunProcess = "curl -X DELETE "+address+"/v2/groups/"+id;
+ BufferedReader stdout = util.executeProcess(completeCommandToRunProcess);
+ util.printLog(stdout);
+ }
+
+ catch (Exception ex) {
+ throw new MarathonException("Exception occured while deleting the group.\n"+ex.toString());
+ }
+ }
+
+ public void launchQueue(String address) throws MarathonException{
+ try{
+
+ String completeCommandToRunProcess = "curl GET "+address+"/v2/queue";
+ BufferedReader stdout = util.executeProcess(completeCommandToRunProcess);
+ util.printLog(stdout);
+ }
+
+ catch (Exception ex) {
+ throw new MarathonException("Exception occured while retrieving the launch queue.\n"+ex.toString());
+ }
+ }
+ public void eventSubscriptionList(String address) throws MarathonException{
+ try{
+
+
+ String completeCommandToRunProcess = "curl GET "+address+"/v2/eventSubscriptions";
+ BufferedReader stdout = util.executeProcess(completeCommandToRunProcess);
+ util.printLog(stdout);
+ }
+
+ catch (Exception ex) {
+ throw new MarathonException("Exception occured while retrieving the list of event subscriptions.\n"+ex.toString());
+ }
+ }
+ public void eventsList(String address) throws MarathonException{
+ try{
+
+ String completeCommandToRunProcess = "curl GET "+address+"/v2/events";
+ BufferedReader stdout = util.executeProcess(completeCommandToRunProcess);
+ util.printLog(stdout);
+ }
+
+ catch (Exception ex) {
+ throw new MarathonException("Exception occured while retrieving the list of events.\n"+ex.toString());
+ }
+ }
+
+ public void deleteMarathonLeader(String address) throws MarathonException{
+ try{
+
+ String completeCommandToRunProcess = "curl -X DELETE "+address+"/v2/leader";
+ BufferedReader stdout = util.executeProcess(completeCommandToRunProcess);
+ util.printLog(stdout);
+ }
+
+ catch (Exception ex) {
+ throw new MarathonException("Exception occured while retrieving the launch queue.\n"+ex.toString());
+ }
+ }
+ public void marathonLeader(String address) throws MarathonException{
+ try{
+
+
+ String completeCommandToRunProcess = "curl GET "+address+"/v2/leader";
+ BufferedReader stdout = util.executeProcess(completeCommandToRunProcess);
+ util.printLog(stdout);
+ }
+
+ catch (Exception ex) {
+ throw new MarathonException("Exception occured while retrieving the leader information.\n"+ex.toString());
+ }
+ }
+ public void marathonInfo(String address) throws MarathonException{
+ try{
+
+
+ String completeCommandToRunProcess = "curl GET "+address+"/v2/info";
+ BufferedReader stdout = util.executeProcess(completeCommandToRunProcess);
+ util.printLog(stdout);
+ }
+
+ catch (Exception ex) {
+ throw new MarathonException("Exception occured while retrieving the marathon information.\n"+ex.toString());
+ }
+ }
+
+
+
+
+
+}
http://git-wip-us.apache.org/repos/asf/airavata-sandbox/blob/1932e865/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/bigDataInjections/AuroraInjectorImpl.java
----------------------------------------------------------------------
diff --git a/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/bigDataInjections/AuroraInjectorImpl.java b/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/bigDataInjections/AuroraInjectorImpl.java
new file mode 100644
index 0000000..109fb99
--- /dev/null
+++ b/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/bigDataInjections/AuroraInjectorImpl.java
@@ -0,0 +1,101 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ *
+ */
+
+package org.apache.airavata.cloud.bigDataInjections;
+
+import java.util.Map;
+import java.util.List;
+
+
+import org.apache.airavata.cloud.bigDataClientSideServices.aurora.auroraClient.AuroraJobSchedulerI;
+import org.apache.airavata.cloud.bigDataClientSideServices.aurora.auroraClient.AuroraJobSchedulerImpl;
+import org.apache.airavata.cloud.exceptions.auroraExceptions.AuroraException;
+
+
+public class AuroraInjectorImpl implements BigDataInjectorI {
+ private AuroraJobSchedulerI auroraJS = null;
+
+ public AuroraInjectorImpl(AuroraJobSchedulerI auroraJSIn) {
+ auroraJS = auroraJSIn;
+ }
+
+
+ public void executeTheBigDataClientSideCommand(Map<String, List<String>> commandLineOptions) {
+
+
+ String commandName = commandLineOptions.get("o").get(0);
+ String RamSize, JobName, CpuCount, DiskSize, Image;
+
+ switch(commandName)
+ {
+ case "kill" :
+ try {
+ auroraJS.jobKill(commandLineOptions.get("n").get(0));
+ } catch(AuroraException ex){
+ } break;
+ case "restart" :
+ try {
+ auroraJS.jobRestart(commandLineOptions.get("n").get(0));
+ } catch (AuroraException ex) {
+ } break;
+ case "update" :
+ try {
+ auroraJS.jobUpdate(commandLineOptions.get("n").get(0));
+ } catch(AuroraException ex){
+ } break;
+ case "update-info" :
+ try {
+ auroraJS.jobUpdateInfo(commandLineOptions.get("n").get(0));
+ } catch(AuroraException ex){
+ } break;
+ case "update-pause" :
+ try {
+ auroraJS.jobUpdatePause(commandLineOptions.get("n").get(0));
+ } catch(AuroraException ex){
+ } break;
+ case "inspect" :
+ try {
+ auroraJS.jobInspect(commandLineOptions.get("n").get(0), commandLineOptions.get("k").get(0));
+ } catch(AuroraException ex){
+ } break;
+ case "quota" :
+ try {
+ auroraJS.clusterQuota(commandLineOptions.get("k").get(0));
+ } catch(AuroraException ex){
+ } break;
+ case "create" :
+ JobName = commandLineOptions.get("n").get(0);
+ RamSize = commandLineOptions.get("r").get(0);
+ CpuCount = commandLineOptions.get("c").get(0);
+ DiskSize = commandLineOptions.get("d").get(0);
+ Image = commandLineOptions.get("i").get(0);
+ try {
+ auroraJS.configCreate(JobName,RamSize,CpuCount,DiskSize,Image);
+ } catch (AuroraException ex) {}
+ try {
+ auroraJS.jobLaunch(JobName);
+ } catch (AuroraException ex) {
+ } break;
+ default :
+ System.out.println("Improper option\nOptions available:\n1) create\n2) kill\n3) restart\n4) update\n 5) update-info\n6) update-pause\n");
+ }
+ } // end of public void executeTheBigDataCommand
+} // end of public class AuroraInjectorImpl
http://git-wip-us.apache.org/repos/asf/airavata-sandbox/blob/1932e865/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/bigDataInjections/BigDataInjectorI.java
----------------------------------------------------------------------
diff --git a/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/bigDataInjections/BigDataInjectorI.java b/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/bigDataInjections/BigDataInjectorI.java
new file mode 100644
index 0000000..a4f2d42
--- /dev/null
+++ b/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/bigDataInjections/BigDataInjectorI.java
@@ -0,0 +1,32 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ *
+ */
+
+package org.apache.airavata.cloud.bigDataInjections;
+
+import java.util.Map;
+import java.util.List;
+
+
+public interface BigDataInjectorI {
+
+ // TODO: this interface should throw an exception
+ public void executeTheBigDataClientSideCommand(Map<String, List<String>> commandLineOptions);
+}
http://git-wip-us.apache.org/repos/asf/airavata-sandbox/blob/1932e865/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/bigDataInjections/MarathonInjectorImpl.java
----------------------------------------------------------------------
diff --git a/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/bigDataInjections/MarathonInjectorImpl.java b/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/bigDataInjections/MarathonInjectorImpl.java
new file mode 100644
index 0000000..f85a5c2
--- /dev/null
+++ b/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/bigDataInjections/MarathonInjectorImpl.java
@@ -0,0 +1,72 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ *
+ */
+
+package org.apache.airavata.cloud.bigDataInjections;
+
+import java.util.Map;
+import java.util.List;
+
+
+import org.apache.airavata.cloud.bigDataClientSideServices.marathon.marathonClient.MarathonJobSchedulerI;
+import org.apache.airavata.cloud.bigDataClientSideServices.marathon.marathonClient.MarathonJobSchedulerImpl;
+import org.apache.airavata.cloud.exceptions.marathonExceptions.MarathonException;
+
+
+public class MarathonInjectorImpl implements BigDataInjectorI {
+ private MarathonJobSchedulerI marathonJS = null;
+
+ public MarathonInjectorImpl(MarathonJobSchedulerI marathonJSIn) {
+ marathonJS = marathonJSIn;
+ }
+
+
+ public void executeTheBigDataClientSideCommand(Map<String, List<String>> commandLineOptions) {
+
+
+ String commandName = commandLineOptions.get("o").get(0);
+ String RamSize, JobName, CpuCount, DiskSize, Image, Command;
+
+ switch(commandName)
+ {
+ case "kill" :
+ try {
+ marathonJS.jobKill(commandLineOptions.get("n").get(0),commandLineOptions.get("a").get(0));
+ } catch(MarathonException ex){
+ } break;
+ case "create" :
+ JobName = commandLineOptions.get("n").get(0);
+ RamSize = commandLineOptions.get("r").get(0);
+ CpuCount = commandLineOptions.get("c").get(0);
+ DiskSize = commandLineOptions.get("d").get(0);
+ Image = commandLineOptions.get("i").get(0);
+ Command = commandLineOptions.get("a").get(0);
+ try {
+ marathonJS.configCreate(JobName,RamSize,CpuCount,DiskSize,Image, Command);
+ } catch (MarathonException ex) {}
+ try {
+ marathonJS.jobLaunch(JobName,commandLineOptions.get("a").get(0));
+ } catch (MarathonException ex) {
+ } break;
+ default :
+ System.out.println("Improper option\nOptions available:\n1) create\n2) kill\n");
+ }
+ } // end of public void executeTheBigDataCommand
+} // end of public class AuroraInjectorImpl
http://git-wip-us.apache.org/repos/asf/airavata-sandbox/blob/1932e865/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/drivers/auroraDriver/AuroraAdminDriver.java
----------------------------------------------------------------------
diff --git a/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/drivers/auroraDriver/AuroraAdminDriver.java b/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/drivers/auroraDriver/AuroraAdminDriver.java
new file mode 100644
index 0000000..0685bcd
--- /dev/null
+++ b/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/drivers/auroraDriver/AuroraAdminDriver.java
@@ -0,0 +1,98 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ *
+ */
+
+package org.apache.airavata.cloud.drivers.auroraDriver;
+
+// TODO: need javadoc documentation at the top of each method
+
+// TODO: individually import the types
+import java.util.*;
+
+import org.apache.airavata.cloud.exceptions.auroraExceptions.AuroraException;
+import org.apache.airavata.cloud.bigDataClientSideServices.aurora.auroraClient.AuroraJobSchedulerI;
+import org.apache.airavata.cloud.bigDataClientSideServices.aurora.auroraClient.AuroraJobSchedulerImpl;
+import org.apache.airavata.cloud.bigDataInjections.AuroraInjectorImpl;
+import org.apache.airavata.cloud.bigDataInjections.BigDataInjectorI;
+
+
+import org.apache.airavata.cloud.exceptions.marathonExceptions.MarathonException;
+import org.apache.airavata.cloud.bigDataClientSideServices.marathon.marathonClient.MarathonJobSchedulerI;
+import org.apache.airavata.cloud.bigDataClientSideServices.marathon.marathonClient.MarathonJobSchedulerImpl;
+import org.apache.airavata.cloud.bigDataInjections.MarathonInjectorImpl;
+
+public class AuroraAdminDriver{
+ public static void main(String[] args) {
+
+ // TODO: do command line validation
+
+ // Processing of the command line arguments should be moved to a different method
+
+
+
+ // This code to add command line arguments is based on Apache Commons
+
+ // TODO: explain why this Map data structure is needed
+ Map<String, List<String>> params = new HashMap<>();
+
+ // TODO: explain what is the purpose of this List
+ List<String> options = null;
+ for (int i = 0; i < args.length; i++) {
+ final String a = args[i];
+ if (a.charAt(0) == '-') {
+ if (a.length() < 2) {
+ // TOOD: need more details in the error statement
+ System.err.println("Error at argument " + a);
+ return;
+ }
+ // TODO: explain the purpose of this ArrayList
+ options = new ArrayList<>();
+ params.put(a.substring(1), options);
+ }
+ // TODO: explain when this "else" branch is taken
+ else if (options != null) {
+ options.add(a);
+ }
+ else {
+
+ System.err.println("Illegal parameter \n[USAGE]\nOptions:\n1) -o\tcreate, kill, restart, update, update-info, update-pause\n2) -n\tname of the job\n 3) -r\tamount of RAM\n 4) -c\tCPU count\n 5) -d\tdisk space\n 6) -k\tname of the task to be killed\n 7) -i\texecutable/image\n ");
+ return;
+ }
+
+
+ }// end of for (int i=0; ...
+
+ // use the code below to decide between injecting Aurora, Marathon, etc. as injections
+
+ AuroraJobSchedulerI auroraJS = new AuroraJobSchedulerImpl();
+ BigDataInjectorI auroraInjector = new AuroraInjectorImpl(auroraJS);
+ auroraInjector.executeTheBigDataClientSideCommand(params);
+
+
+
+ // UNCOMMENT NEXT 3 LINES TO USE MARATHON and comment the 3 lines above to shut AURORA off.
+ /*MarathonJobSchedulerI marathonJS = new MarathonJobSchedulerImpl();
+ BigDataInjectorI marathonInjector = new MarathonInjectorImpl(marathonJS);
+ marathonInjector.executeTheBigDataClientSideCommand(params);*/
+
+
+
+ } // end of public static void main
+} // end of class
http://git-wip-us.apache.org/repos/asf/airavata-sandbox/blob/1932e865/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/exceptions/auroraExceptions/AuroraException.java
----------------------------------------------------------------------
diff --git a/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/exceptions/auroraExceptions/AuroraException.java b/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/exceptions/auroraExceptions/AuroraException.java
new file mode 100644
index 0000000..cb823d9
--- /dev/null
+++ b/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/exceptions/auroraExceptions/AuroraException.java
@@ -0,0 +1,36 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ *
+ */
+
+package org.apache.airavata.cloud.exceptions.auroraExceptions;
+
+public class AuroraException extends Exception {
+ private String exceptionMsg;
+ public AuroraException(){
+ exceptionMsg="";
+ }
+ public AuroraException(String exceptionMsgIn){
+ exceptionMsg=exceptionMsgIn;;
+ }
+
+ public String toString(){
+ return this.exceptionMsg;
+ }
+ }
http://git-wip-us.apache.org/repos/asf/airavata-sandbox/blob/1932e865/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/exceptions/marathonExceptions/MarathonException.java
----------------------------------------------------------------------
diff --git a/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/exceptions/marathonExceptions/MarathonException.java b/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/exceptions/marathonExceptions/MarathonException.java
new file mode 100644
index 0000000..d6010cc
--- /dev/null
+++ b/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/exceptions/marathonExceptions/MarathonException.java
@@ -0,0 +1,36 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ *
+ */
+
+package org.apache.airavata.cloud.exceptions.marathonExceptions;
+
+public class MarathonException extends Exception {
+ private String exceptionMsg;
+ public MarathonException(){
+ exceptionMsg="";
+ }
+ public MarathonException(String exceptionMsgIn){
+ exceptionMsg=exceptionMsgIn;;
+ }
+
+ public String toString(){
+ return this.exceptionMsg;
+ }
+ }
http://git-wip-us.apache.org/repos/asf/airavata-sandbox/blob/1932e865/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/util/auroraUtilities/AuroraUtilI.java
----------------------------------------------------------------------
diff --git a/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/util/auroraUtilities/AuroraUtilI.java b/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/util/auroraUtilities/AuroraUtilI.java
new file mode 100644
index 0000000..b545bef
--- /dev/null
+++ b/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/util/auroraUtilities/AuroraUtilI.java
@@ -0,0 +1,31 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ *
+ */
+
+package org.apache.airavata.cloud.util.auroraUtilities;
+
+import java.io.BufferedReader;
+
+import org.apache.airavata.cloud.exceptions.auroraExceptions.AuroraException;
+
+public interface AuroraUtilI{
+ public void printLog(BufferedReader stdout) throws AuroraException;
+ public BufferedReader executeProcess(String commandToRunProcess) throws AuroraException;
+}
http://git-wip-us.apache.org/repos/asf/airavata-sandbox/blob/1932e865/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/util/auroraUtilities/AuroraUtilImpl.java
----------------------------------------------------------------------
diff --git a/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/util/auroraUtilities/AuroraUtilImpl.java b/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/util/auroraUtilities/AuroraUtilImpl.java
new file mode 100644
index 0000000..dd81285
--- /dev/null
+++ b/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/util/auroraUtilities/AuroraUtilImpl.java
@@ -0,0 +1,63 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ *
+ */
+
+package org.apache.airavata.cloud.util.auroraUtilities;
+
+
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.InputStreamReader;
+
+import org.apache.airavata.cloud.exceptions.auroraExceptions.AuroraException;
+
+public class AuroraUtilImpl implements AuroraUtilI{
+ public void printLog(BufferedReader stdout) throws AuroraException
+ {
+ try{
+ String line;
+ line = stdout.readLine();
+ while (line != null) {
+ System.out.println(line);
+ line = stdout.readLine();
+ }
+ }
+ catch (IOException ex) {
+ throw new AuroraException("IO Exception occured while passing the command.\n"+ex.toString());
+ }
+ }
+
+ public BufferedReader executeProcess(String commandToRunProcess) throws AuroraException
+ {
+ BufferedReader stdout = null;
+ try{
+ Process auroraJob = Runtime.getRuntime().exec(commandToRunProcess);
+ auroraJob.waitFor();
+ stdout = new BufferedReader(new InputStreamReader(auroraJob.getInputStream()));
+ }
+ catch(Exception ex)
+ {
+ throw new AuroraException("Exception occured while passing the command.\n"+ex.toString());
+
+ }
+ return stdout;
+ }
+
+}
http://git-wip-us.apache.org/repos/asf/airavata-sandbox/blob/1932e865/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/util/marathonUtilities/MarathonUtilI.java
----------------------------------------------------------------------
diff --git a/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/util/marathonUtilities/MarathonUtilI.java b/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/util/marathonUtilities/MarathonUtilI.java
new file mode 100644
index 0000000..00b850a
--- /dev/null
+++ b/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/util/marathonUtilities/MarathonUtilI.java
@@ -0,0 +1,33 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ *
+ */
+
+package org.apache.airavata.cloud.util.marathonUtilities;
+
+import java.io.BufferedReader;
+
+import org.apache.airavata.cloud.exceptions.marathonExceptions.MarathonException;
+
+public interface MarathonUtilI{
+
+ public void printLog(BufferedReader stdout) throws MarathonException;
+ public BufferedReader executeProcess(String commandToRunProcess) throws MarathonException;
+
+}
http://git-wip-us.apache.org/repos/asf/airavata-sandbox/blob/1932e865/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/util/marathonUtilities/MarathonUtilImpl.java
----------------------------------------------------------------------
diff --git a/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/util/marathonUtilities/MarathonUtilImpl.java b/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/util/marathonUtilities/MarathonUtilImpl.java
new file mode 100644
index 0000000..3bf0aea
--- /dev/null
+++ b/gsoc2016/Design Changes in Airavata/AuroraMarathonIntegration/src/main/java/org/apache/airavata/cloud/util/marathonUtilities/MarathonUtilImpl.java
@@ -0,0 +1,64 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ *
+ */
+
+package org.apache.airavata.cloud.util.marathonUtilities;
+
+
+import java.io.BufferedReader;
+import java.io.IOException;
+import java.io.InputStreamReader;
+
+import org.apache.airavata.cloud.exceptions.marathonExceptions.MarathonException;
+
+public class MarathonUtilImpl implements MarathonUtilI{
+
+ public void printLog(BufferedReader stdout) throws MarathonException
+ {
+ try{
+ String line;
+ line = stdout.readLine();
+ while (line != null) {
+ System.out.println(line);
+ line = stdout.readLine();
+ }
+ }
+ catch (IOException ex) {
+ throw new MarathonException("IO Exception occured while passing the command.\n"+ex.toString());
+ }
+ }
+
+ public BufferedReader executeProcess(String commandToRunProcess) throws MarathonException
+ {
+ BufferedReader stdout = null;
+ try{
+ Process marathonJob = Runtime.getRuntime().exec(commandToRunProcess);
+ marathonJob.waitFor();
+ stdout = new BufferedReader(new InputStreamReader(marathonJob.getInputStream()));
+ }
+ catch(Exception ex)
+ {
+ throw new MarathonException("Exception occured while passing the command.\n"+ex.toString());
+
+ }
+ return stdout;
+ }
+
+}
http://git-wip-us.apache.org/repos/asf/airavata-sandbox/blob/1932e865/gsoc2016/Design Changes in Airavata/gfac/gfac-core/pom.xml
----------------------------------------------------------------------
diff --git a/gsoc2016/Design Changes in Airavata/gfac/gfac-core/pom.xml b/gsoc2016/Design Changes in Airavata/gfac/gfac-core/pom.xml
new file mode 100644
index 0000000..69253d2
--- /dev/null
+++ b/gsoc2016/Design Changes in Airavata/gfac/gfac-core/pom.xml
@@ -0,0 +1,190 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <parent>
+ <groupId>org.apache.airavata</groupId>
+ <artifactId>airavata-gfac</artifactId>
+ <version>0.17-SNAPSHOT</version>
+ <relativePath>../pom.xml</relativePath>
+ </parent>
+
+ <modelVersion>4.0.0</modelVersion>
+ <artifactId>airavata-gfac-core</artifactId>
+ <name>Airavata GFac Core</name>
+ <description>The core GFAC functionality independent from any webservice implementation.</description>
+ <url>http://airavata.apache.org/</url>
+
+ <dependencies>
+ <!-- Logging -->
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-api</artifactId>
+ </dependency>
+
+ <!-- GFAC schemas -->
+ <dependency>
+ <groupId>org.apache.airavata</groupId>
+ <artifactId>airavata-registry-cpi</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.airavata</groupId>
+ <artifactId>airavata-registry-core</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.xmlbeans</groupId>
+ <artifactId>xmlbeans</artifactId>
+ <version>${xmlbeans.version}</version>
+ </dependency>
+ <!-- Credential Store -->
+ <dependency>
+ <groupId>org.apache.airavata</groupId>
+ <artifactId>airavata-credential-store</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.airavata</groupId>
+ <artifactId>airavata-messaging-core</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+
+ <!-- Configuration file read-->
+ <dependency>
+ <groupId>org.yaml</groupId>
+ <artifactId>snakeyaml</artifactId>
+ <version>${snakeyaml.version}</version>
+ </dependency>
+
+ <!-- Test -->
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.testng</groupId>
+ <artifactId>testng</artifactId>
+ <version>6.1.1</version>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>jcl-over-slf4j</artifactId>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-log4j12</artifactId>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.airavata</groupId>
+ <artifactId>airavata-server-configuration</artifactId>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.airavata</groupId>
+ <artifactId>airavata-client-configuration</artifactId>
+ <scope>test</scope>
+ </dependency>
+
+ <!-- Guava -->
+ <dependency>
+ <groupId>com.google.guava</groupId>
+ <artifactId>guava</artifactId>
+ <version>12.0</version>
+ </dependency>
+ <!-- gsi-ssh api dependencies -->
+ <dependency>
+ <groupId>org.apache.airavata</groupId>
+ <artifactId>airavata-data-models</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <!-- this is the dependency for amqp implementation -->
+ <!-- zookeeper dependencies -->
+ <dependency>
+ <groupId>org.apache.curator</groupId>
+ <artifactId>curator-framework</artifactId>
+ <version>${curator.version}</version>
+ </dependency>
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.codehaus.mojo</groupId>
+ <artifactId>xmlbeans-maven-plugin</artifactId>
+ <version>2.3.3</version>
+ <executions>
+ <execution>
+ <phase>generate-sources</phase>
+ <goals>
+ <goal>xmlbeans</goal>
+ </goals>
+ </execution>
+ </executions>
+ <inherited>true</inherited>
+ <configuration>
+ <schemaDirectory>src/main/resources</schemaDirectory>
+ <xmlConfigs>
+ <xmlConfig implementation="java.io.File">src/main/resources/gsissh-schemas.xsdconfig</xmlConfig>
+ </xmlConfigs>
+ <sourceGenerationDirectory>src/main/java</sourceGenerationDirectory>
+ <outputJar>target/generated/${project.artifactId}-${project.version}.jar</outputJar>
+ </configuration>
+ </plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-surefire-plugin</artifactId>
+ <configuration>
+ <skip>false</skip>
+ <forkMode>always</forkMode>
+ <failIfNoTests>false</failIfNoTests>
+ </configuration>
+ </plugin>
+ <plugin>
+ <groupId>com.mycila</groupId>
+ <artifactId>license-maven-plugin</artifactId>
+ <version>3.0.rc1</version>
+ <configuration>
+ <header>src/main/resources/apache-license.txt</header>
+ <excludes>
+ <exclude>**/README</exclude>
+ <exclude>src/test/resources/**</exclude>
+ <exclude>src/main/resources/**</exclude>
+ </excludes>
+ </configuration>
+ <executions>
+ <execution>
+ <phase>process-sources</phase>
+ <goals>
+ <goal>format</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+
+</project>
http://git-wip-us.apache.org/repos/asf/airavata-sandbox/blob/1932e865/gsoc2016/Design Changes in Airavata/gfac/gfac-core/src/main/java/org/apache/airavata/gfac/core/authentication/GSIAuthenticationInfo.java
----------------------------------------------------------------------
diff --git a/gsoc2016/Design Changes in Airavata/gfac/gfac-core/src/main/java/org/apache/airavata/gfac/core/authentication/GSIAuthenticationInfo.java b/gsoc2016/Design Changes in Airavata/gfac/gfac-core/src/main/java/org/apache/airavata/gfac/core/authentication/GSIAuthenticationInfo.java
new file mode 100644
index 0000000..37752f1
--- /dev/null
+++ b/gsoc2016/Design Changes in Airavata/gfac/gfac-core/src/main/java/org/apache/airavata/gfac/core/authentication/GSIAuthenticationInfo.java
@@ -0,0 +1,43 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.airavata.gfac.core.authentication;
+
+import org.ietf.jgss.GSSCredential;
+
+import java.util.Properties;
+
+/**
+ * Authentication data. Could be MyProxy user name, password, could be GSSCredentials
+ * or could be SSH keys.
+ */
+public abstract class GSIAuthenticationInfo implements AuthenticationInfo {
+
+ public Properties properties = new Properties();
+
+ public abstract GSSCredential getCredentials() throws SecurityException;
+
+ public Properties getProperties() {
+ return properties;
+ }
+
+ public void setProperties(Properties propertiesIn) {
+ properties = propertiesIn;
+ }
+}
http://git-wip-us.apache.org/repos/asf/airavata-sandbox/blob/1932e865/gsoc2016/Design Changes in Airavata/gfac/gfac-core/src/main/java/org/apache/airavata/gfac/core/authentication/SSHKeyAuthentication.java
----------------------------------------------------------------------
diff --git a/gsoc2016/Design Changes in Airavata/gfac/gfac-core/src/main/java/org/apache/airavata/gfac/core/authentication/SSHKeyAuthentication.java b/gsoc2016/Design Changes in Airavata/gfac/gfac-core/src/main/java/org/apache/airavata/gfac/core/authentication/SSHKeyAuthentication.java
new file mode 100644
index 0000000..fae1f32
--- /dev/null
+++ b/gsoc2016/Design Changes in Airavata/gfac/gfac-core/src/main/java/org/apache/airavata/gfac/core/authentication/SSHKeyAuthentication.java
@@ -0,0 +1,90 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.airavata.gfac.core.authentication;
+
+/**
+ * User: AmilaJ (amilaj@apache.org)
+ * Date: 10/4/13
+ * Time: 2:39 PM
+ */
+
+/**
+ * Abstracts out common methods for SSH key authentication.
+ */
+public class SSHKeyAuthentication implements AuthenticationInfo {
+
+ private String userName;
+ private byte[] privateKey;
+ private byte[] publicKey;
+ private String passphrase;
+ private String knownHostsFilePath;
+ private String strictHostKeyChecking; // yes or no
+
+ public SSHKeyAuthentication() {
+ }
+
+ public String getUserName() {
+ return userName;
+ }
+
+ public void setUserName(String userNameIn) {
+ this.userName = userNameIn;
+ }
+
+ public byte[] getPrivateKey() {
+ return privateKey;
+ }
+
+ public void setPrivateKey(byte[] privateKeyIn) {
+ this.privateKey = privateKeyIn;
+ }
+
+ public byte[] getPublicKey() {
+ return publicKey;
+ }
+
+ public void setPublicKey(byte[] publicKeyIn) {
+ this.publicKey = publicKeyIn;
+ }
+
+ public String getPassphrase() {
+ return passphrase;
+ }
+
+ public void setPassphrase(String passphraseIn) {
+ this.passphrase = passphraseIn;
+ }
+
+ public String getKnownHostsFilePath() {
+ return knownHostsFilePath;
+ }
+
+ public void setKnownHostsFilePath(String knownHostsFilePathIn) {
+ this.knownHostsFilePath = knownHostsFilePathIn;
+ }
+
+ public String getStrictHostKeyChecking() {
+ return strictHostKeyChecking;
+ }
+
+ public void setStrictHostKeyChecking(String strictHostKeyCheckingIn) {
+ this.strictHostKeyChecking = strictHostKeyCheckingIn;
+ }
+}
http://git-wip-us.apache.org/repos/asf/airavata-sandbox/blob/1932e865/gsoc2016/Design Changes in Airavata/gfac/gfac-core/src/main/java/org/apache/airavata/gfac/core/authentication/SSHPasswordAuthentication.java
----------------------------------------------------------------------
diff --git a/gsoc2016/Design Changes in Airavata/gfac/gfac-core/src/main/java/org/apache/airavata/gfac/core/authentication/SSHPasswordAuthentication.java b/gsoc2016/Design Changes in Airavata/gfac/gfac-core/src/main/java/org/apache/airavata/gfac/core/authentication/SSHPasswordAuthentication.java
new file mode 100644
index 0000000..b3f64e4
--- /dev/null
+++ b/gsoc2016/Design Changes in Airavata/gfac/gfac-core/src/main/java/org/apache/airavata/gfac/core/authentication/SSHPasswordAuthentication.java
@@ -0,0 +1,42 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.airavata.gfac.core.authentication;
+
+/**
+ * Password authentication for vanilla SSH.
+ */
+public class SSHPasswordAuthentication implements AuthenticationInfo {
+
+ private String userName;
+ private String password;
+
+ public SSHPasswordAuthentication(String userNameIn, String passwordIn) {
+ this.userName = userNameIn;
+ this.password = passwordIn;
+ }
+
+ public String getUserName() {
+ return userName;
+ }
+
+ public String getPassword() {
+ return password;
+ }
+}
http://git-wip-us.apache.org/repos/asf/airavata-sandbox/blob/1932e865/gsoc2016/Design Changes in Airavata/gfac/pom.xml
----------------------------------------------------------------------
diff --git a/gsoc2016/Design Changes in Airavata/gfac/pom.xml b/gsoc2016/Design Changes in Airavata/gfac/pom.xml
new file mode 100644
index 0000000..c0d4ace
--- /dev/null
+++ b/gsoc2016/Design Changes in Airavata/gfac/pom.xml
@@ -0,0 +1,47 @@
+<?xml version="1.0" encoding="UTF-8"?>
+
+<!--Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under
+ the Apache License, Version 2.0 (the� "License"); you may not use this file except in compliance with the License. You may
+ obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to
+ in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF
+ ANY ~ KIND, either express or implied. See the License for the specific language governing permissions and limitations under
+ the License. -->
+
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
+
+ <parent>
+ <groupId>org.apache.airavata</groupId>
+ <artifactId>airavata</artifactId>
+ <version>0.17-SNAPSHOT</version>
+ <relativePath>../../pom.xml</relativePath>
+ </parent>
+
+
+ <modelVersion>4.0.0</modelVersion>
+ <artifactId>airavata-gfac</artifactId>
+ <packaging>pom</packaging>
+ <name>Airavata GFac</name>
+ <url>http://airavata.apache.org/</url>
+
+ <profiles>
+ <profile>
+ <id>default</id>
+ <activation>
+ <activeByDefault>true</activeByDefault>
+ </activation>
+ <modules>
+ <module>gfac-service</module>
+ <module>gfac-client</module>
+ <module>gfac-core</module>
+ <module>gfac-impl</module>
+ <!--<module>gfac-bes</module>-->
+ <module>gfac-application-specific-handlers</module>
+ </modules>
+ </profile>
+ </profiles>
+ <properties>
+ <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
+ <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
+ </properties>
+</project>
http://git-wip-us.apache.org/repos/asf/airavata-sandbox/blob/1932e865/gsoc2016/Design Changes in Airavata/orchestrator/NULL
----------------------------------------------------------------------
diff --git a/gsoc2016/Design Changes in Airavata/orchestrator/NULL b/gsoc2016/Design Changes in Airavata/orchestrator/NULL
new file mode 100644
index 0000000..6c7ca83
--- /dev/null
+++ b/gsoc2016/Design Changes in Airavata/orchestrator/NULL
@@ -0,0 +1 @@
+Device 0 (VID=18d1 and PID=4ee1) is a Google Inc (for LG Electronics/Samsung) Nexus 4/10 (MTP).
http://git-wip-us.apache.org/repos/asf/airavata-sandbox/blob/1932e865/gsoc2016/Design Changes in Airavata/orchestrator/orchestrator-core/pom.xml
----------------------------------------------------------------------
diff --git a/gsoc2016/Design Changes in Airavata/orchestrator/orchestrator-core/pom.xml b/gsoc2016/Design Changes in Airavata/orchestrator/orchestrator-core/pom.xml
new file mode 100644
index 0000000..ee0d23a
--- /dev/null
+++ b/gsoc2016/Design Changes in Airavata/orchestrator/orchestrator-core/pom.xml
@@ -0,0 +1,103 @@
+<?xml version="1.0" encoding="UTF-8"?>
+
+<!--Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under
+the Apache License, Version 2.0 (the� "License"); you may not use this file except in compliance with the License. You may
+obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to
+in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF
+ANY ~ KIND, either express or implied. See the License for the specific language governing permissions and limitations under
+the License. -->
+
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
+
+
+ <parent>
+ <groupId>org.apache.airavata</groupId>
+ <artifactId>orchestrator</artifactId>
+ <version>0.17-SNAPSHOT</version>
+ <relativePath>../pom.xml</relativePath>
+ </parent>
+ <modelVersion>4.0.0</modelVersion>
+ <artifactId>airavata-orchestrator-core</artifactId>
+ <packaging>jar</packaging>
+ <name>Airavata Orchestrator Core</name>
+ <url>http://airavata.apache.org/</url>
+
+ <dependencies>
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-log4j12</artifactId>
+ <version>1.7.5</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.airavata</groupId>
+ <artifactId>airavata-registry-cpi</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.airavata</groupId>
+ <artifactId>airavata-registry-core</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.airavata</groupId>
+ <artifactId>airavata-commons</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.airavata</groupId>
+ <artifactId>airavata-gfac-core</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.airavata</groupId>
+ <artifactId>airavata-gfac-client</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.airavata</groupId>
+ <artifactId>airavata-data-models</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.airavata</groupId>
+ <artifactId>airavata-model-utils</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.derby</groupId>
+ <artifactId>derby</artifactId>
+ <version>${derby.version}</version>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.derby</groupId>
+ <artifactId>derbyclient</artifactId>
+ <version>${derby.version}</version>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.derby</groupId>
+ <artifactId>derbynet</artifactId>
+ <version>${derby.version}</version>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.testng</groupId>
+ <artifactId>testng</artifactId>
+ <version>6.1.1</version>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.airavata</groupId>
+ <artifactId>airavata-server-configuration</artifactId>
+ <scope>test</scope>
+ </dependency>
+ </dependencies>
+
+</project>
[2/3] airavata-sandbox git commit: Added files with the design changes
Posted by sm...@apache.org.
http://git-wip-us.apache.org/repos/asf/airavata-sandbox/blob/1932e865/gsoc2016/Design Changes in Airavata/orchestrator/orchestrator-core/src/main/java/org/apache/airavata/orchestrator/core/OrchestratorConfiguration.java
----------------------------------------------------------------------
diff --git a/gsoc2016/Design Changes in Airavata/orchestrator/orchestrator-core/src/main/java/org/apache/airavata/orchestrator/core/OrchestratorConfiguration.java b/gsoc2016/Design Changes in Airavata/orchestrator/orchestrator-core/src/main/java/org/apache/airavata/orchestrator/core/OrchestratorConfiguration.java
new file mode 100644
index 0000000..da357a4
--- /dev/null
+++ b/gsoc2016/Design Changes in Airavata/orchestrator/orchestrator-core/src/main/java/org/apache/airavata/orchestrator/core/OrchestratorConfiguration.java
@@ -0,0 +1,122 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ *
+*/
+package org.apache.airavata.orchestrator.core;
+
+import java.net.URL;
+import java.util.List;
+
+/**
+ * This keeps configuration of orchestrator, mostly this keep static
+ * configuration, this can be accessed through orchestratorContext object
+ */
+public class OrchestratorConfiguration {
+
+ private String newJobSubmitterClass;
+
+ private String hangedJobSubmitterClass;
+
+ private int submitterInterval = 1000;
+
+ private int threadPoolSize = 10;
+
+ private boolean startSubmitter = false;
+
+ private URL brokerURL;
+
+ private boolean embeddedMode;
+
+ private List<String> validatorClasses;
+
+ private boolean enableValidation;
+
+
+ public List<String> getValidatorClasses() {
+ return validatorClasses;
+ }
+
+ public void setValidatorClasses(List<String> validatorClassesIn) {
+ this.validatorClasses = validatorClassesIn;
+ }
+
+ public boolean isEmbeddedMode() {
+ return embeddedMode;
+ }
+
+ public void setEmbeddedMode(boolean embeddedModeIn) {
+ this.embeddedMode = embeddedModeIn;
+ }
+
+ public URL getBrokerURL() {
+ return brokerURL;
+ }
+
+ public void setBrokerURL(URL brokerURLIn) {
+ this.brokerURL = brokerURLIn;
+ }
+
+ public String getNewJobSubmitterClass() {
+ return newJobSubmitterClass;
+ }
+
+ public int getSubmitterInterval() {
+ return submitterInterval;
+ }
+
+ public int getThreadPoolSize() {
+ return threadPoolSize;
+ }
+
+ public void setNewJobSubmitterClass(String newJobSubmitterClassIn) {
+ this.newJobSubmitterClass = newJobSubmitterClassIn;
+ }
+
+ public void setSubmitterInterval(int submitterIntervalIn) {
+ this.submitterInterval = submitterIntervalIn;
+ }
+
+ public void setThreadPoolSize(int threadPoolSizeIn) {
+ this.threadPoolSize = threadPoolSizeIn;
+ }
+
+ public boolean isStartSubmitter() {
+ return startSubmitter;
+ }
+
+ public void setStartSubmitter(boolean startSubmitterIn) {
+ this.startSubmitter = startSubmitterIn;
+ }
+
+ public String getHangedJobSubmitterClass() {
+ return hangedJobSubmitterClass;
+ }
+
+ public void setHangedJobSubmitterClass(String hangedJobSubmitterClassIn) {
+ this.hangedJobSubmitterClass = hangedJobSubmitterClassIn;
+ }
+
+ public boolean isEnableValidation() {
+ return enableValidation;
+ }
+
+ public void setEnableValidation(boolean enableValidationIn) {
+ this.enableValidation = enableValidationIn;
+ }
+}
http://git-wip-us.apache.org/repos/asf/airavata-sandbox/blob/1932e865/gsoc2016/Design Changes in Airavata/orchestrator/orchestrator-core/src/main/java/org/apache/airavata/orchestrator/core/utils/OrchestratorConstants.java
----------------------------------------------------------------------
diff --git a/gsoc2016/Design Changes in Airavata/orchestrator/orchestrator-core/src/main/java/org/apache/airavata/orchestrator/core/utils/OrchestratorConstants.java b/gsoc2016/Design Changes in Airavata/orchestrator/orchestrator-core/src/main/java/org/apache/airavata/orchestrator/core/utils/OrchestratorConstants.java
new file mode 100644
index 0000000..6a0f04b
--- /dev/null
+++ b/gsoc2016/Design Changes in Airavata/orchestrator/orchestrator-core/src/main/java/org/apache/airavata/orchestrator/core/utils/OrchestratorConstants.java
@@ -0,0 +1,77 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ *
+*/
+package org.apache.airavata.orchestrator.core.utils;
+
+/**
+ * This class contains all the constants in orchestrator-core
+ *
+ */
+/*public class OrchestratorConstants {
+ public static final String AIRAVATA_PROPERTIES = "airavata-server.properties";
+ public static final int hotUpdateInterval=1000;
+ public static final String SUBMIT_INTERVAL = "submitter.interval";
+ public static final String THREAD_POOL_SIZE = "threadpool.size";
+ public static final String START_SUBMITTER = "start.submitter";
+ public static final String EMBEDDED_MODE = "embedded.mode";
+ public static final String ENABLE_VALIDATION = "enable.validation";
+ public static final String JOB_VALIDATOR = "job.validators";
+}*/
+
+
+/**
+ * This enum contains all the constants in orchestrator-core
+ enum is the way about dealing with constants as its very powerful.
+ Hence, a design change has been made to change the class to enum.
+ *
+ */
+public enum OrchestratorConstants {
+ AIRAVATA_PROPERTIES("airavata-server.properties"),
+ hotUpdateInterval(1000),
+ SUBMIT_INTERVAL("submitter.interval"),
+ THREAD_POOL_SIZE("threadpool.size"),
+ START_SUBMITTER("start.submitter"),
+ EMBEDDED_MODE("embedded.mode"),
+ ENABLE_VALIDATION("enable.validation"),
+ JOB_VALIDATOR("job.validators");
+
+
+ private String stringConstant;
+ private int integerConstant;
+
+ OrchestratorConstants(String stringConstantIn)
+ {
+ stringConstant = stringConstantIn;
+ }
+ OrchestratorConstants(int integerConstantIn)
+ {
+ integerConstant = integerConstantIn;
+ }
+
+ public String getOrchestratorStringConstant()
+ {
+ return stringConstant;
+ }
+ public int getOrchestratorIntegerConstant()
+ {
+ return integerConstant;
+ }
+
+}
http://git-wip-us.apache.org/repos/asf/airavata-sandbox/blob/1932e865/gsoc2016/Design Changes in Airavata/orchestrator/orchestrator-core/src/main/java/org/apache/airavata/orchestrator/core/utils/OrchestratorUtils.java
----------------------------------------------------------------------
diff --git a/gsoc2016/Design Changes in Airavata/orchestrator/orchestrator-core/src/main/java/org/apache/airavata/orchestrator/core/utils/OrchestratorUtils.java b/gsoc2016/Design Changes in Airavata/orchestrator/orchestrator-core/src/main/java/org/apache/airavata/orchestrator/core/utils/OrchestratorUtils.java
new file mode 100644
index 0000000..3ee40e0
--- /dev/null
+++ b/gsoc2016/Design Changes in Airavata/orchestrator/orchestrator-core/src/main/java/org/apache/airavata/orchestrator/core/utils/OrchestratorUtils.java
@@ -0,0 +1,327 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ *
+*/
+package org.apache.airavata.orchestrator.core.utils;
+
+import java.io.IOException;
+import java.util.*;
+
+import org.apache.airavata.common.exception.ApplicationSettingsException;
+import org.apache.airavata.common.utils.ServerSettings;
+import org.apache.airavata.model.appcatalog.appinterface.ApplicationInterfaceDescription;
+import org.apache.airavata.model.appcatalog.computeresource.*;
+import org.apache.airavata.model.appcatalog.computeresource.JobSubmissionInterface;
+import org.apache.airavata.model.appcatalog.computeresource.UnicoreJobSubmission;
+import org.apache.airavata.model.appcatalog.gatewayprofile.ComputeResourcePreference;
+import org.apache.airavata.model.appcatalog.gatewayprofile.StoragePreference;
+import org.apache.airavata.model.data.movement.DataMovementInterface;
+import org.apache.airavata.model.data.movement.DataMovementProtocol;
+import org.apache.airavata.model.data.movement.SCPDataMovement;
+import org.apache.airavata.model.data.movement.SecurityProtocol;
+import org.apache.airavata.model.process.ProcessModel;
+import org.apache.airavata.orchestrator.core.OrchestratorConfiguration;
+import org.apache.airavata.orchestrator.core.context.OrchestratorContext;
+import org.apache.airavata.orchestrator.core.exception.OrchestratorException;
+import org.apache.airavata.registry.cpi.*;
+import org.apache.airavata.registry.cpi.ApplicationInterface;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * This contains orchestrator specific utilities
+ */
+public class OrchestratorUtils {
+
+
+ private final static Logger logger = LoggerFactory.getLogger(OrchestratorUtils.class);
+
+ public static String OrchestratorStringConstant(OrchestratorConstants constant)
+ {
+ return constant.getOrchestratorStringConstant();
+ }
+
+ public static int OrchestratorIntegerConstant(OrchestratorConstants constant)
+ {
+ return constant.getOrchestratorIntegerConstant();
+ }
+
+ public static OrchestratorConfiguration loadOrchestratorConfiguration() throws OrchestratorException, IOException, NumberFormatException, ApplicationSettingsException {
+ OrchestratorConfiguration orchestratorConfiguration = new OrchestratorConfiguration();
+ orchestratorConfiguration.setSubmitterInterval(Integer.parseInt((String) ServerSettings.getSetting(OrchestratorStringConstant(OrchestratorConstants.SUBMIT_INTERVAL))));
+ orchestratorConfiguration.setThreadPoolSize(Integer.parseInt((String) ServerSettings.getSetting(OrchestratorStringConstant(OrchestratorConstants.THREAD_POOL_SIZE))));
+ orchestratorConfiguration.setStartSubmitter(Boolean.valueOf(ServerSettings.getSetting(OrchestratorStringConstant(OrchestratorConstants.START_SUBMITTER))));
+ orchestratorConfiguration.setEmbeddedMode(Boolean.valueOf(ServerSettings.getSetting(OrchestratorStringConstant(OrchestratorConstants.EMBEDDED_MODE))));
+ orchestratorConfiguration.setEnableValidation(Boolean.valueOf(ServerSettings.getSetting(OrchestratorStringConstant(OrchestratorConstants.ENABLE_VALIDATION))));
+ if (orchestratorConfiguration.isEnableValidation()) {
+ orchestratorConfiguration.setValidatorClasses(Arrays.asList(ServerSettings.getSetting(OrchestratorStringConstant(OrchestratorConstants.JOB_VALIDATOR)).split(",")));
+ }
+ return orchestratorConfiguration;
+ }
+
+ public static JobSubmissionProtocol getPreferredJobSubmissionProtocol(OrchestratorContext context, ProcessModel model, String gatewayId) throws RegistryException {
+ try {
+ GwyResourceProfile gatewayProfile = context.getRegistry().getAppCatalog().getGatewayProfile();
+ String resourceHostId = model.getComputeResourceId();
+ ComputeResourcePreference preference = gatewayProfile.getComputeResourcePreference(gatewayId
+ , resourceHostId);
+ return preference.getPreferredJobSubmissionProtocol();
+ } catch (AppCatalogException e) {
+ logger.error("Error occurred while initializing app catalog", e);
+ throw new RegistryException("Error occurred while initializing app catalog", e);
+ }
+ }
+
+ public static String getApplicationInterfaceName(OrchestratorContext context, ProcessModel model) throws RegistryException {
+ try {
+ ApplicationInterface applicationInterface = context.getRegistry().getAppCatalog().getApplicationInterface();
+ ApplicationInterfaceDescription appInterface = applicationInterface.getApplicationInterface(model.getApplicationInterfaceId());
+ return appInterface.getApplicationName();
+ } catch (AppCatalogException e) {
+ throw new RegistryException("Error while retrieving application interface", e);
+ }
+ }
+
+ public static DataMovementProtocol getPreferredDataMovementProtocol(OrchestratorContext context, ProcessModel model, String gatewayId) throws RegistryException {
+ try {
+ GwyResourceProfile gatewayProfile = context.getRegistry().getAppCatalog().getGatewayProfile();
+ String resourceHostId = model.getComputeResourceId();
+ ComputeResourcePreference preference = gatewayProfile.getComputeResourcePreference(gatewayId
+ , resourceHostId);
+ return preference.getPreferredDataMovementProtocol();
+ } catch (AppCatalogException e) {
+ logger.error("Error occurred while initializing app catalog", e);
+ throw new RegistryException("Error occurred while initializing app catalog", e);
+ }
+ }
+
+ public static ComputeResourcePreference getComputeResourcePreference(OrchestratorContext context, ProcessModel processModel, String gatewayId) throws RegistryException {
+ try {
+ GwyResourceProfile gatewayProfile = context.getRegistry().getAppCatalog().getGatewayProfile();
+ String resourceHostId = processModel.getComputeResourceId();
+ return gatewayProfile.getComputeResourcePreference(gatewayId, resourceHostId);
+ } catch (AppCatalogException e) {
+ logger.error("Error occurred while initializing app catalog", e);
+ throw new RegistryException("Error occurred while initializing app catalog", e);
+ }
+ }
+
+ public static StoragePreference getStoragePreference(OrchestratorContext context, ProcessModel processModel, String gatewayId) throws RegistryException {
+ try {
+ GwyResourceProfile gatewayProfile = context.getRegistry().getAppCatalog().getGatewayProfile();
+ String resourceHostId = processModel.getComputeResourceId();
+ return gatewayProfile.getStoragePreference(gatewayId, resourceHostId);
+ } catch (AppCatalogException e) {
+ logger.error("Error occurred while initializing app catalog", e);
+ throw new RegistryException("Error occurred while initializing app catalog", e);
+ }
+ }
+
+ public static String getLoginUserName(OrchestratorContext context, ProcessModel processModel, String gatewayId) throws RegistryException {
+ try {
+ String loginUserName = null;
+ String overrideLoginUserName = processModel.getResourceSchedule().getOverrideLoginUserName();
+ if (overrideLoginUserName != null && !overrideLoginUserName.equals("")) {
+ loginUserName = overrideLoginUserName;
+ } else {
+ GwyResourceProfile gatewayProfile = context.getRegistry().getAppCatalog().getGatewayProfile();
+ loginUserName = gatewayProfile.getComputeResourcePreference(gatewayId, processModel.getComputeResourceId()).getLoginUserName();
+ }
+ return loginUserName;
+ } catch (AppCatalogException e) {
+ logger.error("Error occurred while initializing app catalog to fetch login username", e);
+ throw new RegistryException("Error occurred while initializing app catalog to fetch login username", e);
+ }
+ }
+
+ public static String getScratchLocation(OrchestratorContext context, ProcessModel processModel, String gatewayId) throws RegistryException {
+ try {
+ String scratchLocation = null;
+ String overrideScratchLocation = processModel.getResourceSchedule().getOverrideScratchLocation();
+ if (overrideScratchLocation != null && !overrideScratchLocation.equals("")) {
+ scratchLocation = overrideScratchLocation;
+ } else {
+ GwyResourceProfile gatewayProfile = context.getRegistry().getAppCatalog().getGatewayProfile();
+ scratchLocation = gatewayProfile.getComputeResourcePreference(gatewayId, processModel.getComputeResourceId()).getScratchLocation();
+ }
+ return scratchLocation;
+ } catch (AppCatalogException e) {
+ logger.error("Error occurred while initializing app catalog to fetch scratch location", e);
+ throw new RegistryException("Error occurred while initializing app catalog to fetch scratch location", e);
+ }
+ }
+
+ public static JobSubmissionInterface getPreferredJobSubmissionInterface(OrchestratorContext context, ProcessModel processModel, String gatewayId) throws RegistryException {
+ try {
+ String resourceHostId = processModel.getComputeResourceId();
+ ComputeResourcePreference resourcePreference = getComputeResourcePreference(context, processModel, gatewayId);
+ JobSubmissionProtocol preferredJobSubmissionProtocol = resourcePreference.getPreferredJobSubmissionProtocol();
+ ComputeResourceDescription resourceDescription = context.getRegistry().getAppCatalog().getComputeResource().getComputeResource(resourceHostId);
+ List<JobSubmissionInterface> jobSubmissionInterfaces = resourceDescription.getJobSubmissionInterfaces();
+ Map<JobSubmissionProtocol, List<JobSubmissionInterface>> orderedInterfaces = new HashMap<>();
+ List<JobSubmissionInterface> interfaces = new ArrayList<>();
+ if (jobSubmissionInterfaces != null && !jobSubmissionInterfaces.isEmpty()) {
+ for (JobSubmissionInterface submissionInterface : jobSubmissionInterfaces){
+
+ if (preferredJobSubmissionProtocol != null){
+ if (preferredJobSubmissionProtocol.toString().equals(submissionInterface.getJobSubmissionProtocol().toString())){
+ if (orderedInterfaces.containsKey(submissionInterface.getJobSubmissionProtocol())){
+ List<JobSubmissionInterface> interfaceList = orderedInterfaces.get(submissionInterface.getJobSubmissionProtocol());
+ interfaceList.add(submissionInterface);
+ }else {
+ interfaces.add(submissionInterface);
+ orderedInterfaces.put(submissionInterface.getJobSubmissionProtocol(), interfaces);
+ }
+ }
+ }else {
+ Collections.sort(jobSubmissionInterfaces, new Comparator<JobSubmissionInterface>() {
+ @Override
+ public int compare(JobSubmissionInterface jobSubmissionInterface, JobSubmissionInterface jobSubmissionInterface2) {
+ return jobSubmissionInterface.getPriorityOrder() - jobSubmissionInterface2.getPriorityOrder();
+ }
+ });
+ }
+ }
+ interfaces = orderedInterfaces.get(preferredJobSubmissionProtocol);
+ Collections.sort(interfaces, new Comparator<JobSubmissionInterface>() {
+ @Override
+ public int compare(JobSubmissionInterface jobSubmissionInterface, JobSubmissionInterface jobSubmissionInterface2) {
+ return jobSubmissionInterface.getPriorityOrder() - jobSubmissionInterface2.getPriorityOrder();
+ }
+ });
+ } else {
+ throw new RegistryException("Compute resource should have at least one job submission interface defined...");
+ }
+ return interfaces.get(0);
+ } catch (AppCatalogException e) {
+ throw new RegistryException("Error occurred while retrieving data from app catalog", e);
+ }
+ }
+
+ public static DataMovementInterface getPrefferredDataMovementInterface(OrchestratorContext context, ProcessModel processModel, String gatewayId) throws RegistryException {
+ try {
+ String resourceHostId = processModel.getComputeResourceId();
+ ComputeResourcePreference resourcePreference = getComputeResourcePreference(context, processModel, gatewayId);
+ DataMovementProtocol preferredDataMovementProtocol = resourcePreference.getPreferredDataMovementProtocol();
+ ComputeResourceDescription resourceDescription = context.getRegistry().getAppCatalog().getComputeResource().getComputeResource(resourceHostId);
+ List<DataMovementInterface> dataMovementInterfaces = resourceDescription.getDataMovementInterfaces();
+ if (dataMovementInterfaces != null && !dataMovementInterfaces.isEmpty()) {
+ for (DataMovementInterface dataMovementInterface : dataMovementInterfaces){
+ if (preferredDataMovementProtocol != null){
+ if (preferredDataMovementProtocol.toString().equals(dataMovementInterface.getDataMovementProtocol().toString())){
+ return dataMovementInterface;
+ }
+ }
+ }
+ } else {
+ throw new RegistryException("Compute resource should have at least one data movement interface defined...");
+ }
+ } catch (AppCatalogException e) {
+ throw new RegistryException("Error occurred while retrieving data from app catalog", e);
+ }
+ return null;
+ }
+
+ public static int getDataMovementPort(OrchestratorContext context, ProcessModel processModel, String gatewayId) throws RegistryException{
+ try {
+ DataMovementProtocol protocol = getPreferredDataMovementProtocol(context, processModel, gatewayId);
+ DataMovementInterface dataMovementInterface = getPrefferredDataMovementInterface(context, processModel, gatewayId);
+ if (protocol == DataMovementProtocol.SCP ) {
+ SCPDataMovement scpDataMovement = getSCPDataMovement(context, dataMovementInterface.getDataMovementInterfaceId());
+ if (scpDataMovement != null) {
+ return scpDataMovement.getSshPort();
+ }
+ }
+ } catch (RegistryException e) {
+ logger.error("Error occurred while retrieving security protocol", e);
+ }
+ return 0;
+ }
+
+
+ public static SecurityProtocol getSecurityProtocol(OrchestratorContext context, ProcessModel processModel, String gatewayId) throws RegistryException{
+ try {
+ JobSubmissionProtocol submissionProtocol = getPreferredJobSubmissionProtocol(context, processModel, gatewayId);
+ JobSubmissionInterface jobSubmissionInterface = getPreferredJobSubmissionInterface(context, processModel, gatewayId);
+ if (submissionProtocol == JobSubmissionProtocol.SSH ) {
+ SSHJobSubmission sshJobSubmission = getSSHJobSubmission(context, jobSubmissionInterface.getJobSubmissionInterfaceId());
+ if (sshJobSubmission != null) {
+ return sshJobSubmission.getSecurityProtocol();
+ }
+ } else if (submissionProtocol == JobSubmissionProtocol.LOCAL) {
+ LOCALSubmission localJobSubmission = getLocalJobSubmission(context, jobSubmissionInterface.getJobSubmissionInterfaceId());
+ if (localJobSubmission != null) {
+ return localJobSubmission.getSecurityProtocol();
+ }
+ } else if (submissionProtocol == JobSubmissionProtocol.SSH_FORK){
+ SSHJobSubmission sshJobSubmission = getSSHJobSubmission(context, jobSubmissionInterface.getJobSubmissionInterfaceId());
+ if (sshJobSubmission != null) {
+ return sshJobSubmission.getSecurityProtocol();
+ }
+ }
+ } catch (RegistryException e) {
+ logger.error("Error occurred while retrieving security protocol", e);
+ }
+ return null;
+ }
+
+ public static LOCALSubmission getLocalJobSubmission(OrchestratorContext context, String submissionId) throws RegistryException {
+ try {
+ AppCatalog appCatalog = context.getRegistry().getAppCatalog();
+ return appCatalog.getComputeResource().getLocalJobSubmission(submissionId);
+ } catch (Exception e) {
+ String errorMsg = "Error while retrieving local job submission with submission id : " + submissionId;
+ logger.error(errorMsg, e);
+ throw new RegistryException(errorMsg, e);
+ }
+ }
+
+ public static UnicoreJobSubmission getUnicoreJobSubmission(OrchestratorContext context, String submissionId) throws RegistryException {
+ try {
+ AppCatalog appCatalog = context.getRegistry().getAppCatalog();
+ return appCatalog.getComputeResource().getUNICOREJobSubmission(submissionId);
+ } catch (Exception e) {
+ String errorMsg = "Error while retrieving UNICORE job submission with submission id : " + submissionId;
+ logger.error(errorMsg, e);
+ throw new RegistryException(errorMsg, e);
+ }
+ }
+
+ public static SSHJobSubmission getSSHJobSubmission(OrchestratorContext context, String submissionId) throws RegistryException {
+ try {
+ AppCatalog appCatalog = context.getRegistry().getAppCatalog();
+ return appCatalog.getComputeResource().getSSHJobSubmission(submissionId);
+ } catch (Exception e) {
+ String errorMsg = "Error while retrieving SSH job submission with submission id : " + submissionId;
+ logger.error(errorMsg, e);
+ throw new RegistryException(errorMsg, e);
+ }
+ }
+
+ public static SCPDataMovement getSCPDataMovement(OrchestratorContext context, String dataMoveId) throws RegistryException {
+ try {
+ AppCatalog appCatalog = context.getRegistry().getAppCatalog();
+ return appCatalog.getComputeResource().getSCPDataMovement(dataMoveId);
+ } catch (Exception e) {
+ String errorMsg = "Error while retrieving SCP Data movement with submission id : " + dataMoveId;
+ logger.error(errorMsg, e);
+ throw new RegistryException(errorMsg, e);
+ }
+ }
+}
http://git-wip-us.apache.org/repos/asf/airavata-sandbox/blob/1932e865/gsoc2016/Design Changes in Airavata/orchestrator/orchestrator-core/src/main/java/org/apache/airavata/orchestrator/cpi/impl/SimpleOrchestratorImpl.java
----------------------------------------------------------------------
diff --git a/gsoc2016/Design Changes in Airavata/orchestrator/orchestrator-core/src/main/java/org/apache/airavata/orchestrator/cpi/impl/SimpleOrchestratorImpl.java b/gsoc2016/Design Changes in Airavata/orchestrator/orchestrator-core/src/main/java/org/apache/airavata/orchestrator/cpi/impl/SimpleOrchestratorImpl.java
new file mode 100644
index 0000000..0319f27
--- /dev/null
+++ b/gsoc2016/Design Changes in Airavata/orchestrator/orchestrator-core/src/main/java/org/apache/airavata/orchestrator/cpi/impl/SimpleOrchestratorImpl.java
@@ -0,0 +1,625 @@
+/*
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ *
+*/
+package org.apache.airavata.orchestrator.cpi.impl;
+
+import org.apache.airavata.common.utils.AiravataUtils;
+import org.apache.airavata.common.utils.ThriftUtils;
+import org.apache.airavata.gfac.core.task.TaskException;
+import org.apache.airavata.model.appcatalog.appinterface.ApplicationInterfaceDescription;
+import org.apache.airavata.model.appcatalog.computeresource.*;
+import org.apache.airavata.model.appcatalog.gatewayprofile.ComputeResourcePreference;
+import org.apache.airavata.model.application.io.DataType;
+import org.apache.airavata.model.application.io.InputDataObjectType;
+import org.apache.airavata.model.application.io.OutputDataObjectType;
+import org.apache.airavata.model.commons.ErrorModel;
+import org.apache.airavata.model.data.movement.DataMovementProtocol;
+import org.apache.airavata.model.error.LaunchValidationException;
+import org.apache.airavata.model.error.ValidationResults;
+import org.apache.airavata.model.error.ValidatorResult;
+import org.apache.airavata.model.process.ProcessModel;
+import org.apache.airavata.model.experiment.*;
+import org.apache.airavata.model.scheduling.ComputationalResourceSchedulingModel;
+import org.apache.airavata.model.status.TaskState;
+import org.apache.airavata.model.status.TaskStatus;
+import org.apache.airavata.model.task.*;
+import org.apache.airavata.model.util.ExperimentModelUtil;
+import org.apache.airavata.orchestrator.core.exception.OrchestratorException;
+import org.apache.airavata.orchestrator.core.impl.GFACPassiveJobSubmitter;
+import org.apache.airavata.orchestrator.core.job.JobSubmitter;
+import org.apache.airavata.orchestrator.core.utils.OrchestratorUtils;
+import org.apache.airavata.orchestrator.core.validator.JobMetadataValidator;
+import org.apache.airavata.registry.core.experiment.catalog.impl.RegistryFactory;
+import org.apache.airavata.registry.cpi.*;
+import org.apache.airavata.registry.cpi.utils.Constants;
+import org.apache.thrift.TException;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.util.*;
+import java.util.concurrent.ExecutorService;
+
+public class SimpleOrchestratorImpl extends AbstractOrchestrator{
+ private final static Logger logger = LoggerFactory.getLogger(SimpleOrchestratorImpl.class);
+ private ExecutorService executor;
+
+ // this is going to be null unless the thread count is 0
+ private JobSubmitter jobSubmitter = null;
+
+
+ public SimpleOrchestratorImpl() throws OrchestratorException {
+ try {
+ try {
+ // We are only going to use GFacPassiveJobSubmitter
+ jobSubmitter = new GFACPassiveJobSubmitter();
+ jobSubmitter.initialize(this.orchestratorContext);
+
+ } catch (Exception e) {
+ String error = "Error creating JobSubmitter in non threaded mode ";
+ logger.error(error);
+ throw new OrchestratorException(error, e);
+ }
+ } catch (OrchestratorException e) {
+ logger.error("Error Constructing the Orchestrator");
+ throw e;
+ }
+ }
+
+ public boolean launchProcess(ProcessModel processModel, String tokenId) throws OrchestratorException {
+ try {
+ return jobSubmitter.submit(processModel.getExperimentId(), processModel.getProcessId(), tokenId);
+ } catch (Exception e) {
+ throw new OrchestratorException("Error launching the job", e);
+ }
+ }
+
+ public ValidationResults validateExperiment(ExperimentModel experiment) throws OrchestratorException,LaunchValidationException {
+ org.apache.airavata.model.error.ValidationResults validationResults = new org.apache.airavata.model.error.ValidationResults();
+ validationResults.setValidationState(true); // initially making it to success, if atleast one failed them simply mark it failed.
+ String errorMsg = "Validation Errors : ";
+ if (this.orchestratorConfiguration.isEnableValidation()) {
+ List<String> validatorClasses = this.orchestratorContext.getOrchestratorConfiguration().getValidatorClasses();
+ for (String validator : validatorClasses) {
+ try {
+ Class<? extends JobMetadataValidator> vClass = Class.forName(validator.trim()).asSubclass(JobMetadataValidator.class);
+ JobMetadataValidator jobMetadataValidator = vClass.newInstance();
+ validationResults = jobMetadataValidator.validate(experiment, null);
+ if (validationResults.isValidationState()) {
+ logger.info("Validation of " + validator + " is SUCCESSFUL");
+ } else {
+ List<ValidatorResult> validationResultList = validationResults.getValidationResultList();
+ for (ValidatorResult result : validationResultList){
+ if (!result.isResult()){
+ String validationError = result.getErrorDetails();
+ if (validationError != null){
+ errorMsg += validationError + " ";
+ }
+ }
+ }
+ logger.error("Validation of " + validator + " for experiment Id " + experiment.getExperimentId() + " is FAILED:[error]. " + errorMsg);
+ validationResults.setValidationState(false);
+ try {
+ ErrorModel details = new ErrorModel();
+ details.setActualErrorMessage(errorMsg);
+ details.setCreationTime(Calendar.getInstance().getTimeInMillis());
+ orchestratorContext.getRegistry().getExperimentCatalog().add(ExpCatChildDataType.EXPERIMENT_ERROR, details,
+ experiment.getExperimentId());
+ } catch (RegistryException e) {
+ logger.error("Error while saving error details to registry", e);
+ }
+ break;
+ }
+ } catch (ClassNotFoundException | InstantiationException | IllegalAccessException e) {
+ logger.error("Error loading the validation class: ", validator, e);
+ validationResults.setValidationState(false);
+ } /*catch (InstantiationException e) {
+ logger.error("Error loading the validation class: ", validator, e);
+ validationResults.setValidationState(false);
+ } catch (IllegalAccessException e) {
+ logger.error("Error loading the validation class: ", validator, e);
+ validationResults.setValidationState(false);
+ }*/
+ }
+ }
+ if(validationResults.isValidationState()){
+ return validationResults;
+ }else {
+ //atleast one validation has failed, so we throw an exception
+ LaunchValidationException launchValidationException = new LaunchValidationException();
+ launchValidationException.setValidationResult(validationResults);
+ launchValidationException.setErrorMessage("Validation failed refer the validationResults list for detail error. Validation errors : " + errorMsg);
+ throw launchValidationException;
+ }
+ }
+
+ public ValidationResults validateProcess(ExperimentModel experiment, ProcessModel processModel) throws OrchestratorException,LaunchValidationException {
+ org.apache.airavata.model.error.ValidationResults validationResults = new org.apache.airavata.model.error.ValidationResults();
+ validationResults.setValidationState(true); // initially making it to success, if atleast one failed them simply mark it failed.
+ String errorMsg = "Validation Errors : ";
+ if (this.orchestratorConfiguration.isEnableValidation()) {
+ List<String> validatorClzzez = this.orchestratorContext.getOrchestratorConfiguration().getValidatorClasses();
+ for (String validator : validatorClzzez) {
+ try {
+ Class<? extends JobMetadataValidator> vClass = Class.forName(validator.trim()).asSubclass(JobMetadataValidator.class);
+ JobMetadataValidator jobMetadataValidator = vClass.newInstance();
+ validationResults = jobMetadataValidator.validate(experiment, processModel);
+ if (validationResults.isValidationState()) {
+ logger.info("Validation of " + validator + " is SUCCESSFUL");
+ } else {
+ List<ValidatorResult> validationResultList = validationResults.getValidationResultList();
+ for (ValidatorResult result : validationResultList){
+ if (!result.isResult()){
+ String validationError = result.getErrorDetails();
+ if (validationError != null){
+ errorMsg += validationError + " ";
+ }
+ }
+ }
+ logger.error("Validation of " + validator + " for experiment Id " + experiment.getExperimentId() + " is FAILED:[error]. " + errorMsg);
+ validationResults.setValidationState(false);
+ try {
+ ErrorModel details = new ErrorModel();
+ details.setActualErrorMessage(errorMsg);
+ details.setCreationTime(Calendar.getInstance().getTimeInMillis());
+ orchestratorContext.getRegistry().getExperimentCatalog().add(ExpCatChildDataType.PROCESS_ERROR, details,
+ processModel.getProcessId());
+ } catch (RegistryException e) {
+ logger.error("Error while saving error details to registry", e);
+ }
+ break;
+ }
+ } catch (ClassNotFoundException | InstantiationException | IllegalAccessException e) {
+ logger.error("Error loading the validation class: ", validator, e);
+ validationResults.setValidationState(false);
+ } /*catch (InstantiationException e) {
+ logger.error("Error loading the validation class: ", validator, e);
+ validationResults.setValidationState(false);
+ } catch (IllegalAccessException e) {
+ logger.error("Error loading the validation class: ", validator, e);
+ validationResults.setValidationState(false);
+ }*/
+ }
+ }
+ if(validationResults.isValidationState()){
+ return validationResults;
+ }else {
+ //atleast one validation has failed, so we throw an exception
+ LaunchValidationException launchValidationException = new LaunchValidationException();
+ launchValidationException.setValidationResult(validationResults);
+ launchValidationException.setErrorMessage("Validation failed refer the validationResults list for detail error. Validation errors : " + errorMsg);
+ throw launchValidationException;
+ }
+ }
+
+
+ public void cancelExperiment(ExperimentModel experiment, ProcessModel processModel, String tokenId)
+ throws OrchestratorException {
+ // FIXME
+// List<JobDetails> jobDetailsList = task.getJobDetailsList();
+// for(JobDetails jobDetails:jobDetailsList) {
+// JobState jobState = jobDetails.getJobStatus().getJobState();
+// if (jobState.getValue() > 4){
+// logger.error("Cannot cancel the job, because current job state is : " + jobState.toString() +
+// "jobId: " + jobDetails.getJobID() + " Job Name: " + jobDetails.getJobName());
+// return;
+// }
+// }
+// jobSubmitter.terminate(experiment.getExperimentID(),task.getTaskID(),tokenId);
+ }
+
+
+ public ExecutorService getExecutor() {
+ return executor;
+ }
+
+ public void setExecutor(ExecutorService executorIn) {
+ this.executor = executorIn;
+ }
+
+ public JobSubmitter getJobSubmitter() {
+ return jobSubmitter;
+ }
+
+ public void setJobSubmitter(JobSubmitter jobSubmitterIn) {
+ this.jobSubmitter = jobSubmitterIn;
+ }
+
+ public void initialize() throws OrchestratorException {
+
+ }
+
+ public List<ProcessModel> createProcesses (String experimentId, String gatewayId) throws OrchestratorException {
+ List<ProcessModel> processModels = new ArrayList<ProcessModel>();
+ try {
+ Registry registry = orchestratorContext.getRegistry();
+ ExperimentModel experimentModel = (ExperimentModel)registry.getExperimentCatalog().get(ExperimentCatalogModelType.EXPERIMENT, experimentId);
+ List<Object> processList = registry.getExperimentCatalog().get(ExperimentCatalogModelType.PROCESS, Constants.FieldConstants.ExperimentConstants.EXPERIMENT_ID, experimentId);
+ if (processList != null && !processList.isEmpty()) {
+ for (Object processObject : processList) {
+ ProcessModel processModel = (ProcessModel)processObject;
+ processModels.add(processModel);
+ }
+ }else {
+ ProcessModel processModel = ExperimentModelUtil.cloneProcessFromExperiment(experimentModel);
+ String processId = (String)registry.getExperimentCatalog().add(ExpCatChildDataType.PROCESS, processModel, experimentId);
+ processModel.setProcessId(processId);
+ processModels.add(processModel);
+ }
+ } catch (Exception e) {
+ throw new OrchestratorException("Error during creating process");
+ }
+ return processModels;
+ }
+
+ public String createAndSaveTasks(String gatewayId, ProcessModel processModel, boolean autoSchedule) throws OrchestratorException {
+ try {
+ ExperimentCatalog experimentCatalog = orchestratorContext.getRegistry().getExperimentCatalog();
+ AppCatalog appCatalog = orchestratorContext.getRegistry().getAppCatalog();
+ ComputationalResourceSchedulingModel resourceSchedule = processModel.getResourceSchedule();
+ String userGivenQueueName = resourceSchedule.getQueueName();
+ int userGivenWallTime = resourceSchedule.getWallTimeLimit();
+ String resourceHostId = resourceSchedule.getResourceHostId();
+ if (resourceHostId == null){
+ throw new OrchestratorException("Compute Resource Id cannot be null at this point");
+ }
+ ComputeResourceDescription computeResource = appCatalog.getComputeResource().getComputeResource(resourceHostId);
+ JobSubmissionInterface preferredJobSubmissionInterface = OrchestratorUtils.getPreferredJobSubmissionInterface(orchestratorContext, processModel, gatewayId);
+ ComputeResourcePreference resourcePreference = OrchestratorUtils.getComputeResourcePreference(orchestratorContext, processModel, gatewayId);
+ List<String> taskIdList = new ArrayList<>();
+
+ if (resourcePreference.getPreferredJobSubmissionProtocol() == JobSubmissionProtocol.UNICORE) {
+ // TODO - breakdown unicore all in one task to multiple tasks, then we don't need to handle UNICORE here.
+ taskIdList.addAll(createAndSaveSubmissionTasks(gatewayId, preferredJobSubmissionInterface, processModel, userGivenWallTime));
+ } else {
+ taskIdList.addAll(createAndSaveEnvSetupTask(gatewayId, processModel, experimentCatalog));
+ taskIdList.addAll(createAndSaveInputDataStagingTasks(processModel, gatewayId));
+ if (autoSchedule) {
+ List<BatchQueue> definedBatchQueues = computeResource.getBatchQueues();
+ for (BatchQueue batchQueue : definedBatchQueues) {
+ if (batchQueue.getQueueName().equals(userGivenQueueName)) {
+ int maxRunTime = batchQueue.getMaxRunTime();
+ if (maxRunTime < userGivenWallTime) {
+ resourceSchedule.setWallTimeLimit(maxRunTime);
+ // need to create more job submissions
+ int numOfMaxWallTimeJobs = ((int) Math.floor(userGivenWallTime / maxRunTime));
+ for (int i = 1; i <= numOfMaxWallTimeJobs; i++) {
+ taskIdList.addAll(createAndSaveSubmissionTasks(gatewayId,preferredJobSubmissionInterface, processModel, maxRunTime));
+ }
+ int leftWallTime = userGivenWallTime % maxRunTime;
+ if (leftWallTime != 0) {
+ taskIdList.addAll(createAndSaveSubmissionTasks(gatewayId,preferredJobSubmissionInterface, processModel, leftWallTime));
+ }
+ } else {
+ taskIdList.addAll(createAndSaveSubmissionTasks(gatewayId,preferredJobSubmissionInterface, processModel, userGivenWallTime));
+ }
+ }
+ }
+ } else {
+ taskIdList.addAll(createAndSaveSubmissionTasks(gatewayId,preferredJobSubmissionInterface, processModel, userGivenWallTime));
+ }
+ taskIdList.addAll(createAndSaveOutputDataStagingTasks(processModel, gatewayId));
+ }
+ // update process scheduling
+ experimentCatalog.update(ExperimentCatalogModelType.PROCESS, processModel, processModel.getProcessId());
+ return getTaskDag(taskIdList);
+ } catch (Exception e) {
+ throw new OrchestratorException("Error during creating process");
+ }
+ }
+
+ private String getTaskDag(List<String> taskIdList) {
+ if (taskIdList.isEmpty()) {
+ return "";
+ }
+ StringBuilder sb = new StringBuilder();
+ for (String s : taskIdList) {
+ sb.append(s).append(","); // comma separated values
+ }
+ String dag = sb.toString();
+ return dag.substring(0, dag.length() - 1); // remove last comma
+ }
+
+ private List<String> createAndSaveEnvSetupTask(String gatewayId,
+ ProcessModel processModel,
+ ExperimentCatalog experimentCatalog)
+ throws RegistryException, TException {
+ List<String> envTaskIds = new ArrayList<>();
+ TaskModel envSetupTask = new TaskModel();
+ envSetupTask.setTaskType(TaskTypes.ENV_SETUP);
+ envSetupTask.setTaskStatus(new TaskStatus(TaskState.CREATED));
+ envSetupTask.setCreationTime(AiravataUtils.getCurrentTimestamp().getTime());
+ envSetupTask.setParentProcessId(processModel.getProcessId());
+ EnvironmentSetupTaskModel envSetupSubModel = new EnvironmentSetupTaskModel();
+ envSetupSubModel.setProtocol(OrchestratorUtils.getSecurityProtocol(orchestratorContext, processModel, gatewayId));
+ ComputeResourcePreference computeResourcePreference = OrchestratorUtils.getComputeResourcePreference(orchestratorContext, processModel, gatewayId);
+ String scratchLocation = OrchestratorUtils.getScratchLocation(orchestratorContext,processModel, gatewayId);
+ String workingDir = scratchLocation + File.separator + processModel.getProcessId();
+ envSetupSubModel.setLocation(workingDir);
+ byte[] envSetupSub = ThriftUtils.serializeThriftObject(envSetupSubModel);
+ envSetupTask.setSubTaskModel(envSetupSub);
+ String envSetupTaskId = (String) experimentCatalog.add(ExpCatChildDataType.TASK, envSetupTask, processModel.getProcessId());
+ envSetupTask.setTaskId(envSetupTaskId);
+ envTaskIds.add(envSetupTaskId);
+ return envTaskIds;
+ }
+
+ public List<String> createAndSaveInputDataStagingTasks(ProcessModel processModel, String gatewayId) throws RegistryException {
+ List<String> dataStagingTaskIds = new ArrayList<>();
+ List<InputDataObjectType> processInputs = processModel.getProcessInputs();
+
+ sortByInputOrder(processInputs);
+ if (processInputs != null) {
+ for (InputDataObjectType processInput : processInputs) {
+ DataType type = processInput.getType();
+ switch (type) {
+ case STDERR:
+ break;
+ case STDOUT:
+ break;
+ case URI:
+ case URI_COLLECTION:
+ try {
+ TaskModel inputDataStagingTask = getInputDataStagingTask(processModel, processInput, gatewayId);
+ String taskId = (String) orchestratorContext.getRegistry().getExperimentCatalog().add(ExpCatChildDataType.TASK, inputDataStagingTask,
+ processModel.getProcessId());
+ inputDataStagingTask.setTaskId(taskId);
+ dataStagingTaskIds.add(inputDataStagingTask.getTaskId());
+ } catch (TException | AppCatalogException | TaskException e) {
+ throw new RegistryException("Error while serializing data staging sub task model");
+ }
+ break;
+ default:
+ // nothing to do
+ break;
+ }
+ }
+ }
+ return dataStagingTaskIds;
+ }
+
+ public List<String> createAndSaveOutputDataStagingTasks(ProcessModel processModel, String gatewayId) throws RegistryException {
+ List<String> dataStagingTaskIds = new ArrayList<>();
+ List<OutputDataObjectType> processOutputs = processModel.getProcessOutputs();
+ String appName = OrchestratorUtils.getApplicationInterfaceName(orchestratorContext, processModel);
+ if (processOutputs != null) {
+ for (OutputDataObjectType processOutput : processOutputs) {
+ DataType type = processOutput.getType();
+ switch (type) {
+ case STDOUT :
+ processOutput.setValue(appName + ".stdout");
+ createOutputDataSatagingTasks(processModel, gatewayId, dataStagingTaskIds, processOutput);
+ break;
+ case STDERR:
+ processOutput.setValue(appName + ".stderr");
+ createOutputDataSatagingTasks(processModel, gatewayId, dataStagingTaskIds, processOutput);
+ break;
+ case URI:
+ createOutputDataSatagingTasks(processModel, gatewayId, dataStagingTaskIds, processOutput);
+ break;
+ default:
+ // nothing to do
+ break;
+ }
+ }
+ }
+
+ try {
+ if (isArchive(processModel, gatewayId)) {
+ createArchiveDataStatgingTask(processModel, gatewayId, dataStagingTaskIds);
+ }
+ } catch (AppCatalogException e) {
+ throw new RegistryException("Error! Application interface retrieval failed");
+ }
+ return dataStagingTaskIds;
+ }
+
+ private boolean isArchive(ProcessModel processModel, String gatewayId) throws AppCatalogException {
+ AppCatalog appCatalog = RegistryFactory.getAppCatalog();
+ ApplicationInterfaceDescription appInterface = appCatalog.getApplicationInterface().getApplicationInterface(processModel.getApplicationInterfaceId());
+ return appInterface.isArchiveWorkingDirectory();
+ }
+
+ private void createArchiveDataStatgingTask(ProcessModel processModel, String gatewayId, List<String> dataStagingTaskIds) throws RegistryException {
+ TaskModel archiveTask = null;
+ try {
+ archiveTask = getOutputDataStagingTask(processModel, null, gatewayId);
+ } catch (TException e) {
+ throw new RegistryException("Error! DataStaging sub task serialization failed");
+ }
+ String taskId = (String) orchestratorContext.getRegistry().getExperimentCatalog().add(ExpCatChildDataType.TASK, archiveTask,
+ processModel.getProcessId());
+ archiveTask.setTaskId(taskId);
+ dataStagingTaskIds.add(archiveTask.getTaskId());
+
+ }
+
+ private void createOutputDataSatagingTasks(ProcessModel processModel, String gatewayId, List<String> dataStagingTaskIds, OutputDataObjectType processOutput) throws RegistryException {
+ try {
+ TaskModel outputDataStagingTask = getOutputDataStagingTask(processModel, processOutput, gatewayId);
+ String taskId = (String) orchestratorContext.getRegistry().getExperimentCatalog().add(ExpCatChildDataType.TASK, outputDataStagingTask,
+ processModel.getProcessId());
+ outputDataStagingTask.setTaskId(taskId);
+ dataStagingTaskIds.add(outputDataStagingTask.getTaskId());
+ } catch (TException e) {
+ throw new RegistryException("Error while serializing data staging sub task model", e);
+ }
+ }
+
+ private List<String> createAndSaveSubmissionTasks(String gatewayId, JobSubmissionInterface jobSubmissionInterface, ProcessModel processModel, int wallTime)
+ throws TException, RegistryException, OrchestratorException {
+
+ JobSubmissionProtocol jobSubmissionProtocol = jobSubmissionInterface.getJobSubmissionProtocol();
+ MonitorMode monitorMode = null;
+ if (jobSubmissionProtocol == JobSubmissionProtocol.SSH || jobSubmissionProtocol == JobSubmissionProtocol.SSH_FORK) {
+ SSHJobSubmission sshJobSubmission = OrchestratorUtils.getSSHJobSubmission(orchestratorContext, jobSubmissionInterface.getJobSubmissionInterfaceId());
+ monitorMode = sshJobSubmission.getMonitorMode();
+ } else if (jobSubmissionProtocol == JobSubmissionProtocol.UNICORE) {
+ monitorMode = MonitorMode.FORK;
+ } else {
+ logger.error("expId : {}, processId : {} :- Unsupported Job submission protocol {}.",
+ processModel.getExperimentId(), processModel.getProcessId(), jobSubmissionProtocol.name());
+ throw new OrchestratorException("Unsupported Job Submission Protocol " + jobSubmissionProtocol.name());
+ }
+ List<String> submissionTaskIds = new ArrayList<>();
+ TaskModel taskModel = new TaskModel();
+ taskModel.setParentProcessId(processModel.getProcessId());
+ taskModel.setCreationTime(new Date().getTime());
+ taskModel.setLastUpdateTime(taskModel.getCreationTime());
+ TaskStatus taskStatus = new TaskStatus(TaskState.CREATED);
+ taskStatus.setTimeOfStateChange(AiravataUtils.getCurrentTimestamp().getTime());
+ taskModel.setTaskStatus(taskStatus);
+ taskModel.setTaskType(TaskTypes.JOB_SUBMISSION);
+ JobSubmissionTaskModel submissionSubTask = new JobSubmissionTaskModel();
+ submissionSubTask.setMonitorMode(monitorMode);
+ submissionSubTask.setJobSubmissionProtocol(jobSubmissionProtocol);
+ submissionSubTask.setWallTime(wallTime);
+ byte[] bytes = ThriftUtils.serializeThriftObject(submissionSubTask);
+ taskModel.setSubTaskModel(bytes);
+ String taskId = (String) orchestratorContext.getRegistry().getExperimentCatalog().add(ExpCatChildDataType.TASK, taskModel,
+ processModel.getProcessId());
+ taskModel.setTaskId(taskId);
+ submissionTaskIds.add(taskModel.getTaskId());
+
+ // create monitor task for this Email based monitor mode job
+ if (monitorMode == MonitorMode.JOB_EMAIL_NOTIFICATION_MONITOR) {
+ TaskModel monitorTaskModel = new TaskModel();
+ monitorTaskModel.setParentProcessId(processModel.getProcessId());
+ monitorTaskModel.setCreationTime(new Date().getTime());
+ monitorTaskModel.setLastUpdateTime(monitorTaskModel.getCreationTime());
+ TaskStatus monitorTaskStatus = new TaskStatus(TaskState.CREATED);
+ monitorTaskStatus.setTimeOfStateChange(AiravataUtils.getCurrentTimestamp().getTime());
+ monitorTaskModel.setTaskStatus(monitorTaskStatus);
+ monitorTaskModel.setTaskType(TaskTypes.MONITORING);
+ MonitorTaskModel monitorSubTaskModel = new MonitorTaskModel();
+ monitorSubTaskModel.setMonitorMode(monitorMode);
+ monitorTaskModel.setSubTaskModel(ThriftUtils.serializeThriftObject(monitorSubTaskModel));
+ String mTaskId = (String) orchestratorContext.getRegistry().getExperimentCatalog().add(ExpCatChildDataType.TASK, monitorTaskModel, processModel.getProcessId());
+ monitorTaskModel.setTaskId(mTaskId);
+ submissionTaskIds.add(monitorTaskModel.getTaskId());
+ }
+
+ return submissionTaskIds;
+ }
+
+ private void sortByInputOrder(List<InputDataObjectType> processInputs) {
+ Collections.sort(processInputs, new Comparator<InputDataObjectType>() {
+ @Override
+ public int compare(InputDataObjectType inputDT_1, InputDataObjectType inputDT_2) {
+ return inputDT_1.getInputOrder() - inputDT_2.getInputOrder();
+ }
+ });
+ }
+
+ private TaskModel getInputDataStagingTask(ProcessModel processModel, InputDataObjectType processInput, String gatewayId) throws RegistryException, TException, AppCatalogException, TaskException {
+ // create new task model for this task
+ TaskModel taskModel = new TaskModel();
+ taskModel.setParentProcessId(processModel.getProcessId());
+ taskModel.setCreationTime(AiravataUtils.getCurrentTimestamp().getTime());
+ taskModel.setLastUpdateTime(taskModel.getCreationTime());
+ TaskStatus taskStatus = new TaskStatus(TaskState.CREATED);
+ taskStatus.setTimeOfStateChange(AiravataUtils.getCurrentTimestamp().getTime());
+ taskModel.setTaskStatus(taskStatus);
+ taskModel.setTaskType(TaskTypes.DATA_STAGING);
+ // create data staging sub task model
+ DataStagingTaskModel submodel = new DataStagingTaskModel();
+ ComputeResourcePreference computeResourcePreference = OrchestratorUtils.getComputeResourcePreference(orchestratorContext, processModel, gatewayId);
+ ComputeResourceDescription computeResource = orchestratorContext.getRegistry().getAppCatalog().getComputeResource().getComputeResource(processModel.getComputeResourceId());
+ String remoteOutputDir = OrchestratorUtils.getScratchLocation(orchestratorContext,processModel, gatewayId) + File.separator + processModel.getProcessId();
+ remoteOutputDir = remoteOutputDir.endsWith("/") ? remoteOutputDir : remoteOutputDir + "/";
+ URI destination = null;
+ try {
+ DataMovementProtocol dataMovementProtocol = OrchestratorUtils.getPreferredDataMovementProtocol(orchestratorContext, processModel, gatewayId);
+ String loginUserName = OrchestratorUtils.getLoginUserName(orchestratorContext, processModel, gatewayId);
+ destination = new URI(dataMovementProtocol.name(),
+ loginUserName,
+ computeResource.getHostName(),
+ OrchestratorUtils.getDataMovementPort(orchestratorContext, processModel, gatewayId),
+ remoteOutputDir , null, null);
+ } catch (URISyntaxException e) {
+ throw new TaskException("Error while constructing destination file URI");
+ }
+ submodel.setType(DataStageType.INPUT);
+ submodel.setSource(processInput.getValue());
+ submodel.setProcessInput(processInput);
+ submodel.setDestination(destination.toString());
+ taskModel.setSubTaskModel(ThriftUtils.serializeThriftObject(submodel));
+ return taskModel;
+ }
+
+ private TaskModel getOutputDataStagingTask(ProcessModel processModel, OutputDataObjectType processOutput, String gatewayId) throws RegistryException, TException {
+ try {
+
+ // create new task model for this task
+ TaskModel taskModel = new TaskModel();
+ taskModel.setParentProcessId(processModel.getProcessId());
+ taskModel.setCreationTime(AiravataUtils.getCurrentTimestamp().getTime());
+ taskModel.setLastUpdateTime(taskModel.getCreationTime());
+ TaskStatus taskStatus = new TaskStatus(TaskState.CREATED);
+ taskStatus.setTimeOfStateChange(AiravataUtils.getCurrentTimestamp().getTime());
+ taskModel.setTaskStatus(taskStatus);
+ taskModel.setTaskType(TaskTypes.DATA_STAGING);
+ ComputeResourcePreference computeResourcePreference = OrchestratorUtils.getComputeResourcePreference(orchestratorContext, processModel, gatewayId);
+ ComputeResourceDescription computeResource = orchestratorContext.getRegistry().getAppCatalog().getComputeResource().getComputeResource(processModel.getComputeResourceId());
+
+ String remoteOutputDir = OrchestratorUtils.getScratchLocation(orchestratorContext,processModel, gatewayId) + File.separator + processModel.getProcessId();
+ remoteOutputDir = remoteOutputDir.endsWith("/") ? remoteOutputDir : remoteOutputDir + "/";
+ DataStagingTaskModel submodel = new DataStagingTaskModel();
+ DataMovementProtocol dataMovementProtocol = OrchestratorUtils.getPreferredDataMovementProtocol(orchestratorContext, processModel, gatewayId);
+ URI source = null;
+ try {
+ String loginUserName = OrchestratorUtils.getLoginUserName(orchestratorContext, processModel, gatewayId);
+ if (processOutput != null) {
+ submodel.setType(DataStageType.OUPUT);
+ submodel.setProcessOutput(processOutput);
+ source = new URI(dataMovementProtocol.name(),
+ loginUserName,
+ computeResource.getHostName(),
+ OrchestratorUtils.getDataMovementPort(orchestratorContext, processModel, gatewayId),
+ remoteOutputDir + processOutput.getValue(), null, null);
+ } else {
+ // archive
+ submodel.setType(DataStageType.ARCHIVE_OUTPUT);
+ source = new URI(dataMovementProtocol.name(),
+ loginUserName,
+ computeResource.getHostName(),
+ OrchestratorUtils.getDataMovementPort(orchestratorContext, processModel, gatewayId),
+ remoteOutputDir, null, null);
+ }
+ } catch (URISyntaxException e) {
+ throw new TaskException("Error while constructing source file URI");
+ }
+ // We don't know destination location at this time, data staging task will set this.
+ // because destination is required field we set dummy destination
+ submodel.setSource(source.toString());
+ // We don't know destination location at this time, data staging task will set this.
+ // because destination is required field we set dummy destination
+ submodel.setDestination("dummy://temp/file/location");
+ taskModel.setSubTaskModel(ThriftUtils.serializeThriftObject(submodel));
+ return taskModel;
+ } catch (AppCatalogException | TaskException e) {
+ throw new RegistryException("Error occurred while retrieving data movement from app catalog", e);
+ }
+ }
+
+
+}
http://git-wip-us.apache.org/repos/asf/airavata-sandbox/blob/1932e865/gsoc2016/Design Changes in Airavata/orchestrator/pom.xml
----------------------------------------------------------------------
diff --git a/gsoc2016/Design Changes in Airavata/orchestrator/pom.xml b/gsoc2016/Design Changes in Airavata/orchestrator/pom.xml
new file mode 100644
index 0000000..0761b04
--- /dev/null
+++ b/gsoc2016/Design Changes in Airavata/orchestrator/pom.xml
@@ -0,0 +1,45 @@
+<?xml version="1.0" encoding="UTF-8"?>
+
+<!--Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under
+ the Apache License, Version 2.0 (the� "License"); you may not use this file except in compliance with the License. You may
+ obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to
+ in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF
+ ANY ~ KIND, either express or implied. See the License for the specific language governing permissions and limitations under
+ the License. -->
+
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
+
+ <parent>
+ <groupId>org.apache.airavata</groupId>
+ <artifactId>airavata</artifactId>
+ <version>0.17-SNAPSHOT</version>
+ <relativePath>../../pom.xml</relativePath>
+ </parent>
+
+ <modelVersion>4.0.0</modelVersion>
+ <artifactId>orchestrator</artifactId>
+ <packaging>pom</packaging>
+ <name>Airavata Orchestrator</name>
+ <url>http://airavata.apache.org/</url>
+
+ <profiles>
+ <profile>
+ <id>default</id>
+ <activation>
+ <activeByDefault>true</activeByDefault>
+ </activation>
+ <modules>
+ <module>orchestrator-core</module>
+ <module>orchestrator-service</module>
+ <module>orchestrator-client</module>
+ </modules>
+ </profile>
+ </profiles>
+
+ <properties>
+ <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
+ <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
+ </properties>
+
+</project>