[GitHub] rabbah closed pull request #3306: Script to generate IntelliJ Run Configuration for Controller and Invoker

[GitBox <gi...@apache.org>]

rabbah closed pull request #3306: Script to generate IntelliJ Run Configuration for Controller and Invoker
URL: https://github.com/apache/incubator-openwhisk/pull/3306
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/.gitignore b/.gitignore
index 960bb0178b..aab3af088b 100644
--- a/.gitignore
+++ b/.gitignore
@@ -79,3 +79,6 @@ ansible/roles/controller/files/*.p12
 !tests/dat/actions/python_virtualenv_dir.zip
 !tests/dat/actions/python_virtualenv_name.zip
 !tests/dat/actions/zippedaction.zip
+
+# dev
+intellij-run-config.groovy
diff --git a/tools/dev/README.md b/tools/dev/README.md
index d270563e57..351a503ffb 100644
--- a/tools/dev/README.md
+++ b/tools/dev/README.md
@@ -59,3 +59,72 @@ Sample output
     Processing whisks_design_document_for_activations_db_filters_v2.1.0.json
             - whisks-filters.v2.1.0-activations.js
     Generated view json files in /path/too/tools/build/views
+
+## IntelliJ Run Config Generator
+
+This script enables creation of [Intellij Launch Configuration][1] in _<openwhisk home>/.idea/runConfigurations_ 
+with name controller0 and invoker0. For this to work your Intellij project should be [directory based][3]. If your 
+project is file based (uses ipr files) then you can convert it to directory based via _File -> Save as Directory-Based Format_. These run configurations can then be invoked from _Run -> Edit Configurations -> Application_
+
+### Usage
+
+First setup OpenWhisk so that Controller and Invoker containers are up and running. Then run the script:
+
+    ./gradlew -p tools/dev intellij
+    
+It would inspect the running docker containers and then generate the launch configs with name 'controller0' 
+and 'invoker0'.
+
+Key points to note:
+
+1. Uses ~/tmp/openwhisk/controller (or invoker) as working directory.
+2. Changes the PORT to linked one. So controller gets started at 10001 only just like as its done in container.
+
+Now the docker container can be stopped and application can be launched from within the IDE.
+
+**Note** - Currently only the controller can be run from IDE. Invoker posses some [problems][2].
+
+### Configuration
+
+The script allows some local customization of the launch configuration. This can be done by creating a [config][4] file
+`intellij-run-config.groovy` in project root directory. Below is an example of _<openwhisk home>/intellij-run-config.groovy_ 
+file to customize the logging and db port used for CouchDB.
+
+```groovy
+//Configures the settings for controller application
+controller {
+    //Base directory used for controller process
+    workingDir = "/path/to/controller"
+    //System properties to be set
+    props = [
+            'logback.configurationFile':'/path/to/custom/logback.xml'
+    ]
+    //Environment variables to be set
+    env = [
+            'DB_PORT' : '5989',
+            'CONFIG_whisk_controller_protocol' : 'http'
+    ]
+}
+
+invoker {
+    workingDir = "/path/to/invoker"
+    props = [
+            'logback.configurationFile':'/path/to/custom/logback.xml'
+    ]
+    env = [
+            'DB_PORT' : '5989'
+    ]
+}
+
+```
+
+The config allows following properties:
+
+* `workingDir` - Base directory used for controller or invoker process.
+* `props` - Map of system properties which should be passed to the application.
+* `env` - Map of environment variables which should be set for application process.
+
+[1]: https://www.jetbrains.com/help/idea/run-debug-configurations-dialog.html#run_config_common_options
+[2]: https://github.com/apache/incubator-openwhisk/issues/3195
+[3]: https://www.jetbrains.com/help/idea/configuring-projects.html#project-formats
+[4]: http://docs.groovy-lang.org/2.4.2/html/gapi/groovy/util/ConfigSlurper.html
diff --git a/tools/dev/build.gradle b/tools/dev/build.gradle
index af112a86ef..0dde40c92f 100644
--- a/tools/dev/build.gradle
+++ b/tools/dev/build.gradle
@@ -1,9 +1,13 @@
 apply plugin: 'groovy'
 
+repositories {
+    mavenCentral()
+}
+
 def owHome = project.projectDir.parentFile.parentFile
 
 dependencies {
-    compile localGroovy()
+    compile "org.codehaus.groovy:groovy-all:2.4.14"
 }
 
 task couchdbViews(type: JavaExec) {
@@ -12,3 +16,10 @@ task couchdbViews(type: JavaExec) {
     args owHome.absolutePath
     classpath = sourceSets.main.runtimeClasspath
 }
+
+task intellij(type: JavaExec) {
+    description 'Generates Intellij run config for Controller and Invoker'
+    main = 'intellijRunConfig'
+    args owHome.absolutePath
+    classpath = sourceSets.main.runtimeClasspath
+}
diff --git a/tools/dev/src/main/groovy/intellijRunConfig.groovy b/tools/dev/src/main/groovy/intellijRunConfig.groovy
new file mode 100644
index 0000000000..f497b5e898
--- /dev/null
+++ b/tools/dev/src/main/groovy/intellijRunConfig.groovy
@@ -0,0 +1,247 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+import groovy.json.JsonSlurper
+import groovy.text.SimpleTemplateEngine
+
+assert args : "Expecting the OpenWhisk home directory to passed"
+owHome = args[0]
+
+//Launch config template
+def configTemplate = '''<component name="ProjectRunConfigurationManager">
+  <configuration default="false" name="${name}" type="Application" factoryName="Application">
+    <extension name="coverage" enabled="false" merge="false" sample_coverage="true" runner="idea" />
+    <option name="MAIN_CLASS_NAME" value="$main" />
+    <option name="VM_PARAMETERS" value="$sysProps" />
+    <option name="PROGRAM_PARAMETERS" value="0" />
+    <option name="WORKING_DIRECTORY" value="$workingDir" />
+    <option name="ALTERNATIVE_JRE_PATH_ENABLED" value="false" />
+    <option name="ALTERNATIVE_JRE_PATH" />
+    <option name="ENABLE_SWING_INSPECTOR" value="false" />
+    <option name="ENV_VARIABLES" />
+    <option name="PASS_PARENT_ENVS" value="true" />
+    <module name="${type}_main" />
+    <envs>
+      <% env.each { k,v -> %><env name="$k" value="$v" />
+      <% } %>
+    </envs>
+    <method />
+  </configuration>
+</component>
+'''
+
+def meta = [
+        controller : [main:"whisk.core.controller.Controller"],
+        invoker : [main:"whisk.core.invoker.Invoker"]
+]
+
+//Get names of all running containers
+def containerNames = 'docker ps --format {{.Names}}'.execute().text.split("\n")
+
+config = getConfig()
+
+Map controllerEnv = null
+Map invokerEnv = null
+
+containerNames.each{cn ->
+    //Inspect the specific container
+    def inspectResult = "docker inspect $cn".execute().text
+    def json = new JsonSlurper().parseText(inspectResult)
+
+    def imageName = json[0].'Config'.'Image'
+    if (imageName.contains("controller") || imageName.contains("invoker")){
+        def mappedPort = json.'NetworkSettings'.'Ports'.'8080/tcp'[0][0].'HostPort'
+
+        def envBaseMap = getEnvMap(json[0].'Config'.'Env')
+        String type
+        if (imageName.contains("controller")){
+            type = "controller"
+            controllerEnv = envBaseMap
+        } else {
+            type = "invoker"
+            invokerEnv = envBaseMap
+        }
+
+        def overrides = [
+                'PORT' : mappedPort,
+                'WHISK_LOGS_DIR' : "$owHome/core/$type/build/tmp"
+        ]
+
+        def envMap = getEnv(envBaseMap, type, overrides)
+
+        //Prepare system properties
+        def sysProps = getSysProps(envMap,type)
+
+        def templateBinding = [
+                main: meta[type].main,
+                type:type,
+                name:cn,
+                env: encodeForXML(envMap),
+                sysProps : sysProps,
+                USER_HOME : '$USER_HOME$',
+                workingDir : getWorkDir(type)
+        ]
+
+        def engine = new SimpleTemplateEngine()
+        def template = engine.createTemplate(configTemplate).make(templateBinding)
+
+        def launchFile = new File("$owHome/.idea/runConfigurations/${cn}.xml")
+        launchFile.parentFile.mkdirs()
+        launchFile.text = template
+        println "Created ${launchFile.absolutePath}"
+    }
+}
+
+/**
+ * Computes the env values which are common and then specific to controller and invoker
+ * and dumps them to a file. This can be used for docker-compose
+ */
+if (controllerEnv != null && invokerEnv != null){
+    Set<String> commonKeys = controllerEnv.keySet().intersect(invokerEnv.keySet())
+
+    SortedMap commonEnv = new TreeMap()
+    SortedMap controllerSpecificEnv = new TreeMap(controllerEnv)
+    SortedMap invokerSpecificEnv = new TreeMap(invokerEnv)
+    commonKeys.each{ key ->
+        if (controllerEnv[key] == invokerEnv[key]){
+            commonEnv[key] = controllerEnv[key]
+            controllerSpecificEnv.remove(key)
+            invokerSpecificEnv.remove(key)
+        }
+    }
+
+    copyEnvToFile(commonEnv,"whisk-common.env")
+    copyEnvToFile(controllerSpecificEnv,"whisk-controller.env")
+    copyEnvToFile(invokerSpecificEnv,"whisk-invoker.env")
+}
+
+def copyEnvToFile(SortedMap envMap,String envFileName){
+    File envFile = new File(getEnvFileDir(), envFileName)
+    envFile.withPrintWriter {pw ->
+        envMap.each{k,v ->
+            pw.println("$k=$v")
+
+        }
+    }
+    println "Wrote env to ${envFile.absolutePath}"
+}
+
+private File getEnvFileDir() {
+    File dir = new File(new File("build"), "env")
+    dir.mkdirs()
+    return dir
+}
+
+/**
+ * Reads config from intellij-run-config.groovy file
+ */
+def getConfig(){
+    def configFile = new File(owHome, 'intellij-run-config.groovy')
+    def config = configFile.exists() ? new ConfigSlurper().parse(configFile.text) : new ConfigObject()
+    if (configFile.exists()) {
+        println "Reading config from ${configFile.absolutePath}"
+    }
+    config
+}
+
+def getWorkDir(String type){
+    def dir = config[type].workingDir
+    if (dir){
+        File f = new File(dir)
+        if (!f.exists()) {
+            f.mkdirs()
+        }
+        dir
+    }else {
+        'file://$MODULE_DIR$'
+    }
+}
+
+def getSysProps(def envMap, String type){
+    def props = config[type].props
+    def sysProps = transformEnv(envMap)
+    sysProps.putAll(props)
+    sysProps.collect{k,v -> "-D$k='$v'"}.join(' ').replace('\'','')
+}
+
+//Implements the logic from transformEnvironment.sh
+//to ensure comparability as sed -r is not supported on Mac
+def transformEnv(Map<String, String> envMap){
+    def transformedMap = [:]
+    envMap.each{String k,String v ->
+        if (!k.startsWith("CONFIG_") || v.isEmpty()) return
+        k = k.substring("CONFIG_".length())
+        def parts = k.split("\\_")
+        def transformedKey = parts.collect {p ->
+            if (Character.isUpperCase(p[0] as char)){
+                // if the current part starts with an uppercase letter (is PascalCased)
+                // leave it alone
+                return p
+            } else {
+                // rewrite camelCased to kebab-cased
+                return p.replaceAll(/([a-z0-9])([A-Z])/,/$1-$2/).toLowerCase()
+            }
+        }
+
+        //Resolve values which again refer to env variables
+        if (v.startsWith('$')) {
+            def valueAsKey = v.substring(1)
+            if (envMap.containsKey(valueAsKey)){
+                v = envMap.get(valueAsKey)
+            }
+        }
+        transformedMap[transformedKey.join('.')] = v
+    }
+    return transformedMap
+}
+
+/**
+ * Inspect command from docker returns the environment variables as list of string of form key=value
+ * This method converts it to map and add provided overrides with overrides from config
+ */
+def getEnv(Map envMap, String type, Map overrides){
+    def ignoredKeys = ['PATH']
+    def overridesFromConfig = config[type].env
+    Map sortedMap = new TreeMap(envMap)
+    sortedMap.putAll(overrides)
+
+    //Config override come last
+    sortedMap.putAll(overridesFromConfig)
+
+    //Remove ignored keys like PATH which should be inherited
+    ignoredKeys.each {sortedMap.remove(it)}
+    sortedMap
+}
+
+def getEnvMap(def env){
+    def envMap = env.collectEntries {String e ->
+        def eqIndex = e.indexOf('=')
+        def k = e.substring(0, eqIndex)
+        def v = e.substring(eqIndex + 1)
+        [(k):v]
+    }
+    def sortedMap = new TreeMap()
+    sortedMap.putAll(envMap)
+    Collections.unmodifiableSortedMap(sortedMap)
+}
+
+def getEnvAsList(Map envMap){
+    envMap.collect{k,v -> "$k=$v"}
+}
+
+def encodeForXML(Map map){
+    map.collectEntries {k,v -> [(k): v.replace('"', '&quot;')]}
+}


 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services