You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@openwhisk.apache.org by GitBox <gi...@apache.org> on 2018/08/13 12:06:46 UTC
[GitHub] kpavel closed pull request #3886: Proposing Lean OpenWhisk
kpavel closed pull request #3886: Proposing Lean OpenWhisk
URL: https://github.com/apache/incubator-openwhisk/pull/3886
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/.travis.yml b/.travis.yml
index 12551d86e0..86d19d7fc0 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -73,3 +73,19 @@ jobs:
- OPENWHISK_HOST="172.17.0.1" USERS="1" REQUESTS_PER_SEC="1" ./gradlew gatlingRun-ColdBlockingInvokeSimulation
env:
- DESCRIPTION="Execute wrk-performance test suite."
+ - script:
+ # Same System Tests suite (with one KafkaConnectorTest excluded), but when OW deployment is Lean (without Kafka)
+ - ./tools/travis/runLeanSystemTests.sh
+ - ./tools/travis/checkAndUploadLogs.sh system
+ env: DESCRIPTION="Lean System Tests"
+ - script:
+ # Same wrk-performance test suite, but when OW deployment is Lean (without Kafka)
+ - ./tests/performance/preparation/deploy-lean.sh
+ - TERM=dumb ./tests/performance/wrk_tests/latency.sh "https://172.17.0.1:10001" "$(cat ansible/files/auth.guest)" 2m
+ - TERM=dumb ./tests/performance/wrk_tests/throughput.sh "https://172.17.0.1:10001" "$(cat ansible/files/auth.guest)" 4 2 2m
+ - OPENWHISK_HOST="172.17.0.1" CONNECTIONS="100" REQUESTS_PER_SEC="1" ./gradlew gatlingRun-ApiV1Simulation
+ - OPENWHISK_HOST="172.17.0.1" MEAN_RESPONSE_TIME="1000" API_KEY="$(cat ansible/files/auth.guest)" EXCLUDED_KINDS="python:default,java:default,swift:default" PAUSE_BETWEEN_INVOKES="100" ./gradlew gatlingRun-LatencySimulation
+ - OPENWHISK_HOST="172.17.0.1" API_KEY="$(cat ansible/files/auth.guest)" CONNECTIONS="100" REQUESTS_PER_SEC="1" ./gradlew gatlingRun-BlockingInvokeOneActionSimulation
+ - OPENWHISK_HOST="172.17.0.1" USERS="1" REQUESTS_PER_SEC="1" ./gradlew gatlingRun-ColdBlockingInvokeSimulation
+ env:
+ - DESCRIPTION="Execute Lean wrk-performance test suite."
diff --git a/README.md b/README.md
index 821d0773e0..0b4dc03227 100644
--- a/README.md
+++ b/README.md
@@ -32,6 +32,7 @@ OpenWhisk is a cloud-first distributed event-based programming service. It provi
* [Native development](#native-development) (Mac and Ubuntu)
* [Kubernetes](#kubernetes-setup)
* [Vagrant](#vagrant-setup)
+* [Lean](#lean-setup)
* [Learn concepts and commands](#learn-concepts-and-commands)
* [Issues](#issues)
* [Slack](#slack)
@@ -96,6 +97,45 @@ If you plan to make contributions to OpenWhisk, we recommend either a Mac or Ubu
* [Setup Mac for OpenWhisk](tools/macos/README.md)
* [Setup Ubuntu for OpenWhisk](tools/ubuntu-setup/README.md)
+### Lean Setup
+To have a lean setup (no Kafka, Zookeeper and no Invokers as separate entities), consider the following options.
+
+**Docker Compose**
+```
+git clone -b lean https://github.com/kpavel/incubator-openwhisk-devtools.git
+cd incubator-openwhisk-devtools/docker-compose
+make quick-start
+```
+
+For more detailed instructions or if you encounter problems see the [OpenWhisk-dev tools](https://github.com/kpavel/incubator-openwhisk-devtools/tree/lean/docker-compose/README.md) project.
+
+**Native**
+Follow instructions in the [Native development](#native-development) document until the [Deploying Using CouchDB](ansible/README.md#deploying-using-cloudant) step. At this point, replace:
+```
+ansible-playbook -i environments/<environment> openwhisk.yml
+```
+by:
+```
+ansible-playbook -i environments/<environment> openwhisk-lean.yml
+```
+
+**Kubernetes**
+Another path to quickly starting to use Lean OpenWhisk is to install it on a Kubernetes cluster. Lean OpenWhisk can be installed on Minikube, on a managed Kubernetes cluster provisioned from a public cloud provider, or on a Kubernetes cluster you manage yourself. To get started,
+
+```
+git clone -b lean https://github.com/kpavel/incubator-openwhisk-deploy-kube.git
+```
+
+Follow the regular instructions in the [OpenWhisk on Kubernetes README.md](https://github.com/apache/incubator-openwhisk-deploy-kube/blob/master/README.md) until [Customize the Deployment](https://github.com/apache/incubator-openwhisk-deploy-kube/blob/master/README.md#customize-the-deployment) step. At this point add this [Lean configuration](https://github.com/kpavel/incubator-openwhisk-deploy-kube/blob/lean/docs/configurationChoices.md#lean-openwhisk) to your cluster.
+
+**Vagrant**
+Set environment variable LEAN to true
+```
+export LEAN=true
+```
+and continue with [Regular Vagrant Setup](#vagrant-setup) steps
+
+
### Learn concepts and commands
Browse the [documentation](docs/) to learn more. Here are some topics you may be
diff --git a/ansible/controller-lean.yml b/ansible/controller-lean.yml
new file mode 100644
index 0000000000..85cece86e3
--- /dev/null
+++ b/ansible/controller-lean.yml
@@ -0,0 +1,8 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more contributor
+# license agreements; and to You under the Apache License, Version 2.0.
+---
+# This playbook deploys Openwhisk Lean Controller
+
+- hosts: controllers
+ roles:
+ - lean
diff --git a/ansible/group_vars/all b/ansible/group_vars/all
index 21146303c6..b97ec3385e 100644
--- a/ansible/group_vars/all
+++ b/ansible/group_vars/all
@@ -69,7 +69,7 @@ controller:
instances: "{{ groups['controllers'] | length }}"
localBookkeeping: "{{ controller_local_bookkeeping | default('false') }}"
akka:
- provider: cluster
+ provider: "{{ controller_akka_provider | default('cluster') }}"
cluster:
basePort: 8000
host: "{{ groups['controllers'] | map('extract', hostvars, 'ansible_host') | list }}"
diff --git a/ansible/openwhisk-lean.yml b/ansible/openwhisk-lean.yml
new file mode 100644
index 0000000000..9dd87cf25c
--- /dev/null
+++ b/ansible/openwhisk-lean.yml
@@ -0,0 +1,12 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more contributor
+# license agreements; and to You under the Apache License, Version 2.0.
+---
+# This playbook deploys Lean Openwhisk stack.
+# It assumes you have already set up your database with the respective db provider playbook (currently cloudant.yml or couchdb.yml)
+# It assumes that wipe.yml have being deployed at least once
+
+- import_playbook: controller-lean.yml
+
+- import_playbook: edge.yml
+
+- import_playbook: downloadcli.yml
diff --git a/ansible/roles/lean/tasks/clean.yml b/ansible/roles/lean/tasks/clean.yml
new file mode 100644
index 0000000000..0e8805aefc
--- /dev/null
+++ b/ansible/roles/lean/tasks/clean.yml
@@ -0,0 +1,27 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more contributor
+# license agreements; and to You under the Apache License, Version 2.0.
+---
+# Remove lean controller container.
+
+- name: get controller name
+ set_fact:
+ controller_name: "controller-lean"
+
+- name: remove controller
+ docker_container:
+ name: "{{ controller_name }}"
+ image: "{{ docker_registry }}{{ docker.image.prefix }}/controller:{{ docker.image.tag }}"
+ state: absent
+ ignore_errors: True
+
+- name: remove controller log directory
+ file:
+ path: "{{ whisk_logs_dir }}/{{ controller_name }}"
+ state: absent
+ become: "{{ logs.dir.become }}"
+
+- name: remove controller conf directory
+ file:
+ path: "{{ controller.confdir }}/{{ controller_name }}"
+ state: absent
+ become: "{{ controller.dir.become }}"
diff --git a/ansible/roles/lean/tasks/deploy.yml b/ansible/roles/lean/tasks/deploy.yml
new file mode 100644
index 0000000000..b15601f791
--- /dev/null
+++ b/ansible/roles/lean/tasks/deploy.yml
@@ -0,0 +1,262 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more contributor
+# license agreements; and to You under the Apache License, Version 2.0.
+---
+# This role will install Lean Controller in group 'controllers' in the environment
+# inventory
+
+- import_tasks: docker_login.yml
+
+- import_tasks: invoker_lean.yml
+
+- name: get controller name and index
+ set_fact:
+ controller_name: "controller-lean"
+ controller_index: 0
+
+- name: "pull the {{ docker.image.tag }} image of controller"
+ shell: "docker pull {{docker_registry}}{{ docker.image.prefix }}/lean:{{docker.image.tag}}"
+ when: docker_registry != ""
+ register: result
+ until: (result.rc == 0)
+ retries: "{{ docker.pull.retries }}"
+ delay: "{{ docker.pull.delay }}"
+
+- name: ensure controller log directory is created with permissions
+ file:
+ path: "{{ whisk_logs_dir }}/{{ controller_name }}"
+ state: directory
+ mode: 0777
+ become: "{{ logs.dir.become }}"
+
+- name: ensure controller config directory is created with permissions
+ file:
+ path: "{{ controller.confdir }}/{{ controller_name }}"
+ state: directory
+ mode: 0777
+ become: "{{ controller.dir.become }}"
+
+- name: copy jmxremote password file
+ when: jmx.enabled
+ template:
+ src: "jmxremote.password.j2"
+ dest: "{{ controller.confdir }}/{{ controller_name }}/jmxremote.password"
+ mode: 0777
+
+- name: copy jmxremote access file
+ when: jmx.enabled
+ template:
+ src: "jmxremote.access.j2"
+ dest: "{{ controller.confdir }}/{{ controller_name }}/jmxremote.access"
+ mode: 0777
+
+- name: copy nginx certificate keystore
+ when: controller.protocol == 'https'
+ copy:
+ src: "{{ openwhisk_home }}/ansible/roles/controller/files/{{ controllerKeystoreName }}"
+ mode: 0666
+ dest: "{{ controller.confdir }}/{{ controller_name }}"
+ become: "{{ controller.dir.become }}"
+
+- name: copy certificates
+ when: controller.protocol == 'https'
+ copy:
+ src: "{{ openwhisk_home }}/ansible/roles/controller/files/{{ item }}"
+ mode: 0666
+ dest: "{{ controller.confdir }}/{{ controller_name }}"
+ with_items:
+ - "{{ controller.ssl.cert }}"
+ - "{{ controller.ssl.key }}"
+ become: "{{ controller.dir.become }}"
+
+- name: check, that required databases exist
+ include_tasks: "{{ openwhisk_home }}/ansible/tasks/db/checkDb.yml"
+ vars:
+ dbName: "{{ item }}"
+ dbUser: "{{ db.credentials.invoker.user }}"
+ dbPass: "{{ db.credentials.invoker.pass }}"
+ with_items:
+ - "{{ db.whisk.actions }}"
+ - "{{ db.whisk.auth }}"
+ - "{{ db.whisk.activations }}"
+
+- name: prepare controller port
+ set_fact:
+ controller_port: "{{ controller.basePort + (controller_index | int) }}"
+ ports_to_expose:
+ - "{{ controller.basePort + (controller_index | int) }}:8080"
+
+- name: expose additional ports if jmxremote is enabled
+ when: jmx.enabled
+ vars:
+ jmx_remote_port: "{{ jmx.basePortController + (controller_index|int) }}"
+ jmx_remote_rmi_port:
+ "{{ jmx.rmiBasePortController + (controller_index|int) }}"
+ set_fact:
+ ports_to_expose: >-
+ {{ ports_to_expose }} +
+ [ '{{ jmx_remote_port }}:{{ jmx_remote_port }}' ] +
+ [ '{{ jmx_remote_rmi_port }}:{{ jmx_remote_rmi_port }}' ]
+ controller_args: >-
+ {{ controller.arguments }}
+ {{ jmx.jvmCommonArgs }}
+ -Djava.rmi.server.hostname={{ inventory_hostname }}
+ -Dcom.sun.management.jmxremote.rmi.port={{ jmx_remote_rmi_port }}
+ -Dcom.sun.management.jmxremote.port={{ jmx_remote_port }}
+
+- name: populate environment variables for controller
+ set_fact:
+ controller_env:
+ "JAVA_OPTS":
+ -Xmx{{ controller.heap }}
+ -XX:+CrashOnOutOfMemoryError
+ -XX:+UseGCOverheadLimit
+ -XX:ErrorFile=/logs/java_error.log
+ -XX:+HeapDumpOnOutOfMemoryError
+ -XX:HeapDumpPath=/logs
+ "CONTROLLER_OPTS": "{{ controller_args | default(controller.arguments) }}"
+ "CONTROLLER_INSTANCES": 1
+ "JMX_REMOTE": "{{ jmx.enabled }}"
+
+ "PORT": 8080
+
+ "CONFIG_whisk_info_date": "{{ whisk.version.date }}"
+ "CONFIG_whisk_info_buildNo": "{{ docker.image.tag }}"
+
+ "CONFIG_whisk_couchdb_protocol": "{{ db.protocol }}"
+ "CONFIG_whisk_couchdb_host": "{{ db.host }}"
+ "CONFIG_whisk_couchdb_port": "{{ db.port }}"
+ "CONFIG_whisk_couchdb_username": "{{ db.credentials.controller.user }}"
+ "CONFIG_whisk_couchdb_password": "{{ db.credentials.controller.pass }}"
+ "CONFIG_whisk_couchdb_provider": "{{ db.provider }}"
+ "CONFIG_whisk_couchdb_databases_WhiskAuth": "{{ db.whisk.auth }}"
+ "CONFIG_whisk_couchdb_databases_WhiskEntity": "{{ db.whisk.actions }}"
+ "CONFIG_whisk_couchdb_databases_WhiskActivation":
+ "{{ db.whisk.activations }}"
+ "CONFIG_whisk_db_actionsDdoc": "{{ db_whisk_actions_ddoc | default() }}"
+ "CONFIG_whisk_db_activationsDdoc": "{{ db_whisk_activations_ddoc | default() }}"
+ "CONFIG_whisk_db_activationsFilterDdoc": "{{ db_whisk_activations_filter_ddoc | default() }}"
+ "CONFIG_whisk_userEvents_enabled": "{{ user_events }}"
+
+ "LIMITS_ACTIONS_INVOKES_PERMINUTE": "{{ limits.invocationsPerMinute }}"
+ "LIMITS_ACTIONS_INVOKES_CONCURRENT": "{{ limits.concurrentInvocations }}"
+ "LIMITS_TRIGGERS_FIRES_PERMINUTE": "{{ limits.firesPerMinute }}"
+ "LIMITS_ACTIONS_SEQUENCE_MAXLENGTH": "{{ limits.sequenceMaxLength }}"
+
+ "CONFIG_whisk_memory_min": "{{ limit_action_memory_min | default() }}"
+ "CONFIG_whisk_memory_max": "{{ limit_action_memory_max | default() }}"
+ "CONFIG_whisk_memory_std": "{{ limit_action_memory_std | default() }}"
+
+ "CONFIG_whisk_timeLimit_min": "{{ limit_action_time_min | default() }}"
+ "CONFIG_whisk_timeLimit_max": "{{ limit_action_time_max | default() }}"
+ "CONFIG_whisk_timeLimit_std": "{{ limit_action_time_std | default() }}"
+
+ "CONFIG_whisk_activation_payload_max":
+ "{{ limit_activation_payload | default() }}"
+
+ "RUNTIMES_MANIFEST": "{{ runtimesManifest | to_json }}"
+ "CONFIG_whisk_runtimes_defaultImagePrefix":
+ "{{ runtimes_default_image_prefix | default() }}"
+ "CONFIG_whisk_runtimes_defaultImageTag":
+ "{{ runtimes_default_image_tag | default() }}"
+ "CONFIG_whisk_runtimes_bypassPullForLocalImages":
+ "{{ runtimes_bypass_pull_for_local_images | default() }}"
+ "CONFIG_whisk_runtimes_localImagePrefix":
+ "{{ runtimes_local_image_prefix | default() }}"
+
+ "METRICS_KAMON": "{{ metrics.kamon.enabled }}"
+ "METRICS_KAMON_TAGS": "{{ metrics.kamon.tags }}"
+ "METRICS_LOG": "{{ metrics.log.enabled }}"
+ "CONFIG_whisk_controller_protocol": "{{ controller.protocol }}"
+ "CONFIG_whisk_controller_https_keystorePath":
+ "{{ controller.ssl.keystore.path }}"
+ "CONFIG_whisk_controller_https_keystorePassword":
+ "{{ controller.ssl.keystore.password }}"
+ "CONFIG_whisk_controller_https_keystoreFlavor":
+ "{{ controller.ssl.storeFlavor }}"
+ "CONFIG_whisk_controller_https_truststorePath":
+ "{{ controller.ssl.truststore.path }}"
+ "CONFIG_whisk_controller_https_truststorePassword":
+ "{{ controller.ssl.truststore.password }}"
+ "CONFIG_whisk_controller_https_truststoreFlavor":
+ "{{ controller.ssl.storeFlavor }}"
+ "CONFIG_whisk_controller_https_clientAuth":
+ "{{ controller.ssl.clientAuth }}"
+ "CONFIG_whisk_loadbalancer_invokerBusyThreshold":
+ "{{ invoker.busyThreshold }}"
+ "CONFIG_whisk_loadbalancer_blackboxFraction":
+ "{{ controller.blackboxFraction }}"
+ "CONFIG_whisk_loadbalancer_timeoutFactor":
+ "{{ controller.timeoutFactor }}"
+
+ "CONFIG_kamon_statsd_hostname": "{{ metrics.kamon.host }}"
+ "CONFIG_kamon_statsd_port": "{{ metrics.kamon.port }}"
+
+ "CONFIG_whisk_spi_LogStoreProvider": "{{ userLogs.spi }}"
+ "CONFIG_whisk_spi_MessagingProvider": "whisk.connector.lean.LeanMessagingProvider"
+ "CONFIG_whisk_spi_LoadBalancerProvider": "whisk.core.loadBalancer.LeanBalancer"
+
+ "CONFIG_logback_log_level": "{{ controller.loglevel }}"
+
+ "CONFIG_whisk_transactions_header": "{{ transactions.header }}"
+
+- name: merge extra env variables
+ set_fact:
+ controller_env: "{{ controller_env | combine(controller.extraEnv) }}"
+
+- name: include plugins
+ include_tasks: "{{ item }}.yml"
+ with_items: "{{ controller_plugins | default([]) }}"
+
+- name: merge invoker env
+ set_fact:
+ controller_env: "{{ invoker_env | combine(controller_env) }}"
+
+- name: merge invoker ports
+ set_fact:
+ ports_to_expose: "{{ ports_to_expose }} + {{ invoker_ports }}"
+
+- name: set controller volumes
+ set_fact:
+ volumes: "{{ whisk_logs_dir }}/{{ controller_name }}:/logs,\
+ {{ controller.confdir }}/{{ controller_name }}:/conf"
+
+- name: merge invoker volumes
+ set_fact:
+ volumes: "{{ volumes|default('') }},{{ invoker_volumes }}"
+
+- name: (re)start controller
+ docker_container:
+ name: "{{ controller_name }}"
+ image:
+ "{{docker_registry~docker.image.prefix}}/lean:{{docker.image.tag}}"
+ userns_mode: "host"
+ pid_mode: "host"
+ privileged: "yes"
+ state: started
+ recreate: true
+ restart_policy: "{{ docker.restart.policy }}"
+ hostname: "{{ controller_name }}"
+ env: "{{ controller_env }}"
+ volumes: "{{ volumes }}"
+ ports: "{{ ports_to_expose }}"
+ command:
+ /bin/sh -c
+ "exec /init.sh {{ controller_index }}
+ >> /logs/{{ controller_name }}_logs.log 2>&1"
+
+- name: wait until the Controller in this host is up and running
+ uri:
+ url:
+ "{{controller.protocol}}://{{ansible_host}}:{{controller_port}}/ping"
+ validate_certs: "no"
+ client_key:
+ "{{ controller.confdir }}/{{ controller_name }}/{{ controller.ssl.key }}"
+ client_cert:
+ "{{ controller.confdir }}/{{ controller_name }}/{{ controller.ssl.cert }}"
+ register: result
+ until: result.status == 200
+ retries: 12
+ delay: 5
+
+# VIM: let b:syntastic_yaml_yamllint_args="-c '".expand('%:p:h')."../../../yamllint.yml'"
+
diff --git a/ansible/roles/lean/tasks/main.yml b/ansible/roles/lean/tasks/main.yml
new file mode 100644
index 0000000000..9fd95a1388
--- /dev/null
+++ b/ansible/roles/lean/tasks/main.yml
@@ -0,0 +1,12 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more contributor
+# license agreements; and to You under the Apache License, Version 2.0.
+---
+# This role will install lean controller in group 'controllers' in the environment inventory
+# In deploy mode it will deploy lean controller.
+# In clean mode it will remove the lean controller container.
+
+- import_tasks: deploy.yml
+ when: mode == "deploy"
+
+- import_tasks: clean.yml
+ when: mode == "clean"
diff --git a/ansible/tasks/invoker_lean.yml b/ansible/tasks/invoker_lean.yml
new file mode 100644
index 0000000000..da3ddd732f
--- /dev/null
+++ b/ansible/tasks/invoker_lean.yml
@@ -0,0 +1,141 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more contributor
+# license agreements; and to You under the Apache License, Version 2.0.
+---
+
+# Provides invoker data common to Lean controller and invoker (volumes, environment variables). To avoid duplication eventually should be used in invoker/tasks/deploy.yml as well
+
+###
+# This task assumes that the images are local to the invoker host already if there is no prefix or tag
+# which is usually the case for a local deployment. A distributed deployment will specify the prefix, or tag
+# to pull the images from the appropriate registry. If a runtimes_registry is optionally specified, pull images
+# from there; this permits a (private) registry to be used for caching the images. The registry if specified
+# must include a trailing '/'.
+#
+- name: "pull runtime action images per manifest"
+ shell: "docker pull {{runtimes_registry | default()}}{{item.prefix}}/{{item.name}}:{{item.tag | default()}}"
+ loop: "{{ runtimesManifest.runtimes.values() | sum(start=[]) | selectattr('deprecated', 'equalto',false) | map(attribute='image') | list | unique }}"
+ when: skip_pull_runtimes is not defined or not (skip_pull_runtimes == True or skip_pull_runtimes.lower() == "true")
+ register: result
+ until: (result.rc == 0)
+ retries: "{{ docker.pull.retries }}"
+ delay: "{{ docker.pull.delay }}"
+
+###
+# See comment above for pulling other runtime images.
+#
+- name: "pull blackboxes action images per manifest"
+ shell: "docker pull {{runtimes_registry | default()}}{{item.prefix}}/{{item.name}}:{{item.tag | default()}}"
+ loop: "{{ runtimesManifest.blackboxes }}"
+ when: skip_pull_runtimes is not defined or not (skip_pull_runtimes == True or skip_pull_runtimes.lower() == "true")
+ register: result
+ until: (result.rc == 0)
+ retries: "{{ docker.pull.retries }}"
+ delay: "{{ docker.pull.delay }}"
+
+- name: "determine docker root dir on docker-machine"
+ uri: url="http://{{ ansible_host }}:{{ docker.port }}/info" return_content=yes
+ register: dockerInfo_output
+ when: environmentInformation.type == 'docker-machine'
+
+- set_fact:
+ dockerInfo: "{{ dockerInfo_output['json'] }}"
+ when: environmentInformation.type == "docker-machine"
+
+- name: "determine docker root dir"
+ shell: echo -e "GET http:/v1.24/info HTTP/1.0\r\n" | nc -U /var/run/docker.sock | grep "{"
+ args:
+ executable: /bin/bash
+ register: dockerInfo_output
+ when: environmentInformation.type != "docker-machine"
+
+- set_fact:
+ dockerInfo: "{{ dockerInfo_output.stdout|from_json }}"
+ when: environmentInformation.type != "docker-machine"
+
+- name: expose additional ports if jmxremote is enabled
+ when: jmx.enabled
+ set_fact:
+ invoker_ports: "[ \"{{ jmx.basePortInvoker }}:{{ jmx.basePortInvoker }}\" ] + [ \"{{ jmx.rmiBasePortInvoker }}:{{ jmx.rmiBasePortInvoker }}\" ]"
+
+- name: prepare invoker env
+ set_fact:
+ invoker_env:
+ "INVOKER_OPTS": "{{ invoker_args | default(invoker.arguments) }}"
+ "JMX_REMOTE": "{{ jmx.enabled }}"
+ "PORT": 8080
+ "KAFKA_HOSTS": "{{ kafka_connect_string }}"
+ "CONFIG_whisk_kafka_replicationFactor": "{{ kafka.replicationFactor | default() }}"
+ "CONFIG_whisk_kafka_topics_invoker_retentionBytes": "{{ kafka_topics_invoker_retentionBytes | default() }}"
+ "CONFIG_whisk_kafka_topics_invoker_retentionMs": "{{ kafka_topics_invoker_retentionMS | default() }}"
+ "CONFIG_whisk_kakfa_topics_invoker_segmentBytes": "{{ kafka_topics_invoker_segmentBytes | default() }}"
+ "CONFIG_whisk_kafka_common_securityProtocol": "{{ kafka.protocol }}"
+ "CONFIG_whisk_kafka_common_sslTruststoreLocation": "/conf/{{ kafka.ssl.keystore.name }}"
+ "CONFIG_whisk_kafka_common_sslTruststorePassword": "{{ kafka.ssl.keystore.password }}"
+ "CONFIG_whisk_kafka_common_sslKeystoreLocation": "/conf/{{ kafka.ssl.keystore.name }}"
+ "CONFIG_whisk_kafka_common_sslKeystorePassword": "{{ kafka.ssl.keystore.password }}"
+ "CONFIG_whisk_userEvents_enabled": "{{ user_events }}"
+ "ZOOKEEPER_HOSTS": "{{ zookeeper_connect_string }}"
+ "CONFIG_whisk_couchdb_protocol": "{{ db.protocol }}"
+ "CONFIG_whisk_couchdb_host": "{{ db.host }}"
+ "CONFIG_whisk_couchdb_port": "{{ db.port }}"
+ "CONFIG_whisk_couchdb_username": "{{ db.credentials.invoker.user }}"
+ "CONFIG_whisk_couchdb_password": "{{ db.credentials.invoker.pass }}"
+ "CONFIG_whisk_couchdb_provider": "{{ db.provider }}"
+ "CONFIG_whisk_couchdb_databases_WhiskAuth": "{{ db.whisk.auth }}"
+ "CONFIG_whisk_couchdb_databases_WhiskEntity": "{{ db.whisk.actions }}"
+ "CONFIG_whisk_couchdb_databases_WhiskActivation": "{{ db.whisk.activations }}"
+ "DB_WHISK_ACTIONS": "{{ db.whisk.actions }}"
+ "DB_WHISK_ACTIVATIONS": "{{ db.whisk.activations }}"
+ "DB_WHISK_AUTHS": "{{ db.whisk.auth }}"
+ "CONFIG_whisk_db_actionsDdoc": "{{ db_whisk_actions_ddoc | default() }}"
+ "CONFIG_whisk_db_activationsDdoc": "{{ db_whisk_activations_ddoc | default() }}"
+ "CONFIG_whisk_db_activationsFilterDdoc": "{{ db_whisk_activations_filter_ddoc | default() }}"
+ "WHISK_API_HOST_PROTO": "{{ whisk_api_host_proto | default('https') }}"
+ "WHISK_API_HOST_PORT": "{{ whisk_api_host_port | default('443') }}"
+ "WHISK_API_HOST_NAME": "{{ whisk_api_host_name | default(groups['edge'] | first) }}"
+ "RUNTIMES_REGISTRY": "{{ runtimes_registry | default('') }}"
+ "RUNTIMES_MANIFEST": "{{ runtimesManifest | to_json }}"
+ "CONFIG_whisk_runtimes_bypassPullForLocalImages": "{{ runtimes_bypass_pull_for_local_images | default() }}"
+ "CONFIG_whisk_runtimes_localImagePrefix": "{{ runtimes_local_image_prefix | default() }}"
+ "CONFIG_whisk_containerFactory_containerArgs_network": "{{ invoker_container_network_name | default('bridge') }}"
+ "INVOKER_CONTAINER_POLICY": "{{ invoker_container_policy_name | default()}}"
+ "CONFIG_whisk_containerPool_numCore": "{{ invoker.numcore }}"
+ "CONFIG_whisk_containerPool_coreShare": "{{ invoker.coreshare }}"
+ "CONFIG_whisk_docker_containerFactory_useRunc": "{{ invoker.useRunc }}"
+ "WHISK_LOGS_DIR": "{{ whisk_logs_dir }}"
+ "METRICS_KAMON": "{{ metrics.kamon.enabled }}"
+ "METRICS_KAMON_TAGS": "{{ metrics.kamon.tags }}"
+ "METRICS_LOG": "{{ metrics.log.enabled }}"
+ "CONFIG_kamon_statsd_hostname": "{{ metrics.kamon.host }}"
+ "CONFIG_kamon_statsd_port": "{{ metrics.kamon.port }}"
+ "CONFIG_whisk_spi_LogStoreProvider": "{{ userLogs.spi }}"
+ "CONFIG_logback_log_level": "{{ invoker.loglevel }}"
+ "CONFIG_whisk_memory_min": "{{ limit_action_memory_min | default() }}"
+ "CONFIG_whisk_memory_max": "{{ limit_action_memory_max | default() }}"
+ "CONFIG_whisk_memory_std": "{{ limit_action_memory_std | default() }}"
+ "CONFIG_whisk_timeLimit_min": "{{ limit_action_time_min | default() }}"
+ "CONFIG_whisk_timeLimit_max": "{{ limit_action_time_max | default() }}"
+ "CONFIG_whisk_timeLimit_std": "{{ limit_action_time_std | default() }}"
+ "CONFIG_whisk_activation_payload_max": "{{ limit_activation_payload | default() }}"
+ "CONFIG_whisk_transactions_header": "{{ transactions.header }}"
+
+- name: extend invoker dns env
+ set_fact:
+ invoker_env: "{{ invoker_env | default({}) | combine( {'CONFIG_whisk_containerFactory_containerArgs_dnsServers_' ~ item.0: item.1} ) }}"
+ with_indexed_items: "{{ (invoker_container_network_dns_servers | default()).split(' ')}}"
+
+- name: merge extra env variables
+ set_fact:
+ invoker_env: "{{ invoker_env | combine(invoker.extraEnv) }}"
+
+- name: set invoker volumes
+ set_fact:
+ invoker_volumes: "/sys/fs/cgroup:/sys/fs/cgroup,/run/runc:/run/runc,\
+ {{ dockerInfo['DockerRootDir'] }}/containers/:/containers,\
+ {{ docker_sock | default('/var/run/docker.sock') }}:/var/run/docker.sock"
+
+- name: define options when deploying invoker on Ubuntu
+ set_fact:
+ invoker_volumes: "{{ invoker_volumes|default('') }},/usr/lib/x86_64-linux-gnu/libapparmor.so.1:/usr/lib/x86_64-linux-gnu/libapparmor.so.1"
+ when: ansible_distribution == "Ubuntu"
+
diff --git a/common/scala/src/main/scala/whisk/connector/lean/LeanConsumer.scala b/common/scala/src/main/scala/whisk/connector/lean/LeanConsumer.scala
new file mode 100644
index 0000000000..af37923617
--- /dev/null
+++ b/common/scala/src/main/scala/whisk/connector/lean/LeanConsumer.scala
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package whisk.connector.lean
+
+import scala.concurrent.duration._
+import whisk.common.Logging
+import whisk.core.connector.MessageConsumer
+import java.util.concurrent.BlockingQueue
+import java.util.ArrayList
+import scala.collection.JavaConversions._
+
+class LeanConsumer(queue: BlockingQueue[Array[Byte]], override val maxPeek: Int)(implicit logging: Logging)
+ extends MessageConsumer {
+
+ private val gracefulWaitTime = 100.milliseconds
+
+ /**
+ */
+ override def peek(duration: FiniteDuration, retry: Int): Iterable[(String, Int, Long, Array[Byte])] = {
+ val records = new ArrayList[Array[Byte]]()
+ if (queue.drainTo(records, maxPeek) == 0) {
+ if (retry == 0) {
+ Iterable.empty
+ } else {
+ Thread.sleep(gracefulWaitTime.toMillis)
+ peek(duration, retry - 1)
+ }
+ } else {
+ records.map(record => ("", 0, 0L, record))
+ }
+ }
+
+ /**
+ */
+ override def commit(retry: Int): Unit = { /*do nothing*/ }
+
+ override def close(): Unit = {
+ logging.info(this, s"closing lean consumer")
+ }
+}
diff --git a/common/scala/src/main/scala/whisk/connector/lean/LeanMessagingProvider.scala b/common/scala/src/main/scala/whisk/connector/lean/LeanMessagingProvider.scala
new file mode 100644
index 0000000000..b1671d1046
--- /dev/null
+++ b/common/scala/src/main/scala/whisk/connector/lean/LeanMessagingProvider.scala
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package whisk.connector.lean
+
+import java.util.concurrent.BlockingQueue
+import java.util.concurrent.LinkedBlockingQueue
+
+import scala.collection.concurrent.Map
+import scala.collection.concurrent.TrieMap
+import scala.concurrent.duration.FiniteDuration
+import scala.util.Success
+import scala.util.Try
+
+import akka.actor.ActorSystem
+import whisk.common.Logging
+import whisk.core.WhiskConfig
+import whisk.core.connector.MessageConsumer
+import whisk.core.connector.MessageProducer
+import whisk.core.connector.MessagingProvider
+
+/**
+ * A simple implementation of MessagingProvider
+ */
+object LeanMessagingProvider extends MessagingProvider {
+
+ val queues: Map[String, BlockingQueue[Array[Byte]]] =
+ new TrieMap[String, BlockingQueue[Array[Byte]]]
+
+ def getConsumer(config: WhiskConfig, groupId: String, topic: String, maxPeek: Int, maxPollInterval: FiniteDuration)(
+ implicit logging: Logging,
+ actorSystem: ActorSystem): MessageConsumer = {
+
+ var queue = queues.getOrElseUpdate(topic, new LinkedBlockingQueue[Array[Byte]]())
+ new LeanConsumer(queue, maxPeek)
+ }
+
+ def getProducer(config: WhiskConfig)(implicit logging: Logging, actorSystem: ActorSystem): MessageProducer =
+ new LeanProducer(queues)
+
+ def ensureTopic(config: WhiskConfig, topic: String, topicConfig: String)(implicit logging: Logging): Try[Unit] = {
+ if (queues.contains(topic)) {
+ Success(logging.info(this, s"topic $topic already existed"))
+ } else {
+ queues.put(topic, new LinkedBlockingQueue[Array[Byte]](Integer.MAX_VALUE))
+ Success(logging.info(this, s"topic $topic created"))
+ }
+ }
+}
diff --git a/common/scala/src/main/scala/whisk/connector/lean/LeanProducer.scala b/common/scala/src/main/scala/whisk/connector/lean/LeanProducer.scala
new file mode 100644
index 0000000000..4ace4ac2c7
--- /dev/null
+++ b/common/scala/src/main/scala/whisk/connector/lean/LeanProducer.scala
@@ -0,0 +1,60 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package whisk.connector.lean
+
+import akka.actor.ActorSystem
+import scala.concurrent.Future
+import org.apache.kafka.clients.producer.RecordMetadata
+import whisk.common.Counter
+import whisk.common.Logging
+import whisk.core.connector.Message
+import whisk.core.connector.MessageProducer
+
+import java.util.concurrent.{BlockingQueue, LinkedBlockingQueue}
+import scala.collection.concurrent.Map
+import java.nio.charset.StandardCharsets
+import scala.concurrent.ExecutionContext
+
+class LeanProducer(queues: Map[String, BlockingQueue[Array[Byte]]])(implicit logging: Logging, actorSystem: ActorSystem)
+ extends MessageProducer {
+
+ implicit val ec: ExecutionContext = actorSystem.dispatcher
+
+ override def sentCount(): Long = sentCounter.cur
+
+ /** Sends msg to topic. This is an asynchronous operation. */
+ override def send(topic: String, msg: Message, retry: Int = 3): Future[RecordMetadata] = {
+ implicit val transid = msg.transid
+
+ logging.debug(this, s"sending to topic '$topic' msg '$msg'")
+ var queue = queues.getOrElseUpdate(topic, new LinkedBlockingQueue[Array[Byte]]())
+
+ Future {
+ queue.put(msg.serialize.getBytes(StandardCharsets.UTF_8))
+ sentCounter.next()
+ null
+ }
+ }
+
+ /** Closes producer. */
+ override def close(): Unit = {
+ logging.info(this, "closing lean producer")
+ }
+
+ private val sentCounter = new Counter()
+}
diff --git a/core/controller/build.gradle b/core/controller/build.gradle
index b7cfb8ead4..6947db9834 100644
--- a/core/controller/build.gradle
+++ b/core/controller/build.gradle
@@ -43,6 +43,7 @@ dependencies {
compile 'com.lightbend.akka.discovery:akka-discovery-kubernetes-api_2.11:0.11.0'
compile 'com.lightbend.akka.discovery:akka-discovery-marathon-api_2.11:0.11.0'
compile project(':common:scala')
+ compile project(':core:invoker')
scoverage gradle.scoverage.deps
}
diff --git a/core/controller/src/main/scala/whisk/core/loadBalancer/LeanBalancer.scala b/core/controller/src/main/scala/whisk/core/loadBalancer/LeanBalancer.scala
new file mode 100644
index 0000000000..34b78b0fa2
--- /dev/null
+++ b/core/controller/src/main/scala/whisk/core/loadBalancer/LeanBalancer.scala
@@ -0,0 +1,234 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package whisk.core.loadBalancer
+
+import java.nio.charset.StandardCharsets
+import java.util.concurrent.atomic.LongAdder
+
+import akka.actor.{ActorSystem, Props}
+import akka.event.Logging.InfoLevel
+import akka.stream.ActorMaterializer
+import org.apache.kafka.clients.producer.RecordMetadata
+import pureconfig._
+import whisk.spi.SpiLoader
+import whisk.core.entity._
+import whisk.core.entity.size._
+import whisk.common.LoggingMarkers._
+import whisk.common._
+import whisk.core.WhiskConfig._
+import whisk.core.connector._
+import whisk.core.WhiskConfig
+import whisk.core.ConfigKeys
+import scala.collection.concurrent.TrieMap
+import scala.concurrent.duration._
+import scala.concurrent.{ExecutionContext, Future, Promise}
+import scala.util.{Failure, Success}
+import whisk.core.invoker.InvokerReactive
+import whisk.utils.ExecutionContextFactory
+
+/**
+ * Lean loadbalancer implemetation.
+ *
+ * Communicates with Invoker directly without Kafka in the middle. Invoker does not exist as a separate entity, it is built together with Controller
+ * Uses LeanMessagingProvider to use in-memory queue instead of Kafka
+ */
+class LeanBalancer(config: WhiskConfig, controllerInstance: ControllerInstanceId)(implicit val actorSystem: ActorSystem,
+ logging: Logging,
+ materializer: ActorMaterializer)
+ extends LoadBalancer {
+
+ private implicit val executionContext: ExecutionContext = actorSystem.dispatcher
+
+ private val lbConfig = loadConfigOrThrow[ShardingContainerPoolBalancerConfig](ConfigKeys.loadbalancer)
+
+ /** State related to invocations and throttling */
+ private val activations = TrieMap[ActivationId, ActivationEntry]()
+ private val activationsPerNamespace = TrieMap[UUID, LongAdder]()
+ private val totalActivations = new LongAdder()
+ private val totalActivationMemory = new LongAdder()
+
+ actorSystem.scheduler.schedule(0.seconds, 10.seconds) {
+ MetricEmitter.emitHistogramMetric(LOADBALANCER_ACTIVATIONS_INFLIGHT(controllerInstance), totalActivations.longValue)
+ MetricEmitter.emitHistogramMetric(LOADBALANCER_MEMORY_INFLIGHT(controllerInstance), totalActivationMemory.longValue)
+ }
+
+ /** Loadbalancer interface methods */
+ override def invokerHealth(): Future[IndexedSeq[InvokerHealth]] = Future.successful(IndexedSeq.empty[InvokerHealth])
+ override def activeActivationsFor(namespace: UUID): Future[Int] =
+ Future.successful(activationsPerNamespace.get(namespace).map(_.intValue()).getOrElse(0))
+ override def totalActiveActivations: Future[Int] = Future.successful(totalActivations.intValue())
+ override def clusterSize: Int = 1
+
+ val invokerName = InvokerInstanceId(0)
+ val controllerName = ControllerInstanceId("controller-lean")
+
+ /** 1. Publish a message to the loadbalancer */
+ override def publish(action: ExecutableWhiskActionMetaData, msg: ActivationMessage)(
+ implicit transid: TransactionId): Future[Future[Either[ActivationId, WhiskActivation]]] = {
+ val entry = setupActivation(msg, action, invokerName)
+ sendActivationToInvoker(messageProducer, msg, invokerName).map { _ =>
+ entry.promise.future
+ }
+ }
+
+ /** 2. Update local state with the to be executed activation */
+ private def setupActivation(msg: ActivationMessage,
+ action: ExecutableWhiskActionMetaData,
+ instance: InvokerInstanceId): ActivationEntry = {
+
+ totalActivations.increment()
+ totalActivationMemory.add(action.limits.memory.megabytes)
+ activationsPerNamespace.getOrElseUpdate(msg.user.namespace.uuid, new LongAdder()).increment()
+ val timeout = (action.limits.timeout.duration.max(TimeLimit.STD_DURATION) * lbConfig.timeoutFactor) + 1.minute
+ // Install a timeout handler for the catastrophic case where an active ack is not received at all
+ // (because say an invoker is down completely, or the connection to the message bus is disrupted) or when
+ // the active ack is significantly delayed (possibly dues to long queues but the subject should not be penalized);
+ // in this case, if the activation handler is still registered, remove it and update the books.
+ activations.getOrElseUpdate(
+ msg.activationId, {
+ val timeoutHandler = actorSystem.scheduler.scheduleOnce(timeout) {
+ processCompletion(Left(msg.activationId), msg.transid, forced = true, invoker = instance)
+ }
+
+ // please note: timeoutHandler.cancel must be called on all non-timeout paths, e.g. Success
+ ActivationEntry(
+ msg.activationId,
+ msg.user.namespace.uuid,
+ instance,
+ action.limits.memory.megabytes.MB,
+ timeoutHandler,
+ Promise[Either[ActivationId, WhiskActivation]]())
+ })
+ }
+
+ private val messagingProvider = SpiLoader.get[MessagingProvider]
+ private val messageProducer = messagingProvider.getProducer(config)
+
+ /** 3. Send the activation to the invoker */
+ private def sendActivationToInvoker(producer: MessageProducer,
+ msg: ActivationMessage,
+ invoker: InvokerInstanceId): Future[RecordMetadata] = {
+ implicit val transid: TransactionId = msg.transid
+
+ val topic = s"invoker${invoker.toInt}"
+
+ MetricEmitter.emitCounterMetric(LoggingMarkers.LOADBALANCER_ACTIVATION_START)
+ val start = transid.started(
+ this,
+ LoggingMarkers.CONTROLLER_KAFKA,
+ s"posting topic '$topic' with activation id '${msg.activationId}'",
+ logLevel = InfoLevel)
+
+ producer.send(topic, msg).andThen {
+ case Success(status) =>
+ transid.finished(this, start, s"posted to $topic", logLevel = InfoLevel)
+ case Failure(_) =>
+ transid.failed(this, start, s"error on posting to topic $topic")
+ }
+ }
+
+ /**
+ * Subscribes to active acks (completion messages from the invokers), and
+ * registers a handler for received active acks from invokers.
+ */
+ private val activeAckTopic = s"completed${controllerInstance.asString}"
+ private val maxActiveAcksPerPoll = 128
+ private val activeAckPollDuration = 1.second
+ private val activeAckConsumer =
+ messagingProvider.getConsumer(config, activeAckTopic, activeAckTopic, maxPeek = maxActiveAcksPerPoll)
+
+ private val activationFeed = actorSystem.actorOf(Props {
+ new MessageFeed(
+ "activeack",
+ logging,
+ activeAckConsumer,
+ maxActiveAcksPerPoll,
+ activeAckPollDuration,
+ processActiveAck)
+ })
+
+ /** 4. Get the active-ack message and parse it */
+ private def processActiveAck(bytes: Array[Byte]): Future[Unit] = Future {
+ val raw = new String(bytes, StandardCharsets.UTF_8)
+ CompletionMessage.parse(raw) match {
+ case Success(m: CompletionMessage) =>
+ processCompletion(m.response, m.transid, forced = false, invoker = m.invoker)
+ activationFeed ! MessageFeed.Processed
+
+ case Failure(t) =>
+ activationFeed ! MessageFeed.Processed
+ logging.error(this, s"failed processing message: $raw with $t")
+ }
+ }
+
+ /** 5. Process the active-ack and update the state accordingly */
+ private def processCompletion(response: Either[ActivationId, WhiskActivation],
+ tid: TransactionId,
+ forced: Boolean,
+ invoker: InvokerInstanceId): Unit = {
+ val aid = response.fold(l => l, r => r.activationId)
+
+ activations.remove(aid) match {
+ case Some(entry) =>
+ totalActivations.decrement()
+ activationsPerNamespace.get(entry.namespaceId).foreach(_.decrement())
+
+ if (!forced) {
+ entry.timeoutHandler.cancel()
+ entry.promise.trySuccess(response)
+ } else {
+ entry.promise.tryFailure(new Throwable("no active ack received"))
+ }
+
+ logging.info(this, s"${if (!forced) "received" else "forced"} active ack for '$aid'")(tid)
+ // Active acks that are received here are strictly from user actions - health actions are not part of
+ // the load balancer's activation map. Inform the invoker pool supervisor of the user action completion.
+ case None if !forced =>
+ // the entry has already been removed but we receive an active ack for this activation Id.
+ // This happens for health actions, because they don't have an entry in Loadbalancerdata or
+ // for activations that already timed out.
+ logging.info(this, s"received active ack for '$aid' which has no entry")(tid)
+ case None =>
+ // the entry has already been removed by an active ack. This part of the code is reached by the timeout.
+ // As the active ack is already processed we don't have to do anything here.
+ logging.info(this, s"forced active ack for '$aid' which has no entry")(tid)
+ }
+ }
+
+ private def getInvoker() {
+ implicit val ec = ExecutionContextFactory.makeCachedThreadPoolExecutionContext()
+ val actorSystema: ActorSystem =
+ ActorSystem(name = "invoker-actor-system", defaultExecutionContext = Some(ec))
+ val invoker = new InvokerReactive(config, invokerName, messageProducer)(actorSystema, implicitly)
+ }
+
+ getInvoker()
+}
+
+object LeanBalancer extends LoadBalancerProvider {
+
+ override def instance(whiskConfig: WhiskConfig, instance: ControllerInstanceId)(
+ implicit actorSystem: ActorSystem,
+ logging: Logging,
+ materializer: ActorMaterializer): LoadBalancer = new LeanBalancer(whiskConfig, instance)
+
+ def requiredProperties =
+ Map(servicePort -> 8080.toString(), runtimesRegistry -> "") ++
+ ExecManifest.requiredProperties ++
+ wskApiHost
+}
diff --git a/core/lean/Dockerfile b/core/lean/Dockerfile
new file mode 100644
index 0000000000..e1ef8ed56f
--- /dev/null
+++ b/core/lean/Dockerfile
@@ -0,0 +1,48 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more contributor
+# license agreements. See the NOTICE file distributed with this work for additional
+# information regarding copyright ownership. The ASF licenses this file to you
+# under the Apache License, Version 2.0 (the # "License"); you may not use this
+# file except in compliance with the License. You may obtain a copy of the License
+# at:
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software distributed
+# under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
+# CONDITIONS OF ANY KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations under the License.
+#
+
+FROM scala
+
+ENV DOCKER_VERSION 1.12.0
+ENV KUBERNETES_VERSION 1.6.4
+
+RUN apk add --update openssl
+
+# Uncomment to fetch latest version of docker instead: RUN wget -qO- https://get.docker.com | sh
+# Install docker client
+RUN curl -sSL -o docker-${DOCKER_VERSION}.tgz https://get.docker.com/builds/Linux/x86_64/docker-${DOCKER_VERSION}.tgz && \
+tar --strip-components 1 -xvzf docker-${DOCKER_VERSION}.tgz -C /usr/bin docker/docker && \
+tar --strip-components 1 -xvzf docker-${DOCKER_VERSION}.tgz -C /usr/bin docker/docker-runc && \
+rm -f docker-${DOCKER_VERSION}.tgz && \
+chmod +x /usr/bin/docker && \
+chmod +x /usr/bin/docker-runc
+
+# TODO: do we need it in lean? Install swagger-ui
+RUN curl -sSL -o swagger-ui-v3.6.0.tar.gz --no-verbose https://github.com/swagger-api/swagger-ui/archive/v3.6.0.tar.gz && \
+ mkdir swagger-ui && \
+ tar zxf swagger-ui-v3.6.0.tar.gz -C /swagger-ui --strip-components=2 swagger-ui-3.6.0/dist && \
+ rm swagger-ui-v3.6.0.tar.gz && \
+ sed -i s#http://petstore.swagger.io/v2/swagger.json#/api/v1/api-docs#g /swagger-ui/index.html
+
+#
+# Copy app jars
+ADD build/distributions/lean.tar /
+
+COPY init.sh /
+RUN chmod +x init.sh
+
+EXPOSE 8080
+CMD ["./init.sh", "0"]
diff --git a/core/lean/build.gradle b/core/lean/build.gradle
new file mode 100644
index 0000000000..2124780902
--- /dev/null
+++ b/core/lean/build.gradle
@@ -0,0 +1,48 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+apply plugin: 'scala'
+apply plugin: 'application'
+apply plugin: 'eclipse'
+apply plugin: 'org.scoverage'
+
+ext.dockerImageName = 'lean'
+apply from: '../../gradle/docker.gradle'
+distDocker.dependsOn ':common:scala:distDocker', 'distTar'
+
+project.archivesBaseName = "openwhisk-lean"
+
+repositories {
+ mavenCentral()
+}
+
+dependencies {
+ compile "org.scala-lang:scala-library:${gradle.scala.version}"
+ compile 'com.lightbend.akka.management:akka-management-cluster-bootstrap_2.11:0.11.0'
+ compile 'com.lightbend.akka.discovery:akka-discovery-kubernetes-api_2.11:0.11.0'
+ compile 'com.lightbend.akka.discovery:akka-discovery-marathon-api_2.11:0.11.0'
+ compile project(':common:scala')
+ compile project(':core:controller')
+ scoverage gradle.scoverage.deps
+}
+
+tasks.withType(ScalaCompile) {
+ scalaCompileOptions.additionalParameters = gradle.scala.compileFlags
+}
+
+mainClassName = "whisk.core.controller.Controller"
+applicationDefaultJvmArgs = ["-Djava.security.egd=file:/dev/./urandom"]
diff --git a/core/lean/init.sh b/core/lean/init.sh
new file mode 100644
index 0000000000..025318dcbd
--- /dev/null
+++ b/core/lean/init.sh
@@ -0,0 +1,10 @@
+#!/bin/bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more contributor
+# license agreements; and to You under the Apache License, Version 2.0.
+
+./copyJMXFiles.sh
+
+export LEAN_OPTS="$CONTROLLER_OPTS $(./transformEnvironment.sh)"
+
+exec lean/bin/lean "$@"
diff --git a/settings.gradle b/settings.gradle
index 17dd43b631..ba74e2a807 100644
--- a/settings.gradle
+++ b/settings.gradle
@@ -19,6 +19,7 @@ include 'common:scala'
include 'core:controller'
include 'core:invoker'
+include 'core:lean'
include 'tests'
include 'tests:performance:gatling_tests'
diff --git a/tests/build.gradle b/tests/build.gradle
index 15dd333b5d..6cb70d62db 100644
--- a/tests/build.gradle
+++ b/tests/build.gradle
@@ -51,6 +51,21 @@ def leanExcludes = [
'**/MaxActionDurationTests*',
]
+def systemIncludes = [
+ "apigw/healthtests/**",
+ "ha/**",
+ "limits/**",
+ "services/**",
+ "system/basic/**",
+ "system/rest/**",
+ "whisk/core/admin/**",
+ "whisk/core/cli/test/**",
+ "whisk/core/apigw/actions/test/**",
+ "whisk/core/limits/**",
+ "whisk/core/database/test/*CacheConcurrencyTests*",
+ "whisk/core/controller/test/*ControllerApiTests*",
+]
+
ext.testSets = [
"REQUIRE_ONLY_DB" : [
"includes" : [
@@ -66,23 +81,17 @@ ext.testSets = [
]
],
"REQUIRE_SYSTEM" : [
- "includes" : [
- "apigw/healthtests/**",
- "ha/**",
- "limits/**",
- "services/**",
- "system/basic/**",
- "system/rest/**",
- "whisk/core/admin/**",
- "whisk/core/cli/test/**",
- "whisk/core/apigw/actions/test/**",
- "whisk/core/limits/**",
- "whisk/core/database/test/*CacheConcurrencyTests*",
- "whisk/core/controller/test/*ControllerApiTests*",
- ]
+ "includes" : systemIncludes
],
"LEAN" : [
"excludes" : leanExcludes
+ ],
+ "REQUIRE_LEAN_SYSTEM" : [
+ "includes" : systemIncludes,
+
+ // Lean OpenWhisk doesn't use Kafka and has no Kafka installed as part of the setup.
+ // Tests suit below validating KafkaConnector so have to be excluded for Lean System tests
+ "excludes" : ["**/*KafkaConnectorTests*"]
]
]
diff --git a/tests/performance/preparation/deploy-lean.sh b/tests/performance/preparation/deploy-lean.sh
new file mode 100755
index 0000000000..1d99cf5ffd
--- /dev/null
+++ b/tests/performance/preparation/deploy-lean.sh
@@ -0,0 +1,38 @@
+#!/bin/sh
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+set -e
+SCRIPTDIR="$(cd "$(dirname "$0")"; pwd)"
+ROOTDIR="$SCRIPTDIR/../../.."
+
+# Build Openwhisk
+cd $ROOTDIR
+TERM=dumb ./gradlew distDocker -PdockerImagePrefix=testing $GRADLE_PROJS_SKIP
+
+# Deploy Openwhisk
+cd $ROOTDIR/ansible
+ANSIBLE_CMD="$ANSIBLE_CMD -e limit_invocations_per_minute=999999 -e limit_invocations_concurrent=999999 -e controller_client_auth=false"
+
+$ANSIBLE_CMD setup.yml
+
+$ANSIBLE_CMD prereq.yml
+$ANSIBLE_CMD couchdb.yml
+$ANSIBLE_CMD initdb.yml
+$ANSIBLE_CMD wipe.yml
+
+$ANSIBLE_CMD controller-lean.yml
+$ANSIBLE_CMD edge.yml
diff --git a/tools/travis/runLeanSystemTests.sh b/tools/travis/runLeanSystemTests.sh
new file mode 100755
index 0000000000..c7198961cf
--- /dev/null
+++ b/tools/travis/runLeanSystemTests.sh
@@ -0,0 +1,36 @@
+#!/usr/bin/env bash
+
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+set -e
+
+SCRIPTDIR=$(cd $(dirname "$0") && pwd)
+ROOTDIR="$SCRIPTDIR/../.."
+
+cd $ROOTDIR/tools/travis
+
+export ORG_GRADLE_PROJECT_testSetName="REQUIRE_LEAN_SYSTEM"
+export GRADLE_COVERAGE=true
+
+./setupPrereq.sh
+
+./distDocker.sh
+
+./setupLeanSystem.sh
+
+./runTests.sh
diff --git a/tools/travis/setupLeanSystem.sh b/tools/travis/setupLeanSystem.sh
new file mode 100755
index 0000000000..e4afcd392a
--- /dev/null
+++ b/tools/travis/setupLeanSystem.sh
@@ -0,0 +1,32 @@
+#!/bin/bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+set -e
+
+# Build script for Travis-CI.
+SECONDS=0
+SCRIPTDIR=$(cd $(dirname "$0") && pwd)
+ROOTDIR="$SCRIPTDIR/../.."
+
+cd $ROOTDIR/ansible
+
+$ANSIBLE_CMD apigateway.yml
+$ANSIBLE_CMD openwhisk-lean.yml
+$ANSIBLE_CMD routemgmt.yml
+
+echo "Time taken for ${0##*/} is $SECONDS secs"
diff --git a/tools/vagrant/Vagrantfile b/tools/vagrant/Vagrantfile
index 2466e35238..3de3d92228 100644
--- a/tools/vagrant/Vagrantfile
+++ b/tools/vagrant/Vagrantfile
@@ -87,7 +87,7 @@ Vagrant.configure('2') do |config|
echo PATH=${PATH}:${HOME}/bin:${OPENWHISK_HOME}/tools/build >> /etc/environment
cd ${OPENWHISK_HOME}
cd tools/ubuntu-setup
- su vagrant -c 'source all.sh oracle'
+ su vagrant -c 'source all.sh'
echo "`date`: ubuntu-setup-end" >> /tmp/vagrant-times.txt
SCRIPT
@@ -149,7 +149,16 @@ Vagrant.configure('2') do |config|
echo "`date`: deploy-start" >> /tmp/vagrant-times.txt
cd ${ANSIBLE_HOME}
su vagrant -c 'ansible-playbook -i environments/vagrant wipe.yml'
- su vagrant -c 'ansible-playbook -i environments/vagrant openwhisk.yml -e invoker_use_runc=False'
+
+ export LEAN=#{ENV['LEAN']} || "false"
+ if [[ $LEAN == "true" ]]; then
+ # Deploy Lean Openwhisk (consolidated controller + invoker without kafka, zookeeper etc.)
+ su vagrant -c 'ansible-playbook -i environments/vagrant openwhisk-lean.yml -e invoker_use_runc=False -e controller_akka_provider=local'
+ else
+ # Deploy full Openwhisk stack
+ su vagrant -c 'ansible-playbook -i environments/vagrant openwhisk.yml -e invoker_use_runc=False'
+ fi
+
su vagrant -c 'ansible-playbook -i environments/vagrant postdeploy.yml'
su vagrant -c 'ansible-playbook -i environments/vagrant apigateway.yml'
su vagrant -c 'ansible-playbook -i environments/vagrant routemgmt.yml'
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services