You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@openwhisk.apache.org by ra...@apache.org on 2017/06/28 00:22:20 UTC
[incubator-openwhisk] branch master updated: Make OW run with
docker for mac (#1790)
This is an automated email from the ASF dual-hosted git repository.
rabbah pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-openwhisk.git
The following commit(s) were added to refs/heads/master by this push:
new 55e693a Make OW run with docker for mac (#1790)
55e693a is described below
commit 55e693a4d91ea18aed0a01740db0ac2b9f56cddb
Author: Dominic Kim <st...@gmail.com>
AuthorDate: Wed Jun 28 09:22:18 2017 +0900
Make OW run with docker for mac (#1790)
* Add documentation for docker for mac (#1431)
* Replace akka-http client to play-ws client (temporarily)
* Introduce docker-machine environment
---
.gitignore | 2 +-
ansible/README.md | 70 +++++++++++-------
.../{mac => docker-machine}/group_vars/all | 3 -
.../{mac => docker-machine}/hosts.j2.ini | 0
ansible/environments/mac/group_vars/all | 5 --
ansible/environments/mac/hosts | 33 +++++++++
ansible/roles/cli/tasks/download_cli.yml | 6 +-
ansible/setup.yml | 10 +--
.../scala/whisk/core/container/HttpUtils.scala | 1 +
.../whisk/core/container/WhiskContainer.scala | 63 -----------------
tests/build.gradle | 3 +
.../scala/actionContainers/ActionContainer.scala | 10 +--
.../src/test/scala/common}/AkkaHttpUtils.scala | 82 +++++++++++-----------
tools/build/redo | 4 +-
tools/macos/README.md | 82 +++-------------------
tools/macos/{ => docker-machine}/README.md | 47 +++++++++++--
.../macos/{ => docker-machine}/tweak-dockerhost.sh | 0
.../{ => docker-machine}/tweak-dockermachine.sh | 0
18 files changed, 189 insertions(+), 232 deletions(-)
diff --git a/.gitignore b/.gitignore
index 76fd0f8..437ce4c 100644
--- a/.gitignore
+++ b/.gitignore
@@ -43,7 +43,7 @@ node_modules
.idea
# Ansible
-ansible/environments/mac/hosts
+ansible/environments/docker-machine/hosts
ansible/db_local.ini*
ansible/tmp/*
ansible/roles/nginx/files/*.csr
diff --git a/ansible/README.md b/ansible/README.md
index 02ed0ee..2d30324 100644
--- a/ansible/README.md
+++ b/ansible/README.md
@@ -17,40 +17,55 @@ Nothing to be done, Ansible is already installed during vagrant provisioning.
You can skip setup and prereq steps as those have been done by vagrant for you.
You may jump directly to [Deploying Using CouchDB](#deploying-using-couchdb)
-#### Mac users
-It is assumed that a VM has been provisioned using [Docker Machine](../tools/macos/README.md).
-
+#### Docker for Mac users
+```
+sudo easy_install pip
+sudo pip install ansible==2.3.0.0
```
-brew install python
-pip install ansible==2.3.0.0
+Docker for Mac does not provide any official ways to meet some requirements for OpenWhisk.
+You need to depends on the workarounds until Docker provides official methods.
-cd ansible
-ansible-playbook -i environments/mac setup.yml [-e docker_machine_name=whisk]
+If you prefer [Docker-machine](https://docs.docker.com/machine/) to [Docker for mac](https://docs.docker.com/docker-for-mac/), you can follow instructions in [docker-machine/README.md](../tools/macos/docker-machine/README.md).
+
+##### Enable Docker remote API
+
+During the deployment steps, each component communicates with Docker via Docker remote API.
+Currently however, Docker for Mac does not support such a feature.
+
+There are many workarounds for this.
+One way is to use `socat` command to setup proxy for the UNIX socket.
+
+```
+docker run -d -v /var/run/docker.sock:/var/run/docker.sock -p 4243:2375 bobrik/socat TCP4-LISTEN:2375,fork,reuseaddr UNIX-CONNECT:/var/run/docker.sock
```
-**Hint:** If you omit the optional `-e docker_machine_name` parameter, it will default to "whisk".
-If your docker-machine VM has a different name you may pass it via the `-e docker_machine_name` parameter.
+If you want to deploy OpenWhisk in distributed environment, you are required to enable Docker remote API in all Mac hosts.
-After this there should be a `hosts` file in the `ansible/environments/mac` directory.
+##### Activate docker0 network
+This is an optional step for local deployment.
+The OpenWhisk deployment via Ansible uses the `docker0` network interface to deploy OpenWhisk and it does not exist on Docker for Mac environment.
-To verify the hosts file you can do a quick ping to the docker machine:
+An expedient workaround is to add alias for `docker0` network to loopback interface.
```
-cd ansible
-ansible all -i environments/mac -m ping
+sudo ifconfig lo0 alias 172.17.0.1/24
```
-Should result in something like:
+##### Setup proxy container to run unit tests (optional)
+
+This step is only required to run tests with Docker for Mac.
+If you do not run tests locally, you can just skip this step.
```
-ansible | SUCCESS => {
- "changed": false,
- "ping": "pong"
-}
-192.168.99.100 | SUCCESS => {
- "changed": false,
- "ping": "pong"
-}
+docker run -d -p 3128:3128 style95/squid:3.5.26-p1
+```
+
+You need to configure gradle proxy settings.
+
+**~/.gradle/gradle.properties**
+```
+systemProp.http.proxyHost=localhost
+systemProp.http.proxyPort=3128
```
### Using Ansible
@@ -127,6 +142,7 @@ cd ansible
ansible-playbook -i environments/<environment> couchdb.yml
ansible-playbook -i environments/<environment> initdb.yml
ansible-playbook -i environments/<environment> wipe.yml
+ansible-playbook -i environments/<environment> apigateway.yml
ansible-playbook -i environments/<environment> openwhisk.yml
ansible-playbook -i environments/<environment> postdeploy.yml
```
@@ -145,6 +161,7 @@ cd <openwhisk_home>
cd ansible
ansible-playbook -i environments/<environment> initdb.yml
ansible-playbook -i environments/<environment> wipe.yml
+ansible-playbook -i environments/<environment> apigateway.yml
ansible-playbook -i environments/<environment> openwhisk.yml
ansible-playbook -i environments/<environment> postdeploy.yml
```
@@ -176,7 +193,7 @@ cd ansible
ansible-playbook -i environments/<environment> invoker.yml -e docker_image_tag=myNewInvoker
```
-**Hint:** You can omit the docker image tag parameters in which case `latest` will be used implicitly.
+**Hint:** You can omit the Docker image tag parameters in which case `latest` will be used implicitly.
### Cleaning a Single Component
You can remove a single component just as you would remove the entire deployment stack.
@@ -244,6 +261,12 @@ An expedient workaround is to create a link to the expected location:
ln -s $(which python) /usr/local/bin/python
```
+Alternatively, you can also configure the location of Python interpreter in `environments/<environment>/group_vars`.
+
+```
+ansible_python_interpreter: "/usr/local/bin/python"
+```
+
#### Spaces in Paths
Ansible 2.1.0.0 and earlier versions do not support a space in file paths.
Many file imports and roles will not work correctly when included from a path that contains spaces.
@@ -274,4 +297,3 @@ limits:
- The `concurrent` under `limits->actions->invokes` represents the maximum concurrent invocations allowed per namespace.
- The `concurrentInSystem` under `limits->actions->invokes` represents the maximum concurrent invocations the system will allow across all namespaces.
- The `perMinute` under `limits->triggers-fires` represents the allowed namespace trigger firings per minute.
-
diff --git a/ansible/environments/mac/group_vars/all b/ansible/environments/docker-machine/group_vars/all
similarity index 87%
copy from ansible/environments/mac/group_vars/all
copy to ansible/environments/docker-machine/group_vars/all
index 051358f..6d14d1d 100644
--- a/ansible/environments/mac/group_vars/all
+++ b/ansible/environments/docker-machine/group_vars/all
@@ -38,6 +38,3 @@ apigw_host_v2: "http://{{ groups['apigateway']|first }}:{{apigateway.port.api}}/
# RunC enablement
invoker_use_runc: true
invoker_use_reactive_pool: false
-
-controller_arguments: '-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.port=1098'
-invoker_arguments: "{{ controller_arguments }}"
diff --git a/ansible/environments/mac/hosts.j2.ini b/ansible/environments/docker-machine/hosts.j2.ini
similarity index 100%
rename from ansible/environments/mac/hosts.j2.ini
rename to ansible/environments/docker-machine/hosts.j2.ini
diff --git a/ansible/environments/mac/group_vars/all b/ansible/environments/mac/group_vars/all
old mode 100644
new mode 100755
index 051358f..8d9a71c
--- a/ansible/environments/mac/group_vars/all
+++ b/ansible/environments/mac/group_vars/all
@@ -4,11 +4,6 @@ whisk_logs_dir: /Users/Shared/wsklogs
docker_registry: ""
docker_dns: ""
-# The whisk_api_localhost_name is used to configure nginx to permit vanity URLs for web actions.
-# It is also used for the SSL certificate generation. For a local deployment, this is typically
-# a hostname that is resolved on the client, via /etc/hosts for example.
-whisk_api_localhost_name: "openwhisk"
-
db_prefix: "{{ ansible_user_id|lower }}_{{ ansible_hostname|lower }}_"
# Auto lookup to find the db credentials
diff --git a/ansible/environments/mac/hosts b/ansible/environments/mac/hosts
new file mode 100644
index 0000000..0cbc0d5
--- /dev/null
+++ b/ansible/environments/mac/hosts
@@ -0,0 +1,33 @@
+; the first parameter in a host is the inventory_hostname which has to be
+; either an ip
+; or a resolvable hostname
+
+; used for local actions only
+ansible ansible_connection=local
+
+[edge]
+172.17.0.1 ansible_connection=local
+
+[controllers]
+172.17.0.1 ansible_connection=local
+
+[kafka]
+172.17.0.1 ansible_connection=local
+
+; the consul_servers group has maximum 5 machines
+[consul_servers]
+172.17.0.1 ansible_connection=local
+
+[invokers]
+172.17.0.1 ansible_connection=local
+
+; db group is only used if db_provider is CouchDB
+[db]
+172.17.0.1 ansible_connection=local
+
+[redis]
+172.17.0.1 ansible_connection=local
+
+[apigateway]
+172.17.0.1 ansible_connection=local
+
diff --git a/ansible/roles/cli/tasks/download_cli.yml b/ansible/roles/cli/tasks/download_cli.yml
index df18b3d..7ac9209 100644
--- a/ansible/roles/cli/tasks/download_cli.yml
+++ b/ansible/roles/cli/tasks/download_cli.yml
@@ -9,13 +9,13 @@
dest="{{ openwhisk_home }}/bin/wsk"
mode=0755
validate_certs=False
- when: "'environments/mac' not in inventory_dir"
+ when: "'environments/docker-machine' not in inventory_dir"
-- name: "download cli (mac) to openwhisk home at {{ openwhisk_home }}"
+- name: "download cli (docker-machine) to openwhisk home at {{ openwhisk_home }}"
local_action: >
get_url
url="https://{{ groups['edge'] | first }}/cli/go/download/mac/amd64/wsk"
dest="{{ openwhisk_home }}/bin/wsk"
mode=0755
validate_certs=False
- when: "'environments/mac' in inventory_dir"
+ when: "'environments/docker-machine' in inventory_dir"
diff --git a/ansible/setup.yml b/ansible/setup.yml
index dd5561e..78e187a 100644
--- a/ansible/setup.yml
+++ b/ansible/setup.yml
@@ -6,21 +6,21 @@
- name: find the ip of docker-machine
local_action: shell "docker-machine" "ip" "{{docker_machine_name | default('whisk')}}"
register: result
- when: "'environments/mac' in inventory_dir"
+ when: "'environments/docker-machine' in inventory_dir"
- name: get the docker-machine ip
set_fact:
docker_machine_ip: "{{ result.stdout }}"
- when: "'environments/mac' in inventory_dir"
+ when: "'environments/docker-machine' in inventory_dir"
- name: gen hosts for docker-machine
- local_action: template src="{{playbook_dir}}/environments/mac/hosts.j2.ini" dest="{{ playbook_dir }}/environments/mac/hosts"
- when: "'environments/mac' in inventory_dir"
+ local_action: template src="{{playbook_dir}}/environments/docker-machine/hosts.j2.ini" dest="{{ playbook_dir }}/environments/docker-machine/hosts"
+ when: "'environments/docker-machine' in inventory_dir"
# this step is needed to generate db_local.ini which is required by later steps
- name: add new db host on-the-fly
add_host: name={{ docker_machine_ip }} groups=db
- when: "'environments/mac' in inventory_dir"
+ when: "'environments/docker-machine' in inventory_dir"
- name: check if db_local.ini exists?
stat: path="{{ playbook_dir }}/db_local.ini"
diff --git a/core/invoker/src/main/scala/whisk/core/container/HttpUtils.scala b/core/invoker/src/main/scala/whisk/core/container/HttpUtils.scala
index e508a26..e53ddb2 100644
--- a/core/invoker/src/main/scala/whisk/core/container/HttpUtils.scala
+++ b/core/invoker/src/main/scala/whisk/core/container/HttpUtils.scala
@@ -132,5 +132,6 @@ protected[core] class HttpUtils(
private val connection = HttpClientBuilder
.create
.setDefaultRequestConfig(httpconfig)
+ .useSystemProperties()
.build
}
diff --git a/core/invoker/src/main/scala/whisk/core/container/WhiskContainer.scala b/core/invoker/src/main/scala/whisk/core/container/WhiskContainer.scala
index 2e0e378..4685693 100644
--- a/core/invoker/src/main/scala/whisk/core/container/WhiskContainer.scala
+++ b/core/invoker/src/main/scala/whisk/core/container/WhiskContainer.scala
@@ -21,18 +21,9 @@ import java.time.Clock
import java.time.Instant
import java.util.concurrent.atomic.AtomicInteger
-import scala.concurrent.Await
-import scala.concurrent.Future
import scala.concurrent.duration.DurationInt
import scala.concurrent.duration.FiniteDuration
-
import akka.actor.ActorSystem
-import akka.http.scaladsl.Http
-import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport._
-import akka.http.scaladsl.marshalling._
-import akka.http.scaladsl.model._
-import akka.http.scaladsl.unmarshalling._
-import akka.stream.ActorMaterializer
import spray.json._
import whisk.common.Logging
import whisk.common.LoggingMarkers
@@ -144,60 +135,6 @@ class WhiskContainer(
sendPayloadApache(endpoint, msg, timeout, retry)
}
- private def sendPayloadAkka(endpoint: String, msg: JsObject, timeout: FiniteDuration)(implicit system: ActorSystem): RunResult = {
- import system.dispatcher
-
- val start = ContainerCounter.now()
-
- val f = sendPayloadAsync(endpoint, msg, timeout)
-
- f.onFailure {
- case t: Throwable =>
- logging.warn(this, s"Exception while posting to action container ${t.getMessage}")
- }
-
- // Should never timeout because the future has a built-in timeout.
- // Keeping a finite duration for safety.
- Await.ready(f, timeout + 1.minute)
-
- val end = ContainerCounter.now()
-
- val r = f.value.get.toOption.flatten
- RunResult(Interval(start, end), ???)
- }
-
- /**
- * Asynchronously posts a message to the container.
- *
- * @param msg the message to post
- * @return response from the container if any
- */
- private def sendPayloadAsync(endpoint: String, msg: JsObject, timeout: FiniteDuration)(implicit system: ActorSystem): Future[Option[(Int, String)]] = {
- implicit val ec = system.dispatcher
- implicit val materializer = ActorMaterializer()
-
- containerHostAndPort map { hp =>
-
- val flow = Http().outgoingConnection(hp.host, hp.port)
-
- val uri = Uri(
- scheme = "http",
- authority = Uri.Authority(host = Uri.Host(hp.host), port = hp.port),
- path = Uri.Path(endpoint))
-
- for (
- entity <- Marshal(msg).to[MessageEntity];
- request = HttpRequest(method = HttpMethods.POST, uri = uri, entity = entity);
- response <- AkkaHttpUtils.singleRequest(request, timeout, retryOnTCPErrors = true, retryInterval = 100.milliseconds);
- responseBody <- Unmarshal(response.entity).to[String]
- ) yield {
- Some((response.status.intValue, responseBody))
- }
- } getOrElse {
- Future.successful(None)
- }
- }
-
private def sendPayloadApache(endpoint: String, msg: JsObject, timeout: FiniteDuration, retry: Boolean): RunResult = {
val start = ContainerCounter.now()
diff --git a/tests/build.gradle b/tests/build.gradle
index f80e47e..3a07e70 100644
--- a/tests/build.gradle
+++ b/tests/build.gradle
@@ -7,6 +7,8 @@ repositories {
}
tasks.withType(Test) {
+ systemProperties(System.getProperties())
+
testLogging {
events "passed", "skipped", "failed"
showStandardStreams = true
@@ -50,6 +52,7 @@ dependencies {
compile 'com.typesafe.akka:akka-testkit_2.11:2.4.16'
compile 'com.google.code.gson:gson:2.3.1'
compile 'org.scalamock:scalamock-scalatest-support_2.11:3.4.2'
+ compile 'com.typesafe.play:play-ws_2.11:2.5.11'
compile project(':common:scala')
compile project(':core:controller')
diff --git a/tests/src/test/scala/actionContainers/ActionContainer.scala b/tests/src/test/scala/actionContainers/ActionContainer.scala
index f74f3c4..978114d 100644
--- a/tests/src/test/scala/actionContainers/ActionContainer.scala
+++ b/tests/src/test/scala/actionContainers/ActionContainer.scala
@@ -161,12 +161,10 @@ object ActionContainer {
private def syncPost(host: String, port: Int, endPoint: String, content: JsValue)(
implicit actorSystem: ActorSystem): (Int, Option[JsObject]) = {
- import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport._
- import akka.http.scaladsl.marshalling._
import akka.http.scaladsl.model._
import akka.http.scaladsl.unmarshalling._
import akka.stream.ActorMaterializer
- import whisk.core.container.AkkaHttpUtils
+ import common.AkkaHttpUtils
implicit val materializer = ActorMaterializer()
@@ -176,10 +174,8 @@ object ActionContainer {
path = Uri.Path(endPoint))
val f = for (
- entity <- Marshal(content).to[MessageEntity];
- request = HttpRequest(method = HttpMethods.POST, uri = uri, entity = entity);
- response <- AkkaHttpUtils.singleRequest(request, 90.seconds, retryOnTCPErrors = true);
- responseBody <- Unmarshal(response.entity).to[String]
+ response <- AkkaHttpUtils.singleRequest(uri.toString(), content, 90.seconds, retryOnTCPErrors = true);
+ responseBody <- Unmarshal(response.body).to[String]
) yield (response.status.intValue, Try(responseBody.parseJson.asJsObject).toOption)
Await.result(f, 90.seconds)
diff --git a/core/invoker/src/main/scala/whisk/core/container/AkkaHttpUtils.scala b/tests/src/test/scala/common/AkkaHttpUtils.scala
similarity index 60%
rename from core/invoker/src/main/scala/whisk/core/container/AkkaHttpUtils.scala
rename to tests/src/test/scala/common/AkkaHttpUtils.scala
index ab48a9a..46faa69 100644
--- a/core/invoker/src/main/scala/whisk/core/container/AkkaHttpUtils.scala
+++ b/tests/src/test/scala/common/AkkaHttpUtils.scala
@@ -15,34 +15,45 @@
* limitations under the License.
*/
-package whisk.core.container
-
-import scala.concurrent.Await
-import scala.concurrent.Future
-import scala.concurrent.Promise
-import scala.concurrent.duration.Duration
-import scala.concurrent.duration.DurationInt
-import scala.concurrent.duration.FiniteDuration
-import scala.util.Try
+package common
+
+import java.util.concurrent.TimeoutException
+
import akka.actor.ActorSystem
-import akka.http.scaladsl._
import akka.http.scaladsl.model._
import akka.stream.ActorMaterializer
-import akka.stream.scaladsl._
-import java.util.concurrent.TimeoutException
+import play.api.http.{ContentTypeOf, Writeable}
+import play.api.libs.ws.WSResponse
+import play.api.libs.ws.ahc.AhcWSClient
+import play.api.mvc.Codec
+import spray.json.JsValue
+
+import scala.concurrent.{Await, Future, Promise}
+import scala.concurrent.duration.{Duration, DurationInt, FiniteDuration}
+import scala.util.Try
object AkkaHttpUtils {
+
+ // Writable for spray-json is required
+ implicit def sprayJsonContentType(implicit codec: Codec): ContentTypeOf[JsValue] = {
+ ContentTypeOf[JsValue](Some(ContentTypes.`application/json`.toString()))
+ }
+ implicit def sprayJsonWriteable(implicit codec: Codec): Writeable[JsValue] = {
+ Writeable(message => codec.encode(message.toString()))
+ }
+
def singleRequestBlocking(
- request: HttpRequest,
+ uri: String,
+ content: JsValue,
timeout: FiniteDuration,
retryOnTCPErrors: Boolean = false,
retryOn4xxErrors: Boolean = false,
retryOn5xxErrors: Boolean = false,
retryInterval: FiniteDuration = 100.milliseconds)
- (implicit system: ActorSystem) : Try[HttpResponse] = {
+ (implicit system: ActorSystem) : Try[WSResponse] = {
val f = singleRequest(
- request, timeout, retryOnTCPErrors, retryOn4xxErrors, retryOn5xxErrors, retryInterval
+ uri, content, timeout, retryOnTCPErrors, retryOn4xxErrors, retryOn5xxErrors, retryInterval
)
// Duration.Inf is not an issue, since singleRequest has a built-in timeout mechanism.
@@ -54,24 +65,21 @@ object AkkaHttpUtils {
// Makes a request, expects a successful within timeout, retries on selected
// errors until timeout has passed.
def singleRequest(
- request: HttpRequest,
- timeout: FiniteDuration,
- retryOnTCPErrors: Boolean = false,
- retryOn4xxErrors: Boolean = false,
- retryOn5xxErrors: Boolean = false,
- retryInterval: FiniteDuration = 100.milliseconds)
- (implicit system: ActorSystem) : Future[HttpResponse] = {
-
+ uri: String,
+ content: JsValue,
+ timeout: FiniteDuration,
+ retryOnTCPErrors: Boolean = false,
+ retryOn4xxErrors: Boolean = false,
+ retryOn5xxErrors: Boolean = false,
+ retryInterval: FiniteDuration = 100.milliseconds)
+ (implicit system: ActorSystem) : Future[WSResponse] = {
implicit val executionContext = system.dispatcher
implicit val materializer = ActorMaterializer()
+ val wsClient = AhcWSClient()
- val timeoutException = new TimeoutException(s"Request to ${request.uri.authority} could not be completed in time.")
-
- val authority = request.uri.authority
- val relativeRequest = request.copy(uri = request.uri.toRelative)
- val flow = Http().outgoingConnection(authority.host.address, authority.port)
+ val timeoutException = new TimeoutException(s"Request to ${uri} could not be completed in time.")
- val promise = Promise[HttpResponse]
+ val promise = Promise[WSResponse]
// Timeout includes all retries.
system.scheduler.scheduleOnce(timeout) {
@@ -79,25 +87,19 @@ object AkkaHttpUtils {
}
def tryOnce() : Unit = if(!promise.isCompleted) {
- val f = Source.single(relativeRequest).via(flow).runWith(Sink.head)
-
+ val f = wsClient.url(uri).withRequestTimeout(timeout).post(content)
f.onSuccess {
- case r if r.status.intValue >= 400 && r.status.intValue < 500 && retryOn4xxErrors =>
- // need to drain the response to close the connection
- r.entity.dataBytes.runWith(Sink.ignore)
+ case r if r.status >= 400 && r.status < 500 && retryOn4xxErrors =>
system.scheduler.scheduleOnce(retryInterval) { tryOnce() }
-
- case r if r.status.intValue >= 500 && r.status.intValue < 600 && retryOn5xxErrors =>
- // need to drain the response to close the connection
- r.entity.dataBytes.runWith(Sink.ignore)
+ case r if r.status >= 500 && r.status < 600 && retryOn5xxErrors =>
system.scheduler.scheduleOnce(retryInterval) { tryOnce() }
-
case r =>
+ wsClient.close()
promise.trySuccess(r)
}
f.onFailure {
- case s : akka.stream.StreamTcpException if retryOnTCPErrors =>
+ case s : java.net.ConnectException if retryOnTCPErrors =>
// TCP error (e.g. connection couldn't be opened)
system.scheduler.scheduleOnce(retryInterval) { tryOnce() }
diff --git a/tools/build/redo b/tools/build/redo
index 321b60b..0396648 100755
--- a/tools/build/redo
+++ b/tools/build/redo
@@ -74,7 +74,7 @@ def getArgs():
if osname == 'Linux':
return 'local'
elif osname == 'Darwin':
- return 'mac'
+ return 'docker-machine'
else:
return None
@@ -82,7 +82,7 @@ def getArgs():
parser.add_argument('-b', '--build', help='build component', action='store_const', const=True, default=False)
parser.add_argument('-x', '--teardown', help='teardown component', action='store_const', const=True, default=False)
parser.add_argument('-d', '--deploy', help='deploy component', action='store_const', const=True, default=False)
- parser.add_argument('-t', '--target', help='deploy target (one of [mac, local])', default=detectDeployTarget())
+ parser.add_argument('-t', '--target', help='deploy target (one of [mac, docker-machine, local])', default=detectDeployTarget())
parser.add_argument('-y', '--yaml', help='deploy target using inferred YAML file if component is not one of known targets', action='store_const', const=True, default=False)
parser.add_argument('-g', '--gradle', help='use target using inferred gradle file if component is not one of known targets', action='store_const', const=True, default=False)
parser.add_argument('-n', '--just-print', help='prints the component configuration but does not run any targets', action='store_const', const=True, default=False, dest='skiprun')
diff --git a/tools/macos/README.md b/tools/macos/README.md
index d3b46dd..c16541a 100644
--- a/tools/macos/README.md
+++ b/tools/macos/README.md
@@ -1,18 +1,18 @@
-# How to setup a Mac host for OpenWhisk
+# Setting up OpenWhisk with Docker for Mac
-One way to develop or deploy OpenWhisk on a Mac is to use [docker-machine](https://docs.docker.com/machine/install-machine/) which runs the Docker daemon inside a virtual machine accessible from the Mac host.
+OpenWhisk can run on a Mac host with [Docker for Mac](https://docs.docker.com/docker-for-mac/).
+If you prefer to use Docker-machine, you can follow instructions in [docker-machine/README.md](docker-machine/README.md)
# Prerequisites
The following are required to build and deploy OpenWhisk from a Mac host:
-- [Oracle VM VirtualBox](https://www.virtualbox.org/wiki/Downloads)
-- [Docker 1.12.0](https://docs.docker.com/engine/installation/mac/) (including `docker-machine`)
+- [Docker 1.12.0](https://docs.docker.com/docker-for-mac/)
- [Java 8](http://www.oracle.com/technetwork/java/javase/downloads/index.html)
- [Scala 2.11](http://scala-lang.org/download/)
-- [Ansible 2.2.1.0](http://docs.ansible.com/ansible/intro_installation.html)
+- [Ansible 2.3.0.0](http://docs.ansible.com/ansible/intro_installation.html)
-**Tip** Versions of Docker and Ansible are lower than the latest released versions, the versions used in OpenWhisk are pinned to have stability during continues integration and deployment.
+**Tip** Versions of Docker and Ansible are lower than the latest released versions, the versions used in OpenWhisk are pinned to have stability during continuous integration and deployment.
[Homebrew](http://brew.sh/) is an easy way to install all of these and prepare your Mac to build and deploy OpenWhisk. The following shell command is provided for your convenience to install `brew` with [Cask](https://github.com/caskroom/homebrew-cask) and bootstraps these to complete the setup. Copy the entire section below and paste it into your terminal to run it.
@@ -23,82 +23,18 @@ echo '
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
# install cask
brew tap caskroom/cask
-# install virtualbox
-brew cask install virtualbox
-# install docker 1.12.0
-brew install https://raw.githubusercontent.com/Homebrew/homebrew-core/33301827c3d770bfd49f0e50d84e0b125b06b0b7/Formula/docker.rb
-# install docker-machine
-brew install docker-machine
# install java 8
brew cask install java
# install scala
brew install scala
# install pip
sudo easy_install pip
+# install docker for python
+/usr/local/bin/pip install docker=2.2.1
# install script prerequisites
sudo -H pip install ansible==2.3.0.0 jsonschema couchdb' | bash
```
-# Create and configure Docker machine
-
-It is recommended that you create a virtual machine `whisk` with at least 4GB of RAM.
-
-```
-docker-machine create -d virtualbox \
- --virtualbox-memory 4096 \
- --virtualbox-boot2docker-url=https://github.com/boot2docker/boot2docker/releases/download/v1.12.0/boot2docker.iso \
- whisk # the name of your docker machine
-```
-
-The Docker virtual machine requires some tweaking to work from the Mac host with OpenWhisk.
-The following [script](./tweak-dockermachine.sh) will disable TLS, add port forwarding
-within the VM and routes `172.17.x.x` from the Mac host to the Docker virtual machine.
-Enter your sudo Mac password when prompted.
-
-```
-cd /your/path/to/openwhisk
-./tools/macos/tweak-dockermachine.sh
-```
-
-The final output of the script should resemble the following two lines.
-```
-Run the following:
-export DOCKER_HOST="tcp://192.168.99.100:4243" # your Docker virtual machine IP may vary
-```
-
-The Docker host reported by `docker-machine ip whisk` will give you the IP address.
-Currently, the system requires that you use port `4243` to communicate with the Docker host
-from OpenWhisk.
-
-Ignore errors messages from `docker-machine ls` for the `whisk` virtual machine, this is due
-to the configuration of the port `4243` vs. `2376`
-```
-NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS
-whisk - virtualbox Running tcp://192.168.99.100:2376 Unknown Unable to query docker version: Cannot connect to the docker engine endpoint
-```
-
-To verify that docker is configure properly with `docker-machine` run `docker ps`, you should not see any errors. Here is an example output:
-```
-CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
-
-```
-
-You may find it convenient to set these environment variables in your bash profile (e.g., `~/.bash_profile` or `~/.profile`).
-```
-export OPENWHISK_HOME=/your/path/to/openwhisk
-export DOCKER_HOST=tcp://$(docker-machine ip whisk):4243
-```
-
-The tweaks to the Docker machine persist across reboots.
-However one of the tweaks is applied on the Mac host and must be applied
-again if you reboot your Mac. Without it, some tests which require direct
-communication with Docker containers will fail. To run just the Mac host tweaks,
-run the following [script](./tweak-dockerhost.sh). Enter your sudo Mac password when prompted.
-```
-cd /your/path/to/openwhisk
-./tools/macos/tweak-dockerhost.sh
-```
-
# Build
```
cd /your/path/to/openwhisk
@@ -110,7 +46,7 @@ cd /your/path/to/openwhisk
Follow instructions in [ansible/README.md](../../ansible/README.md)
### Configure the CLI
-Follow instructions in [Configure CLI](../../docs/README.md#setting-up-the-openwhisk-cli)
+Follow instructions in [Configure CLI](../../../docs/README.md#setting-up-the-openwhisk-cli)
### Use the wsk CLI
```
diff --git a/tools/macos/README.md b/tools/macos/docker-machine/README.md
similarity index 76%
copy from tools/macos/README.md
copy to tools/macos/docker-machine/README.md
index d3b46dd..65f9f43 100644
--- a/tools/macos/README.md
+++ b/tools/macos/docker-machine/README.md
@@ -1,6 +1,7 @@
-# How to setup a Mac host for OpenWhisk
+# Setting up OpenWhisk with Docker-machine
-One way to develop or deploy OpenWhisk on a Mac is to use [docker-machine](https://docs.docker.com/machine/install-machine/) which runs the Docker daemon inside a virtual machine accessible from the Mac host.
+OpenWhisk can on a Mac using a virtual machine in which Docker daemon is running.
+You will make provision of a virtual machine with Docker-machine and communicate with them via Docker remote API.
# Prerequisites
@@ -57,7 +58,7 @@ Enter your sudo Mac password when prompted.
```
cd /your/path/to/openwhisk
-./tools/macos/tweak-dockermachine.sh
+./tools/macos/docker-machine/tweak-dockermachine.sh
```
The final output of the script should resemble the following two lines.
@@ -96,7 +97,7 @@ communication with Docker containers will fail. To run just the Mac host tweaks,
run the following [script](./tweak-dockerhost.sh). Enter your sudo Mac password when prompted.
```
cd /your/path/to/openwhisk
-./tools/macos/tweak-dockerhost.sh
+./tools/macos/docker-machine/tweak-dockerhost.sh
```
# Build
@@ -107,10 +108,44 @@ cd /your/path/to/openwhisk
**Tip** Using `gradlew` handles the installation of the correct version of gradle to use.
# Deploy
-Follow instructions in [ansible/README.md](../../ansible/README.md)
+
+```
+brew install python
+pip install ansible==2.3.0.0
+
+cd ansible
+ansible-playbook -i environments/docker-machine setup.yml [-e docker_machine_name=whisk]
+```
+
+**Hint:** If you omit the optional `-e docker_machine_name` parameter, it will default to "whisk".
+If your docker-machine VM has a different name you may pass it via the `-e docker_machine_name` parameter.
+
+After this there should be a `hosts` file in the `ansible/environments/docker-machine` directory.
+
+To verify the hosts file you can do a quick ping to the docker machine:
+
+```
+cd ansible
+ansible all -i environments/docker-machine -m ping
+```
+
+Should result in something like:
+
+```
+ansible | SUCCESS => {
+ "changed": false,
+ "ping": "pong"
+}
+192.168.99.100 | SUCCESS => {
+ "changed": false,
+ "ping": "pong"
+}
+```
+
+Follow remaining instructions from [Using Ansible](../../../ansible/README.md#using-ansible) section in [ansible/README.md](../../../ansible/README.md)
### Configure the CLI
-Follow instructions in [Configure CLI](../../docs/README.md#setting-up-the-openwhisk-cli)
+Follow instructions in [Configure CLI](../../../docs/README.md#setting-up-the-openwhisk-cli)
### Use the wsk CLI
```
diff --git a/tools/macos/tweak-dockerhost.sh b/tools/macos/docker-machine/tweak-dockerhost.sh
old mode 100755
new mode 100644
similarity index 100%
rename from tools/macos/tweak-dockerhost.sh
rename to tools/macos/docker-machine/tweak-dockerhost.sh
diff --git a/tools/macos/tweak-dockermachine.sh b/tools/macos/docker-machine/tweak-dockermachine.sh
old mode 100755
new mode 100644
similarity index 100%
rename from tools/macos/tweak-dockermachine.sh
rename to tools/macos/docker-machine/tweak-dockermachine.sh
--
To stop receiving notification emails like this one, please contact
['"commits@openwhisk.apache.org" <co...@openwhisk.apache.org>'].