You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@openwhisk.apache.org by 张海勇 <zh...@mail.ustc.edu.cn> on 2017/07/12 12:34:31 UTC
question about resource allocation and schedule with deploying
openwhisk on kubernetes
Hi Daniel Lavine,
when deploying openwhisk on kubernetes, if a job runs on openwhisk , how to alloc and free resource , openwhisk is a manager or kubernetes is a manager?
if there are too many jobs ,how to load balance? how to span job to other node? restart openwhisk components or restart jobs thread?
Best regards,
Zhang Haiyong
-----原始邮件-----
发件人:"张海勇" <zh...@mail.ustc.edu.cn>
发送时间:2017-07-10 17:51:30 (星期一)
收件人: "张海勇" <zh...@mail.ustc.edu.cn>, "Daniel Lavine" <dl...@us.ibm.com>
抄送: "Daniel Lavine" <dl...@us.ibm.com>, dev@openwhisk.incubator.apache.org
主题: Re: issue about invoker
Hi Daniel Lavine,
I add invoker to the script file .travis/build.sh
kubectl apply -f configure/configure_whisk.ym
#setup invoker
kubectl apply -f ansible/environments/kube/files/invoker-service.yml
kubectl apply -f ansible/environments/kube/files/invoker.yml
The controller is failed to connect to invoker with the image adobeapiplatform/whisk-invoker
[2017-07-10T09:33:31.771Z] [INFO] [#tid_40] [ActionsApi] action activation id: 61551b15635f43268c31cd708db1966d [marker:controller_loadbalancer_start:22]
[2017-07-10T09:33:31.775Z] [ERROR] [#sid_121] [LoadBalancerService] all invokers down
[2017-07-10T09:33:31.775Z] [ERROR] [#tid_40] [ActionsApi] [POST] action activation failed: no invokers available
[2017-07-10T09:33:31.779Z] [INFO] [#tid_40] [BasicHttpService] [marker:http_post.500_count:31:31]
The controller is setup successfully with the image openwhisk/invoker
[2017-07-10T09:41:35.452Z] [INFO] [#sid_120] [LoadBalancerService] test action for invoker health now exists
[2017-07-10T09:41:35.502Z] [INFO] Bound to /0.0.0.0:8080
[2017-07-10T09:41:35.515Z] [INFO] [#sid_120] [CouchDbRestStore] [PUT] 'openwhisk_kube_whisks' completed document: 'id: whisk.system/invokerHealthTestAction, rev: null', response: '{"ok":true,"id":"whisk.system/invokerHealthTestAction","rev":"1-6e6565bab3cb9030d9d2df48413a688c"}' [marker:database_saveDocument_finish:1581:78]
[2017-07-10T09:41:35.522Z] [INFO] [#sid_120] [WhiskAction] write all done, caching id: whisk.system/invokerHealthTestAction Cached
[2017-07-10T09:41:35.523Z] [INFO] [#sid_130] [RateThrottler] actions per minute: maxPerMinute = 120
[2017-07-10T09:41:35.523Z] [INFO] [#sid_130] [RateThrottler] triggers per minute: maxPerMinute = 60
[2017-07-10T09:41:35.524Z] [INFO] [??] [ActivationThrottler] concurrencyLimit = 100, systemOverloadLimit = 5000
[2017-07-10T09:41:35.606Z] [INFO] [??] [ActionsApi] actionSequenceLimit '50'
But the invoker is not configured
kubectl -n openwhisk logs invoker-0
[2017-07-10T09:42:35.001Z] [INFO] Slf4jLogger started
[2017-07-10T09:42:35.044Z] [INFO] [??] [Config] environment set value for consulserver.host
[2017-07-10T09:42:35.046Z] [INFO] [??] [Config] environment set value for self.docker.endpoint
[2017-07-10T09:42:35.046Z] [INFO] [??] [Config] environment set value for consul.host.port4
[2017-07-10T09:42:35.046Z] [INFO] [??] [Config] environment set value for port
[2017-07-10T09:42:35.047Z] [INFO] [??] [WhiskConfig] reading properties from consul at consul.openwhisk:8500
[2017-07-10T09:42:36.078Z] [ERROR] [??] [Config] required property invoker.instances still not set
[2017-07-10T09:42:36.079Z] [ERROR] [??] [Invoker] Bad configuration, cannot start.
How to config the invoker ?
Best regards,
Zhang Haiyong
-----原始邮件-----
发件人:"张海勇" <zh...@mail.ustc.edu.cn>
发送时间:2017-07-10 14:09:39 (星期一)
收件人: "Daniel Lavine" <dl...@us.ibm.com>
抄送:dev@openwhisk.incubator.apache.org
主题: issue about invoker
Hi Daniel Lavine,
I set up some components successfully , but there is no invoker component.
so I install it by hand like this:
kubectl apply -f environments/kube/files/invoker.yml
when I run "wsk -i action invoke hello"
there is an error like this:
error: Unable to invoke action 'hello1': There was an internal server error. (code 15)
I check the log by cli "kubectl -n openwhisk logs invoker-0":
[2017-07-10T03:58:06.929Z] [INFO] Slf4jLogger started
[2017-07-10T03:58:06.961Z] [INFO] [??] [Config] environment set value for consulserver.host
[2017-07-10T03:58:06.963Z] [INFO] [??] [Config] environment set value for self.docker.endpoint
[2017-07-10T03:58:06.963Z] [INFO] [??] [Config] environment set value for consul.host.port4
[2017-07-10T03:58:06.963Z] [INFO] [??] [Config] environment set value for port
[2017-07-10T03:58:06.963Z] [INFO] [??] [WhiskConfig] reading properties from consul at consul.openwhisk:8500
[2017-07-10T03:58:07.888Z] [ERROR] [??] [Config] required property whisk.api.host still not set
How can I fix this issue about required property whisk.api.host still not set?
notes: invoker image is adobeapiplatform/whisk-invoker, the openwhisk/invoker image seems not work well
Best regards,
Zhang Haiyong
-----原始邮件-----
发件人:"张海勇" <zh...@mail.ustc.edu.cn>
发送时间:2017-07-05 15:36:59 (星期三)
收件人: "Daniel Lavine" <dl...@us.ibm.com>
抄送:dev@openwhisk.incubator.apache.org
主题: isuue about deployment without ansible
Hi Daniel Lavine,
when I set up Kubernetes environment , just run
1. kubectl apply -f environments/kube/files/xxx.yml
2. kubectl apply -f environments/kube/files/xxx-service.yml
Can I deploy XXX component successfully instead of run "ansible-playbook -i environments/kube openwhisk.yml"?
Best regards,
Zhang Haiyong
-----原始邮件-----
发件人:"张海勇" <zh...@mail.ustc.edu.cn>
发送时间:2017-07-05 15:04:19 (星期三)
收件人: "Daniel Lavine" <dl...@us.ibm.com>
抄送:dev@openwhisk.incubator.apache.org
主题: issue about mountPath
Hi Daniel Lavine,
When I deploy components such as consul nginx by CLI "ansible-playbook -i environments/kube openwhisk.yml",
there are some mountPath in the yml config file , where can I get these files ?
for example: nginx
best regards,
Zhang Haiyong
-----原始邮件-----
发件人:"Daniel Lavine" <dl...@us.ibm.com>
发送时间:2017-06-23 00:12:21 (星期五)
收件人: "张海勇" <zh...@mail.ustc.edu.cn>
抄送:dev@openwhisk.incubator.apache.org
主题: Re: issue about Image danlavine/whisk_config:v1.5.6
Hey Zhanghaiyong,
For theses types of issues, I would open up an issue against the github repo directly so that it can be better addressed by the people who know most about that code base. Also, we would be able to integrate any code changes that might need to happen. That being said, I would take a look at these instructions https://github.com/apache/incubator-openwhisk-deploy-kube#configure-openwhisk. Whenever deploying Docker images --> Kubernetes Pods you should always be using the Kubernetes cli "kubectl", not Docker commands directly.
git repo: https://github.com/apache/incubator-openwhisk-deploy-kube/issues
Thanks,
Dan Lavine
张海勇 ---06/22/2017 07:36:07 AM---Hi Daniel Lavine, I am a learner about deploying openwhisk on kubernetes, when I run the script
From: 张海勇 <zh...@mail.ustc.edu.cn>
To: dlavine@us.ibm.com
Cc: dev@openwhisk.incubator.apache.org
Date: 06/22/2017 07:36 AM
Subject: issue about Image danlavine/whisk_config:v1.5.6
Hi Daniel Lavine,
I am a learner about deploying openwhisk on kubernetes, when I run the script
$TRAVIS_KUBE_VERSION=v1.5.6 TRAVIS_ETCD_VERSION=v3.0.14 ./setup.sh
$TRAVIS_KUBE_VERSION=v1.5.6 ./build.sh
$kubectl -n openwhisk logs configure-openwhisk-xxxxx
there is no log , then I download the image whisk_config:v1.5.6, and run
$docker run danlavine/whisk_config:v1.5.6 /incubator-openwhisk-deploy-kube/configure/configure.sh
and noted that the command kubectl did not run successfully, the log is as below:
The connection to the server localhost:8080 was refused - did you specify the right host or port?
so I think the kubernetes environments in the image (danlavine/whisk_config:v1.5.6) is not complete.
need you to confirm , thanks !
Best regards,
zhanghaiyong
Re: question about resource allocation and schedule with deploying
openwhisk on kubernetes
Posted by Daniel Lavine <dl...@us.ibm.com>.
Hey Zhang,
When OpenWhisk is deployed on Kubernetes, the Invokers mount the host's
Docker socket. This means that the Invoker talks directly to docker when
running any jobs and does not use Kubernetes to create OpenWhisk jobs.
These jobs are all scheduled through the OpenWhisk Controller and it is
responsible for load balancing the jobs as well. Now if there is a resource
constraint you should be able to scale the number of invoker instances.
They should be able to register themselves to the Controller by Kafka
automatically.
Dan Lavine
From: 张海勇 <zh...@mail.ustc.edu.cn>
To: "Daniel Lavine" <dl...@us.ibm.com>
Cc: dev@openwhisk.incubator.apache.org
Date: 07/12/2017 08:39 AM
Subject: question about resource allocation and schedule with deploying
openwhisk on kubernetes
Hi Daniel Lavine,
when deploying openwhisk on kubernetes, if a job runs on openwhisk ,
how to alloc and free resource , openwhisk is a manager or kubernetes is a
manager?
if there are too many jobs ,how to load balance? how to span job to
other node? restart openwhisk components or restart jobs thread?
Best regards,
Zhang Haiyong
-----原始邮件-----
发件人:"张海勇" <zh...@mail.ustc.edu.cn>
发送时间:2017-07-10 17:51:30 (星期一)
收件人: "张海勇" <zh...@mail.ustc.edu.cn>, "Daniel Lavine"
<dl...@us.ibm.com>
抄送: "Daniel Lavine" <dl...@us.ibm.com>,
dev@openwhisk.incubator.apache.org
主题: Re: issue about invoker
Hi Daniel Lavine,
I add invoker to the script file .travis/build.sh
kubectl apply -f configure/configure_whisk.ym
#setup invoker
kubectl apply -f ansible/environments/kube/files/invoker-service.yml
kubectl apply -f ansible/environments/kube/files/invoker.yml
The controller is failed to connect to invoker with the image
adobeapiplatform/whisk-invoker
[2017-07-10T09:33:31.771Z] [INFO] [#tid_40] [ActionsApi] action
activation id: 61551b15635f43268c31cd708db1966d
[marker:controller_loadbalancer_start:22]
[2017-07-10T09:33:31.775Z] [ERROR] [#sid_121] [LoadBalancerService]
all invokers down
[2017-07-10T09:33:31.775Z] [ERROR] [#tid_40] [ActionsApi] [POST]
action activation failed: no invokers available
[2017-07-10T09:33:31.779Z] [INFO] [#tid_40] [BasicHttpService]
[marker:http_post.500_count:31:31]
The controller is setup successfully with the image openwhisk/invoker
[2017-07-10T09:41:35.452Z] [INFO] [#sid_120] [LoadBalancerService]
test action for invoker health now exists
[2017-07-10T09:41:35.502Z] [INFO] Bound to /0.0.0.0:8080
[2017-07-10T09:41:35.515Z] [INFO] [#sid_120] [CouchDbRestStore] [PUT]
'openwhisk_kube_whisks' completed document: 'id:
whisk.system/invokerHealthTestAction, rev: null', response:
'{"ok":true,"id":"whisk.system/invokerHealthTestAction","rev":"1-6e6565bab3cb9030d9d2df48413a688c"}' [marker:database_saveDocument_finish:1581:78]
[2017-07-10T09:41:35.522Z] [INFO] [#sid_120] [WhiskAction] write all
done, caching id: whisk.system/invokerHealthTestAction Cached
[2017-07-10T09:41:35.523Z] [INFO] [#sid_130] [RateThrottler] actions
per minute: maxPerMinute = 120
[2017-07-10T09:41:35.523Z] [INFO] [#sid_130] [RateThrottler] triggers
per minute: maxPerMinute = 60
[2017-07-10T09:41:35.524Z] [INFO] [??] [ActivationThrottler]
concurrencyLimit = 100, systemOverloadLimit = 5000
[2017-07-10T09:41:35.606Z] [INFO] [??] [ActionsApi]
actionSequenceLimit '50'
But the invoker is not configured
kubectl -n openwhisk logs invoker-0
[2017-07-10T09:42:35.001Z] [INFO] Slf4jLogger started
[2017-07-10T09:42:35.044Z] [INFO] [??] [Config] environment set value
for consulserver.host
[2017-07-10T09:42:35.046Z] [INFO] [??] [Config] environment set value
for self.docker.endpoint
[2017-07-10T09:42:35.046Z] [INFO] [??] [Config] environment set value
for consul.host.port4
[2017-07-10T09:42:35.046Z] [INFO] [??] [Config] environment set value
for port
[2017-07-10T09:42:35.047Z] [INFO] [??] [WhiskConfig] reading
properties from consul at consul.openwhisk:8500
[2017-07-10T09:42:36.078Z] [ERROR] [??] [Config] required property
invoker.instances still not set
[2017-07-10T09:42:36.079Z] [ERROR] [??] [Invoker] Bad configuration,
cannot start.
How to config the invoker ?
Best regards,
Zhang Haiyong
-----原始邮件-----
发件人:"张海勇" <zh...@mail.ustc.edu.cn>
发送时间:2017-07-10 14:09:39 (星期一)
收件人: "Daniel Lavine" <dl...@us.ibm.com>
抄送: dev@openwhisk.incubator.apache.org
主题: issue about invoker
Hi Daniel Lavine,
I set up some components successfully , but there is no invoker component.
so I install it by hand like this:
kubectl apply -f environments/kube/files/invoker.yml
when I run "wsk -i action invoke hello"
there is an error like this:
error: Unable to invoke action 'hello1': There was an internal server
error. (code 15)
I check the log by cli "kubectl -n openwhisk logs invoker-0":
[2017-07-10T03:58:06.929Z] [INFO] Slf4jLogger started
[2017-07-10T03:58:06.961Z] [INFO] [??] [Config] environment set value
for consulserver.host
[2017-07-10T03:58:06.963Z] [INFO] [??] [Config] environment set value
for self.docker.endpoint
[2017-07-10T03:58:06.963Z] [INFO] [??] [Config] environment set value
for consul.host.port4
[2017-07-10T03:58:06.963Z] [INFO] [??] [Config] environment set value
for port
[2017-07-10T03:58:06.963Z] [INFO] [??] [WhiskConfig] reading
properties from consul at consul.openwhisk:8500
[2017-07-10T03:58:07.888Z] [ERROR] [??] [Config] required property
whisk.api.host still not set
How can I fix this issue about required property whisk.api.host still not
set?
notes: invoker image is adobeapiplatform/whisk-invoker, the
openwhisk/invoker image seems not work well
Best regards,
Zhang Haiyong
-----原始邮件-----
发件人:"张海勇" <zh...@mail.ustc.edu.cn>
发送时间:2017-07-05 15:36:59 (星期三)
收件人: "Daniel Lavine" <dl...@us.ibm.com>
抄送: dev@openwhisk.incubator.apache.org
主题: isuue about deployment without ansible
Hi Daniel Lavine,
when I set up Kubernetes environment , just run
1. kubectl apply -f environments/kube/files/xxx.yml
2. kubectl apply -f environments/kube/files/xxx-service.yml
Can I deploy XXX component successfully instead of run "ansible-playbook
-i environments/kube openwhisk.yml"?
Best regards,
Zhang Haiyong
-----原始邮件-----
发件人:"张海勇" <zh...@mail.ustc.edu.cn>
发送时间:2017-07-05 15:04:19 (星期三)
收件人: "Daniel Lavine" <dl...@us.ibm.com>
抄送: dev@openwhisk.incubator.apache.org
主题: issue about mountPath
Hi Daniel Lavine,
When I deploy components such as consul nginx by CLI "ansible-playbook -i
environments/kube openwhisk.yml",
there are some mountPath in the yml config file , where can I get these
files ?
for example: nginx
best regards,
Zhang Haiyong
-----原始邮件-----
发件人:"Daniel Lavine" <dl...@us.ibm.com>
发送时间:2017-06-23 00:12:21 (星期五)
收件人: "张海勇" <zh...@mail.ustc.edu.cn>
抄送: dev@openwhisk.incubator.apache.org
主题: Re: issue about Image danlavine/whisk_config:v1.5.6
Hey Zhanghaiyong,
For theses types of issues, I would open up an issue against the github
repo directly so that it can be better addressed by the people who know
most about that code base. Also, we would be able to integrate any code
changes that might need to happen. That being said, I would take a look
at these instructions
https://github.com/apache/incubator-openwhisk-deploy-kube#configure-openwhisk
. Whenever deploying Docker images --> Kubernetes Pods you should always
be using the Kubernetes cli "kubectl", not Docker commands directly.
git repo:
https://github.com/apache/incubator-openwhisk-deploy-kube/issues
Thanks,
Dan Lavine
张海勇 ---06/22/2017 07:36:07 AM---Hi Daniel Lavine, I am a learner
about deploying openwhisk on kubernetes, when I run the script
From: 张海勇 <zh...@mail.ustc.edu.cn>
To: dlavine@us.ibm.com
Cc: dev@openwhisk.incubator.apache.org
Date: 06/22/2017 07:36 AM
Subject: issue about Image danlavine/whisk_config:v1.5.6
Hi Daniel Lavine,
I am a learner about deploying openwhisk on kubernetes, when I run the
script
$TRAVIS_KUBE_VERSION=v1.5.6 TRAVIS_ETCD_VERSION=v3.0.14 ./setup.sh
$TRAVIS_KUBE_VERSION=v1.5.6 ./build.sh
$kubectl -n openwhisk logs configure-openwhisk-xxxxx
there is no log , then I download the image whisk_config:v1.5.6, and run
$docker run
danlavine/whisk_config:v1.5.6 /incubator-openwhisk-deploy-kube/configure/configure.sh
and noted that the command kubectl did not run successfully, the log is
as below:
The connection to the server localhost:8080 was refused - did you
specify the right host or port?
so I think the kubernetes environments in the image
(danlavine/whisk_config:v1.5.6) is not complete.
need you to confirm , thanks !
Best regards,
zhanghaiyong