You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by GitBox <gi...@apache.org> on 2021/07/07 10:23:26 UTC
[GitHub] [camel-k] stepan111 opened a new issue #2485: ImagePullBackOff when node failed
stepan111 opened a new issue #2485:
URL: https://github.com/apache/camel-k/issues/2485
Hello,
First of all thanks for great project !
I am running camel-k integration on GKE cluster with preemptible nodes.(nodes recreated daily)
Currently I have to delete and apply integration each time k8s migrate pod to other node because of `ImagePullBackOff` error:
```
$ k get po -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
camel-k-operator-6f848d7b5c-kgrrp 0/2 Shutdown 0 30h <none> gke-sandbox-blue-sandbox-blue-default-f0ed42c1-3p5d <none> <none>
camel-k-operator-6f848d7b5c-wjd9n 2/2 Running 2 9h 10.100.18.144 gke-sandbox-blue-sandbox-blue-default-ee794f56-16s7 <none> <none>
gmail-5d5b644958-hx9gm 1/2 ImagePullBackOff 0 5m1s 10.100.20.131 gke-sandbox-blue-sandbox-blue-default-f0ed42c1-3p5d <none> <none>
gmail-5d5b644958-sb2xt 0/2 Shutdown 0 25h <none> gke-sandbox-blue-sandbox-blue-default-ee794f56-16s7 <none> <none>
```
I've added next triats to integration:
```
traits:
pull-secret:
configuration:
auto: false
enabled: true
secretName: registry
```
But it don't take affect on deployment made for this integration.
```
$ k get deploy/integration | grep imagePullSecret
$
```
Operator version:
```
$ k get deploy/camel-k-operator -o jsonpath={.spec.template.spec.containers..image}
docker.io/apache/camel-k:1.4.0
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-k] nicolaferraro commented on issue #2485: ImagePullBackOff when node failed
Posted by GitBox <gi...@apache.org>.
nicolaferraro commented on issue #2485:
URL: https://github.com/apache/camel-k/issues/2485#issuecomment-875588689
So, technically the pull secret is stored on the namespace and it should be shared among all nodes, so if a node is deleted and the pod (here we are in the Kubernetes domain, the Camel K operator should not interfere with that behavior) recreated on another node, the same pull secret used by previous node should be used by the new one. So I assume you're currently not using them but want to..
The trait you've configured should be correct.
Did you try to apply that configuration upon creation of the integration, e.g. using `-t pull-secret.enabled=true -t pull-secret.secret-name=registry` ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel-k] stepan111 commented on issue #2485: ImagePullBackOff when node failed
Posted by GitBox <gi...@apache.org>.
stepan111 commented on issue #2485:
URL: https://github.com/apache/camel-k/issues/2485#issuecomment-876940651
Yes @nicolaferraro , I am using pull-secret trait for integration. Actually I see 2 pods created when I apply integration.
And I found that there are 2 replica sets :
```
$ k get po
NAME READY STATUS RESTARTS AGE
camel-k-operator-6f848d7b5c-9jrdn 2/2 Running 2 11h
gmail-7758f9577d-bhfz6 0/2 PodInitializing 0 13s
gmail-867d487988-sk9g9 1/2 ImagePullBackOff 0 13s
$ k get rs
NAME DESIRED CURRENT READY AGE
camel-k-operator-6f848d7b5c 1 1 1 13d
gmail-7758f9577d 0 0 0 3m46s
gmail-867d487988 1 1 1 3m46s
```
And rs that contain imagPullSecret is scaled to 0:
```
$ k get rs gmail-7758f9577d -o yaml | grep -A1 imagePullSecret
f:imagePullSecrets:
.: {}
--
imagePullSecrets:
- name: registry
$ k get rs gmail-867d487988 -o yaml | grep -A1 imagePullSecret
$
```
Seems that both replicaSets are revisions of deployment gmail made by integration :
```
$ k rollout history deployment gmail
deployment.apps/gmail
REVISION CHANGE-CAUSE
1 <none>
2 <none>
```
And second revision doesn't contain imagePull secrets(maybe it is patched not properly or so).
From events I see that both pods assigned to one node:
```
12m Normal Scheduled pod/gmail-7758f9577d-bhfz6 Successfully assigned camel-k/gmail-7758f9577d-bhfz6 to gke-sandbox-blue-sandbox-blue-default-f0ed42c1-d6fq
12m Normal Scheduled pod/gmail-867d487988-sk9g9 Successfully assigned camel-k/gmail-867d487988-sk9g9 to gke-sandbox-blue-sandbox-blue-default-f0ed42c1-d6fq
```
So I am thinking that first pod download image on node and second using this image while node is up. And when node crashes active replicaSet can't pull image.
Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org