You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/05/07 05:38:45 UTC
[GitHub] [airflow] j-y-matsubara opened a new pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
j-y-matsubara opened a new pull request #8757:
URL: https://github.com/apache/airflow/pull/8757
In versions less than 2.0 using kubernetes_request_factory.py, can not add sidecar container by using pod_mutation_hook.
---
Make sure to mark the boxes below before creating PR: [x]
- [x] Description above provides context of the change
- [x] Unit tests coverage for changes (not needed for documentation changes)
- [x] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)"
- [x] Relevant documentation is updated including usage instructions.
- [ ] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
---
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
j-y-matsubara commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422455491
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is it used for sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers (sidecars), excluding the init container ( in the case of using kubernetes_request_factory.py, pod object of argument of pod_mutation_hook is not V1Pod※. **Part of the contents of** req object creating in pod_request_factory.py is overwritten with this pod object, and launch "Pod" based on **req object.** ).
In the version 2.0, We can directly edit the pod object used to launch "Pod". ( the pod object of argument of pod_mutation_hook is V1Pod ) Therefore, we can add any containaers (sidecar).
If my understanding is mistaken, please tell me.
※Supplementary information (version 1.10.10)
If we set the following to the operator,
```
k2 = KubernetesPodOperator(namespace='airflow01',
task_id='test-kwsk',
name='test-kwsk',
image="airflowworker:1.0.0",
image_pull_policy="IfNotPresent",
cmds=["bash", "-cx"],
arguments=['sleep 10000'],
do_xcom_push=True,
....
..
```
The pod object of argument of pod_mutation_hook is already set as follows.
```
affinity: {}
annotations: {}
args: ['sleep 10000']
cmds: ['bash', '-cx']
configmaps: []
dnspolicy: None
envs: {}
hostnetwork: False
image: airflowworker:1.0.0
image_pull_policy: IfNotPresent
...
..
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
j-y-matsubara commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422455491
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is is made and used for sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers, excluding the init container ( in the case of using kubernetes_request_factory.py, pod object is not V1Pod※. Overwrite **part of the contents of** req object creating in pod_request_factory.py with pod object, and launch "Pod" based on req object. ).
In the version 2.0, We can directly edit the pod object (V1Pod) used to launch "Pod". Therefore, we can add any containaers (sidecar).
※Supplementary information
If we set the following to the operator,
```
k2 = KubernetesPodOperator(namespace='airflow01',
task_id='test-kwsk',
name='test-kwsk',
image="airflowworker:1.0.0",
image_pull_policy="IfNotPresent",
cmds=["bash", "-cx"],
arguments=['sleep 10000'],
do_xcom_push=True,
....
..
```
The pod object of argument of pod_mutation_hook is already set as follows.
```
affinity: {}
annotations: {}
args: ['sleep 10000']
cmds: ['bash', '-cx']
configmaps: []
dnspolicy: None
envs: {}
hostnetwork: False
image: airflowworker:1.0.0
image_pull_policy: IfNotPresent
...
..
```
If my understanding is mistaken, please tell me.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
j-y-matsubara commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422455491
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is is made and used for sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers, excluding the init container ( in the case of using kubernetes_request_factory.py, pod object is not V1Pod. Overwrite part of the contents of req object creating in pod_request_factory.py with pod object, and launch "Pod" by using req object. ).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
j-y-matsubara commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422455491
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is is made and used for sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers, excluding the init container ( in the case of using kubernetes_request_factory.py, pod object is not V1Pod. Overwrite **part of the contents of** req object creating in pod_request_factory.py with pod object, and launch "Pod" based on req object. ).
In the version 2.0, We can directly edit the pod objects (V1Pod) used to launch "Pod". Therefore, we can add any containaers (sidecar).
If my understanding is mistaken, please tell me.
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is is made and used for sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers, excluding the init container ( in the case of using kubernetes_request_factory.py, pod object is not V1Pod. Overwrite **part of the contents of** req object creating in pod_request_factory.py with pod object, and launch "Pod" based on req object. ).
In the version 2.0, We can directly edit the pod object (V1Pod) used to launch "Pod". Therefore, we can add any containaers (sidecar).
If my understanding is mistaken, please tell me.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
j-y-matsubara commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422455491
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is it used for sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers (sidecars), excluding the init container.
In the case of using kubernetes_request_factory.py※, pod object of argument of pod_mutation_hook is not V1Pod.
**Part of the contents of** req object creating in pod_request_factory.py is overwritten with this pod object, in kubernetes_request_factory.py, and launch "Pod" based on **req object.**
In this overwriting, I think there is no function to add new containers (sidecars) excluding the init container.
In the version 2.0, We can directly edit the pod object used to launch "Pod". ( the pod object of argument of pod_mutation_hook is V1Pod ) Therefore, we can add any containaers (sidecar).
If my understanding is mistaken, please tell me.
※Supplementary information (version 1.10.10)
If we set the following to the operator,
```
k2 = KubernetesPodOperator(namespace='airflow01',
task_id='test-kwsk',
name='test-kwsk',
image="airflowworker:1.0.0",
image_pull_policy="IfNotPresent",
cmds=["bash", "-cx"],
arguments=['sleep 10000'],
do_xcom_push=True,
....
..
```
The pod object of argument of pod_mutation_hook is already set as follows.
```
affinity: {}
annotations: {}
args: ['sleep 10000']
cmds: ['bash', '-cx']
configmaps: []
dnspolicy: None
envs: {}
hostnetwork: False
image: airflowworker:1.0.0
image_pull_policy: IfNotPresent
...
..
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
j-y-matsubara commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422455491
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is is made and used for sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers, excluding the init container ( in the case of using kubernetes_request_factory.py, pod object is not V1Pod. Overwrite **part of the contents of** req object creating in pod_request_factory.py with pod object, and launch "Pod" by using req object. ).
In the version 2.0, We can directly edit the pod objects (V1Pod) used to launch "Pod". Therefore, we can add any containaers (sidecar).
If my understanding is mistaken, please tell me.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
j-y-matsubara commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422455491
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is is made and used for sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers, excluding the init container ( in the case of using kubernetes_request_factory.py, pod object is not V1Pod※. Overwrite **part of the contents of** req object creating in pod_request_factory.py with pod object, and launch "Pod" based on req object. ).
In the version 2.0, We can directly edit the pod object (V1Pod) used to launch "Pod". Therefore, we can add any containaers (sidecar).
※ If you set the following to the operator,
```
k2 = KubernetesPodOperator(namespace='airflow01',
task_id='test-kwsk',
name='test-kwsk',
image="airflowworker:1.0.0",
image_pull_policy="IfNotPresent",
cmds=["bash", "-cx"],
arguments=['sleep 10000'],
do_xcom_push=True,
....
..
```
The pod object of argument of pod_mutation_hook is already set as follows.
```
affinity: {}
annotations: {}
args: ['sleep 10000']
cmds: ['bash', '-cx']
configmaps: []
dnspolicy: None
envs: {}
hostnetwork: False
image: airflowworker:1.0.0
image_pull_policy: IfNotPresent
...
..
```
If my understanding is mistaken, please tell me.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
j-y-matsubara commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422455491
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is it used for sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers, excluding the init container ( in the case of using kubernetes_request_factory.py, pod object of argument of pod_mutation_hook is not V1Pod※. Overwrite **part of the contents of** req object, for example req['spec']['containers'][0]['image'] = pod.image , creating in pod_request_factory.py with pod object, and launch "Pod" based on req object. ).
In the version 2.0, We can directly edit the pod object (V1Pod) used to launch "Pod". ( argument of pod_mutation_hook is V1Pod ) Therefore, we can add any containaers (sidecar).
If my understanding is mistaken, please tell me.
※Supplementary information (version 1.10.10)
If we set the following to the operator,
```
k2 = KubernetesPodOperator(namespace='airflow01',
task_id='test-kwsk',
name='test-kwsk',
image="airflowworker:1.0.0",
image_pull_policy="IfNotPresent",
cmds=["bash", "-cx"],
arguments=['sleep 10000'],
do_xcom_push=True,
....
..
```
The pod object of argument of pod_mutation_hook is already set as follows.
```
affinity: {}
annotations: {}
args: ['sleep 10000']
cmds: ['bash', '-cx']
configmaps: []
dnspolicy: None
envs: {}
hostnetwork: False
image: airflowworker:1.0.0
image_pull_policy: IfNotPresent
...
..
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
j-y-matsubara commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422455491
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is is made and used for sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers, excluding the init container ( in the case of using kubernetes_request_factory.py, pod object is not V1Pod※. Overwrite **part of the contents of** req object creating in pod_request_factory.py with pod object, and launch "Pod" based on req object. ).
In the version 2.0, We can directly edit the pod object (V1Pod) used to launch "Pod". Therefore, we can add any containaers (sidecar).
If my understanding is mistaken, please tell me.
※Supplementary information
If we set the following to the operator,
```
k2 = KubernetesPodOperator(namespace='airflow01',
task_id='test-kwsk',
name='test-kwsk',
image="airflowworker:1.0.0",
image_pull_policy="IfNotPresent",
cmds=["bash", "-cx"],
arguments=['sleep 10000'],
do_xcom_push=True,
....
..
```
The pod object of argument of pod_mutation_hook is already set as follows.
```
affinity: {}
annotations: {}
args: ['sleep 10000']
cmds: ['bash', '-cx']
configmaps: []
dnspolicy: None
envs: {}
hostnetwork: False
image: airflowworker:1.0.0
image_pull_policy: IfNotPresent
...
..
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
j-y-matsubara commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422455491
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is it used for sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers (sidecars), excluding the init container ( in the case of using kubernetes_request_factory.py, pod object of argument of pod_mutation_hook is not V1Pod※. **Part of the contents of** req object creating in pod_request_factory.py is overwritten with this pod object, and launch "Pod" based on req object. ).
In the version 2.0, We can directly edit the pod object (V1Pod) used to launch "Pod". ( argument of pod_mutation_hook is V1Pod ) Therefore, we can add any containaers (sidecar).
If my understanding is mistaken, please tell me.
※Supplementary information (version 1.10.10)
If we set the following to the operator,
```
k2 = KubernetesPodOperator(namespace='airflow01',
task_id='test-kwsk',
name='test-kwsk',
image="airflowworker:1.0.0",
image_pull_policy="IfNotPresent",
cmds=["bash", "-cx"],
arguments=['sleep 10000'],
do_xcom_push=True,
....
..
```
The pod object of argument of pod_mutation_hook is already set as follows.
```
affinity: {}
annotations: {}
args: ['sleep 10000']
cmds: ['bash', '-cx']
configmaps: []
dnspolicy: None
envs: {}
hostnetwork: False
image: airflowworker:1.0.0
image_pull_policy: IfNotPresent
...
..
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
j-y-matsubara commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422455491
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is it used for sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers (sidecars), excluding the init container.
In the case of using kubernetes_request_factory.py※, pod object of argument of pod_mutation_hook is not V1Pod.
**Part of the contents of** req object creating in pod_request_factory.py is overwritten with this pod object, in kubernetes_request_factory.py, and launch "Pod" based on **req object.**
In this overwriting, I think there is no function to add new containers (sidecars) excluding the init container.
In the version 2.0, We can directly edit the pod object used to launch "Pod". ( the pod object of argument of pod_mutation_hook is V1Pod ) Therefore, we can add any containaers (sidecar).
If my understanding is mistaken, please tell me.
Thank you
※Supplementary information (version 1.10.10)
If we set the following to the operator,
```
k2 = KubernetesPodOperator(namespace='airflow01',
task_id='test-kwsk',
name='test-kwsk',
image="airflowworker:1.0.0",
image_pull_policy="IfNotPresent",
cmds=["bash", "-cx"],
arguments=['sleep 10000'],
do_xcom_push=True,
....
..
```
The pod object of argument of pod_mutation_hook is already set as follows.
```
affinity: {}
annotations: {}
args: ['sleep 10000']
cmds: ['bash', '-cx']
configmaps: []
dnspolicy: None
envs: {}
hostnetwork: False
image: airflowworker:1.0.0
image_pull_policy: IfNotPresent
...
..
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
j-y-matsubara commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422455491
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is it used for sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers (sidecars), excluding the init container ( in the case of using kubernetes_request_factory.py, pod object of argument of pod_mutation_hook is not V1Pod※. **Part of the contents of** req object creating in pod_request_factory.py is overwritten with this pod object, and launch "Pod" based on **req object.** ).
In the version 2.0, We can directly edit the pod object ( this object is V1Pod) used to launch "Pod". ( argument of pod_mutation_hook is V1Pod ) Therefore, we can add any containaers (sidecar).
If my understanding is mistaken, please tell me.
※Supplementary information (version 1.10.10)
If we set the following to the operator,
```
k2 = KubernetesPodOperator(namespace='airflow01',
task_id='test-kwsk',
name='test-kwsk',
image="airflowworker:1.0.0",
image_pull_policy="IfNotPresent",
cmds=["bash", "-cx"],
arguments=['sleep 10000'],
do_xcom_push=True,
....
..
```
The pod object of argument of pod_mutation_hook is already set as follows.
```
affinity: {}
annotations: {}
args: ['sleep 10000']
cmds: ['bash', '-cx']
configmaps: []
dnspolicy: None
envs: {}
hostnetwork: False
image: airflowworker:1.0.0
image_pull_policy: IfNotPresent
...
..
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
j-y-matsubara commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422455491
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is it used for sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers (sidecars), excluding the init container.
In the case of using kubernetes_request_factory.py ( airflow/contrib/kubernetes/ ),version 1.X.X, pod object of argument of pod_mutation_hook is not V1Pod.
**Part of the contents of** req object creating in pod_request_factory.py is overwritten with this pod object, in kubernetes_request_factory.py, and launch "Pod" based on **req object.**
In this overwriting, I think there is no function to add new containers (sidecars) excluding the init container.
In the version 2.0 (airflow/kubernetes/), We can directly edit the pod object used to launch "Pod". ( the pod object of argument of pod_mutation_hook is V1Pod ) Therefore, we can add any containaers (sidecar).
If my understanding is mistaken, please tell me.
Thank you in advance.
※Supplementary information (version 1.10.10)
If we set the following to the operator,
```
k2 = KubernetesPodOperator(namespace='airflow01',
task_id='test-kwsk',
name='test-kwsk',
image="airflowworker:1.0.0",
image_pull_policy="IfNotPresent",
cmds=["bash", "-cx"],
arguments=['sleep 10000'],
do_xcom_push=True,
....
..
```
The pod object of argument of pod_mutation_hook is already set as follows.
```
affinity: {}
annotations: {}
args: ['sleep 10000']
cmds: ['bash', '-cx']
configmaps: []
dnspolicy: None
envs: {}
hostnetwork: False
image: airflowworker:1.0.0
image_pull_policy: IfNotPresent
...
..
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
j-y-matsubara commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422455491
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is it used for sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers (sidecars), excluding the init container ( in the case of using kubernetes_request_factory.py, pod object of argument of pod_mutation_hook is not V1Pod※. **Part of the contents of** req object creating in pod_request_factory.py is overwritten with this pod object, and launch "Pod" based on **req object.** ).
In the version 2.0, We can directly edit the pod object used to launch "Pod". ( the pod object (argument of pod_mutation_hook) is V1Pod ) Therefore, we can add any containaers (sidecar).
If my understanding is mistaken, please tell me.
※Supplementary information (version 1.10.10)
If we set the following to the operator,
```
k2 = KubernetesPodOperator(namespace='airflow01',
task_id='test-kwsk',
name='test-kwsk',
image="airflowworker:1.0.0",
image_pull_policy="IfNotPresent",
cmds=["bash", "-cx"],
arguments=['sleep 10000'],
do_xcom_push=True,
....
..
```
The pod object of argument of pod_mutation_hook is already set as follows.
```
affinity: {}
annotations: {}
args: ['sleep 10000']
cmds: ['bash', '-cx']
configmaps: []
dnspolicy: None
envs: {}
hostnetwork: False
image: airflowworker:1.0.0
image_pull_policy: IfNotPresent
...
..
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
j-y-matsubara commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422455491
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is it used for sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers (sidecars), excluding the init container. ( In the case of using kubernetes_request_factory.py※, pod object of argument of pod_mutation_hook is not V1Pod. **Part of the contents of** req object creating in pod_request_factory.py is overwritten with this pod object, in kubernetes_request_factory.py, and launch "Pod" based on **req object.** To this overwrite, I think there is no function to add new containers (sidecars) excluding the init container.)
In the version 2.0, We can directly edit the pod object used to launch "Pod". ( the pod object of argument of pod_mutation_hook is V1Pod ) Therefore, we can add any containaers (sidecar).
If my understanding is mistaken, please tell me.
※Supplementary information (version 1.10.10)
If we set the following to the operator,
```
k2 = KubernetesPodOperator(namespace='airflow01',
task_id='test-kwsk',
name='test-kwsk',
image="airflowworker:1.0.0",
image_pull_policy="IfNotPresent",
cmds=["bash", "-cx"],
arguments=['sleep 10000'],
do_xcom_push=True,
....
..
```
The pod object of argument of pod_mutation_hook is already set as follows.
```
affinity: {}
annotations: {}
args: ['sleep 10000']
cmds: ['bash', '-cx']
configmaps: []
dnspolicy: None
envs: {}
hostnetwork: False
image: airflowworker:1.0.0
image_pull_policy: IfNotPresent
...
..
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
j-y-matsubara commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422455491
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is is made and used for sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers, excluding the init container ( in the case of using kubernetes_request_factory.py ).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422377856
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
It can be used for sidecar containers too can't it?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
j-y-matsubara commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422455491
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is it used for sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers (sidecars), excluding the init container. ( In the case of using kubernetes_request_factory.py※, pod object of argument of pod_mutation_hook is not V1Pod. **Part of the contents of** req object creating in pod_request_factory.py is overwritten with this pod object, and launch "Pod" based on **req object.** To this overwrite, I think there is no function to add new containers (sidecars) excluding the init container.)
In the version 2.0, We can directly edit the pod object used to launch "Pod". ( the pod object of argument of pod_mutation_hook is V1Pod ) Therefore, we can add any containaers (sidecar).
If my understanding is mistaken, please tell me.
※Supplementary information (version 1.10.10)
If we set the following to the operator,
```
k2 = KubernetesPodOperator(namespace='airflow01',
task_id='test-kwsk',
name='test-kwsk',
image="airflowworker:1.0.0",
image_pull_policy="IfNotPresent",
cmds=["bash", "-cx"],
arguments=['sleep 10000'],
do_xcom_push=True,
....
..
```
The pod object of argument of pod_mutation_hook is already set as follows.
```
affinity: {}
annotations: {}
args: ['sleep 10000']
cmds: ['bash', '-cx']
configmaps: []
dnspolicy: None
envs: {}
hostnetwork: False
image: airflowworker:1.0.0
image_pull_policy: IfNotPresent
...
..
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
j-y-matsubara commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422455491
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is is made and used for sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers, excluding the init container ( in the case of using kubernetes_request_factory.py, pod object is not V1Pod. Overwrite part of the contents of req object creating in pod_request_factory.py with pod object, and launch "Pod" by using req object. ).
In the version 2.0, We can directly edit the pod objects (V1Pod) used launching Pod. Therefore, we can add any containaers (sidecar).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
j-y-matsubara commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422455491
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is this function used to add sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers, excluding the init container ( in the case of using kubernetes_request_factory.py, pod object of argument of pod_mutation_hook is not V1Pod※. Overwrite **part of the contents of** req object, for example req['spec']['containers'][0]['image'] = pod.image , creating in pod_request_factory.py with pod object, and launch "Pod" based on req object. ).
In the version 2.0, We can directly edit the pod object (V1Pod) used to launch "Pod". ( argument of pod_mutation_hook is V1Pod ) Therefore, we can add any containaers (sidecar).
If my understanding is mistaken, please tell me.
※Supplementary information (version 1.10.10)
If we set the following to the operator,
```
k2 = KubernetesPodOperator(namespace='airflow01',
task_id='test-kwsk',
name='test-kwsk',
image="airflowworker:1.0.0",
image_pull_policy="IfNotPresent",
cmds=["bash", "-cx"],
arguments=['sleep 10000'],
do_xcom_push=True,
....
..
```
The pod object of argument of pod_mutation_hook is already set as follows.
```
affinity: {}
annotations: {}
args: ['sleep 10000']
cmds: ['bash', '-cx']
configmaps: []
dnspolicy: None
envs: {}
hostnetwork: False
image: airflowworker:1.0.0
image_pull_policy: IfNotPresent
...
..
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
j-y-matsubara commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422455491
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is it used for sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers (sidecars), excluding the init container.
In the case of using kubernetes_request_factory.py ( airflow/contrib/kubernetes/ ) , pod object of argument of pod_mutation_hook is not V1Pod.
**Part of the contents of** req object creating in pod_request_factory.py is overwritten with this pod object, in kubernetes_request_factory.py, and launch "Pod" based on **req object.**
In this overwriting, I think there is no function to add new containers (sidecars) excluding the init container.
In the version 2.0 (airflow/kubernetes/), We can directly edit the pod object used to launch "Pod". ( the pod object of argument of pod_mutation_hook is V1Pod ) Therefore, we can add any containaers (sidecar).
If my understanding is mistaken, please tell me.
Thank you in advance.
※Supplementary information (version 1.10.10)
If we set the following to the operator,
```
k2 = KubernetesPodOperator(namespace='airflow01',
task_id='test-kwsk',
name='test-kwsk',
image="airflowworker:1.0.0",
image_pull_policy="IfNotPresent",
cmds=["bash", "-cx"],
arguments=['sleep 10000'],
do_xcom_push=True,
....
..
```
The pod object of argument of pod_mutation_hook is already set as follows.
```
affinity: {}
annotations: {}
args: ['sleep 10000']
cmds: ['bash', '-cx']
configmaps: []
dnspolicy: None
envs: {}
hostnetwork: False
image: airflowworker:1.0.0
image_pull_policy: IfNotPresent
...
..
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
j-y-matsubara commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422455491
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is it used for sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers (sidecars), excluding the init container ( in the case of using kubernetes_request_factory.py, pod object of argument of pod_mutation_hook is not V1Pod※. **Part of the contents of** req object creating in pod_request_factory.py is overwritten with this pod object, and launch "Pod" based on **req object.** ).
In the version 2.0, We can directly edit the pod object (V1Pod) used to launch "Pod". ( argument of pod_mutation_hook is V1Pod ) Therefore, we can add any containaers (sidecar).
If my understanding is mistaken, please tell me.
※Supplementary information (version 1.10.10)
If we set the following to the operator,
```
k2 = KubernetesPodOperator(namespace='airflow01',
task_id='test-kwsk',
name='test-kwsk',
image="airflowworker:1.0.0",
image_pull_policy="IfNotPresent",
cmds=["bash", "-cx"],
arguments=['sleep 10000'],
do_xcom_push=True,
....
..
```
The pod object of argument of pod_mutation_hook is already set as follows.
```
affinity: {}
annotations: {}
args: ['sleep 10000']
cmds: ['bash', '-cx']
configmaps: []
dnspolicy: None
envs: {}
hostnetwork: False
image: airflowworker:1.0.0
image_pull_policy: IfNotPresent
...
..
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
j-y-matsubara commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422455718
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
to every worker pod launched by KubernetesExecutor or KubernetesPodOperator.
.. code:: python
def pod_mutation_hook(pod: Pod):
pod.annotations['airflow.apache.org/launched-by'] = 'Tests'
+
+
+The structure of 'pod' that is an argument of pod_mutation_hook are as follows.
+
+.. code:: python
+
+ affinity: {}
+ annotations: {}
+ args: []
+ cmds: []
+ configmaps: []
+ dnspolicy:
+ Noneenvs: {}
Review comment:
Done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
j-y-matsubara commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422455491
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is is made and used for sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers, excluding the init container ( in the case of using kubernetes_request_factory.py, pod object is not V1Pod. Overwrite part of the contents of req object creating in pod_request_factory.py with pod object, and launch "Pod" by using req object. ).
In the version 2.0, We can directly edit the pod objects (V1Pod) used by launching "Pod". Therefore, we can add any containaers (sidecar).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
j-y-matsubara commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422455491
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is it used for sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers (sidecars), excluding the init container. ( In the case of using kubernetes_request_factory.py※, pod object of argument of pod_mutation_hook is not V1Pod. **Part of the contents of** req object creating in pod_request_factory.py is overwritten with this pod object, and launch "Pod" based on **req object.** To this overwrite, there is no function to add new containers (sidecars).)
In the version 2.0, We can directly edit the pod object used to launch "Pod". ( the pod object of argument of pod_mutation_hook is V1Pod ) Therefore, we can add any containaers (sidecar).
If my understanding is mistaken, please tell me.
※Supplementary information (version 1.10.10)
If we set the following to the operator,
```
k2 = KubernetesPodOperator(namespace='airflow01',
task_id='test-kwsk',
name='test-kwsk',
image="airflowworker:1.0.0",
image_pull_policy="IfNotPresent",
cmds=["bash", "-cx"],
arguments=['sleep 10000'],
do_xcom_push=True,
....
..
```
The pod object of argument of pod_mutation_hook is already set as follows.
```
affinity: {}
annotations: {}
args: ['sleep 10000']
cmds: ['bash', '-cx']
configmaps: []
dnspolicy: None
envs: {}
hostnetwork: False
image: airflowworker:1.0.0
image_pull_policy: IfNotPresent
...
..
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
j-y-matsubara commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422455491
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is is made and used for sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers, excluding the init container ( in the case of using kubernetes_request_factory.py, pod object is not V1Pod. Overwrite part of the contents of req object creating in pod_request_factory.py with pod object, and launch "Pod" by using req object. ).
In the version 2.0, We can directly edit the pod objects (V1Pod) used to launch "Pod". Therefore, we can add any containaers (sidecar).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
j-y-matsubara commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422455491
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is it used for sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers (sidecars), excluding the init container ( in the case of using kubernetes_request_factory.py, pod object of argument of pod_mutation_hook is not V1Pod※. Overwrite **part of the contents of** req object, for example req['spec']['containers'][0]['image'] = pod.image , creating in pod_request_factory.py with pod object, and launch "Pod" based on req object. ).
In the version 2.0, We can directly edit the pod object (V1Pod) used to launch "Pod". ( argument of pod_mutation_hook is V1Pod ) Therefore, we can add any containaers (sidecar).
If my understanding is mistaken, please tell me.
※Supplementary information (version 1.10.10)
If we set the following to the operator,
```
k2 = KubernetesPodOperator(namespace='airflow01',
task_id='test-kwsk',
name='test-kwsk',
image="airflowworker:1.0.0",
image_pull_policy="IfNotPresent",
cmds=["bash", "-cx"],
arguments=['sleep 10000'],
do_xcom_push=True,
....
..
```
The pod object of argument of pod_mutation_hook is already set as follows.
```
affinity: {}
annotations: {}
args: ['sleep 10000']
cmds: ['bash', '-cx']
configmaps: []
dnspolicy: None
envs: {}
hostnetwork: False
image: airflowworker:1.0.0
image_pull_policy: IfNotPresent
...
..
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
j-y-matsubara commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422455491
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is it used for sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers (sidecars), excluding the init container.
In the case of using kubernetes_request_factory.py※, pod object of argument of pod_mutation_hook is not V1Pod.
**Part of the contents of** req object creating in pod_request_factory.py is overwritten with this pod object, in kubernetes_request_factory.py, and launch "Pod" based on **req object.**
In this overwriting, I think there is no function to add new containers (sidecars) excluding the init container.
In the version 2.0, We can directly edit the pod object used to launch "Pod". ( the pod object of argument of pod_mutation_hook is V1Pod ) Therefore, we can add any containaers (sidecar).
If my understanding is mistaken, please tell me.
Thank you in advance.
※Supplementary information (version 1.10.10)
If we set the following to the operator,
```
k2 = KubernetesPodOperator(namespace='airflow01',
task_id='test-kwsk',
name='test-kwsk',
image="airflowworker:1.0.0",
image_pull_policy="IfNotPresent",
cmds=["bash", "-cx"],
arguments=['sleep 10000'],
do_xcom_push=True,
....
..
```
The pod object of argument of pod_mutation_hook is already set as follows.
```
affinity: {}
annotations: {}
args: ['sleep 10000']
cmds: ['bash', '-cx']
configmaps: []
dnspolicy: None
envs: {}
hostnetwork: False
image: airflowworker:1.0.0
image_pull_policy: IfNotPresent
...
..
```
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is it used for sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers (sidecars), excluding the init container.
In the case of using kubernetes_request_factory.py※, pod object of argument of pod_mutation_hook is not V1Pod.
**Part of the contents of** req object creating in pod_request_factory.py is overwritten with this pod object, in kubernetes_request_factory.py, and launch "Pod" based on **req object.**
In this overwriting, I think there is no function to add new containers (sidecars) excluding the init container.
In the version 2.0, We can directly edit the pod object used to launch "Pod". ( the pod object of argument of pod_mutation_hook is V1Pod ) Therefore, we can add any containaers (sidecar).
If my understanding is mistaken, please tell me.
Thank you in advance.
※Supplementary information (version 1.10.10)
If we set the following to the operator,
```
k2 = KubernetesPodOperator(namespace='airflow01',
task_id='test-kwsk',
name='test-kwsk',
image="airflowworker:1.0.0",
image_pull_policy="IfNotPresent",
cmds=["bash", "-cx"],
arguments=['sleep 10000'],
do_xcom_push=True,
....
..
```
The pod object of argument of pod_mutation_hook is already set as follows.
```
affinity: {}
annotations: {}
args: ['sleep 10000']
cmds: ['bash', '-cx']
configmaps: []
dnspolicy: None
envs: {}
hostnetwork: False
image: airflowworker:1.0.0
image_pull_policy: IfNotPresent
...
..
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
j-y-matsubara commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422455491
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is it used for sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers (sidecars), excluding the init container. ( In the case of using kubernetes_request_factory.py※, pod object of argument of pod_mutation_hook is not V1Pod. **Part of the contents of** req object creating in pod_request_factory.py is overwritten with this pod object, and launch "Pod" based on **req object.** There is no function to add new containers (sidecars) to this overwrite. )
In the version 2.0, We can directly edit the pod object used to launch "Pod". ( the pod object of argument of pod_mutation_hook is V1Pod ) Therefore, we can add any containaers (sidecar).
If my understanding is mistaken, please tell me.
※Supplementary information (version 1.10.10)
If we set the following to the operator,
```
k2 = KubernetesPodOperator(namespace='airflow01',
task_id='test-kwsk',
name='test-kwsk',
image="airflowworker:1.0.0",
image_pull_policy="IfNotPresent",
cmds=["bash", "-cx"],
arguments=['sleep 10000'],
do_xcom_push=True,
....
..
```
The pod object of argument of pod_mutation_hook is already set as follows.
```
affinity: {}
annotations: {}
args: ['sleep 10000']
cmds: ['bash', '-cx']
configmaps: []
dnspolicy: None
envs: {}
hostnetwork: False
image: airflowworker:1.0.0
image_pull_policy: IfNotPresent
...
..
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
j-y-matsubara commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422455491
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is is made and used for sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers, excluding the init container ( in the case of using kubernetes_request_factory.py, pod object is not V1Pod. Overwrite part of the contents of req object creating in pod_request_factory.py with pod object, and launch "Pod" by using req object. ).
In the version 2.0, We can directly edit the pod objects (V1Pod) used by launching Pod. Therefore, we can add any containaers (sidecar).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
j-y-matsubara commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422455491
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is is made and used for sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers, excluding the init container ( in the case of using kubernetes_request_factory.py, pod object is not V1Pod. Overwrite **part of the contents of** req object creating in pod_request_factory.py with pod object, and launch "Pod" by using req object. ).
In the version 2.0, We can directly edit the pod objects (V1Pod) used to launch "Pod". Therefore, we can add any containaers (sidecar).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] stale[bot] closed pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
stale[bot] closed pull request #8757:
URL: https://github.com/apache/airflow/pull/8757
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422377934
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
to every worker pod launched by KubernetesExecutor or KubernetesPodOperator.
.. code:: python
def pod_mutation_hook(pod: Pod):
pod.annotations['airflow.apache.org/launched-by'] = 'Tests'
+
+
+The structure of 'pod' that is an argument of pod_mutation_hook are as follows.
+
+.. code:: python
+
+ affinity: {}
+ annotations: {}
+ args: []
+ cmds: []
+ configmaps: []
+ dnspolicy:
+ Noneenvs: {}
Review comment:
I don't thin this name is right.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
j-y-matsubara commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422455491
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is is made and used for sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers, excluding the init container ( in the case of using kubernetes_request_factory.py, pod object of argument of pod_mutation_hook is not V1Pod※. Overwrite **part of the contents of** req object, for example req['spec']['containers'][0]['image'] = pod.image , creating in pod_request_factory.py with pod object, and launch "Pod" based on req object. ).
In the version 2.0, We can directly edit the pod object (V1Pod) used to launch "Pod". ( argument of pod_mutation_hook is V1Pod ) Therefore, we can add any containaers (sidecar).
If my understanding is mistaken, please tell me.
※Supplementary information (version 1.10.10)
If we set the following to the operator,
```
k2 = KubernetesPodOperator(namespace='airflow01',
task_id='test-kwsk',
name='test-kwsk',
image="airflowworker:1.0.0",
image_pull_policy="IfNotPresent",
cmds=["bash", "-cx"],
arguments=['sleep 10000'],
do_xcom_push=True,
....
..
```
The pod object of argument of pod_mutation_hook is already set as follows.
```
affinity: {}
annotations: {}
args: ['sleep 10000']
cmds: ['bash', '-cx']
configmaps: []
dnspolicy: None
envs: {}
hostnetwork: False
image: airflowworker:1.0.0
image_pull_policy: IfNotPresent
...
..
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
j-y-matsubara commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422455491
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is it used for sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers (sidecars), excluding the init container. ( In the case of using kubernetes_request_factory.py※, pod object of argument of pod_mutation_hook is not V1Pod. **Part of the contents of** req object creating in pod_request_factory.py is overwritten with this pod object, and launch "Pod" based on **req object.** To this overwrite, I think there is no function to add new containers (sidecars) excluding the init container.)
In the version 2.0, We can directly edit the pod object used to launch "Pod". ( the pod object of argument of pod_mutation_hook is V1Pod ) Therefore, we can add any containaers (sidecar).
If my understanding is mistaken, please tell me.
※Supplementary information (version 1.10.10)
If we set the following to the operator,
```
k2 = KubernetesPodOperator(namespace='airflow01',
task_id='test-kwsk',
name='test-kwsk',
image="airflowworker:1.0.0",
image_pull_policy="IfNotPresent",
cmds=["bash", "-cx"],
arguments=['sleep 10000'],
do_xcom_push=True,
....
..
```
The pod object of argument of pod_mutation_hook is already set as follows.
```
affinity: {}
annotations: {}
args: ['sleep 10000']
cmds: ['bash', '-cx']
configmaps: []
dnspolicy: None
envs: {}
hostnetwork: False
image: airflowworker:1.0.0
image_pull_policy: IfNotPresent
...
..
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
j-y-matsubara commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422455491
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is is made and used for sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers, excluding the init container ( in the case of using kubernetes_request_factory.py, pod object of argument of pod_mutation_hook is not V1Pod※. Overwrite **part of the contents of** req object, for example req['spec']['containers'][0]['image'] = pod.image , creating in pod_request_factory.py with pod object, and launch "Pod" based on req object. ).
In the version 2.0, We can directly edit the pod object (V1Pod) used to launch "Pod". ( argument of pod_mutation_hook is V1Pod ) Therefore, we can add any containaers (sidecar).
If my understanding is mistaken, please tell me.
※Supplementary information
If we set the following to the operator,
```
k2 = KubernetesPodOperator(namespace='airflow01',
task_id='test-kwsk',
name='test-kwsk',
image="airflowworker:1.0.0",
image_pull_policy="IfNotPresent",
cmds=["bash", "-cx"],
arguments=['sleep 10000'],
do_xcom_push=True,
....
..
```
The pod object of argument of pod_mutation_hook is already set as follows.
```
affinity: {}
annotations: {}
args: ['sleep 10000']
cmds: ['bash', '-cx']
configmaps: []
dnspolicy: None
envs: {}
hostnetwork: False
image: airflowworker:1.0.0
image_pull_policy: IfNotPresent
...
..
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
j-y-matsubara commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422455491
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is it used for sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers (sidecars), excluding the init container. ( In the case of using kubernetes_request_factory.py※, pod object of argument of pod_mutation_hook is not V1Pod. **Part of the contents of** req object creating in pod_request_factory.py is overwritten with this pod object, in kubernetes_request_factory.py, and launch "Pod" based on **req object.** In this overwriting, I think there is no function to add new containers (sidecars) excluding the init container.)
In the version 2.0, We can directly edit the pod object used to launch "Pod". ( the pod object of argument of pod_mutation_hook is V1Pod ) Therefore, we can add any containaers (sidecar).
If my understanding is mistaken, please tell me.
※Supplementary information (version 1.10.10)
If we set the following to the operator,
```
k2 = KubernetesPodOperator(namespace='airflow01',
task_id='test-kwsk',
name='test-kwsk',
image="airflowworker:1.0.0",
image_pull_policy="IfNotPresent",
cmds=["bash", "-cx"],
arguments=['sleep 10000'],
do_xcom_push=True,
....
..
```
The pod object of argument of pod_mutation_hook is already set as follows.
```
affinity: {}
annotations: {}
args: ['sleep 10000']
cmds: ['bash', '-cx']
configmaps: []
dnspolicy: None
envs: {}
hostnetwork: False
image: airflowworker:1.0.0
image_pull_policy: IfNotPresent
...
..
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
j-y-matsubara commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422455491
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is it used for sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers (sidecars), excluding the init container. ( In the case of using kubernetes_request_factory.py※, pod object of argument of pod_mutation_hook is not V1Pod. **Part of the contents of** req object creating in pod_request_factory.py is overwritten with this pod object in kubernetes_request_factory.py, and launch "Pod" based on **req object.** To this overwrite, I think there is no function to add new containers (sidecars) excluding the init container.)
In the version 2.0, We can directly edit the pod object used to launch "Pod". ( the pod object of argument of pod_mutation_hook is V1Pod ) Therefore, we can add any containaers (sidecar).
If my understanding is mistaken, please tell me.
※Supplementary information (version 1.10.10)
If we set the following to the operator,
```
k2 = KubernetesPodOperator(namespace='airflow01',
task_id='test-kwsk',
name='test-kwsk',
image="airflowworker:1.0.0",
image_pull_policy="IfNotPresent",
cmds=["bash", "-cx"],
arguments=['sleep 10000'],
do_xcom_push=True,
....
..
```
The pod object of argument of pod_mutation_hook is already set as follows.
```
affinity: {}
annotations: {}
args: ['sleep 10000']
cmds: ['bash', '-cx']
configmaps: []
dnspolicy: None
envs: {}
hostnetwork: False
image: airflowworker:1.0.0
image_pull_policy: IfNotPresent
...
..
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
j-y-matsubara commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422455491
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is is made and used for sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers, excluding the init container ( in the case of using kubernetes_request_factory.py, pod object is not V1Pod※. Overwrite **part of the contents of** req object, for example req['spec']['containers'][0]['image'] = pod.image , creating in pod_request_factory.py with pod object, and launch "Pod" based on req object. ).
In the version 2.0, We can directly edit the pod object (V1Pod) used to launch "Pod". ( argument of pod_mutation_hook is V1Pod ) Therefore, we can add any containaers (sidecar).
If my understanding is mistaken, please tell me.
※Supplementary information
If we set the following to the operator,
```
k2 = KubernetesPodOperator(namespace='airflow01',
task_id='test-kwsk',
name='test-kwsk',
image="airflowworker:1.0.0",
image_pull_policy="IfNotPresent",
cmds=["bash", "-cx"],
arguments=['sleep 10000'],
do_xcom_push=True,
....
..
```
The pod object of argument of pod_mutation_hook is already set as follows.
```
affinity: {}
annotations: {}
args: ['sleep 10000']
cmds: ['bash', '-cx']
configmaps: []
dnspolicy: None
envs: {}
hostnetwork: False
image: airflowworker:1.0.0
image_pull_policy: IfNotPresent
...
..
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] stale[bot] commented on pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
stale[bot] commented on pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#issuecomment-649939413
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
j-y-matsubara commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422455491
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is is made and used for sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers, excluding the init container ( in the case of using kubernetes_request_factory.py, pod object is not V1Pod).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
j-y-matsubara commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422455491
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is it used for sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers (sidecars), excluding the init container.
In the case of using kubernetes_request_factory.py ( airflow/contrib/kubernetes/ ),in the version 1.X.X, pod object of argument of pod_mutation_hook is not V1Pod.
**Part of the contents of** req object creating in pod_request_factory.py is overwritten with this pod object, in kubernetes_request_factory.py, and launch "Pod" based on **req object.**
In this overwriting, I think there is no function to add new containers (sidecars) excluding the init container.
In the version 2.0 (airflow/kubernetes/), We can directly edit the pod object used to launch "Pod". ( the pod object of argument of pod_mutation_hook is V1Pod ) Therefore, we can add any containaers (sidecar).
If my understanding is mistaken, please tell me.
Thank you in advance.
※Supplementary information (version 1.10.10)
If we set the following to the operator,
```
k2 = KubernetesPodOperator(namespace='airflow01',
task_id='test-kwsk',
name='test-kwsk',
image="airflowworker:1.0.0",
image_pull_policy="IfNotPresent",
cmds=["bash", "-cx"],
arguments=['sleep 10000'],
do_xcom_push=True,
....
..
```
The pod object of argument of pod_mutation_hook is already set as follows.
```
affinity: {}
annotations: {}
args: ['sleep 10000']
cmds: ['bash', '-cx']
configmaps: []
dnspolicy: None
envs: {}
hostnetwork: False
image: airflowworker:1.0.0
image_pull_policy: IfNotPresent
...
..
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
j-y-matsubara commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422455491
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is is made and used for sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers, excluding the init container ( in the case of using kubernetes_request_factory.py, pod object is not V1Pod※. Overwrite **part of the contents of** req object, for example req['spec']['containers'][0]['image'] = pod.image , creating in pod_request_factory.py with pod object, and launch "Pod" based on req object. ).
In the version 2.0, We can directly edit the pod object (V1Pod) used to launch "Pod". Therefore, we can add any containaers (sidecar).
If my understanding is mistaken, please tell me.
※Supplementary information
If we set the following to the operator,
```
k2 = KubernetesPodOperator(namespace='airflow01',
task_id='test-kwsk',
name='test-kwsk',
image="airflowworker:1.0.0",
image_pull_policy="IfNotPresent",
cmds=["bash", "-cx"],
arguments=['sleep 10000'],
do_xcom_push=True,
....
..
```
The pod object of argument of pod_mutation_hook is already set as follows.
```
affinity: {}
annotations: {}
args: ['sleep 10000']
cmds: ['bash', '-cx']
configmaps: []
dnspolicy: None
envs: {}
hostnetwork: False
image: airflowworker:1.0.0
image_pull_policy: IfNotPresent
...
..
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] j-y-matsubara commented on a change in pull request #8757: Fix the incorrect description of pod_mutation_hook in kubernetes.rst
Posted by GitBox <gi...@apache.org>.
j-y-matsubara commented on a change in pull request #8757:
URL: https://github.com/apache/airflow/pull/8757#discussion_r422455491
##########
File path: docs/kubernetes.rst
##########
@@ -40,11 +40,44 @@ that has the ability to mutate pod objects before sending them to the Kubernetes
for scheduling. It receives a single argument as a reference to pod objects, and
is expected to alter its attributes.
-This could be used, for instance, to add sidecar or init containers
+This could be used, for instance, to add init containers
Review comment:
How is it used for sidecar?
We can mutation pod object by using pod_mutation_hook, but I think we can not add any containers (sidecars), excluding the init container ( in the case of using kubernetes_request_factory.py※, pod object of argument of pod_mutation_hook is not V1Pod. **Part of the contents of** req object creating in pod_request_factory.py is overwritten with this pod object, and launch "Pod" based on **req object.** ).
In the version 2.0, We can directly edit the pod object used to launch "Pod". ( the pod object of argument of pod_mutation_hook is V1Pod ) Therefore, we can add any containaers (sidecar).
If my understanding is mistaken, please tell me.
※Supplementary information (version 1.10.10)
If we set the following to the operator,
```
k2 = KubernetesPodOperator(namespace='airflow01',
task_id='test-kwsk',
name='test-kwsk',
image="airflowworker:1.0.0",
image_pull_policy="IfNotPresent",
cmds=["bash", "-cx"],
arguments=['sleep 10000'],
do_xcom_push=True,
....
..
```
The pod object of argument of pod_mutation_hook is already set as follows.
```
affinity: {}
annotations: {}
args: ['sleep 10000']
cmds: ['bash', '-cx']
configmaps: []
dnspolicy: None
envs: {}
hostnetwork: False
image: airflowworker:1.0.0
image_pull_policy: IfNotPresent
...
..
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org