You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/03/01 21:43:54 UTC
[GitHub] [airflow] ashb opened a new pull request #21916: Add map_index label to mapped KubernetesPodOperator
ashb opened a new pull request #21916:
URL: https://github.com/apache/airflow/pull/21916
This seems remarkable simple. Have I missed something?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #21916: Add map_index label to mapped KubernetesPodOperator
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #21916:
URL: https://github.com/apache/airflow/pull/21916#issuecomment-1055903320
The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the core of Airflow. If the committers decide that the full tests matrix is needed, they will add the label 'full tests needed'. Then you should rebase to the latest main or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb merged pull request #21916: Add map_index label to mapped KubernetesPodOperator
Posted by GitBox <gi...@apache.org>.
ashb merged pull request #21916:
URL: https://github.com/apache/airflow/pull/21916
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #21916: Add map_index label to mapped KubernetesPodOperator
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #21916:
URL: https://github.com/apache/airflow/pull/21916#discussion_r817787199
##########
File path: airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py
##########
@@ -282,13 +282,17 @@ def _get_ti_pod_labels(context: Optional[dict] = None, include_try_number: bool
if not context:
return {}
- labels = {
- 'dag_id': context['dag'].dag_id,
- 'task_id': context['task'].task_id,
- 'execution_date': context['ts'],
- }
+ ti = context['ti']
+
+ labels = {'dag_id': ti.dag_id, 'task_id': ti.task_id, 'execution_date': context['ts']}
Review comment:
I had that at first here but it makes the change about task mapping hard to see, so I've got it in a separate commit/pr
(If you haven't noticed I'm a big fan of Pr=one thing)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dstandish commented on a change in pull request #21916: Add map_index label to mapped KubernetesPodOperator
Posted by GitBox <gi...@apache.org>.
dstandish commented on a change in pull request #21916:
URL: https://github.com/apache/airflow/pull/21916#discussion_r817784022
##########
File path: airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py
##########
@@ -282,13 +282,17 @@ def _get_ti_pod_labels(context: Optional[dict] = None, include_try_number: bool
if not context:
return {}
- labels = {
- 'dag_id': context['dag'].dag_id,
- 'task_id': context['task'].task_id,
- 'execution_date': context['ts'],
- }
+ ti = context['ti']
+
+ labels = {'dag_id': ti.dag_id, 'task_id': ti.task_id, 'execution_date': context['ts']}
Review comment:
probably we should replace `execution_date` with `run_id` here, since that is now the "real" PK component right?. the signifigance of these labels, AFAIK, is just for "reattaching" to a pod after interruption, so it's just meant to be attrs that uniquely identify the TI. though it needn't necessarily be done in this PR of course.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dstandish commented on a change in pull request #21916: Add map_index label to mapped KubernetesPodOperator
Posted by GitBox <gi...@apache.org>.
dstandish commented on a change in pull request #21916:
URL: https://github.com/apache/airflow/pull/21916#discussion_r818016620
##########
File path: airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py
##########
@@ -282,13 +282,17 @@ def _get_ti_pod_labels(context: Optional[dict] = None, include_try_number: bool
if not context:
return {}
- labels = {
- 'dag_id': context['dag'].dag_id,
- 'task_id': context['task'].task_id,
- 'execution_date': context['ts'],
- }
+ ti = context['ti']
+
+ labels = {'dag_id': ti.dag_id, 'task_id': ti.task_id, 'execution_date': context['ts']}
Review comment:
yup, makes sense
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dstandish commented on a change in pull request #21916: Add map_index label to mapped KubernetesPodOperator
Posted by GitBox <gi...@apache.org>.
dstandish commented on a change in pull request #21916:
URL: https://github.com/apache/airflow/pull/21916#discussion_r817784022
##########
File path: airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py
##########
@@ -282,13 +282,17 @@ def _get_ti_pod_labels(context: Optional[dict] = None, include_try_number: bool
if not context:
return {}
- labels = {
- 'dag_id': context['dag'].dag_id,
- 'task_id': context['task'].task_id,
- 'execution_date': context['ts'],
- }
+ ti = context['ti']
+
+ labels = {'dag_id': ti.dag_id, 'task_id': ti.task_id, 'execution_date': context['ts']}
Review comment:
probably we should replace `execution_date` with `run_id` here, since that is now the "real" PK component right?. the signifigance of these labels, AFAIK, is just for "reattaching" to a pod after scheduler death, so it's just meant to be attrs that uniquely identify the TI. though it needn't necessarily be done in this PR of course.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #21916: Add map_index label to mapped KubernetesPodOperator
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #21916:
URL: https://github.com/apache/airflow/pull/21916#discussion_r817787199
##########
File path: airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py
##########
@@ -282,13 +282,17 @@ def _get_ti_pod_labels(context: Optional[dict] = None, include_try_number: bool
if not context:
return {}
- labels = {
- 'dag_id': context['dag'].dag_id,
- 'task_id': context['task'].task_id,
- 'execution_date': context['ts'],
- }
+ ti = context['ti']
+
+ labels = {'dag_id': ti.dag_id, 'task_id': ti.task_id, 'execution_date': context['ts']}
Review comment:
I had that at first here but it makes the change about task mapping hard to see, so I've got it in a separate commit/pr
(If you haven't noticed I'm a big fan of Pr=one thing, especially as changelog comes from commit messages.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on a change in pull request #21916: Add map_index label to mapped KubernetesPodOperator
Posted by GitBox <gi...@apache.org>.
ashb commented on a change in pull request #21916:
URL: https://github.com/apache/airflow/pull/21916#discussion_r818711208
##########
File path: airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py
##########
@@ -282,13 +282,17 @@ def _get_ti_pod_labels(context: Optional[dict] = None, include_try_number: bool
if not context:
return {}
- labels = {
- 'dag_id': context['dag'].dag_id,
- 'task_id': context['task'].task_id,
- 'execution_date': context['ts'],
- }
+ ti = context['ti']
+
+ labels = {'dag_id': ti.dag_id, 'task_id': ti.task_id, 'execution_date': context['ts']}
Review comment:
https://github.com/apache/airflow/pull/21960
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org