You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "mrk-andreev (via GitHub)" <gi...@apache.org> on 2024/11/09 16:25:24 UTC
[PR] Add random_name_suffix to SparkKubernetesOperator (#43800) [airflow]
mrk-andreev opened a new pull request, #43847:
URL: https://github.com/apache/airflow/pull/43847
Prior to this change, `random_name_suffix` was only documented but not implemented as a configurable option. Passing this value as an argument had no effect. This commit introduces a `false` option for `random_name_suffix`, which prevents the generation of a random suffix for the pod name. For compatibility, the default value is set to `true`, ensuring the pod name will still conform to `MAX_LABEL_LEN = 63`.
Fixes: #43800
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add random_name_suffix to SparkKubernetesOperator (#43800) [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on code in PR #43847:
URL: https://github.com/apache/airflow/pull/43847#discussion_r1835446143
##########
providers/tests/cncf/kubernetes/operators/test_spark_kubernetes.py:
##########
@@ -828,3 +830,26 @@ def test_resolve_application_file_real_file_not_exists(create_task_instance_of_o
task: SparkKubernetesOperator = ti.task
with pytest.raises(TypeError, match="application_file body can't transformed into the dictionary"):
_ = task.template_body
+
+
+@pytest.mark.parametrize(
+ "random_name_suffix",
+ [pytest.param(True, id="use-random_name_suffix"), pytest.param(False, id="skip-random_name_suffix")],
+)
+def test_create_job_name(random_name_suffix: bool):
+ name = f"{uuid4()}"
+ op = SparkKubernetesOperator(task_id="task_id", name=name, random_name_suffix=random_name_suffix)
+ pod_name = op.create_job_name()
+
+ if random_name_suffix:
+ assert pod_name.startswith(name)
Review Comment:
```suggestion
assert pod_name.startswith(name)
```
we could also check if the pod_name is different than name
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add random_name_suffix to SparkKubernetesOperator (#43800) [airflow]
Posted by "mrk-andreev (via GitHub)" <gi...@apache.org>.
mrk-andreev commented on code in PR #43847:
URL: https://github.com/apache/airflow/pull/43847#discussion_r1835447587
##########
providers/tests/cncf/kubernetes/operators/test_spark_kubernetes.py:
##########
@@ -828,3 +830,26 @@ def test_resolve_application_file_real_file_not_exists(create_task_instance_of_o
task: SparkKubernetesOperator = ti.task
with pytest.raises(TypeError, match="application_file body can't transformed into the dictionary"):
_ = task.template_body
+
+
+@pytest.mark.parametrize(
+ "random_name_suffix",
+ [pytest.param(True, id="use-random_name_suffix"), pytest.param(False, id="skip-random_name_suffix")],
+)
+def test_create_job_name(random_name_suffix: bool):
+ name = f"{uuid4()}"
+ op = SparkKubernetesOperator(task_id="task_id", name=name, random_name_suffix=random_name_suffix)
+ pod_name = op.create_job_name()
+
+ if random_name_suffix:
+ assert pod_name.startswith(name)
Review Comment:
Withed with
```
assert pod_name.startswith(name)
assert pod_name != name
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Add random_name_suffix to SparkKubernetesOperator (#43800) [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk merged PR #43847:
URL: https://github.com/apache/airflow/pull/43847
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org