You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "jscheffl (via GitHub)" <gi...@apache.org> on 2025/11/01 21:45:12 UTC

[PR] Enable ruff PLW2901 rule [airflow]

jscheffl opened a new pull request, #57700:
URL: https://github.com/apache/airflow/pull/57700

   While implementing https://github.com/apache/airflow/pull/57294 I realized that a couple of more ruff PLW rules might be interesting, this PR enables PLW2901
   
   See also https://docs.astral.sh/ruff/rules/#warning-plw


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Enable ruff PLW2101,PLW2901,PLW3301 rule [airflow]

Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl commented on code in PR #57700:
URL: https://github.com/apache/airflow/pull/57700#discussion_r2506865508


##########
airflow-core/src/airflow/serialization/serde.py:
##########
@@ -365,30 +365,29 @@ def _register():
     _stringifiers.clear()
 
     with Stats.timer("serde.load_serializers") as timer:
-        for _, name, _ in iter_namespace(airflow.serialization.serializers):
-            name = import_module(name)
-            for s in getattr(name, "serializers", ()):
-                if not isinstance(s, str):
-                    s = qualname(s)
-                if s in _serializers and _serializers[s] != name:
-                    raise AttributeError(f"duplicate {s} for serialization in {name} and {_serializers[s]}")
+        for _, module_name, _ in iter_namespace(airflow.serialization.serializers):
+            module = import_module(module_name)
+            for serializers in getattr(module, "serializers", ()):
+                s = serializers if isinstance(serializers, str) else qualname(serializers)

Review Comment:
   Good point, applied!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Enable ruff PLW2101,PLW2901,PLW3301 rule [airflow]

Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W commented on code in PR #57700:
URL: https://github.com/apache/airflow/pull/57700#discussion_r2501477294


##########
airflow-core/src/airflow/serialization/serde.py:
##########
@@ -365,30 +365,29 @@ def _register():
     _stringifiers.clear()
 
     with Stats.timer("serde.load_serializers") as timer:
-        for _, name, _ in iter_namespace(airflow.serialization.serializers):
-            name = import_module(name)
-            for s in getattr(name, "serializers", ()):
-                if not isinstance(s, str):
-                    s = qualname(s)
-                if s in _serializers and _serializers[s] != name:
-                    raise AttributeError(f"duplicate {s} for serialization in {name} and {_serializers[s]}")
+        for _, module_name, _ in iter_namespace(airflow.serialization.serializers):
+            module = import_module(module_name)
+            for serializers in getattr(module, "serializers", ()):
+                s = serializers if isinstance(serializers, str) else qualname(serializers)

Review Comment:
   ```suggestion
                   serializers_qualname = serializers if isinstance(serializers, str) else qualname(serializers)
   ```
   
   nit: `s` or `d` don't fit my taste. or `s_qualname` might be better 🤔 



##########
airflow-core/src/airflow/executors/executor_loader.py:
##########
@@ -74,11 +74,11 @@ def _get_executor_names(cls) -> list[ExecutorName]:
 
         all_executor_names: list[tuple[str | None, list[str]]] = cls._get_team_executor_configs()
 
-        executor_names = []
+        executor_names: list[ExecutorName] = []
         for team_name, executor_names_config in all_executor_names:
             executor_names_per_team = []
-            for name in executor_names_config:
-                if len(split_name := name.split(":")) == 1:
+            for executor_name_str in executor_names_config:

Review Comment:
   do we need the `_str` here? ignore me if it's needed 👀
   
   ```suggestion
               for executor_name in executor_names_config:
   ```



##########
airflow-core/src/airflow/serialization/serde.py:
##########
@@ -365,30 +365,29 @@ def _register():
     _stringifiers.clear()
 
     with Stats.timer("serde.load_serializers") as timer:
-        for _, name, _ in iter_namespace(airflow.serialization.serializers):
-            name = import_module(name)
-            for s in getattr(name, "serializers", ()):
-                if not isinstance(s, str):
-                    s = qualname(s)
-                if s in _serializers and _serializers[s] != name:
-                    raise AttributeError(f"duplicate {s} for serialization in {name} and {_serializers[s]}")
+        for _, module_name, _ in iter_namespace(airflow.serialization.serializers):
+            module = import_module(module_name)
+            for serializers in getattr(module, "serializers", ()):
+                s = serializers if isinstance(serializers, str) else qualname(serializers)
+                if s in _serializers and _serializers[s] != module:
+                    raise AttributeError(f"duplicate {s} for serialization in {module} and {_serializers[s]}")
                 log.debug("registering %s for serialization", s)
-                _serializers[s] = name
-            for d in getattr(name, "deserializers", ()):
-                if not isinstance(d, str):
-                    d = qualname(d)
-                if d in _deserializers and _deserializers[d] != name:
-                    raise AttributeError(f"duplicate {d} for deserialization in {name} and {_serializers[d]}")
+                _serializers[s] = module
+            for deserializers in getattr(module, "deserializers", ()):
+                d = deserializers if isinstance(deserializers, str) else qualname(deserializers)
+                if d in _deserializers and _deserializers[d] != module:
+                    raise AttributeError(
+                        f"duplicate {d} for deserialization in {module} and {_deserializers[d]}"
+                    )
                 log.debug("registering %s for deserialization", d)
-                _deserializers[d] = name
+                _deserializers[d] = module
                 _extra_allowed.add(d)
-            for c in getattr(name, "stringifiers", ()):
-                if not isinstance(c, str):
-                    c = qualname(c)
-                if c in _deserializers and _deserializers[c] != name:
-                    raise AttributeError(f"duplicate {c} for stringifiers in {name} and {_stringifiers[c]}")
+            for stringifiers in getattr(module, "stringifiers", ()):

Review Comment:
   Probably not something we need to do it this PR. but we maybe can duplicate these few for loop 🤔 



##########
airflow-core/tests/unit/plugins/test_plugin_ignore.py:
##########
@@ -104,8 +104,8 @@ def test_find_not_should_ignore_path_glob(self, tmp_path):
         }
         ignore_list_file = ".airflowignore_glob"
         print("-" * 20)
-        for file_path in find_path_from_directory(plugin_folder_path, ignore_list_file, "glob"):
-            file_path = Path(file_path)
+        for file_path_raw in find_path_from_directory(plugin_folder_path, ignore_list_file, "glob"):
+            file_path = Path(file_path_raw)

Review Comment:
   ```suggestion
           for raw_file_path in find_path_from_directory(plugin_folder_path, ignore_list_file, "glob"):
               file_path = Path(raw_file_path)
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Enable ruff PLW2101,PLW2901,PLW3301 rule [airflow]

Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl commented on PR #57700:
URL: https://github.com/apache/airflow/pull/57700#issuecomment-3508864720

   I think there is no value in backporting, would skip this


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Enable ruff PLW2101,PLW2901,PLW3301 rule [airflow]

Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl commented on PR #57700:
URL: https://github.com/apache/airflow/pull/57700#issuecomment-3499287871

   > the descirption was inaccurate, just updated it.
   
   Thanks @Lee-W for the great feedback! Adjusted names to your very good proposals!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Enable ruff PLW2101,PLW2901,PLW3301 rule [airflow]

Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl commented on code in PR #57700:
URL: https://github.com/apache/airflow/pull/57700#discussion_r2495777888


##########
airflow-core/src/airflow/serialization/serialized_objects.py:
##########
@@ -2565,40 +2572,43 @@ def _deserialize_dag_internal(
 
         # Note: Context is passed explicitly through method parameters, no class attributes needed
 
-        for k, v in encoded_dag.items():
-            if k == "_downstream_task_ids":
-                v = set(v)
-            elif k == "tasks":
+        for k_in, v_in in encoded_dag.items():
+            v: Any
+            k = k_in
+            if k_in == "_downstream_task_ids":
+                v = set(v_in)
+            elif k_in == "tasks":
                 SerializedBaseOperator._load_operator_extra_links = cls._load_operator_extra_links
                 tasks = {}
-                for obj in v:
+                for obj in v_in:
                     if obj.get(Encoding.TYPE) == DAT.OP:
                         deser = SerializedBaseOperator.deserialize_operator(
                             obj[Encoding.VAR], client_defaults
                         )
                         tasks[deser.task_id] = deser
                 k = "task_dict"
                 v = tasks
-            elif k == "timezone":
-                v = cls._deserialize_timezone(v)
-            elif k == "dagrun_timeout":
-                v = cls._deserialize_timedelta(v)
-            elif k.endswith("_date"):
-                v = cls._deserialize_datetime(v)
-            elif k == "edge_info":
+            elif k_in == "timezone":
+                v = cls._deserialize_timezone(v_in)
+            elif k_in == "dagrun_timeout":
+                v = cls._deserialize_timedelta(v_in)
+            elif k_in.endswith("_date"):
+                v = cls._deserialize_datetime(v_in)
+            elif k_in == "edge_info":
                 # Value structure matches exactly
-                pass
-            elif k == "timetable":
-                v = decode_timetable(v)
-            elif k == "weight_rule":
-                v = decode_priority_weight_strategy(v)
-            elif k in cls._decorated_fields:
-                v = cls.deserialize(v)
-            elif k == "params":
-                v = cls._deserialize_params_dict(v)
-            elif k == "tags":
-                v = set(v)
-            # else use v as it is
+                v = v_in
+            elif k_in == "timetable":
+                v = decode_timetable(v_in)
+            elif k_in == "weight_rule":
+                v = decode_priority_weight_strategy(v_in)
+            elif k_in in cls._decorated_fields:
+                v = cls.deserialize(v_in)
+            elif k_in == "params":
+                v = cls._deserialize_params_dict(v_in)
+            elif k_in == "tags":
+                v = set(v_in)
+            else:
+                v = v_in

Review Comment:
   Hi @uranusjr are you referring to this specific case of serialized objects (where we could exclude with `# noqa: ...` or is it a general concern?
   
   In rework making the codebase compatible I also had one other point in https://github.com/apache/airflow/pull/57700/files#diff-94532ce5cd8778da565396e51f54f7a10078725a1f61656615fbb173b7503fceR1818 where making code compliant made me a hard time.
   
   So do you mean in these specific cases we should exclude rule where it makes code not better? Or do you dislike the rule in general (I think in 90% of cases it prevents failures).
   
   I can also raise a discussion in devlist if you think we should have a larger round of forming an opinion on the rule.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Enable ruff PLW2101,PLW2901,PLW3301 rule [airflow]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #57700:
URL: https://github.com/apache/airflow/pull/57700#issuecomment-3508861006

   ### Backport failed to create: v3-1-test. View the failure log <a href='https://github.com/apache/airflow/actions/runs/19214835439'> Run details </a>
   
   <table>
       <tr>
           <th>Status</th>
           <th>Branch</th>
           <th>Result</th>
       </tr>
       <tr>
           <td>❌</td>
           <td>v3-1-test</td>
           <td><a href="https://github.com/apache/airflow/commit/4d596bcd08b3b9e04e853d6c212ba8d79e1f29fb"><img src='https://img.shields.io/badge/Commit-4d596bc-red' alt='Commit Link'></a></td>
       </tr>
   </table>
   
   You can attempt to backport this manually by running:
   
   ```bash
   cherry_picker 4d596bc v3-1-test
   ```
   
   This should apply the commit to the v3-1-test branch and leave the commit in conflict state marking
   the files that need manual conflict resolution.
   
   After you have resolved the conflicts, you can continue the backport process by running:
   
   ```bash
   cherry_picker --continue
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Enable ruff PLW2101,PLW2901,PLW3301 rule [airflow]

Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl commented on code in PR #57700:
URL: https://github.com/apache/airflow/pull/57700#discussion_r2506865693


##########
airflow-core/tests/unit/plugins/test_plugin_ignore.py:
##########
@@ -104,8 +104,8 @@ def test_find_not_should_ignore_path_glob(self, tmp_path):
         }
         ignore_list_file = ".airflowignore_glob"
         print("-" * 20)
-        for file_path in find_path_from_directory(plugin_folder_path, ignore_list_file, "glob"):
-            file_path = Path(file_path)
+        for file_path_raw in find_path_from_directory(plugin_folder_path, ignore_list_file, "glob"):
+            file_path = Path(file_path_raw)

Review Comment:
   Ah another naming nit, thanks! Applied



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Enable ruff PLW2101,PLW2901,PLW3301 rule [airflow]

Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl commented on code in PR #57700:
URL: https://github.com/apache/airflow/pull/57700#discussion_r2492211788


##########
pyproject.toml:
##########
@@ -609,6 +609,9 @@ extend-select = [
     "PLW1508", # Invalid type for environment variable default; expected str or None
     "PLW1510", # subprocess.run without explicit check argument
     "PLW1641", # Object does not implement __hash__ method
+    "PLW2101", # Threading lock directly created in with statement has no effect
+    "PLW2901", # Outer {outer_kind} variable {name} overwritten by inner {inner_kind} target
+    "PLW3301", # Nested {func} calls can be flattened

Review Comment:
   Yeah good point. Was batching the others up as well as no changes needed for them. Adjusted title



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Enable ruff PLW2101,PLW2901,PLW3301 rule [airflow]

Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W commented on PR #57700:
URL: https://github.com/apache/airflow/pull/57700#issuecomment-3494833165

   the descirption was inaccurate, just updated it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Enable ruff PLW2101,PLW2901,PLW3301 rule [airflow]

Posted by "Lee-W (via GitHub)" <gi...@apache.org>.
Lee-W commented on code in PR #57700:
URL: https://github.com/apache/airflow/pull/57700#discussion_r2497177221


##########
airflow-core/src/airflow/api_fastapi/core_api/services/public/variables.py:
##########
@@ -148,9 +148,11 @@ def handle_bulk_update(self, action: BulkUpdateAction, results: BulkActionRespon
             for variable in action.entities:
                 if variable.key not in update_keys:
                     continue
-                variable = update_orm_from_pydantic(variable.key, variable, action.update_mask, self.session)
+                variable_py = update_orm_from_pydantic(

Review Comment:
   ```suggestion
                   updated_variable = update_orm_from_pydantic(
   ```
   
   



##########
airflow-core/src/airflow/models/pool.py:
##########
@@ -198,9 +197,9 @@ def slots_stats(
         )
 
         # calculate queued and running metrics
-        for pool_name, state, count in state_count_by_pool:
+        for pool_name, state, count_raw in state_count_by_pool:

Review Comment:
   ```suggestion
           for pool_name, state, decimal_count in state_count_by_pool:
   ```
   
   maybe `decimal_count`, it expresses why we need to do it better



##########
providers/amazon/src/airflow/providers/amazon/aws/hooks/athena.py:
##########
@@ -40,11 +40,15 @@
 
 def query_params_to_string(params: dict[str, str | Collection[str]]) -> str:
     result = ""
-    for key, value in params.items():
+    for key, value_org in params.items():

Review Comment:
   original_value?



##########
airflow-core/tests/unit/plugins/test_plugin_ignore.py:
##########
@@ -77,8 +77,8 @@ def test_find_not_should_ignore_path_regexp(self, tmp_path):
             "test_load_sub1.py",
         }
         ignore_list_file = ".airflowignore"
-        for file_path in find_path_from_directory(plugin_folder_path, ignore_list_file, "regexp"):
-            file_path = Path(file_path)
+        for file_path_raw in find_path_from_directory(plugin_folder_path, ignore_list_file, "regexp"):

Review Comment:
   ```suggestion
           for raw_file_path in find_path_from_directory(plugin_folder_path, ignore_list_file, "regexp"):
   ```
   
   non-blocking



##########
airflow-core/tests/unit/core/test_impersonation_tests.py:
##########
@@ -58,9 +58,8 @@ def set_permissions(settings: dict[Path | str, int]):
     orig_permissions = []
     try:
         print(" Change file/directory permissions ".center(72, "+"))
-        for path, mode in settings.items():
-            if isinstance(path, str):
-                path = Path(path)
+        for path_raw, mode in settings.items():

Review Comment:
   ```suggestion
           for raw_path, mode in settings.items():
   ```
   
   non-blocking



##########
airflow-core/src/airflow/executors/executor_loader.py:
##########
@@ -77,8 +77,8 @@ def _get_executor_names(cls) -> list[ExecutorName]:
         executor_names = []
         for team_name, executor_names_config in all_executor_names:
             executor_names_per_team = []
-            for name in executor_names_config:
-                if len(split_name := name.split(":")) == 1:
+            for name_raw in executor_names_config:

Review Comment:
   ```suggestion
               for executor_name in executor_names_config:
   ```



##########
airflow-core/src/airflow/serialization/serde.py:
##########
@@ -365,26 +365,23 @@ def _register():
     _stringifiers.clear()
 
     with Stats.timer("serde.load_serializers") as timer:
-        for _, name, _ in iter_namespace(airflow.serialization.serializers):
-            name = import_module(name)
-            for s in getattr(name, "serializers", ()):
-                if not isinstance(s, str):
-                    s = qualname(s)
+        for _, module_name, _ in iter_namespace(airflow.serialization.serializers):
+            name = import_module(module_name)
+            for attr_s in getattr(name, "serializers", ()):

Review Comment:
   ```suggestion
               for serializers in getattr(name, "serializers", ()):
   ```



##########
airflow-core/src/airflow/serialization/serde.py:
##########
@@ -365,26 +365,23 @@ def _register():
     _stringifiers.clear()
 
     with Stats.timer("serde.load_serializers") as timer:
-        for _, name, _ in iter_namespace(airflow.serialization.serializers):
-            name = import_module(name)
-            for s in getattr(name, "serializers", ()):
-                if not isinstance(s, str):
-                    s = qualname(s)
+        for _, module_name, _ in iter_namespace(airflow.serialization.serializers):
+            name = import_module(module_name)

Review Comment:
   ```suggestion
               module = import_module(module_name)
   ```
   
   I think it should be a module instead



##########
airflow-core/src/airflow/dag_processing/collection.py:
##########
@@ -637,8 +637,8 @@ def _get_dag_assets(
     inlets: bool = True,
     outlets: bool = True,
 ) -> Iterable[tuple[str, AssetT]]:
-    for task in dag.data["dag"]["tasks"]:
-        task = task[Encoding.VAR]
+    for task_raw in dag.data["dag"]["tasks"]:

Review Comment:
   ```suggestion
       for raw_task in dag.data["dag"]["tasks"]:
   ```
   
   `raw_task` fits my taste better, but non-blocking



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Enable ruff PLW2101,PLW2901,PLW3301 rule [airflow]

Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on code in PR #57700:
URL: https://github.com/apache/airflow/pull/57700#discussion_r2492579874


##########
airflow-core/src/airflow/serialization/serialized_objects.py:
##########
@@ -2565,40 +2572,43 @@ def _deserialize_dag_internal(
 
         # Note: Context is passed explicitly through method parameters, no class attributes needed
 
-        for k, v in encoded_dag.items():
-            if k == "_downstream_task_ids":
-                v = set(v)
-            elif k == "tasks":
+        for k_in, v_in in encoded_dag.items():
+            v: Any
+            k = k_in
+            if k_in == "_downstream_task_ids":
+                v = set(v_in)
+            elif k_in == "tasks":
                 SerializedBaseOperator._load_operator_extra_links = cls._load_operator_extra_links
                 tasks = {}
-                for obj in v:
+                for obj in v_in:
                     if obj.get(Encoding.TYPE) == DAT.OP:
                         deser = SerializedBaseOperator.deserialize_operator(
                             obj[Encoding.VAR], client_defaults
                         )
                         tasks[deser.task_id] = deser
                 k = "task_dict"
                 v = tasks
-            elif k == "timezone":
-                v = cls._deserialize_timezone(v)
-            elif k == "dagrun_timeout":
-                v = cls._deserialize_timedelta(v)
-            elif k.endswith("_date"):
-                v = cls._deserialize_datetime(v)
-            elif k == "edge_info":
+            elif k_in == "timezone":
+                v = cls._deserialize_timezone(v_in)
+            elif k_in == "dagrun_timeout":
+                v = cls._deserialize_timedelta(v_in)
+            elif k_in.endswith("_date"):
+                v = cls._deserialize_datetime(v_in)
+            elif k_in == "edge_info":
                 # Value structure matches exactly
-                pass
-            elif k == "timetable":
-                v = decode_timetable(v)
-            elif k == "weight_rule":
-                v = decode_priority_weight_strategy(v)
-            elif k in cls._decorated_fields:
-                v = cls.deserialize(v)
-            elif k == "params":
-                v = cls._deserialize_params_dict(v)
-            elif k == "tags":
-                v = set(v)
-            # else use v as it is
+                v = v_in
+            elif k_in == "timetable":
+                v = decode_timetable(v_in)
+            elif k_in == "weight_rule":
+                v = decode_priority_weight_strategy(v_in)
+            elif k_in in cls._decorated_fields:
+                v = cls.deserialize(v_in)
+            elif k_in == "params":
+                v = cls._deserialize_params_dict(v_in)
+            elif k_in == "tags":
+                v = set(v_in)
+            else:
+                v = v_in

Review Comment:
   This one is a -0 from me since they arguably not only don’t help, but acually make this kind of operations _easier to get wrong_.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Enable ruff PLW2101,PLW2901,PLW3301 rule [airflow]

Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on code in PR #57700:
URL: https://github.com/apache/airflow/pull/57700#discussion_r2496848512


##########
airflow-core/src/airflow/serialization/serialized_objects.py:
##########
@@ -2565,40 +2572,43 @@ def _deserialize_dag_internal(
 
         # Note: Context is passed explicitly through method parameters, no class attributes needed
 
-        for k, v in encoded_dag.items():
-            if k == "_downstream_task_ids":
-                v = set(v)
-            elif k == "tasks":
+        for k_in, v_in in encoded_dag.items():
+            v: Any
+            k = k_in
+            if k_in == "_downstream_task_ids":
+                v = set(v_in)
+            elif k_in == "tasks":
                 SerializedBaseOperator._load_operator_extra_links = cls._load_operator_extra_links
                 tasks = {}
-                for obj in v:
+                for obj in v_in:
                     if obj.get(Encoding.TYPE) == DAT.OP:
                         deser = SerializedBaseOperator.deserialize_operator(
                             obj[Encoding.VAR], client_defaults
                         )
                         tasks[deser.task_id] = deser
                 k = "task_dict"
                 v = tasks
-            elif k == "timezone":
-                v = cls._deserialize_timezone(v)
-            elif k == "dagrun_timeout":
-                v = cls._deserialize_timedelta(v)
-            elif k.endswith("_date"):
-                v = cls._deserialize_datetime(v)
-            elif k == "edge_info":
+            elif k_in == "timezone":
+                v = cls._deserialize_timezone(v_in)
+            elif k_in == "dagrun_timeout":
+                v = cls._deserialize_timedelta(v_in)
+            elif k_in.endswith("_date"):
+                v = cls._deserialize_datetime(v_in)
+            elif k_in == "edge_info":
                 # Value structure matches exactly
-                pass
-            elif k == "timetable":
-                v = decode_timetable(v)
-            elif k == "weight_rule":
-                v = decode_priority_weight_strategy(v)
-            elif k in cls._decorated_fields:
-                v = cls.deserialize(v)
-            elif k == "params":
-                v = cls._deserialize_params_dict(v)
-            elif k == "tags":
-                v = set(v)
-            # else use v as it is
+                v = v_in
+            elif k_in == "timetable":
+                v = decode_timetable(v_in)
+            elif k_in == "weight_rule":
+                v = decode_priority_weight_strategy(v_in)
+            elif k_in in cls._decorated_fields:
+                v = cls.deserialize(v_in)
+            elif k_in == "params":
+                v = cls._deserialize_params_dict(v_in)
+            elif k_in == "tags":
+                v = set(v_in)
+            else:
+                v = v_in

Review Comment:
   It’s -0 so I’d not block this, but personally I’m not into this loop variable reuse rule in general.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Enable ruff PLW2101,PLW2901,PLW3301 rule [airflow]

Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl merged PR #57700:
URL: https://github.com/apache/airflow/pull/57700


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Enable ruff PLW2901 rule [airflow]

Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on code in PR #57700:
URL: https://github.com/apache/airflow/pull/57700#discussion_r2491043169


##########
pyproject.toml:
##########
@@ -609,6 +609,9 @@ extend-select = [
     "PLW1508", # Invalid type for environment variable default; expected str or None
     "PLW1510", # subprocess.run without explicit check argument
     "PLW1641", # Object does not implement __hash__ method
+    "PLW2101", # Threading lock directly created in with statement has no effect
+    "PLW2901", # Outer {outer_kind} variable {name} overwritten by inner {inner_kind} target
+    "PLW3301", # Nested {func} calls can be flattened

Review Comment:
   This enables more than just `PLW2901` should we update the PR description or is that a mistake ?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Enable ruff PLW2101,PLW2901,PLW3301 rule [airflow]

Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl commented on code in PR #57700:
URL: https://github.com/apache/airflow/pull/57700#discussion_r2500762272


##########
airflow-core/src/airflow/serialization/serialized_objects.py:
##########
@@ -2565,40 +2572,43 @@ def _deserialize_dag_internal(
 
         # Note: Context is passed explicitly through method parameters, no class attributes needed
 
-        for k, v in encoded_dag.items():
-            if k == "_downstream_task_ids":
-                v = set(v)
-            elif k == "tasks":
+        for k_in, v_in in encoded_dag.items():
+            v: Any
+            k = k_in
+            if k_in == "_downstream_task_ids":
+                v = set(v_in)
+            elif k_in == "tasks":
                 SerializedBaseOperator._load_operator_extra_links = cls._load_operator_extra_links
                 tasks = {}
-                for obj in v:
+                for obj in v_in:
                     if obj.get(Encoding.TYPE) == DAT.OP:
                         deser = SerializedBaseOperator.deserialize_operator(
                             obj[Encoding.VAR], client_defaults
                         )
                         tasks[deser.task_id] = deser
                 k = "task_dict"
                 v = tasks
-            elif k == "timezone":
-                v = cls._deserialize_timezone(v)
-            elif k == "dagrun_timeout":
-                v = cls._deserialize_timedelta(v)
-            elif k.endswith("_date"):
-                v = cls._deserialize_datetime(v)
-            elif k == "edge_info":
+            elif k_in == "timezone":
+                v = cls._deserialize_timezone(v_in)
+            elif k_in == "dagrun_timeout":
+                v = cls._deserialize_timedelta(v_in)
+            elif k_in.endswith("_date"):
+                v = cls._deserialize_datetime(v_in)
+            elif k_in == "edge_info":
                 # Value structure matches exactly
-                pass
-            elif k == "timetable":
-                v = decode_timetable(v)
-            elif k == "weight_rule":
-                v = decode_priority_weight_strategy(v)
-            elif k in cls._decorated_fields:
-                v = cls.deserialize(v)
-            elif k == "params":
-                v = cls._deserialize_params_dict(v)
-            elif k == "tags":
-                v = set(v)
-            # else use v as it is
+                v = v_in
+            elif k_in == "timetable":
+                v = decode_timetable(v_in)
+            elif k_in == "weight_rule":
+                v = decode_priority_weight_strategy(v_in)
+            elif k_in in cls._decorated_fields:
+                v = cls.deserialize(v_in)
+            elif k_in == "params":
+                v = cls._deserialize_params_dict(v_in)
+            elif k_in == "tags":
+                v = set(v_in)
+            else:
+                v = v_in

Review Comment:
   I reverted the usage in serialized_objects and made it minimal invasive. Hope this improves to have a 0.000001 instead of -0 :-D



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Enable ruff PLW2101,PLW2901,PLW3301 rule [airflow]

Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl commented on code in PR #57700:
URL: https://github.com/apache/airflow/pull/57700#discussion_r2506845464


##########
airflow-core/src/airflow/executors/executor_loader.py:
##########
@@ -74,11 +74,11 @@ def _get_executor_names(cls) -> list[ExecutorName]:
 
         all_executor_names: list[tuple[str | None, list[str]]] = cls._get_team_executor_configs()
 
-        executor_names = []
+        executor_names: list[ExecutorName] = []
         for team_name, executor_names_config in all_executor_names:
             executor_names_per_team = []
-            for name in executor_names_config:
-                if len(split_name := name.split(":")) == 1:
+            for executor_name_str in executor_names_config:

Review Comment:
   Attempted this before but then `executor_name` is re-used in line 129ff in the loop and generated a clash there. First I renamed below but then realized that the name below is a very good fit - so decided rather finding an alternative here.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org