You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/09/14 00:40:47 UTC
[GitHub] [airflow] mik-laj opened a new issue, #26378: Introduce flake8-simplify to simplify your code.
mik-laj opened a new issue, #26378:
URL: https://github.com/apache/airflow/issues/26378
### Description
We should consider introducing [flake8-simplify](https://github.com/MartinThoma/flake8-simplify) to enforce some best programming practice on CI.
This is the error list that detects this tool for the `scripts/` directory:
```
scripts/ci/pre_commit/pre_commit_chart_schema.py:65:5: SIM110 Use 'return any(path.startswith(prefix) for prefix in VENDORED_PATHS)'
scripts/ci/pre_commit/pre_commit_check_2_2_compatibility.py:49:9: SIM102 Use a single if-statement instead of nested if-statements
scripts/ci/pre_commit/pre_commit_decorator_operator_implements_custom_name.py:44:9: SIM102 Use a single if-statement instead of nested if-statements
scripts/ci/pre_commit/pre_commit_vendor_k8s_json_schema.py:41:9: SIM102 Use a single if-statement instead of nested if-statements
scripts/ci/pre_commit/pre_commit_version_heads_map.py:32:44: SIM115 Use context handler for opening files
scripts/ci/pre_commit/pre_commit_version_heads_map.py:47:25: SIM115 Use context handler for opening files
scripts/in_container/verify_providers.py:272:5: SIM111 Use 'return all(not (msg_string == m and warn.filename.find('/importlib/') != -1) for m in KNOWN_DEPRECATED_DIRECT_IMPORTS)'
scripts/in_container/verify_providers.py:280:5: SIM111 Use 'return all(not msg_string == m for m in KNOWN_COMMON_DEPRECATED_MESSAGES)'
```
This is the error list for the `airflow` package excluding `airflow.providers` package.
```
airflow/api/__init__.py:31:5: SIM105 Use 'contextlib.suppress(AirflowConfigException)'
airflow/api_connexion/endpoints/log_endpoint.py:93:9: SIM105 Use 'contextlib.suppress(TaskNotFound)'
airflow/api_connexion/endpoints/pool_endpoint.py:91:5: SIM105 Use 'contextlib.suppress(KeyError)'
airflow/api_connexion/endpoints/role_and_permission_endpoint.py:129:34: SIM201 Use 'field != "permissions"' instead of 'not field == "permissions"'
airflow/api_connexion/endpoints/user_endpoint.py:142:5: SIM102 Use a single if-statement instead of nested if-statements
airflow/api_connexion/endpoints/user_endpoint.py:148:5: SIM102 Use a single if-statement instead of nested if-statements
airflow/api_connexion/schemas/task_instance_schema.py:133:9: SIM102 Use a single if-statement instead of nested if-statements
airflow/callbacks/pipe_callback_sink.py:42:9: SIM105 Use 'contextlib.suppress(ConnectionError)'
airflow/cli/commands/connection_command.py:178:38: SIM201 Use 'filetype != ".env"' instead of 'not filetype == ".env"'
airflow/cli/commands/connection_command.py:215:21: SIM203 Use 'args.conn_type not in _get_connection_types()' instead of 'not args.conn_type in _get_connection_types()'
airflow/cli/commands/scheduler_command.py:93:5: SIM102 Use a single if-statement instead of nested if-statements
airflow/cli/commands/webserver_command.py:164:9: SIM113 Use enumerate for 'excess'
airflow/config_templates/default_celery.py:35:1: SIM102 Use a single if-statement instead of nested if-statements
airflow/configuration.py:82:29: SIM907 Use 'Optional[str]' instead of 'Union[str, None]'
airflow/configuration.py:82:59: SIM907 Use 'Optional[str]' instead of 'Union[str, None]'
airflow/configuration.py:458:9: SIM102 Use a single if-statement instead of nested if-statements
airflow/configuration.py:464:9: SIM102 Use a single if-statement instead of nested if-statements
airflow/configuration.py:472:9: SIM102 Use a single if-statement instead of nested if-statements
airflow/configuration.py:497:9: SIM102 Use a single if-statement instead of nested if-statements
airflow/configuration.py:612:9: SIM102 Use a single if-statement instead of nested if-statements
airflow/configuration.py:904:9: SIM102 Use a single if-statement instead of nested if-statements
airflow/configuration.py:1192:13: SIM102 Use a single if-statement instead of nested if-statements
airflow/dag_processing/manager.py:176:9: SIM105 Use 'contextlib.suppress(ConnectionError)'
airflow/dag_processing/manager.py:329:13: SIM105 Use 'contextlib.suppress(ConnectionError)'
airflow/dag_processing/manager.py:1000:16: SIM118 Use 'file_path in self._processors' instead of 'file_path in self._processors.keys()'
airflow/decorators/base.py:88:8: SIM118 Use '"self" in inspect.signature(python_callable).parameters' instead of '"self" in inspect.signature(python_callable).parameters.keys()'
airflow/decorators/base.py:267:16: SIM109 Use 'ttype in (dict, Dict)' instead of 'ttype == dict or ttype == Dict'
airflow/executors/celery_executor.py:202:5: SIM105 Use 'contextlib.suppress(ImportError)'
airflow/executors/celery_executor.py:207:5: SIM105 Use 'contextlib.suppress(ImportError)'
airflow/jobs/backfill_job.py:860:17: SIM102 Use a single if-statement instead of nested if-statements
airflow/jobs/local_task_job.py:165:9: SIM102 Use a single if-statement instead of nested if-statements
airflow/kubernetes/kube_config.py:84:9: SIM102 Use a single if-statement instead of nested if-statements
airflow/migrations/env.py:30:5: SIM103 Return the condition type_ == 'table' and (name.startswith('celery_') or name == 'session') directly
airflow/migrations/versions/0069_2_0_0_add_scheduling_decision_to_dagrun_and_.py:43:17: SIM901 Use 'conn.dialect.name == 'sqlite'' instead of 'bool(conn.dialect.name == 'sqlite')'
airflow/migrations/versions/0069_2_0_0_add_scheduling_decision_to_dagrun_and_.py:44:16: SIM901 Use 'conn.dialect.name == 'mssql'' instead of 'bool(conn.dialect.name == 'mssql')'
airflow/migrations/versions/0069_2_0_0_add_scheduling_decision_to_dagrun_and_.py:93:17: SIM901 Use 'conn.dialect.name == 'sqlite'' instead of 'bool(conn.dialect.name == 'sqlite')'
airflow/migrations/versions/0088_2_2_0_improve_mssql_compatibility.py:172:9: SIM102 Use a single if-statement instead of nested if-statements
airflow/migrations/versions/0090_2_2_0_rename_concurrency_column_in_dag_table_.py:41:17: SIM901 Use 'conn.dialect.name == 'sqlite'' instead of 'bool(conn.dialect.name == 'sqlite')'
airflow/migrations/versions/0098_2_3_0_added_timetable_description_column.py:46:17: SIM901 Use 'op.get_bind().dialect.name == 'sqlite'' instead of 'bool(op.get_bind().dialect.name == 'sqlite')'
airflow/models/baseoperator.py:934:16: SIM201 Use 'self != other' instead of 'not self == other'
airflow/models/baseoperator.py:1054:14: SIM114 Use logical or ((self.task_id not in dag.task_dict) or (self.task_id in dag.task_dict and dag.task_dict[self.task_id] is not self)) and a single body
airflow/models/baseoperator.py:1054:14: SIM114 Use logical or ((self.task_id not in dag.task_dict) or (self.task_id in dag.task_dict and dag.task_dict[self.task_id] is not self)) and a single body
airflow/models/connection.py:203:13: SIM401 Use 'self.extra = query.get(self.EXTRA_KEY, json.dumps(query))' instead of an if-block
airflow/models/dag.py:459:9: SIM102 Use a single if-statement instead of nested if-statements
airflow/models/dag.py:655:16: SIM201 Use 'self != other' instead of 'not self == other'
airflow/models/dag.py:2242:9: SIM904 Initialize dictionary 'd' directly
airflow/models/dagbag.py:643:13: SIM110 Use 'return any(not session.query(Permission).join(Action).join(Resource).filter(Action.
name == permission_name).filter(Resource.name == dag_resource_name).
one_or_none() for permission_name in DAG_ACTIONS)'
airflow/models/dagrun.py:1007:41: SIM907 Use 'Optional[Sequence[int]]' instead of 'Union[Sequence[int], None]'
airflow/models/serialized_dag.py:136:9: SIM102 Use a single if-statement instead of nested if-statements
airflow/models/xcom.py:688:8: SIM201 Use 'set(base_xcom_params) != set(xcom_params)' instead of 'not set(base_xcom_params) == set(xcom_params)'
airflow/sensors/base.py:141:9: SIM102 Use a single if-statement instead of nested if-statements
airflow/sensors/bash.py:58:9: SIM117 Use 'with TemporaryDirectory(prefix='airflowtmp') as tmp_dir, NamedTemporaryFile(dir=tmp_dir, prefix=self.task_id) as f:' instead of multiple with statements
airflow/serialization/serialized_objects.py:825:18: SIM114 Use logical or ((k in cls._decorated_fields or k not in op.get_serialized_fields()) or (k in ('outlets', 'inlets'))) and a single body
airflow/serialization/serialized_objects.py:825:18: SIM114 Use logical or ((k in cls._decorated_fields or k not in op.get_serialized_fields()) or (k in ('outlets', 'inlets'))) and a single body
airflow/serialization/serialized_objects.py:825:18: SIM114 Use logical or ((k in cls._decorated_fields or k not in op.get_serialized_fields()) or (k in ('outlets', 'inlets'))) and a single body
airflow/serialization/serialized_objects.py:825:18: SIM114 Use logical or ((k in cls._decorated_fields or k not in op.get_serialized_fields()) or (k in ('outlets', 'inlets'))) and a single body
airflow/serialization/serialized_objects.py:825:18: SIM114 Use logical or ((k in cls._decorated_fields or k not in op.get_serialized_fields()) or (k in ('outlets', 'inlets'))) and a single body
airflow/serialization/serialized_objects.py:825:18: SIM114 Use logical or ((k in cls._decorated_fields or k not in op.get_serialized_fields()) or (k in ('outlets', 'inlets'))) and a single body
airflow/serialization/serialized_objects.py:825:18: SIM114 Use logical or ((k in cls._decorated_fields or k not in op.get_serialized_fields()) or (k in ('outlets', 'inlets'))) and a single body
airflow/serialization/serialized_objects.py:825:18: SIM114 Use logical or ((k in cls._decorated_fields or k not in op.get_serialized_fields()) or (k in ('outlets', 'inlets'))) and a single body
airflow/serialization/serialized_objects.py:825:18: SIM114 Use logical or ((k in cls._decorated_fields or k not in op.get_serialized_fields()) or (k in ('outlets', 'inlets'))) and a single body
airflow/serialization/serialized_objects.py:825:18: SIM114 Use logical or ((k in cls._decorated_fields or k not in op.get_serialized_fields()) or (k in ('outlets', 'inlets'))) and a single body
airflow/serialization/serialized_objects.py:825:18: SIM114 Use logical or ((k in cls._decorated_fields or k not in op.get_serialized_fields()) or (k in ('outlets', 'inlets'))) and a single body
airflow/serialization/serialized_objects.py:825:18: SIM114 Use logical or ((k in cls._decorated_fields or k not in op.get_serialized_fields()) or (k in ('outlets', 'inlets'))) and a single body
airflow/serialization/serialized_objects.py:825:18: SIM114 Use logical or ((k in cls._decorated_fields or k not in op.get_serialized_fields()) or (k in ('outlets', 'inlets'))) and a single body
airflow/serialization/serialized_objects.py:1293:12: SIM201 Use 'self.dependency_type != "dataset"' instead of 'not self.dependency_type == "dataset"'
airflow/settings.py:87:1: SIM909 Remove reflexive assignment 'json = json'
airflow/ti_deps/deps/not_previously_skipped_dep.py:60:21: SIM114 Use logical or ((XCOM_SKIPMIXIN_FOLLOWED in prev_result and ti.task_id not in prev_result[
XCOM_SKIPMIXIN_FOLLOWED]) or (XCOM_SKIPMIXIN_SKIPPED in prev_result and ti.task_id in prev_result[
XCOM_SKIPMIXIN_SKIPPED])) and a single body
airflow/ti_deps/deps/trigger_rule_dep.py:188:13: SIM102 Use a single if-statement instead of nested if-statements
airflow/ti_deps/deps/trigger_rule_dep.py:244:14: SIM114 Use logical or ((trigger_rule == TR.NONE_FAILED) or (trigger_rule == TR.NONE_FAILED_MIN_ONE_SUCCESS)) and a single body
airflow/ti_deps/deps/trigger_rule_dep.py:244:14: SIM114 Use logical or ((trigger_rule == TR.NONE_FAILED) or (trigger_rule == TR.NONE_FAILED_MIN_ONE_SUCCESS)) and a single body
airflow/ti_deps/deps/trigger_rule_dep.py:244:14: SIM114 Use logical or ((trigger_rule == TR.NONE_FAILED) or (trigger_rule == TR.NONE_FAILED_MIN_ONE_SUCCESS)) and a single body
airflow/ti_deps/deps/trigger_rule_dep.py:244:14: SIM114 Use logical or ((trigger_rule == TR.NONE_FAILED) or (trigger_rule == TR.NONE_FAILED_MIN_ONE_SUCCESS)) and a single body
airflow/ti_deps/deps/trigger_rule_dep.py:244:14: SIM114 Use logical or ((trigger_rule == TR.NONE_FAILED) or (trigger_rule == TR.NONE_FAILED_MIN_ONE_SUCCESS)) and a single body
airflow/ti_deps/deps/trigger_rule_dep.py:244:14: SIM114 Use logical or ((trigger_rule == TR.NONE_FAILED) or (trigger_rule == TR.NONE_FAILED_MIN_ONE_SUCCESS)) and a single body
airflow/utils/code_utils.py:41:9: SIM105 Use 'contextlib.suppress(TypeError)'
airflow/utils/code_utils.py:47:9: SIM105 Use 'contextlib.suppress(TypeError, AttributeError)'
airflow/utils/dates.py:98:10: SIM114 Use logical or ((isinstance(delta, timedelta)) or (isinstance(delta, relativedelta))) and a single body
airflow/utils/dates.py:98:10: SIM114 Use logical or ((isinstance(delta, timedelta)) or (isinstance(delta, relativedelta))) and a single body
airflow/utils/dot_renderer.py:120:8: SIM101 Multiple isinstance-calls which can be merged into a single call for variable 'node'
airflow/utils/file.py:196:16: SIM115 Use context handler for opening files
airflow/utils/log/file_processor_handler.py:144:13: SIM115 Use context handler for opening files
airflow/utils/log/file_task_handler.py:173:21: SIM102 Use a single if-statement instead of nested if-statements
airflow/utils/log/file_task_handler.py:203:17: SIM105 Use 'contextlib.suppress(AirflowConfigException, ValueError)'
airflow/utils/log/file_task_handler.py:311:13: SIM115 Use context handler for opening files
airflow/utils/log/secrets_masker.py:148:9: SIM105 Use 'contextlib.suppress(AttributeError)'
airflow/utils/process_utils.py:120:13: SIM105 Use 'contextlib.suppress(OSError)'
airflow/www/api/experimental/endpoints.py:87:5: SIM908 Use 'data.get("run_id")' instead of 'if "run_id" in data: data["run_id"]'
airflow/www/api/experimental/endpoints.py:225:17: SIM901 Use 'paused == 'true'' instead of 'bool(paused == 'true')'
airflow/www/extensions/init_appbuilder.py:613:9: SIM110 Use 'return any(baseview.__class__ == view.__class__ for baseview in self.baseviews)'
airflow/www/extensions/init_security.py:47:5: SIM105 Use 'contextlib.suppress(AirflowConfigException)'
airflow/www/fab_security/manager.py:540:16: SIM201 Use 'type(ret) != dict' instead of 'not type(ret) == dict'
airflow/www/fab_security/manager.py:1341:9: SIM110 Use 'return any(re.match(_resource_name, resource_name) and re.match(_action_name,
action_name) for _resource_name, _action_name in perms)'
airflow/www/fab_security/sqla/manager.py:549:9: SIM110 Use 'return any(perm.action and perm.action.name == action_name for perm in perms)'
airflow/www/security.py:455:9: SIM113 Use enumerate for 'deleted_count'
airflow/www/views.py:3971:14: SIM101 Multiple isinstance-calls which can be merged into a single call for variable 'items'
airflow/www/views.py:4917:13: SIM113 Use enumerate for 'count'
airflow/www/views.py:4942:13: SIM113 Use enumerate for 'count'
airflow/www/views.py:4970:13: SIM113 Use enumerate for 'count'
airflow/www/views.py:4993:13: SIM113 Use enumerate for 'count'
```
### Use case/motivation
_No response_
### Related issues
_No response_
### Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on issue #26378: Introduce flake8-simplify to simplify our code.
Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on issue #26378:
URL: https://github.com/apache/airflow/issues/26378#issuecomment-1419239789
We removed flake8 when switched to ruff https://github.com/apache/airflow/pull/28893
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] pierrejeambrun commented on issue #26378: Introduce flake8-simplify to simplify our code.
Posted by "pierrejeambrun (via GitHub)" <gi...@apache.org>.
pierrejeambrun commented on issue #26378:
URL: https://github.com/apache/airflow/issues/26378#issuecomment-1419579965
Yep, I think a ruff implementation of `flake8-simplify` might just be around the corner, this might still be relevant.
https://github.com/charliermarsh/ruff/issues/998
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #26378: Introduce flake8-simplify to simplify our code.
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #26378:
URL: https://github.com/apache/airflow/issues/26378#issuecomment-1250404980
I like it, but I think this one (like most of the tooling updates) needs devlist proposal and likely lasy consensus there (and then introduction on pre-commit level). Maybe you would like to follow that through @mik-laj ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org