You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/10/05 16:21:47 UTC
[GitHub] [airflow] jacobmhoffman opened a new pull request #11285: [AIRFLOW-11041] Create AirflowMacroPluginRemovedRule to ease upgrade to Airflow 2.0
jacobmhoffman opened a new pull request #11285:
URL: https://github.com/apache/airflow/pull/11285
Adds a rule to check that `airflow.AirflowMacroPlugin` is not referenced anywhere in the dag folder since it will be removed in Airflow 2.0.
closes: #11041
Example:
```
======================================================== STATUS =======================================================
Remove airflow.AirflowMacroPlugin class............................................................................FAIL
Found 1 problem.
=================================================== RECOMMENDATIONS ===================================================
Remove airflow.AirflowMacroPlugin class
---------------------------------------
The airflow.AirflowMacroPlugin class has been removed.
Problems:
1. airflow.AirflowMacroPlugin will be removed. Affected file: /home/user/airflow/dags/foo.py (line 1)
```
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jacobmhoffman commented on pull request #11285: [AIRFLOW-11041] Create AirflowMacroPluginRemovedRule to ease upgrade to Airflow 2.0
Posted by GitBox <gi...@apache.org>.
jacobmhoffman commented on pull request #11285:
URL: https://github.com/apache/airflow/pull/11285#issuecomment-705329443
> @jacobmhoffman please rebase the PR as we had to fix some side effects in tests: #11326
@turbaszek thanks for the update! Rebased on latest `v1-10-test`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jacobmhoffman commented on a change in pull request #11285: [AIRFLOW-11041] Create AirflowMacroPluginRemovedRule to ease upgrade to Airflow 2.0
Posted by GitBox <gi...@apache.org>.
jacobmhoffman commented on a change in pull request #11285:
URL: https://github.com/apache/airflow/pull/11285#discussion_r500019774
##########
File path: tests/upgrade/rules/test_airflow_macro_plugin_removed.py
##########
@@ -0,0 +1,73 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from unittest import TestCase
+
+from tempfile import NamedTemporaryFile
+from tests.compat import mock
+
+from airflow.upgrade.rules.airflow_macro_plugin_removed import AirflowMacroPluginRemovedRule
+
+
+@mock.patch("airflow.upgrade.rules.airflow_macro_plugin_removed.list_py_file_paths")
+class TestAirflowMacroPluginRemovedRule(TestCase):
+ def setUp(self):
+ self.temp_file = NamedTemporaryFile("w+")
+
+ def create_temp_file(self, mock_list_files, lines):
+ mock_list_files.return_value = [self.temp_file.name]
+ for line in lines:
+ self.temp_file.write(line)
+ self.temp_file.write("\n")
+ self.temp_file.flush()
Review comment:
Thanks for the suggestion!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #11285: [AIRFLOW-11041] Create AirflowMacroPluginRemovedRule to ease upgrade to Airflow 2.0
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #11285:
URL: https://github.com/apache/airflow/pull/11285#issuecomment-703742210
The CI and PROD Docker Images for the build are prepared in a separate "Build Image" workflow,
that you will not see in the list of checks (you will see "Wait for images" jobs instead).
You can checks the status of those images in [The workflow run](https://github.com/apache/airflow/actions/runs/289762900)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek commented on a change in pull request #11285: [AIRFLOW-11041] Create AirflowMacroPluginRemovedRule to ease upgrade to Airflow 2.0
Posted by GitBox <gi...@apache.org>.
turbaszek commented on a change in pull request #11285:
URL: https://github.com/apache/airflow/pull/11285#discussion_r500367473
##########
File path: tests/upgrade/rules/test_airflow_macro_plugin_removed.py
##########
@@ -0,0 +1,70 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from contextlib import contextmanager
+from unittest import TestCase
+
+from tempfile import NamedTemporaryFile
+from tests.compat import mock
+
+from airflow.upgrade.rules.airflow_macro_plugin_removed import AirflowMacroPluginRemovedRule
+
+
+@contextmanager
+def create_temp_file(mock_list_files, lines):
+ temp_file = NamedTemporaryFile("w+")
+ mock_list_files.return_value = [temp_file.name]
+ for line in lines:
+ temp_file.write(line)
+ temp_file.write("\n")
+ temp_file.flush()
+ yield temp_file
Review comment:
I'm afraid that the temp file is not deleted (but I'm not sure), how about:
```suggestion
with NamedTemporaryFile("w+") as temp.file:
mock_list_files.return_value = [temp_file.name]
temp_file.writelines(lines)
temp_file.flush()
yield temp_file
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jacobmhoffman commented on a change in pull request #11285: [AIRFLOW-11041] Create AirflowMacroPluginRemovedRule to ease upgrade to Airflow 2.0
Posted by GitBox <gi...@apache.org>.
jacobmhoffman commented on a change in pull request #11285:
URL: https://github.com/apache/airflow/pull/11285#discussion_r500705163
##########
File path: airflow/upgrade/rules/airflow_macro_plugin_removed.py
##########
@@ -0,0 +1,55 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from __future__ import absolute_import
+
+from airflow import conf
+from airflow.upgrade.rules.base_rule import BaseRule
+from airflow.utils.dag_processing import list_py_file_paths
+
+
+class AirflowMacroPluginRemovedRule(BaseRule):
+
+ title = "Remove airflow.AirflowMacroPlugin class"
+
+ description = """\
+The airflow.AirflowMacroPlugin class has been removed.\
+"""
Review comment:
yep much better :sweat_smile:
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek commented on pull request #11285: [AIRFLOW-11041] Create AirflowMacroPluginRemovedRule to ease upgrade to Airflow 2.0
Posted by GitBox <gi...@apache.org>.
turbaszek commented on pull request #11285:
URL: https://github.com/apache/airflow/pull/11285#issuecomment-705073687
@jacobmhoffman please rebase the PR as we had to fix some side effects in tests: #11326
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek commented on a change in pull request #11285: [AIRFLOW-11041] Create AirflowMacroPluginRemovedRule to ease upgrade to Airflow 2.0
Posted by GitBox <gi...@apache.org>.
turbaszek commented on a change in pull request #11285:
URL: https://github.com/apache/airflow/pull/11285#discussion_r499741673
##########
File path: airflow/upgrade/rules/airflow_macro_plugin_removed.py
##########
@@ -0,0 +1,55 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from __future__ import absolute_import
+
+from airflow import conf
+from airflow.upgrade.rules.base_rule import BaseRule
+from airflow.utils.dag_processing import list_py_file_paths
+
+
+class AirflowMacroPluginRemovedRule(BaseRule):
+
+ title = "Remove airflow.AirflowMacroPlugin class"
+
+ description = """\
+The airflow.AirflowMacroPlugin class has been removed.\
+"""
+
+ MACRO_PLUGIN_CLASS = "airflow.AirflowMacroPlugin"
+
+ def change_info(self, file_path, line_number):
Review comment:
```suggestion
def _change_info(self, file_path, line_number):
```
A nit but I think that we can make it private
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on pull request #11285: [AIRFLOW-11041] Create AirflowMacroPluginRemovedRule to ease upgrade to Airflow 2.0
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on pull request #11285:
URL: https://github.com/apache/airflow/pull/11285#issuecomment-703738906
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst)
Here are some useful points:
- Pay attention to the quality of your code (flake8, pylint and type annotations). Our [pre-commits]( https://github.com/apache/airflow/blob/master/STATIC_CODE_CHECKS.rst#prerequisites-for-pre-commit-hooks) will help you with that.
- In case of a new feature add useful documentation (in docstrings or in `docs/` directory). Adding a new operator? Check this short [guide](https://github.com/apache/airflow/blob/master/docs/howto/custom-operator.rst) Consider adding an example DAG that shows how users should use it.
- Consider using [Breeze environment](https://github.com/apache/airflow/blob/master/BREEZE.rst) for testing locally, itβs a heavy docker but it ships with a working Airflow and a lot of integrations.
- Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
- Please follow [ASF Code of Conduct](https://www.apache.org/foundation/policies/conduct) for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
- Be sure to read the [Airflow Coding style]( https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#coding-style-and-best-practices).
Apache Airflow is a community-driven project and together we are making it better π.
In case of doubts contact the developers at:
Mailing List: dev@airflow.apache.org
Slack: https://s.apache.org/airflow-slack
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek merged pull request #11285: [AIRFLOW-11041] Create AirflowMacroPluginRemovedRule to ease upgrade to Airflow 2.0
Posted by GitBox <gi...@apache.org>.
turbaszek merged pull request #11285:
URL: https://github.com/apache/airflow/pull/11285
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek commented on a change in pull request #11285: [AIRFLOW-11041] Create AirflowMacroPluginRemovedRule to ease upgrade to Airflow 2.0
Posted by GitBox <gi...@apache.org>.
turbaszek commented on a change in pull request #11285:
URL: https://github.com/apache/airflow/pull/11285#discussion_r499739588
##########
File path: airflow/upgrade/rules/airflow_macro_plugin_removed.py
##########
@@ -0,0 +1,55 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from __future__ import absolute_import
+
+from airflow import conf
+from airflow.upgrade.rules.base_rule import BaseRule
+from airflow.utils.dag_processing import list_py_file_paths
+
+
+class AirflowMacroPluginRemovedRule(BaseRule):
+
+ title = "Remove airflow.AirflowMacroPlugin class"
+
+ description = """\
+The airflow.AirflowMacroPlugin class has been removed.\
+"""
+
+ MACRO_PLUGIN_CLASS = "airflow.AirflowMacroPlugin"
+
+ def change_info(self, file_path, line_number):
+ return "{} will be removed. Affected file: {} (line {})".format(
+ self.MACRO_PLUGIN_CLASS, file_path, line_number
+ )
+
+ def _check_file(self, file_path):
+ problems = []
+ class_name_to_check = self.MACRO_PLUGIN_CLASS.split(".")[-1]
+ with open(file_path, "r") as f:
+ for line_number, line in enumerate(f):
+ if class_name_to_check in line:
+ problems.append(self.change_info(file_path, line_number + 1))
Review comment:
```suggestion
for line_number, line in enumerate(f, 1):
if class_name_to_check in line:
problems.append(self.change_info(file_path, line_number))
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on pull request #11285: [AIRFLOW-11041] Create AirflowMacroPluginRemovedRule to ease upgrade to Airflow 2.0
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on pull request #11285:
URL: https://github.com/apache/airflow/pull/11285#issuecomment-706530229
Awesome work, congrats on your first merged pull request!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek commented on a change in pull request #11285: [AIRFLOW-11041] Create AirflowMacroPluginRemovedRule to ease upgrade to Airflow 2.0
Posted by GitBox <gi...@apache.org>.
turbaszek commented on a change in pull request #11285:
URL: https://github.com/apache/airflow/pull/11285#discussion_r500364970
##########
File path: airflow/upgrade/rules/airflow_macro_plugin_removed.py
##########
@@ -0,0 +1,55 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from __future__ import absolute_import
+
+from airflow import conf
+from airflow.upgrade.rules.base_rule import BaseRule
+from airflow.utils.dag_processing import list_py_file_paths
+
+
+class AirflowMacroPluginRemovedRule(BaseRule):
+
+ title = "Remove airflow.AirflowMacroPlugin class"
+
+ description = """\
+The airflow.AirflowMacroPlugin class has been removed.\
+"""
Review comment:
```suggestion
description = "The airflow.AirflowMacroPlugin class has been removed."
```
WDYT? π
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jacobmhoffman edited a comment on pull request #11285: [AIRFLOW-11041] Create AirflowMacroPluginRemovedRule to ease upgrade to Airflow 2.0
Posted by GitBox <gi...@apache.org>.
jacobmhoffman edited a comment on pull request #11285:
URL: https://github.com/apache/airflow/pull/11285#issuecomment-706495442
what's the process for getting this merged? @turbaszek
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek commented on a change in pull request #11285: [AIRFLOW-11041] Create AirflowMacroPluginRemovedRule to ease upgrade to Airflow 2.0
Posted by GitBox <gi...@apache.org>.
turbaszek commented on a change in pull request #11285:
URL: https://github.com/apache/airflow/pull/11285#discussion_r499741937
##########
File path: airflow/upgrade/rules/airflow_macro_plugin_removed.py
##########
@@ -0,0 +1,55 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from __future__ import absolute_import
+
+from airflow import conf
+from airflow.upgrade.rules.base_rule import BaseRule
+from airflow.utils.dag_processing import list_py_file_paths
+
+
+class AirflowMacroPluginRemovedRule(BaseRule):
+
+ title = "Remove airflow.AirflowMacroPlugin class"
+
+ description = """\
+The airflow.AirflowMacroPlugin class has been removed.\
+"""
+
+ MACRO_PLUGIN_CLASS = "airflow.AirflowMacroPlugin"
+
+ def change_info(self, file_path, line_number):
+ return "{} will be removed. Affected file: {} (line {})".format(
+ self.MACRO_PLUGIN_CLASS, file_path, line_number
+ )
+
+ def _check_file(self, file_path):
+ problems = []
+ class_name_to_check = self.MACRO_PLUGIN_CLASS.split(".")[-1]
+ with open(file_path, "r") as f:
+ for line_number, line in enumerate(f):
+ if class_name_to_check in line:
+ problems.append(self.change_info(file_path, line_number + 1))
+ return problems
+
+ def check(self, session=None):
Review comment:
```suggestion
def check(self):
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jacobmhoffman commented on pull request #11285: [AIRFLOW-11041] Create AirflowMacroPluginRemovedRule to ease upgrade to Airflow 2.0
Posted by GitBox <gi...@apache.org>.
jacobmhoffman commented on pull request #11285:
URL: https://github.com/apache/airflow/pull/11285#issuecomment-705329443
> @jacobmhoffman please rebase the PR as we had to fix some side effects in tests: #11326
@turbaszek thanks for the update! Rebased on latest `v1-10-test`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek commented on a change in pull request #11285: [AIRFLOW-11041] Create AirflowMacroPluginRemovedRule to ease upgrade to Airflow 2.0
Posted by GitBox <gi...@apache.org>.
turbaszek commented on a change in pull request #11285:
URL: https://github.com/apache/airflow/pull/11285#discussion_r499744010
##########
File path: tests/upgrade/rules/test_airflow_macro_plugin_removed.py
##########
@@ -0,0 +1,73 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from unittest import TestCase
+
+from tempfile import NamedTemporaryFile
+from tests.compat import mock
+
+from airflow.upgrade.rules.airflow_macro_plugin_removed import AirflowMacroPluginRemovedRule
+
+
+@mock.patch("airflow.upgrade.rules.airflow_macro_plugin_removed.list_py_file_paths")
+class TestAirflowMacroPluginRemovedRule(TestCase):
+ def setUp(self):
+ self.temp_file = NamedTemporaryFile("w+")
+
+ def create_temp_file(self, mock_list_files, lines):
+ mock_list_files.return_value = [self.temp_file.name]
+ for line in lines:
+ self.temp_file.write(line)
+ self.temp_file.write("\n")
+ self.temp_file.flush()
Review comment:
Please create here the temporary file and close it. Otherwise, there's is side effect between tests. You may consider creating context manger (using `@contextmanager` decorator) that with handle it and then you can do
```python
with create_temp_file(lines):
# do something here
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] turbaszek commented on a change in pull request #11285: [AIRFLOW-11041] Create AirflowMacroPluginRemovedRule to ease upgrade to Airflow 2.0
Posted by GitBox <gi...@apache.org>.
turbaszek commented on a change in pull request #11285:
URL: https://github.com/apache/airflow/pull/11285#discussion_r499740657
##########
File path: airflow/upgrade/rules/airflow_macro_plugin_removed.py
##########
@@ -0,0 +1,55 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from __future__ import absolute_import
+
+from airflow import conf
+from airflow.upgrade.rules.base_rule import BaseRule
+from airflow.utils.dag_processing import list_py_file_paths
+
+
+class AirflowMacroPluginRemovedRule(BaseRule):
+
+ title = "Remove airflow.AirflowMacroPlugin class"
+
+ description = """\
+The airflow.AirflowMacroPlugin class has been removed.\
+"""
+
+ MACRO_PLUGIN_CLASS = "airflow.AirflowMacroPlugin"
+
+ def change_info(self, file_path, line_number):
+ return "{} will be removed. Affected file: {} (line {})".format(
+ self.MACRO_PLUGIN_CLASS, file_path, line_number
+ )
+
+ def _check_file(self, file_path):
+ problems = []
+ class_name_to_check = self.MACRO_PLUGIN_CLASS.split(".")[-1]
+ with open(file_path, "r") as f:
Review comment:
```suggestion
with open(file_path, "r") as f:
```
Let's use something more informative than `f`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #11285: [AIRFLOW-11041] Create AirflowMacroPluginRemovedRule to ease upgrade to Airflow 2.0
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #11285:
URL: https://github.com/apache/airflow/pull/11285#issuecomment-706517029
There were some intermittent failures I restarted it
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jacobmhoffman commented on pull request #11285: [AIRFLOW-11041] Create AirflowMacroPluginRemovedRule to ease upgrade to Airflow 2.0
Posted by GitBox <gi...@apache.org>.
jacobmhoffman commented on pull request #11285:
URL: https://github.com/apache/airflow/pull/11285#issuecomment-706495442
what's the process for getting this merged?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org