You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/01/21 06:14:18 UTC
[GitHub] [airflow] uranusjr opened a new pull request #21003: Squelch more deprecation warnings
uranusjr opened a new pull request #21003:
URL: https://github.com/apache/airflow/pull/21003
Should be mostly straightforward.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on a change in pull request #21003: Squelch more deprecation warnings
Posted by GitBox <gi...@apache.org>.
eladkal commented on a change in pull request #21003:
URL: https://github.com/apache/airflow/pull/21003#discussion_r789677056
##########
File path: airflow/providers/google/cloud/transfers/s3_to_gcs.py
##########
@@ -21,9 +21,13 @@
from airflow.exceptions import AirflowException
from airflow.providers.amazon.aws.hooks.s3 import S3Hook
-from airflow.providers.amazon.aws.operators.s3_list import S3ListOperator
from airflow.providers.google.cloud.hooks.gcs import GCSHook, _parse_gcs_url, gcs_object_is_directory
+try:
+ from airflow.providers.amazon.aws.operators.s3 import S3ListOperator
+except ImportError:
+ from airflow.providers.amazon.aws.operators.s3_list import S3ListOperator
Review comment:
It is more user friendly. I just mentioned that this is the first time we do that...
On the other hand we have so many deprecated classes/functions/params right now supporting all of them is not that easy. Maybe we should consider some cleanup. Google provider for example had several major releases without removing deprecated code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #21003: Squelch more deprecation warnings
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #21003:
URL: https://github.com/apache/airflow/pull/21003#discussion_r789676342
##########
File path: airflow/providers/google/cloud/transfers/s3_to_gcs.py
##########
@@ -21,9 +21,13 @@
from airflow.exceptions import AirflowException
from airflow.providers.amazon.aws.hooks.s3 import S3Hook
-from airflow.providers.amazon.aws.operators.s3_list import S3ListOperator
from airflow.providers.google.cloud.hooks.gcs import GCSHook, _parse_gcs_url, gcs_object_is_directory
+try:
+ from airflow.providers.amazon.aws.operators.s3 import S3ListOperator
+except ImportError:
+ from airflow.providers.amazon.aws.operators.s3_list import S3ListOperator
Review comment:
Actually we have the technicall capability and we already used it (min dependencies between sftp and ssh I believe) but in this case I agree breaking cross-provider compatibility is not a good idea.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on a change in pull request #21003: Squelch more deprecation warnings
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #21003:
URL: https://github.com/apache/airflow/pull/21003#discussion_r789672917
##########
File path: airflow/providers/google/cloud/transfers/s3_to_gcs.py
##########
@@ -21,9 +21,13 @@
from airflow.exceptions import AirflowException
from airflow.providers.amazon.aws.hooks.s3 import S3Hook
-from airflow.providers.amazon.aws.operators.s3_list import S3ListOperator
from airflow.providers.google.cloud.hooks.gcs import GCSHook, _parse_gcs_url, gcs_object_is_directory
+try:
+ from airflow.providers.amazon.aws.operators.s3 import S3ListOperator
+except ImportError:
+ from airflow.providers.amazon.aws.operators.s3_list import S3ListOperator
Review comment:
This will be problematic as we do not yet have the technical ability to enforce a specific provider version. We would have to add some code to add this ability. I think the current solution is sufficient and even more end-user friendly. It is also in line with how we approach backward compatibility with the airflow core.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on a change in pull request #21003: Squelch more deprecation warnings
Posted by GitBox <gi...@apache.org>.
eladkal commented on a change in pull request #21003:
URL: https://github.com/apache/airflow/pull/21003#discussion_r789495471
##########
File path: airflow/providers/google/cloud/transfers/s3_to_gcs.py
##########
@@ -21,9 +21,13 @@
from airflow.exceptions import AirflowException
from airflow.providers.amazon.aws.hooks.s3 import S3Hook
-from airflow.providers.amazon.aws.operators.s3_list import S3ListOperator
from airflow.providers.google.cloud.hooks.gcs import GCSHook, _parse_gcs_url, gcs_object_is_directory
+try:
+ from airflow.providers.amazon.aws.operators.s3 import S3ListOperator
+except ImportError:
+ from airflow.providers.amazon.aws.operators.s3_list import S3ListOperator
Review comment:
We dont do that in other operators.
We always import from most updated path
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr merged pull request #21003: Squelch more deprecation warnings
Posted by GitBox <gi...@apache.org>.
uranusjr merged pull request #21003:
URL: https://github.com/apache/airflow/pull/21003
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #21003: Squelch more deprecation warnings
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #21003:
URL: https://github.com/apache/airflow/pull/21003#issuecomment-1018337320
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #21003: Squelch more deprecation warnings
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #21003:
URL: https://github.com/apache/airflow/pull/21003#discussion_r789676342
##########
File path: airflow/providers/google/cloud/transfers/s3_to_gcs.py
##########
@@ -21,9 +21,13 @@
from airflow.exceptions import AirflowException
from airflow.providers.amazon.aws.hooks.s3 import S3Hook
-from airflow.providers.amazon.aws.operators.s3_list import S3ListOperator
from airflow.providers.google.cloud.hooks.gcs import GCSHook, _parse_gcs_url, gcs_object_is_directory
+try:
+ from airflow.providers.amazon.aws.operators.s3 import S3ListOperator
+except ImportError:
+ from airflow.providers.amazon.aws.operators.s3_list import S3ListOperator
Review comment:
Actually we have the technicall capability and we already used it (min dependencies between sftp and ssh I believe) but in this case I agree breaking cross-compatibility is not a good idea.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on a change in pull request #21003: Squelch more deprecation warnings
Posted by GitBox <gi...@apache.org>.
eladkal commented on a change in pull request #21003:
URL: https://github.com/apache/airflow/pull/21003#discussion_r789677056
##########
File path: airflow/providers/google/cloud/transfers/s3_to_gcs.py
##########
@@ -21,9 +21,13 @@
from airflow.exceptions import AirflowException
from airflow.providers.amazon.aws.hooks.s3 import S3Hook
-from airflow.providers.amazon.aws.operators.s3_list import S3ListOperator
from airflow.providers.google.cloud.hooks.gcs import GCSHook, _parse_gcs_url, gcs_object_is_directory
+try:
+ from airflow.providers.amazon.aws.operators.s3 import S3ListOperator
+except ImportError:
+ from airflow.providers.amazon.aws.operators.s3_list import S3ListOperator
Review comment:
It is more user friendly I'm very OK with it!
I just mentioned that this is the first time we do that...
On the other hand we have so many deprecated classes/functions/params right now supporting all of them is not that easy. Maybe we should consider some cleanup. Google provider for example had several major releases without removing deprecated code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org