You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/09/11 16:49:39 UTC
[GitHub] [airflow] eladkal opened a new pull request #18168: Rename LocalToAzureDataLakeStorageOperator to LocalFilesystemToADLSOperator
eladkal opened a new pull request #18168:
URL: https://github.com/apache/airflow/pull/18168
Renaming for consistency:.
Local -> LocalFilesystem based on: `LocalFilesystemToS3Operator`, `LocalFilesystemToGCSOperator`
shorting AzureDataLakeStorage to ADLS based on: `ADLSToGCSOperator`.
Noting that we have other operators that use the full service name like: `AzureDataLakeStorageDeleteOperator`, `AzureDataLakeStorageListOperator` but I think the ADLS is preferred just like we use GCS rather than GoogleCloudStorage - This falls within [case 6 Isolated cases of AIP-21](https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths#AIP21:Changesinimportpaths-Case#6Otherisolatedcases) I will create PRs to handle them as well if we go with the shorter ADLS version.
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/main/UPDATING.md).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #18168: Rename LocalToAzureDataLakeStorageOperator to LocalFilesystemToADLSOperator
Posted by GitBox <gi...@apache.org>.
potiuk merged pull request #18168:
URL: https://github.com/apache/airflow/pull/18168
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on a change in pull request #18168: Rename LocalToAzureDataLakeStorageOperator to LocalFilesystemToADLSOperator
Posted by GitBox <gi...@apache.org>.
eladkal commented on a change in pull request #18168:
URL: https://github.com/apache/airflow/pull/18168#discussion_r706773824
##########
File path: airflow/providers/microsoft/azure/transfers/local_to_adls.py
##########
@@ -100,3 +100,20 @@ def execute(self, context: dict) -> None:
blocksize=self.blocksize,
**self.extra_upload_options,
)
+
+
+class LocalToAzureDataLakeStorageOperator(LocalFilesystemToADLSOperator):
Review comment:
To my knowledge this file is used to test the classes migrated from contrib to providers. It's being used in
https://github.com/apache/airflow/blob/8505d2f0a4524313e3eff7a4f16b9a9439c7a79f/tests/core/test_core_to_contrib.py#L29
The `LocalFilesystemToADLSOperator` wasn't part of contrib. It was added directly to providers in https://github.com/apache/airflow/pull/10814
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on a change in pull request #18168: Rename LocalToAzureDataLakeStorageOperator to LocalFilesystemToADLSOperator
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #18168:
URL: https://github.com/apache/airflow/pull/18168#discussion_r706663960
##########
File path: airflow/providers/microsoft/azure/transfers/local_to_adls.py
##########
@@ -100,3 +100,20 @@ def execute(self, context: dict) -> None:
blocksize=self.blocksize,
**self.extra_upload_options,
)
+
+
+class LocalToAzureDataLakeStorageOperator(LocalFilesystemToADLSOperator):
Review comment:
Can you add this class to file: https://github.com/apache/airflow/blob/main/tests/deprecated_classes.py?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #18168: Rename LocalToAzureDataLakeStorageOperator to LocalFilesystemToADLSOperator
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #18168:
URL: https://github.com/apache/airflow/pull/18168#issuecomment-917712057
The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the core of Airflow. If the committers decide that the full tests matrix is needed, they will add the label 'full tests needed'. Then you should rebase to the latest main or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org