You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/07/13 09:08:07 UTC
[GitHub] [airflow] pyerbiz opened a new pull request #16964: [AIRFLOW 16730] Added docs & doc ref's for AWS transfer operators between SFTP & S3
pyerbiz opened a new pull request #16964:
URL: https://github.com/apache/airflow/pull/16964
…tp #16730 (#1)
* created files
* added s3tosftp and examples
* vault var removed
* updated s3_to_sftp example and doc
* fixed directives
* ref error fixed
closes: #16730
related:
-->
Added docs and example dags for SFTPToS3Operator and S3ToSFTPOperator. Added ref in s3_to_sftp.py and sftp_to_s3.py
---
**^ Add meaningful description above**
-
-->
Test:
./breeze build-docs -- --package-filter apache-airflow-providers-amazon
---
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #16964: [AIRFLOW 16730] Added docs & doc ref's for AWS transfer operators between SFTP & S3
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #16964:
URL: https://github.com/apache/airflow/pull/16964#issuecomment-879610914
Can you please @pyerbiz :
1) rebase to latest main (there were an error in main fixed in #16971
2) fix static checks (the easiest way is to install pre-commit as suggested in the error message - it will then auto-correct everything for you.
Having `pre-commit` installed for airflow contribution is HEARTILY recommended . It saves a lot of time on iterations and a lot of CI build time if you run the checks locally :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] pyerbiz commented on pull request #16964: [AIRFLOW 16730] Added docs & doc ref's for AWS transfer operators between SFTP & S3
Posted by GitBox <gi...@apache.org>.
pyerbiz commented on pull request #16964:
URL: https://github.com/apache/airflow/pull/16964#issuecomment-879813662
> Can you please @pyerbiz :
>
> 1. rebase to latest main (there were an error in main fixed in [Fixes typo in the name of file for Breeze docker compose env #16971](https://github.com/apache/airflow/pull/16971)
-- rebased local branch
> 2. fix static checks (the easiest way is to install pre-commit as suggested in the error message - it will then auto-correct everything for you.
-- fixed static check
> Having `pre-commit` installed for airflow contribution is HEARTILY recommended . It saves a lot of time on iterations and a lot of CI build time if you run the checks locally :)
-- installed and passed pre-commit static check all files. Ran through breeze.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] pyerbiz commented on pull request #16964: [AIRFLOW 16730] Added docs & doc ref's for AWS transfer operators between SFTP & S3
Posted by GitBox <gi...@apache.org>.
pyerbiz commented on pull request #16964:
URL: https://github.com/apache/airflow/pull/16964#issuecomment-879660944
Converting to draft to prevent mistakenly triggering CI/CD
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #16964: [AIRFLOW 16730] Added docs & doc ref's for AWS transfer operators between SFTP & S3
Posted by GitBox <gi...@apache.org>.
potiuk merged pull request #16964:
URL: https://github.com/apache/airflow/pull/16964
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #16964: [AIRFLOW 16730] Added docs & doc ref's for AWS transfer operators between SFTP & S3
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #16964:
URL: https://github.com/apache/airflow/pull/16964#issuecomment-880127339
```
Validate providers.yaml files............................................................Failed
- hook id: provider-yamls
- exit code: 1
Checking integration duplicates
Checking completeness of list of {sensors, hooks, operators}
Checking for duplicates in list of {sensors, hooks, operators}
Checking completeness of list of transfers
Checking for duplicates in list of transfers
Checking connection classes belong to package
Checking doc files
```
Above is cool ^^
Below does not tell too much vv
```
Items in the first set but not the second:
'/docs/apache-airflow-providers-amazon/operators/transfer/s3_to_sftp.rst'
'/docs/apache-airflow-providers-amazon/operators/transfer/sftp_to_s3.rst'
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on pull request #16964: [AIRFLOW 16730] Added docs & doc ref's for AWS transfer operators between SFTP & S3
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on pull request #16964:
URL: https://github.com/apache/airflow/pull/16964#issuecomment-882090279
Awesome work, congrats on your first merged pull request!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #16964: [AIRFLOW 16730] Added docs & doc ref's for AWS transfer operators between SFTP & S3
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #16964:
URL: https://github.com/apache/airflow/pull/16964#issuecomment-882090211
Wooho! all good. Merging.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #16964: [AIRFLOW 16730] Added docs & doc ref's for AWS transfer operators between SFTP & S3
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #16964:
URL: https://github.com/apache/airflow/pull/16964#issuecomment-879604168
The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the core of Airflow. If the committers decide that the full tests matrix is needed, they will add the label 'full tests needed'. Then you should rebase to the latest main or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] pyerbiz edited a comment on pull request #16964: [AIRFLOW 16730] Added docs & doc ref's for AWS transfer operators between SFTP & S3
Posted by GitBox <gi...@apache.org>.
pyerbiz edited a comment on pull request #16964:
URL: https://github.com/apache/airflow/pull/16964#issuecomment-879813662
@potiuk
> 1. rebase to latest main (there were an error in main fixed in [Fixes typo in the name of file for Breeze docker compose env #16971](https://github.com/apache/airflow/pull/16971)
-- rebased local branch
> 2. fix static checks (the easiest way is to install pre-commit as suggested in the error message - it will then auto-correct everything for you.
-- fixed static check
> Having `pre-commit` installed for airflow contribution is HEARTILY recommended . It saves a lot of time on iterations and a lot of CI build time if you run the checks locally :)
-- installed and passed pre-commit static check all files. Ran through breeze.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on pull request #16964: [AIRFLOW 16730] Added docs & doc ref's for AWS transfer operators between SFTP & S3
Posted by GitBox <gi...@apache.org>.
potiuk commented on pull request #16964:
URL: https://github.com/apache/airflow/pull/16964#issuecomment-880126533
You just need to add the new guides to https://github.com/apache/airflow/blob/main/airflow/providers/amazon/provider.yaml (look at other .rst files there and do it in the same fashion (and yeah @mik-laj maybe worth to improve the error message in this case ;) )
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] pyerbiz commented on pull request #16964: [AIRFLOW 16730] Added docs & doc ref's for AWS transfer operators between SFTP & S3
Posted by GitBox <gi...@apache.org>.
pyerbiz commented on pull request #16964:
URL: https://github.com/apache/airflow/pull/16964#issuecomment-880103580
its failing static check on `hook id: provider-yamls` for providers.yaml file. I did not edit that file, which means I have to add something to it? I did come across a doc I think that talked about `providers.yaml` file -- but right now I couldn't find it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] pyerbiz edited a comment on pull request #16964: [AIRFLOW 16730] Added docs & doc ref's for AWS transfer operators between SFTP & S3
Posted by GitBox <gi...@apache.org>.
pyerbiz edited a comment on pull request #16964:
URL: https://github.com/apache/airflow/pull/16964#issuecomment-879813662
> Can you please @pyerbiz :
>
> 1. rebase to latest main (there were an error in main fixed in [Fixes typo in the name of file for Breeze docker compose env #16971](https://github.com/apache/airflow/pull/16971)
-- rebased local branch
> 2. fix static checks (the easiest way is to install pre-commit as suggested in the error message - it will then auto-correct everything for you.
-- fixed static check
> Having `pre-commit` installed for airflow contribution is HEARTILY recommended . It saves a lot of time on iterations and a lot of CI build time if you run the checks locally :)
-- installed and passed pre-commit static check all files. Ran through breeze.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org