You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "melugoyal (via GitHub)" <gi...@apache.org> on 2023/06/01 03:25:51 UTC
[GitHub] [airflow] melugoyal opened a new pull request, #31659: allow anonymous AWS access
melugoyal opened a new pull request, #31659:
URL: https://github.com/apache/airflow/pull/31659
allow anonymous AWS access. this allows the consumer of any AWS operator to use the equivalent of `aws s3 cp <s3 path> . --no-sign-request`, to eg write a DAG that reads from a public S3 bucket. to leverage this, an AWS connection can be created as such:
```
conn = Connection(
conn_id="aws_demo",
conn_type="aws",
extra={
"config_kwargs": {
"signature_version": "unsigned",
},
},
)
```
this is necessary because botocore's `UNSIGNED` is not JSON-serializable, so using `botocore.UNSIGNED` when instantiating the connection will cause the connection processing to fail. instead we can use a string in the connection instantiation and convert it to botocore's expected value later. more history about this issue here: https://github.com/boto/botocore/issues/2442
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a newsfragment file, named `{pr_number}.significant.rst` or `{issue_number}.significant.rst`, in [newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on pull request #31659: allow anonymous AWS access
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on PR #31659:
URL: https://github.com/apache/airflow/pull/31659#issuecomment-1574286279
Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jedcunningham commented on pull request #31659: allow anonymous AWS access
Posted by "jedcunningham (via GitHub)" <gi...@apache.org>.
jedcunningham commented on PR #31659:
URL: https://github.com/apache/airflow/pull/31659#issuecomment-1574249452
@melugoyal I'd suggest you [set up the pre-commit hooks](https://github.com/apache/airflow/blob/main/STATIC_CODE_CHECKS.rst#pre-commit-hooks), they help catch these issues early 👍.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jedcunningham merged pull request #31659: allow anonymous AWS access
Posted by "jedcunningham (via GitHub)" <gi...@apache.org>.
jedcunningham merged PR #31659:
URL: https://github.com/apache/airflow/pull/31659
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] o-nikolas commented on pull request #31659: allow anonymous AWS access
Posted by "o-nikolas (via GitHub)" <gi...@apache.org>.
o-nikolas commented on PR #31659:
URL: https://github.com/apache/airflow/pull/31659#issuecomment-1574234666
Other than the static check failures, lgtm :+1:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] vincbeck commented on pull request #31659: allow anonymous AWS access
Posted by "vincbeck (via GitHub)" <gi...@apache.org>.
vincbeck commented on PR #31659:
URL: https://github.com/apache/airflow/pull/31659#issuecomment-1574186073
> Overall looks good. Should we document this somehow as well?
Good callout! I'd say yes :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] jedcunningham commented on pull request #31659: allow anonymous AWS access
Posted by "jedcunningham (via GitHub)" <gi...@apache.org>.
jedcunningham commented on PR #31659:
URL: https://github.com/apache/airflow/pull/31659#issuecomment-1574287465
Thanks @melugoyal! Congrats on your first commit 🎉
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] melugoyal commented on pull request #31659: allow anonymous AWS access
Posted by "melugoyal (via GitHub)" <gi...@apache.org>.
melugoyal commented on PR #31659:
URL: https://github.com/apache/airflow/pull/31659#issuecomment-1574197214
@jedcunningham @vincbeck good call. added a sentence to the AWS connection docs. lmk how that looks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org