You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2021/12/02 22:51:39 UTC
[GitHub] [iceberg] jackye1995 opened a new pull request #3658: AWS: support S3FileIO alternative endpoint and credentials
jackye1995 opened a new pull request #3658:
URL: https://github.com/apache/iceberg/pull/3658
As discussed in the mailing list: https://mail-archives.apache.org/mod_mbox/iceberg-dev/202112.mbox/%3CCAMwmD1_P2PCMpzUai5RV%2B4a9Bv62ZFqrrDdxCtLcTSvPnUxk6g%40mail.gmail.com%3E
For `DefaultClientFactory`, we allow configuration of S3 static credentials and endpoint.
For `AssumeRoleClientFactory`, we only allow configuration of S3 endpoint, because credential is always assume role credential.
@rdblue @danielcweeks @findepi @mayursrivastava
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] jackye1995 merged pull request #3658: AWS: support S3FileIO alternative endpoint and credentials
Posted by GitBox <gi...@apache.org>.
jackye1995 merged pull request #3658:
URL: https://github.com/apache/iceberg/pull/3658
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] jackye1995 commented on pull request #3658: AWS: support S3FileIO alternative endpoint and credentials
Posted by GitBox <gi...@apache.org>.
jackye1995 commented on pull request #3658:
URL: https://github.com/apache/iceberg/pull/3658#issuecomment-991164029
Thanks for the review Daniel!
For anyone looking into this, we are adding these configs for users who want to use S3 compatible storage services and frameworks such as MinIO through the AWS S3 client. But ultimately the right way to go is to have a native FileIO, like in the case of GCS, which can unlock all the features of a storage service.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] rdblue commented on pull request #3658: AWS: support S3FileIO alternative endpoint and credentials
Posted by GitBox <gi...@apache.org>.
rdblue commented on pull request #3658:
URL: https://github.com/apache/iceberg/pull/3658#issuecomment-986294543
Looks good to me, but I'm not very familiar with this part. It would be great if @danielcweeks could take a look, too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] jackye1995 commented on pull request #3658: AWS: support S3FileIO alternative endpoint and credentials
Posted by GitBox <gi...@apache.org>.
jackye1995 commented on pull request #3658:
URL: https://github.com/apache/iceberg/pull/3658#issuecomment-985251614
tests added, and I've run AWS integration tests to verify all tests pass.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] jackye1995 commented on pull request #3658: AWS: support S3FileIO alternative endpoint and credentials
Posted by GitBox <gi...@apache.org>.
jackye1995 commented on pull request #3658:
URL: https://github.com/apache/iceberg/pull/3658#issuecomment-985068078
This is a bit hard to test, I will try to add integration tests, meanwhile @findepi @mayursrivastava could you check if the added configurations are enough for using S3FileIO in GCS, MinIO and other services?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org