You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "kevinjqliu (via GitHub)" <gi...@apache.org> on 2024/07/04 18:37:46 UTC

[I] [Bug] Load the proper AWS credential for glue/dynamodb catalog [iceberg-python]

kevinjqliu opened a new issue, #892:
URL: https://github.com/apache/iceberg-python/issues/892

   ### Apache Iceberg version
   
   None
   
   ### Please describe the bug 🐞
   
   `s3.access-key-id` and `s3.secret-access-key` are the pyiceberg way to pass AWS credentials as properties to the catalog. 
   However, in the `glue` and `dynamodb` catalog, `aws_access_key_id ` and `aws_secret_access_key` is expected
   https://github.com/search?q=repo%3Aapache%2Ficeberg-python+aws_secret_access_key+path%3A.py+-path%3Atests&type=code
   
   Also see https://github.com/apache/iceberg-python/issues/515#issuecomment-2208697240 
   
   We should standardize on a using `s3.access-key-id` and `s3.secret-access-key` for glue/dynamodb catalog. And perhaps also fall back to `aws_access_key_id`/`aws_secret_access_key`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [I] [Bug] Load the proper AWS credential for glue/dynamodb catalog [iceberg-python]

Posted by "HonahX (via GitHub)" <gi...@apache.org>.
HonahX commented on issue #892:
URL: https://github.com/apache/iceberg-python/issues/892#issuecomment-2226614416

   Hi @kevinjqliu @jayceslesar. Thanks for the issue and valuable discussion. I would like to give a try with my proposal in https://github.com/apache/iceberg-python/issues/570#issuecomment-2033588669. Draft PR: https://github.com/apache/iceberg-python/pull/922.
   
   > What does the java iceberg do when looking for s3 credentials?
   
   Java implementation supports `s3.access-key-id` and other `s3.*` properties to config `S3FileIO`, but it does not support config glue/dynamodb by explicitly passing the credentials. Instead, it allows user to specify a [custom credential provider implementation](https://github.com/apache/iceberg/blob/c68abfc9fd3956077b43aba20441f089bb8b93d6/aws/src/main/java/org/apache/iceberg/aws/AwsClientProperties.java#L52). 
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [I] [Bug] Load the proper AWS credential for glue/dynamodb catalog [iceberg-python]

Posted by "jayceslesar (via GitHub)" <gi...@apache.org>.
jayceslesar commented on issue #892:
URL: https://github.com/apache/iceberg-python/issues/892#issuecomment-2226257388

   maybe its fine to just break compatibility leading up to 1.0.0? What does the java iceberg do when looking for s3 credentials?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [I] [Bug] Load the proper AWS credential for glue/dynamodb catalog [iceberg-python]

Posted by "kevinjqliu (via GitHub)" <gi...@apache.org>.
kevinjqliu commented on issue #892:
URL: https://github.com/apache/iceberg-python/issues/892#issuecomment-2224334703

   Another consideration is that there can be 2 separate s3 credentials. One to pass to glue/dynamo. Another for the catalog itself. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [I] [Bug] Load the proper AWS credential for glue/dynamodb catalog [iceberg-python]

Posted by "Fokko (via GitHub)" <gi...@apache.org>.
Fokko closed issue #892: [Bug] Load the proper AWS credential for glue/dynamodb catalog
URL: https://github.com/apache/iceberg-python/issues/892


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [I] [Bug] Load the proper AWS credential for glue/dynamodb catalog [iceberg-python]

Posted by "jayceslesar (via GitHub)" <gi...@apache.org>.
jayceslesar commented on issue #892:
URL: https://github.com/apache/iceberg-python/issues/892#issuecomment-2224262949

   seems like a misnomer for dynamo/glue to expect something called `s3.some-aws-access-key`?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org