You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "ajantha-bhat (via GitHub)" <gi...@apache.org> on 2023/09/27 07:02:30 UTC
[GitHub] [iceberg] ajantha-bhat opened a new issue, #8662: Document Azure and GCP integration
ajantha-bhat opened a new issue, #8662:
URL: https://github.com/apache/iceberg/issues/8662
We have a nice documentation for AWS (https://iceberg.apache.org/docs/latest/aws/) which explains if the warehouse path is S3, what configurations are needed for catalog and what dependencies are needed.
We need a similar documentation for Azure and GCP.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] ldacey commented on issue #8662: Document Azure and GCP integration
Posted by "ldacey (via GitHub)" <gi...@apache.org>.
ldacey commented on issue #8662:
URL: https://github.com/apache/iceberg/issues/8662#issuecomment-1738228279
I was confused with where to start on GCP. I haven't dug too deep because I know there is no write support yet for pyiceberg (and I do not have Spark installed at all), but I did want to test a few things.
I know I need a catalog and it looks like the easiest one might be my PostgreSQL cloud SQL database:
```
catalog:
default:
type: sql
uri: postgresql+psycopg2://username:password@localhost/mydatabase
```
But I did not see a description of how to create the actual table, if the database needs to be named specifically, what is the schema of the database table? Does it get created automatically if I just create a "iceberg" database with a iceberg user and password?
Since I am on GCP, I know of Dataproc as well and I have explored Dataplex before, but I am not sure if I can use that as a catalog for pyiceberg. If I can use this, I am not sure if there is a benefit compared to the PostgreSQL approach above either way.
I might be missing something - I am basing this information off of this documentation: https://py.iceberg.apache.org/configuration/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Document Azure and GCP integration [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #8662:
URL: https://github.com/apache/iceberg/issues/8662#issuecomment-2362461758
This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] ajantha-bhat commented on issue #8662: Document Azure and GCP integration
Posted by "ajantha-bhat (via GitHub)" <gi...@apache.org>.
ajantha-bhat commented on issue #8662:
URL: https://github.com/apache/iceberg/issues/8662#issuecomment-1736832206
I am clearly not the expert here on this to add documentation.
I saw that we have `ADLSFileIO` and `GCSFileIO` classes in code.
But didn't find any catalog level examples or test cases for the same.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org