You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "JunseoChoJJ (via GitHub)" <gi...@apache.org> on 2024/04/22 02:34:16 UTC
[I] does iceberg can run on k8s? [iceberg]
JunseoChoJJ opened a new issue, #10196:
URL: https://github.com/apache/iceberg/issues/10196
### Query engine
I am using spark
### Question
hi
I'm a newbee to iceberg. and it might be stupid question but please tell me
does iceberg can run on k8s?
My think was spark can run on k8s so icberg can also run on k8s
other spark, superset, flink has docs on how to install on k8s
but iceberg doesn't have docs on how to install on k8s. so I'm in struggle.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] does iceberg can run on k8s? [iceberg]
Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra closed issue #10196: does iceberg can run on k8s?
URL: https://github.com/apache/iceberg/issues/10196
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] does iceberg can run on k8s? [iceberg]
Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on issue #10196:
URL: https://github.com/apache/iceberg/issues/10196#issuecomment-2068886431
@JunseoChoJJ yes Iceberg can run on k8s. There are no special installation docs required for k8s. Iceberg is integrated through your engine of choice (Flink, Spark, Trino, ..), so all you need to do is properly set up Iceberg with your engine according to the docs. (e.g. setting up Spark with Iceberg is documented [here](https://iceberg.apache.org/docs/latest/spark-configuration/))
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] does iceberg can run on k8s? [iceberg]
Posted by "JunseoChoJJ (via GitHub)" <gi...@apache.org>.
JunseoChoJJ commented on issue #10196:
URL: https://github.com/apache/iceberg/issues/10196#issuecomment-2071620951
@nastra I am using naver cloud storage. it is same like s3, gcs. I want to figure out besides using s3, gcs, or minio. Can I use other storage to configure with iceberg
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] does iceberg can run on k8s? [iceberg]
Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on issue #10196:
URL: https://github.com/apache/iceberg/issues/10196#issuecomment-2071634712
If Naver Cloud Storage has an S3-compatible API, then you can use it with Iceberg by using `S3FileIO` in your Spark catalog config: `--conf spark.sql.catalog.my_catalog.io-impl=org.apache.iceberg.aws.s3.S3FileIO`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] does iceberg can run on k8s? [iceberg]
Posted by "JunseoChoJJ (via GitHub)" <gi...@apache.org>.
JunseoChoJJ commented on issue #10196:
URL: https://github.com/apache/iceberg/issues/10196#issuecomment-2071683150
@nastra Thanks a lot
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] does iceberg can run on k8s? [iceberg]
Posted by "nastra (via GitHub)" <gi...@apache.org>.
nastra commented on issue #10196:
URL: https://github.com/apache/iceberg/issues/10196#issuecomment-2071584139
> Can I put this data into our Company storage as well using rest api?
I'm not exactly sure I understand. Are you trying to replicate the data that lands in S3/GCS/Minio? Can you please re-phrase the question?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] does iceberg can run on k8s? [iceberg]
Posted by "JunseoChoJJ (via GitHub)" <gi...@apache.org>.
JunseoChoJJ commented on issue #10196:
URL: https://github.com/apache/iceberg/issues/10196#issuecomment-2071196674
@nastra thank you can I ask you one more After they etl in spark using iceberg, they put data in to storage like s3, Gcs, minio. Can I put this data into our storage as well using rest api?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org