You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by GitBox <gi...@apache.org> on 2021/10/20 12:21:00 UTC
[GitHub] [iceberg] SusurHe opened a new issue #3327: How do I need to merge small files with the new API
SusurHe opened a new issue #3327:
URL: https://github.com/apache/iceberg/issues/3327
I find the class `org.apache.iceberg.actions.Actions` was deprecated, when i want to use it to merge small files;
Now I find the new class `SparkActions`, but I don't know how to set target files size?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] ashoksri0 commented on issue #3327: How do I need to merge small files with the new API
Posted by GitBox <gi...@apache.org>.
ashoksri0 commented on issue #3327:
URL: https://github.com/apache/iceberg/issues/3327#issuecomment-952180759
can we do the same thing in java api without spark
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] pan3793 commented on issue #3327: How do I need to merge small files with the new API
Posted by GitBox <gi...@apache.org>.
pan3793 commented on issue #3327:
URL: https://github.com/apache/iceberg/issues/3327#issuecomment-948219131
You can find demo from [TestNewRewriteDataFilesAction](https://github.com/apache/iceberg/blob/119d20d22e66655bce1590041e4ba25ae48874fb/spark/v3.0/spark3/src/test/java/org/apache/iceberg/spark/actions/TestNewRewriteDataFilesAction.java)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] RussellSpitzer commented on issue #3327: How do I need to merge small files with the new API
Posted by GitBox <gi...@apache.org>.
RussellSpitzer commented on issue #3327:
URL: https://github.com/apache/iceberg/issues/3327#issuecomment-952183278
Short answer: No
Long answer: You can do similar things with the pure java api but this would involve reading all the files you want to your local machine, combining them, then writing them from your local machine. I'm not sure if it would be easier to just manually implement that or to implement a "Local" RewriteDataAction
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] RussellSpitzer commented on issue #3327: How do I need to merge small files with the new API
Posted by GitBox <gi...@apache.org>.
RussellSpitzer commented on issue #3327:
URL: https://github.com/apache/iceberg/issues/3327#issuecomment-949740468
Specifically see
https://github.com/apache/iceberg/blob/119d20d22e66655bce1590041e4ba25ae48874fb/spark/v3.0/spark3/src/test/java/org/apache/iceberg/spark/actions/TestNewRewriteDataFilesAction.java#L228-L230
For setting options like target file size
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org