You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "mxm (via GitHub)" <gi...@apache.org> on 2026/02/20 12:43:36 UTC

[PR] Spark: Add ExecutorService support to RewriteTablePath [iceberg]

mxm opened a new pull request, #15381:
URL: https://github.com/apache/iceberg/pull/15381

   This adds `executeWith(ExecutorService)` to RewriteTablePath to enable parallel metadata rewriting for version files and manifest lists.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark: Add ExecutorService support to RewriteTablePath [iceberg]

Posted by "mxm (via GitHub)" <gi...@apache.org>.
mxm commented on code in PR #15381:
URL: https://github.com/apache/iceberg/pull/15381#discussion_r2833179480


##########
api/src/main/java/org/apache/iceberg/actions/RewriteTablePath.java:
##########
@@ -99,6 +101,18 @@ default RewriteTablePath createFileList(boolean createFileList) {
     return this;
   }
 
+  /**
+   * Passes an alternative executor service that will be used for version file and manifest list
+   * rewriting. If this method is not called, these operations will be performed sequentially.
+   *
+   * @param executorService an executor service to parallelize metadata rewriting
+   * @return this for method chaining
+   */
+  default RewriteTablePath executeWith(ExecutorService executorService) {
+    throw new UnsupportedOperationException(
+        "This implementation does not support providing an ExecutorService.");
+  }

Review Comment:
   I saw that other PRs just did a `return this` here, but it feels wrong to ignore the provided ExecutorService entirely.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark: Add ExecutorService support to RewriteTablePath [iceberg]

Posted by "mxm (via GitHub)" <gi...@apache.org>.
mxm commented on PR #15381:
URL: https://github.com/apache/iceberg/pull/15381#issuecomment-3935964283

   CC @huaxiangsun @anuragmantri @pvary


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark: Add ExecutorService support to RewriteTablePath [iceberg]

Posted by "mxm (via GitHub)" <gi...@apache.org>.
mxm commented on PR #15381:
URL: https://github.com/apache/iceberg/pull/15381#issuecomment-4029742761

   Thanks @pvary!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark: Add ExecutorService support to RewriteTablePath [iceberg]

Posted by "pvary (via GitHub)" <gi...@apache.org>.
pvary commented on PR #15381:
URL: https://github.com/apache/iceberg/pull/15381#issuecomment-3991546431

   Let's give folks a few more days to see if there are any objections, then I will merge.
   CC: @szehon-ho, @RussellSpitzer 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark: Add ExecutorService support to RewriteTablePath [iceberg]

Posted by "pvary (via GitHub)" <gi...@apache.org>.
pvary commented on PR #15381:
URL: https://github.com/apache/iceberg/pull/15381#issuecomment-4024803354

   Merged to main.
   Thanks for @mxm for the PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org


Re: [PR] Spark: Add ExecutorService support to RewriteTablePath [iceberg]

Posted by "pvary (via GitHub)" <gi...@apache.org>.
pvary merged PR #15381:
URL: https://github.com/apache/iceberg/pull/15381


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org