You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "Zouxxyy (via GitHub)" <gi...@apache.org> on 2025/07/09 07:31:09 UTC

[PR] [spark] Introduce write.distribution-mode conf [paimon]

Zouxxyy opened a new pull request, #5856:
URL: https://github.com/apache/paimon/pull/5856

   <!-- Please specify the module before the PR name: [core] ... or [flink] ... -->
   
   ### Purpose
   
   Currently, in v2 write, writing to a partitioned table with bucket = -1 defaults to repartitioning the data by partition columns, which is inconsistent with the default behavior of v1 write. So a new option `spark.paimon.write.distribution-mode` has been added to control this behavior and align it between v1 and v2 writes.
   
   This configuration only applies to bucket-unaware tables, because bucket tables always need repartition now.
   Maybe in the furure, repartitioning for bucketed tables may also be optional — for example, when the write mode is set to write-only.
   
   ### Tests
   
   <!-- List UT and IT cases to verify this change -->
   
   ### API and Format
   
   <!-- Does this change affect API or storage format -->
   
   ### Documentation
   
   <!-- Does this change introduce a new feature -->
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [spark] Apply partition.sink-strategy to spark write [paimon]

Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi merged PR #5856:
URL: https://github.com/apache/paimon/pull/5856


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [spark] Introduce write.distribution-mode conf [paimon]

Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on PR #5856:
URL: https://github.com/apache/paimon/pull/5856#issuecomment-3051525998

   Can we reuse `partition.sink-strategy` option?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org