You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "JoshRosen (via GitHub)" <gi...@apache.org> on 2025/02/11 02:30:15 UTC

[PR] [MINOR][DOCS] Fix incorrect description of constraint on spark.sql.adaptive.coalescePartitions.minPartitionSize [spark]

JoshRosen opened a new pull request, #49872:
URL: https://github.com/apache/spark/pull/49872

   ### What changes were proposed in this pull request?
   
   This PR addresses a minor problem in the SQL performance guide's description of the `spark.sql.adaptive.coalescePartitions.minPartitionSize` configuration.
   
   Currently, the guide says:
   
   > The minimum size of shuffle partitions after coalescing. Its value can be at most 20% of `spark.sql.adaptive.advisoryPartitionSizeInBytes`. This is useful when the target size is ignored during partition coalescing, which is the default case.
   
   but the second sentence is not true because the 20% limitation was removed in https://github.com/apache/spark/pull/33655#discussion_r683582184 (SPARK-36430), which shipped in 3.2.0. That PR updated the in-code description but not the guide on the web.
   
   ### Why are the changes needed?
   
   Incorrect doc is misleading.
   
   ### Does this PR introduce _any_ user-facing change?
   
   No.
   
   ### How was this patch tested?
   
   n/a
   
   ### Was this patch authored or co-authored using generative AI tooling?
   
   No.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [MINOR][DOCS] Fix incorrect description of constraint on spark.sql.adaptive.coalescePartitions.minPartitionSize [spark]

Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan commented on PR #49872:
URL: https://github.com/apache/spark/pull/49872#issuecomment-2649725958

   thanks, merging to master/3.5


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Re: [PR] [MINOR][DOCS] Fix incorrect description of constraint on spark.sql.adaptive.coalescePartitions.minPartitionSize [spark]

Posted by "cloud-fan (via GitHub)" <gi...@apache.org>.
cloud-fan closed pull request #49872: [MINOR][DOCS] Fix incorrect description of constraint on spark.sql.adaptive.coalescePartitions.minPartitionSize
URL: https://github.com/apache/spark/pull/49872


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org