You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "toujours33 (via GitHub)" <gi...@apache.org> on 2025/11/14 08:44:58 UTC
Re: [PR] [SPARK-20236][SQL] dynamic partition overwrite [spark]
toujours33 commented on PR #18714:
URL: https://github.com/apache/spark/pull/18714#issuecomment-3531623707
@cloud-fan Could you please help me understand why this piece of code in InsertIntoHadoopFsRelationCommand needs?
`enableDynamicOverwrite && mode == SaveMode.Overwrite &&
staticPartitions.size < partitionColumns.length`
If dynamic overwrite is enabled but we're specifying all static partitions in the INSERT overwrite statement, it will still delete related partitions first.
And as described in [SPARK-30112](https://github.com/apache/spark/pull/26752), we're still not allowed to insert overwrite same table if all static partitions are specified.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org