You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "sundapeng (via GitHub)" <gi...@apache.org> on 2026/09/01 08:26:22 UTC

[PR] [spark] Let MSCK see the null partition of a value-only format table [paimon]

sundapeng opened a new pull request, #9521:
URL: https://github.com/apache/paimon/pull/9521

   ### Purpose
   
   Fix `MSCK REPAIR TABLE` for catalog-managed Format Tables that use value-only partition paths.
   
   The default partition directory (normally `__DEFAULT_PARTITION__`) starts with `_`. Repair did not pass the configured default partition name to filesystem discovery, so that directory was treated as hidden. `ADD PARTITIONS` could not discover it, while `SYNC` or `DROP PARTITIONS` could unregister a live partition and make its rows invisible even though the data files remained.
   
   Pass the table's default partition name to the existing discovery overload. Keep raw path discovery unchanged so values such as `month=01` are not cast and rewritten.
   
   ### Tests
   
   - `mvn -pl paimon-spark/paimon-spark-common -DskipITs -Dtest=FormatTablePartitionRepairTest -DfailIfNoTests=false test`
     - JUnit: 22 passed
     - ScalaTest executed by the module lifecycle: 36 passed
   - `mvn -pl paimon-spark/paimon-spark-common spotless:check`
   
   The new regression cases cover ADD, SYNC, a custom default partition name, a default partition on a non-leaf value-only level, and the unaffected key-value layout.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [spark] Let MSCK see the null partition of a value-only format table [paimon]

Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi merged PR #9521:
URL: https://github.com/apache/paimon/pull/9521


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [spark] Let MSCK see the null partition of a value-only format table [paimon]

Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on PR #9521:
URL: https://github.com/apache/paimon/pull/9521#issuecomment-5505566945

   +1


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org