[I] [BUG] Partition DDL commands ignore slash.separated.date.partitioning: ADD PARTITION creates a stray directory, DROP/TRUNCATE PARTITION silently do nothing [hudi]

["SEPURI-SAI-KRISHNA (via GitHub)" <gi...@apache.org>]

SEPURI-SAI-KRISHNA opened a new issue, #19702:
URL: https://github.com/apache/hudi/issues/19702

   ## Bug Description
   
   **What happened:**
   
   `HoodieSqlCommonUtils#makePartitionPath` builds the on-disk partition directory for the partition
   DDL commands from the table config, but it only reads two of the three configs that determine the
   layout:
   
   ```scala
   def makePartitionPath(hoodieCatalogTable: HoodieCatalogTable,
                         normalizedSpecs: Map[String, String]): String = {
     val tableConfig = hoodieCatalogTable.tableConfig
     val enableHiveStylePartitioning = java.lang.Boolean.parseBoolean(tableConfig.getHiveStylePartitioningEnable)
     val enableEncodeUrl = java.lang.Boolean.parseBoolean(tableConfig.getUrlEncodePartitioning)
   
     makePartitionPath(hoodieCatalogTable.partitionFields, normalizedSpecs, enableEncodeUrl, enableHiveStylePartitioning)
   }
   ```
   (`HoodieSqlCommonUtils.scala:420-427`)
   
   `HoodieTableConfig#getSlashSeparatedDatePartitioning` (`HoodieTableConfig.java:1344`) is never
   consulted, so on a table written with
   `hoodie.datasource.write.slash.separated.date.partitioning=true` the DDL commands compute
   `2026-01-05` while the writer produced `2026/01/05`. The two disagree for every partition value
   containing a dash.
   
   Three commands are affected:
   
   | command | path | consequence |
   |---|---|---|
   | `ALTER TABLE ... ADD PARTITION` | `AlterHoodieTableAddPartitionCommand.scala:69` | creates a bogus `<base>/2026-01-05/` directory with a `.hoodie_partition_metadata` in it, alongside the real `<base>/2026/01/05/`. `hasPartitionMetadata` checks the wrong path too, so `IF NOT EXISTS` never sees the existing partition |
   | `ALTER TABLE ... DROP PARTITION` | `AlterHoodieTableDropPartitionCommand.scala:63` via `getPartitionPathToDrop` (`HoodieSqlCommonUtils.scala:400-404`) | targets `2026-01-05`, which does not exist, so the real partition is not dropped |
   | `TRUNCATE TABLE ... PARTITION` | `TruncateHoodieTableCommand.scala:94` via `getPartitionPathToDrop` | same: truncates nothing |
   
   `ADD PARTITION` is the damaging one, since it leaves a stray partition directory behind that
   subsequent listings can pick up. `DROP`/`TRUNCATE` fail silently, which is its own problem: the
   user is told the partition was dropped and the data is still there.
   
   **What you expected:**
   
   The DDL commands should derive the same partition directory the writer does, i.e.
   `makePartitionPath` should apply the `-` -> `/` substitution when the table config has slash
   separated date partitioning enabled, just as it already honours hive-style partitioning and URL
   encoding.
   
   **Steps to reproduce:**
   
   ```sql
   CREATE TABLE t (id INT, name STRING, date_col STRING) USING hudi
     PARTITIONED BY (date_col)
     TBLPROPERTIES (hoodie.datasource.write.slash.separated.date.partitioning = 'true');
   
   INSERT INTO t VALUES (1, 'a', '2026-01-05');
   -- writer creates <base>/2026/01/05/
   
   ALTER TABLE t DROP PARTITION (date_col = '2026-01-05');
   -- reports success, <base>/2026/01/05/ is untouched
   
   ALTER TABLE t ADD PARTITION (date_col = '2026-02-06');
   -- creates <base>/2026-02-06/ rather than <base>/2026/02/06/
   ```
   
   **Suggested fix:**
   
   Read `tableConfig.getSlashSeparatedDatePartitioning` in `makePartitionPath` and apply the
   substitution to the encoded value, mirroring what the write path does in
   `PartitionPathFormatterBase#combine` / `KeyGenUtils#getRecordPartitionPath` — including the
   single-partition-field restriction and the leading-dash guard those two use, so DDL and writer stay
   in agreement rather than trading one mismatch for another.
   
   Raised during review of #19648.
   
   ## Environment
   
   **Hudi version:** master (1.3.0-SNAPSHOT)
   **Query engine:** Spark SQL (DDL partition commands)
   **Relevant configs:** `hoodie.datasource.write.slash.separated.date.partitioning=true`
   
   ## Logs and Stack Trace
   
   No stack trace: `DROP`/`TRUNCATE PARTITION` report success while doing nothing, and `ADD PARTITION`
   succeeds while creating the wrong directory.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [BUG] Partition DDL commands ignore slash.separated.date.partitioning: ADD PARTITION creates a stray directory, DROP/TRUNCATE PARTITION silently do nothing [hudi]

["danny0405 (via GitHub)" <gi...@apache.org>]

danny0405 closed issue #19702: [BUG] Partition DDL commands ignore slash.separated.date.partitioning: ADD PARTITION creates a stray directory, DROP/TRUNCATE PARTITION silently do nothing
URL: https://github.com/apache/hudi/issues/19702


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org