Re: [I] [1.0.2] Record Level Index being dropped by clustering + cleaning [hudi]

["vaibhavk1992 (via GitHub)" <gi...@apache.org>]

vaibhavk1992 commented on issue #13996:
URL: https://github.com/apache/hudi/issues/13996#issuecomment-3476652723

   @DamianoErnesti-Anritsu I tried reproduce the issue mentioned above. I am using spark datasource API and the below code
   **Record level index+partition/column stats+Secondary index+inline clustering**
   `inserts.write.format("hudi")
     .option("hoodie.datasource.write.partitionpath.field", "city")
     .option("hoodie.table.name", tableName)
     .option("hoodie.datasource.write.table.type", "MERGE_ON_READ")
     # Partition Stats
     .option("hoodie.metadata.enable", "true")
     .option("hoodie.metadata.partition.stats.enable", "true")
     .option("hoodie.metadata.column.stats.enable", "true")
     .option("hoodie.metadata.column.stats.column.list", "rider,fare")
     # Record Level Index
     .option("hoodie.metadata.record.index.enable", "true")
     .option("hoodie.datasource.write.recordkey.field", "uuid")
     .option("hoodie.metadata.index.secondary.enable", "true")
     .option("hoodie.datasource.write.secondarykey.column", "city")
     .option("hoodie.parquet.small.file.limit", "0")
     .option("hoodie.clustering.inline", "true")
     .option("hoodie.clustering.inline.max.commits", "4")
     .option("hoodie.clustering.plan.strategy.target.file.max.bytes", "1073741824")
     .option("hoodie.clustering.plan.strategy.small.file.limit", "629145600")
     .option("hoodie.clustering.plan.strategy.sort.columns", "city")
     .mode("overwrite")
     .save(basePath)`
   
   When I clean up using spark submit **I am able to see** the record index directory. May be if you can share the yours sample code and commands I will try again. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org