[GitHub] [hudi] xushiyan commented on pull request #4083: [HUDI-2837] The original hoodie.table.name should be maintained in Spark SQL

[GitBox <gi...@apache.org>]

xushiyan commented on pull request #4083:
URL: https://github.com/apache/hudi/pull/4083#issuecomment-1003661058


   > @dongkelun @xushiyan I offer another solution to discuss.
   > 
   > Query incrementally in hive need to set `hoodie.%s.consume.start.timestamp` which is used in `HoodieHiveUtils.readStartCommitTime`。Currently, we pass the `hoodie.table.name` named `tableName` to this function. We can add configs `hoodie.datasource.write.database.name` in `DataSourceWriteOptions` and `hoodie.database.name` in `HoodieTableConfig`. And if `database.name` provided, we joint the `database.name` and `table.name` and pass it to `readStartCommitTime`. And then, use can set `hoodie.dbName.tableName.consume.start.timestamp` in hive and query.
   > 
   > Also, `hoodie.datasource.write.database.name` and `hoodie.database.name` can reuse in other scene.
   > 
   > @xushiyan what do you think.
   
   @YannByron @dongkelun Sorry for the late reply. Instead of setting a switch to use database name, setting the config itself and checking its value is cleaner. The idea sounds good to me. Thanks.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org