[GitHub] [hudi] gnailJC commented on a diff in pull request #5616: [MINOR]: Optimize the judgment logic of `SparkDataSourceOptions Key`

[GitBox <gi...@apache.org>]

gnailJC commented on code in PR #5616:
URL: https://github.com/apache/hudi/pull/5616#discussion_r875489218


##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/HoodieSqlCommonUtils.scala:
##########
@@ -250,7 +250,7 @@ object HoodieSqlCommonUtils extends SparkAdapterSupport {
                    (baseConfig: Map[String, String] = Map.empty): Map[String, String] = {
     baseConfig ++ DFSPropertiesConfiguration.getGlobalProps.asScala ++ // Table options has the highest priority
       (spark.sessionState.conf.getAllConfs ++ HoodieOptionConfig.mappingSqlOptionToHoodieParam(options))
-        .filterKeys(_.startsWith("hoodie."))
+        .filterKeys(DataSourceOptionsHelper.isHoodieConfigKey)

Review Comment:
   > The logic has changed ?
   
   Yes, if only use `startWith(''hoodie.'')` to determine whether it is the hoodie config key, then "as.of.instant" will be misjudged and cannot achieve the expected purpose.
   
   for example:
   create a hoodie_tbl table by sql: 
   `
   CREATE external TABLE hoodie_tbl
   using hudi
   options (
       hoodie.datasource.query.type='snapshot',
       as.of.instant='20220512165111404'
   ) LOCATION 'xxx';
   `
   hoodie_tbl will always query the latest snapshot, not historical snapshots(20220512165111404), because `as.of.instant `will be filtered and will not take effect.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org