You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2020/07/30 07:27:16 UTC

[GitHub] [hudi] zhedoubushishi commented on issue #1586: [SUPPORT] DMS with 2 key example

zhedoubushishi commented on issue #1586:
URL: https://github.com/apache/hudi/issues/1586#issuecomment-665856107


   I added a log print to checked the config read by JCommander:
   
   ```
   20/07/25 00:50:06 INFO HoodieDeltaStreamer: debug => config is hoodie.datasource.write.recordkey.field=f1
   20/07/25 00:50:06 INFO HoodieDeltaStreamer: debug => config is f2,,f3
   20/07/25 00:50:06 INFO HoodieDeltaStreamer: debug => config is hoodie.datasource.write.keygenerator.class=org.apache.hudi.keygen.ComplexKeyGenerator
   20/07/25 00:50:06 INFO HoodieDeltaStreamer: debug => config is hoodie.datasource.write.partitionpath.field=dt
   20/07/25 00:50:06 INFO HoodieDeltaStreamer: debug => config is hoodie.datasource.hive_sync.database=default
   20/07/25 00:50:06 INFO HoodieDeltaStreamer: debug => config is hoodie.datasource.hive_sync.table=hudi_table
   20/07/25 00:50:06 INFO HoodieDeltaStreamer: debug => config is hoodie.datasource.hive_sync.partition_fields=dt
   20/07/25 00:50:06 INFO HoodieDeltaStreamer: debug => config is hoodie.datasource.hive_sync.partition_extractor_class=org.apache.hudi.hive.MultiPartKeysValueExtractor
   20/07/25 00:50:06 INFO HoodieDeltaStreamer: debug => config is hoodie.deltastreamer.source.dfs.root=s3://hudi/data
   ```
   
   It shows that JCommander will automatically split the string by ",":
   ```
   20/07/25 00:50:06 INFO HoodieDeltaStreamer: debug => config is hoodie.datasource.write.recordkey.field=f1
   20/07/25 00:50:06 INFO HoodieDeltaStreamer: debug => config is f2,,f3
   ```
   That's why it unable to read multiple fields as record key.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org