[GitHub] [hudi] noobarcitect commented on issue #1586: [SUPPORT] DMS with 2 key example

[GitBox <gi...@apache.org>]

noobarcitect commented on issue #1586:
URL: https://github.com/apache/hudi/issues/1586#issuecomment-683269914


   @bvaradar 
   is this issue resolved in any of Hudi release ? 
   I am currently using 0.6.0 with scala 2.11 and previously tried with 0.5.2-incubating/0.5.3 + Scala 2.11 as well the result is same Below is the description of the spark job that I wrote.
   
   My usecase is somewhat same where I am using 2 columns of a dataset to make an upsert. but the comma separated values don't work. Below is the code I am using to make an upsert in a dataset where albumId and trackId are 2 keys which identifies a unique record : 
   
   albumDf.write
         .format("hudi")
         .option(DataSourceWriteOptions.TABLE_TYPE_OPT_KEY, DataSourceWriteOptions.COW_TABLE_TYPE_OPT_VAL)
         .option(DataSourceWriteOptions.RECORDKEY_FIELD_OPT_KEY, "albumId, trackId")
         .option(DataSourceWriteOptions.PRECOMBINE_FIELD_OPT_KEY, combineKey)
         .option(HoodieWriteConfig.TABLE_NAME, tableName)
         .option(DataSourceWriteOptions.OPERATION_OPT_KEY, DataSourceWriteOptions.UPSERT_OPERATION_OPT_VAL)      
         .option("hoodie.upsert.shuffle.parallelism", "2")
         .mode(SaveMode.Append)
         .save(s"$basePath/$tableName/")
   
   And below is the error that I get : 
   	... 5 more
   Caused by: org.apache.hudi.exception.HoodieKeyException: recordKey value: "null" for field: "albumId, trackId" cannot be null or empty.
   	at org.apache.hudi.keygen.SimpleKeyGenerator.getKe
   
   
   
   Can you please help ? 
   
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org