[jira] [Reopened] (HUDI-2343) Fix the exception for mergeInto when the primaryKey and preCombineField of source table and target table differ in case only

["sivabalan narayanan (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HUDI-2343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

sivabalan narayanan reopened HUDI-2343:
---------------------------------------

> Fix the exception for mergeInto when the primaryKey and preCombineField of source table and target table differ in case only
> ----------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HUDI-2343
>                 URL: https://issues.apache.org/jira/browse/HUDI-2343
>             Project: Apache Hudi
>          Issue Type: Sub-task
>          Components: Spark Integration
>            Reporter: 董可伦
>            Assignee: 董可伦
>            Priority: Blocker
>              Labels: pull-request-available
>             Fix For: 0.10.0
>
>
> Example:
> {code:java}
> val tableName = "test_hudi_table"
> spark.sql(
> s"""
> create table ${tableName} (
> id int,
> name string,
> price double,
> ts long
> ) using hudi
> options (
> primaryKey = 'id',
> type = 'cow'
> )
> location '/tmp/${tableName}'
> """.stripMargin)
> spark.sql(
>   s"""
>      | merge into $tableName
>      | using (
>      |  select 1 as ID, 'a1' as NAME, 10 as PRICE, 1000 as TS, '1' as FLAG
>      | ) s0
>      | on s0.ID = $tableName.id
>      | when matched and FLAG = '1' then update set
>      | id = s0.ID, name = s0.NAME, price = s0.PRICE, ts = s0.TS
>      | when not matched and FLAG = '1' then insert *
>  """.stripMargin){code}
> It will throw an exception:
> {code:java}
> org.apache.hudi.exception.HoodieException: ts(Part -ts) field not found in record. Acceptable fields were :[ID, NAME, PRICE, TS, FLAG]
> {code}
> and:
> {code:java}
> spark.sql(
>   s"""
>      | merge into $tableName
>      | using (
>      |  select 1 as ID, 'a1' as NAME, 10 as PRICE, 1000 as ts, '1' as FLAG
>      | ) s0
>      | on s0.ID = $tableName.id
>      | when matched and FLAG = '1' then update set
>      | id = s0.ID, name = s0.NAME, price = s0.PRICE, ts = s0.ts
>      | when not matched and FLAG = '1' then insert *
>  """.stripMargin)
> {code}
> It will throw the exception:
> {code:java}
> org.apache.hudi.exception.HoodieKeyException: recordKey values: "id:__null__" for fields: [id] cannot be entirely null or empty.
> {code}
>  
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)