[jira] [Updated] (HUDI-4797) Merge Into Table Failed when Source Table Has Different Column Order

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HUDI-4797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ASF GitHub Bot updated HUDI-4797:
---------------------------------
    Labels: pull-request-available  (was: )

> Merge Into Table Failed when Source Table Has Different Column Order
> --------------------------------------------------------------------
>
>                 Key: HUDI-4797
>                 URL: https://issues.apache.org/jira/browse/HUDI-4797
>             Project: Apache Hudi
>          Issue Type: Bug
>          Components: spark-sql
>            Reporter: jimmyz
>            Assignee: jimmyz
>            Priority: Major
>              Labels: pull-request-available
>
> h2. reproduce steps:
> 1. create table t1 (id int, data string) using hudi tblproperties (type ='mor', primaryKey = 'id')
> 2. merge into t1 as t0 using (select 'a1' as data, 1 as id) as s0 on s0.id = t0.id when not matched and s0.id % 2 = 1 then insert *
> h2. exception:
> {code:java}
> Caused by: java.lang.ClassCastException: org.apache.spark.unsafe.types.UTF8String cannot be cast to java.lang.Integer
> 	at org.apache.hudi.sql.payload.ExpressionPayloadEvaluator_09f7a7b7_0021_4d57_a806_f0a8089b35ed.eval(Unknown Source)
> 	at org.apache.spark.sql.hudi.command.payload.ExpressionPayload.org$apache$spark$sql$hudi$command$payload$ExpressionPayload$$evaluate(ExpressionPayload.scala:244)
> 	... 18 more
> {code}
> h2. root cause:
> Spark join result in different column order, join key will be promoted to head.
> When it came to 'not match' condition, The position of the id in the condition is different from the position in the actual record



--
This message was sent by Atlassian Jira
(v8.20.10#820010)