You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "jimmyz (Jira)" <ji...@apache.org> on 2022/09/07 07:51:00 UTC
[jira] [Assigned] (HUDI-4797) Merge Into Table Failed when Source Table Has Different Column Order
[ https://issues.apache.org/jira/browse/HUDI-4797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
jimmyz reassigned HUDI-4797:
----------------------------
Assignee: jimmyz
> Merge Into Table Failed when Source Table Has Different Column Order
> --------------------------------------------------------------------
>
> Key: HUDI-4797
> URL: https://issues.apache.org/jira/browse/HUDI-4797
> Project: Apache Hudi
> Issue Type: Bug
> Components: spark-sql
> Reporter: jimmyz
> Assignee: jimmyz
> Priority: Major
>
> h2. reproduce steps:
> 1. create table t1 (id int, data string) using hudi tblproperties (type ='mor', primaryKey = 'id')
> 2. merge into t1 as t0 using (select 'a1' as data, 1 as id) as s0 on s0.id = t0.id when not matched and s0.id % 2 = 1 then insert *
> h2. exception:
> {code:java}
> Caused by: java.lang.ClassCastException: org.apache.spark.unsafe.types.UTF8String cannot be cast to java.lang.Integer
> at org.apache.hudi.sql.payload.ExpressionPayloadEvaluator_09f7a7b7_0021_4d57_a806_f0a8089b35ed.eval(Unknown Source)
> at org.apache.spark.sql.hudi.command.payload.ExpressionPayload.org$apache$spark$sql$hudi$command$payload$ExpressionPayload$$evaluate(ExpressionPayload.scala:244)
> ... 18 more
> {code}
> h2. root cause:
> Spark join result in different column order, join key will be promoted to head.
> When it came to 'not match' condition, The position of the id in the condition is different from the position in the actual record
--
This message was sent by Atlassian Jira
(v8.20.10#820010)