You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "Davis Zhang (Jira)" <ji...@apache.org> on 2025/01/06 18:45:00 UTC

[jira] [Assigned] (HUDI-8321) SQL insert mode needs to set merger as well to support non avro records

     [ https://issues.apache.org/jira/browse/HUDI-8321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Davis Zhang reassigned HUDI-8321:
---------------------------------

    Assignee: Lin Liu  (was: Jonathan Vexler)

> SQL insert mode needs to set merger as well to support non avro records
> -----------------------------------------------------------------------
>
>                 Key: HUDI-8321
>                 URL: https://issues.apache.org/jira/browse/HUDI-8321
>             Project: Apache Hudi
>          Issue Type: Sub-task
>          Components: spark, spark-sql, writer-core
>            Reporter: Jonathan Vexler
>            Assignee: Lin Liu
>            Priority: Blocker
>             Fix For: 1.0.1
>
>
> org.apache.spark.sql.hudi.ProvidesHoodieConfig deduces the payload to sometimes be ValidateDuplicateKeyPayload. This is only relevant for avro merging, and the validation will not happen if spark records are used. 
> HoodieSparkValidateDuplicateKeyRecordMerger should also be set here so that validation happens



--
This message was sent by Atlassian Jira
(v8.20.10#820010)