You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "Udit Mehrotra (Jira)" <ji...@apache.org> on 2020/01/10 22:45:00 UTC

[jira] [Created] (HUDI-519) Document the need for Avro dependency shading/relocation for custom payloads

Udit Mehrotra created HUDI-519:
----------------------------------

             Summary: Document the need for Avro dependency shading/relocation for custom payloads 
                 Key: HUDI-519
                 URL: https://issues.apache.org/jira/browse/HUDI-519
             Project: Apache Hudi (incubating)
          Issue Type: Task
          Components: Docs, Usability
            Reporter: Udit Mehrotra


In [https://github.com/apache/incubator-hudi/pull/1005] we are migrating Hudi to Spark 2.4.4. As part of this migration, we also had to migrate Hudi to use Avro 1.8.2 (required by spark), while Hive still uses older version of Avro.

This has resulted in the need to shade Avro in *hadoop-mr-bundle*. This has implications on users of Hudi, who implement custom record payloads. They would have start shading Avro in there custom jars, similar to how it shaded in *hadoop-mr-bundle*.

This Jira is to track the documentation of this caveat in release notes, and if needed at other places like website etc.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)