You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "Alexander Filipchik (Jira)" <ji...@apache.org> on 2020/04/17 16:49:00 UTC

[jira] [Created] (HUDI-801) Add a way to postprocess schema after it is loaded from the schema provider

Alexander Filipchik created HUDI-801:
----------------------------------------

             Summary: Add a way to postprocess schema after it is loaded from the schema provider
                 Key: HUDI-801
                 URL: https://issues.apache.org/jira/browse/HUDI-801
             Project: Apache Hudi (incubating)
          Issue Type: Improvement
          Components: DeltaStreamer
            Reporter: Alexander Filipchik
             Fix For: 0.6.0


Sometimes it is needed to postprocess schemas after they are fetched from the external sources. Some examples of postprocessing:
 * make sure all the defaults are set correctly, and update schema if not.
 * insert marker columns into records with no fields (no writable as parquest)
 * ...

Would be great to have a way to plug in custom post processors.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)