You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@gobblin.apache.org by "Zihan Li (Jira)" <ji...@apache.org> on 2020/08/21 22:47:00 UTC

[jira] [Created] (GOBBLIN-1248) Fix discrepancy between table schema and file schema

Zihan Li created GOBBLIN-1248:
---------------------------------

             Summary: Fix discrepancy between table schema and file schema
                 Key: GOBBLIN-1248
                 URL: https://issues.apache.org/jira/browse/GOBBLIN-1248
             Project: Apache Gobblin
          Issue Type: Task
            Reporter: Zihan Li


Previously in streaming pipeline, to avoid race condition on metadata schema, when we do hive registration, we always fetch the latest schema from Kafka SchemaRegistry, since gobblin converter may change the schema, this introduce discrepancy between hive table schema and real file schema, so we need a better way to solve this problem



--
This message was sent by Atlassian Jira
(v8.3.4#803005)