You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@gobblin.apache.org by "Zihan Li (Jira)" <ji...@apache.org> on 2020/10/13 21:33:00 UTC

[jira] [Resolved] (GOBBLIN-1248) Fix discrepancy between table schema and file schema

     [ https://issues.apache.org/jira/browse/GOBBLIN-1248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Zihan Li resolved GOBBLIN-1248.
-------------------------------
    Fix Version/s: 0.15.0
       Resolution: Fixed

> Fix discrepancy between table schema and file schema
> ----------------------------------------------------
>
>                 Key: GOBBLIN-1248
>                 URL: https://issues.apache.org/jira/browse/GOBBLIN-1248
>             Project: Apache Gobblin
>          Issue Type: Task
>            Reporter: Zihan Li
>            Priority: Major
>             Fix For: 0.15.0
>
>          Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> Previously in streaming pipeline, to avoid race condition on metadata schema, when we do hive registration, we always fetch the latest schema from Kafka SchemaRegistry, since gobblin converter may change the schema, this introduce discrepancy between hive table schema and real file schema, so we need a better way to solve this problem



--
This message was sent by Atlassian Jira
(v8.3.4#803005)