[jira] [Created] (FLINK-17909) Make the GenericInMemoryCatalog to hold the serialized meta data to uncover more potential bugs

["Jark Wu (Jira)" <ji...@apache.org>]

Jark Wu created FLINK-17909:
-------------------------------

             Summary: Make the GenericInMemoryCatalog to hold the serialized meta data to uncover more potential bugs
                 Key: FLINK-17909
                 URL: https://issues.apache.org/jira/browse/FLINK-17909
             Project: Flink
          Issue Type: Improvement
          Components: Table SQL / API, Table SQL / Planner
            Reporter: Jark Wu
             Fix For: 1.11.0


Currently, the builtin {{GenericInMemoryCatalog}} hold the meta objects in HashMap. However, this lead to many bugs when users switch to some persisted catalogs, e.g. Hive Metastore. For example, FLINK-17189, FLINK-17868, FLINK-16021. 

That is because the builtin {{GenericInMemoryCatalog}} doesn't cover the important path of serialization and deserialization of meta data. We missed some important meta information (PK, time attributes) when serialization and deserialization which lead to bugs. 

So I propose to hold the serialized meta data in {{GenericInMemoryCatalog}} to cover the serialization and deserializtion path. 

We may lose some performance here, but {{GenericInMemoryCatalog}} is mostly used in demo/experiment/testing, so I think it's fine. 








--
This message was sent by Atlassian Jira
(v8.3.4#803005)