[GitHub] [incubator-iceberg] shawnding opened a new pull request #679: [WIP] [ISSUE #672] Add namespace metadata

[GitBox <gi...@apache.org>]

shawnding opened a new pull request #679: [WIP] [ISSUE #672] Add namespace metadata 
URL: https://github.com/apache/incubator-iceberg/pull/679
 
 
   **Create a namespace metadata file  and save it into** _${WAREHOUSE_BASE}/namespace_metadata_  
   **Then :** 
   1. The commands of `list namespace` or `list table` read the namespace metadata file. Reading from the file could have better performance than listing directories on file system, especially  on Object Store, like S3。
   2. Move the functions of namespace manager out of catalog. Hive catalog could also make good use of the namespace manager to support multi part namespace.
   3. It could help to manage the Iceberg tables and namespaces without the help from a center meta store service like Hive MetaStore.
    
   **TODO: To guarantee the atomicity of create/drop table,  which includes the operations on both 'metadata.json' and 'namespace_metadata.json'**
   
   The followings demonstrate the content of 'namespace_metadata.json'
   ```
   {
     "format-version" : 1,
     "uuid" : "628e40f3-93dc-4b4f-bd00-dee78d7073d1",
     "location" : "/data/tmp/namespace_metadata",
     "last-updated-ms" : 1575278464343,
     "namespaces" : [
                      {
                         "namespace" : "default",
                          "last-updated-ms" : 1575278464343,
                          "tables" : ["tbl1", "tbl2" ],
                          "properties" : { 
                             "owner" : "apache" 
                           }
                      }, 
                      {
                            "namespace" : "db.n2.n3",
                             "last-updated-ms" : 1575278464343,
                             "tables" : [ ],
                             "properties" : { }
                      } 
       ]
   }
   ```
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org