You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@parquet.apache.org by "Carlos Diogo (Jira)" <ji...@apache.org> on 2021/02/20 10:00:27 UTC

[jira] [Created] (PARQUET-1987) Document how a schema can have columns splitted over different files

Carlos Diogo created PARQUET-1987:
-------------------------------------

             Summary: Document how a schema can have columns splitted over different files 
                 Key: PARQUET-1987
                 URL: https://issues.apache.org/jira/browse/PARQUET-1987
             Project: Parquet
          Issue Type: Improvement
            Reporter: Carlos Diogo


In the design overview is stated that the format supports that columns of a given schema are splitted over several files . 



To date ,other than the reference in the parquet homepage there are no other references to these feature , not even if it is actually implemented. 

As it currently stands , I believe that for a given row group you cannot have some columns in one file and other column group in another file belonging to the same schema .

If these feature was actually implemented it would open the door to faster data updates which affect a single column in very wide tables, which is typical on big data use cases .

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)