You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@parquet.apache.org by Mohammad Islam <mi...@yahoo.com.INVALID> on 2015/10/15 18:24:52 UTC
Projection pushdown with nested data type
Hi,We have data which is few level nested. If we want to read only one column from some lower level structure, does Parquet read the complete higher level structure? Or it only reads that column and assemble the structure and return.
For example, my conceptual schema is as follows struct Level1 { struct Level21{ x : int; struct level3 { ............... } } struct Level22{ ..... } }
Now if I need to read only Level.Level21.x, what will parquet do? read everything under Level1 or Level2 or only Level1.level2.x?
If yes, are there any pointers( like example, test class, JIRA or doc) around?
Thanks,Mohammad
Projection pushdown with nested data type
Posted by Mohammad Islam <mi...@yahoo.com.INVALID>.
Hi,We have data which is few level nested. If we want to read only one column from some lower level structure, does Parquet read the complete higher level structure? Or it only reads that column and assemble the structure and return.
For example, my conceptual schema is as follows struct Level1 { struct Level21{ x : int; struct level3 { ............... } } struct Level22{ ..... } }
Now if I need to read only Level.Level21.x, what will parquet do? read everything under Level1 or Level2 or only Level1.level2.x?
If yes, are there any pointers( like example, test class, JIRA or doc) around?
Thanks,Mohammad