You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@hudi.apache.org by "vbalaji@apache.org" <vb...@apache.org> on 2019/05/01 19:23:00 UTC

[DISCUSS] HIP 3: Timeline Service with Incremental File System View Syncing

Hi All,

I created a HIP for file-system view caching and metadata management.
https://cwiki.apache.org/confluence/display/HUDI/HIP-3

Please review and let me know your comments.

Thanks,Balaji.V

Re: [DISCUSS] HIP 3: Timeline Service with Incremental File System View Syncing

Posted by Vinoth Chandar <vi...@apache.org>.

Left some comments. Asking here since its more discoverable. Have you
thought about also storing key range info for each file group in the
timeline datastore? This will be very handy for indexing and could speed
things up since we need not open 100s of parquet file  footers?

On Wed, May 1, 2019 at 12:23 PM vbalaji@apache.org <vb...@apache.org>
wrote:

> Hi All,
>
> I created a HIP for file-system view caching and metadata management.
> https://cwiki.apache.org/confluence/display/HUDI/HIP-3
>
> Please review and let me know your comments.
>
> Thanks,Balaji.V