[DISCUSS] Implementing Parquet read optimisations

["Suhail, Ahmar" <ah...@amazon.co.uk>]

Hey everyone,

Last month I started a thread about creating a new Apache project which would implement read optimisations for Parquet (eg: readVectored, prefetching and caching the footer) in a single place, so we don’t need to duplicate code for this across cloud providers. This is original email thread: https://lists.apache.org/thread/nbksq32cs8h1ldj8762y6wh9zzp8gqx6

There was some consensus there that the Parquet java would be the best place to start.  Before we start looking at code changes, I wanted to briefly outline the motivation and the changes required to the project. I have outlined a more detailed plan in this doc: https://docs.google.com/document/d/1Xdlh23tmCs-KvzHhY2RuwFYmc3xntUKcmwb8yxEl78Y/edit?usp=sharing


Would be great to get your thoughts on this. I’ll also join the Parquet sync tomorrow and hope to discuss this there.

Thanks,
Ahmar

Re: [DISCUSS] Implementing Parquet read optimisations

["Suhail, Ahmar" <ah...@amazon.co.uk>]

Hey Micah, 

I've just enabled the comments on the doc, link here. 

I haven't received any feedback on the doc, but I did bring this up in the parquet sync and early Jan where there was alignment that the parquet-java project would be ok to accept PR's for these changes. Before I submit any code though, it would be great to get some opinions on the architecture.

Thank you!

On 01/02/2026, 21:36, "Micah Kornfield" <emkornfield@gmail.com <ma...@gmail.com>> wrote:


CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.






Hi Ahmar,
Sorry for the late reply. Two questions:
1. The doc doesn't seem to have comment access enabled.
2. Have you received any feedback on this? Do you still want to proceed?


Thanks,
Micah


On Tue, Jan 6, 2026 at 6:17 AM Suhail, Ahmar <ahmarsu@amazon.co.uk <ma...@amazon.co.uk>> wrote:


> Hey everyone,
>
> Last month I started a thread about creating a new Apache project which
> would implement read optimisations for Parquet (eg: readVectored,
> prefetching and caching the footer) in a single place, so we don’t need to
> duplicate code for this across cloud providers. This is original email
> thread: https://lists.apache.org/thread/nbksq32cs8h1ldj8762y6wh9zzp8gqx6 <https://lists.apache.org/thread/nbksq32cs8h1ldj8762y6wh9zzp8gqx6>
>
> There was some consensus there that the Parquet java would be the best
> place to start. Before we start looking at code changes, I wanted to
> briefly outline the motivation and the changes required to the project. I
> have outlined a more detailed plan in this doc:
> https://docs.google.com/document/d/1Xdlh23tmCs-KvzHhY2RuwFYmc3xntUKcmwb8yxEl78Y/edit?usp=sharing <https://docs.google.com/document/d/1Xdlh23tmCs-KvzHhY2RuwFYmc3xntUKcmwb8yxEl78Y/edit?usp=sharing>
>
>
> Would be great to get your thoughts on this. I’ll also join the Parquet
> sync tomorrow and hope to discuss this there.
>
> Thanks,
> Ahmar
>
>

Re: [DISCUSS] Implementing Parquet read optimisations

[Micah Kornfield <em...@gmail.com>]

Hi Ahmar,
Sorry for the late reply.  Two questions:
1.  The doc doesn't seem to have comment access enabled.
2.  Have you received any feedback on this?  Do you still want to proceed?

Thanks,
Micah

On Tue, Jan 6, 2026 at 6:17 AM Suhail, Ahmar <ah...@amazon.co.uk> wrote:

> Hey everyone,
>
> Last month I started a thread about creating a new Apache project which
> would implement read optimisations for Parquet (eg: readVectored,
> prefetching and caching the footer) in a single place, so we don’t need to
> duplicate code for this across cloud providers. This is original email
> thread: https://lists.apache.org/thread/nbksq32cs8h1ldj8762y6wh9zzp8gqx6
>
> There was some consensus there that the Parquet java would be the best
> place to start.  Before we start looking at code changes, I wanted to
> briefly outline the motivation and the changes required to the project. I
> have outlined a more detailed plan in this doc:
> https://docs.google.com/document/d/1Xdlh23tmCs-KvzHhY2RuwFYmc3xntUKcmwb8yxEl78Y/edit?usp=sharing
>
>
> Would be great to get your thoughts on this. I’ll also join the Parquet
> sync tomorrow and hope to discuss this there.
>
> Thanks,
> Ahmar
>
>