Re: [I] incorporate fields later for projectable table scans [wayang]

["AjayBoddeda4 (via GitHub)" <gi...@apache.org>]

AjayBoddeda4 commented on issue #297:
URL: https://github.com/apache/wayang/issues/297#issuecomment-4131212704

   Hi, I am Ajay Boddeda, a GSoC 2026 applicant working on the DataFrames API proposal for Apache Wayang.
   This TODO is directly relevant to my proposal. The fields parameter in JavaCSVTableSource represents column projection — which is exactly what the select() operation in the DataFrame API needs to implement. Currently all columns are read even when only a subset is needed, which is inefficient.
   When the DataFrame API executes a df.select('name', 'age') operation, it should push the field projection down to the source level so that only the required columns are read from the CSV file. Incorporating the ImmutableIntList fields properly would enable this projection pushdown optimization.
   This is a key performance feature for the DataFrame API and I would love to investigate this further as part of my GSoC project.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@wayang.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org