You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "andygrove (via GitHub)" <gi...@apache.org> on 2026/05/13 03:39:03 UTC
[PR] feat(parquet): expose ParquetReadOptions via registerParquet and readParquet [datafusion-java]
andygrove opened a new pull request, #18:
URL: https://github.com/apache/datafusion-java/pull/18
Closes #17.
Exposes DataFusion's `ParquetReadOptions` on the Java surface and adds `SessionContext.readParquet(...)`.
## What's in this PR
- `org.apache.datafusion.ParquetReadOptions` — mutable fluent class with five setters: `fileExtension`, `parquetPruning`, `skipMetadata`, `metadataSizeHint`, `schema` (Arrow Java `Schema`).
- `SessionContext.registerParquet(name, path, options)` — overload of the existing method. Existing `registerParquet(name, path)` is preserved (delegates to the overload with default options).
- `SessionContext.readParquet(path[, options]) -> DataFrame` — analog of Rust's `ctx.read_parquet`, returns a DataFrame without registering.
- Schema is marshalled across JNI as Arrow IPC stream bytes (reuses the IPC mechanism added in #13's `tableSchema`).
- 7 new tests: 3 unit tests for the options class, 4 integration tests (default options row count, custom file extension, explicit schema, metadata size hint).
## Not in this PR
- `table_partition_cols` (Hive-style partition columns) — requires per-column ArrowType marshalling.
- `file_sort_order` — requires logical-Expr serialization.
- `file_decryption_properties` — encryption; niche.
- A static-factory `builder()` — fluent setters suffice.
- CSV/JSON/Avro analog option classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat(parquet): expose ParquetReadOptions via registerParquet and readParquet [datafusion-java]
Posted by "andygrove (via GitHub)" <gi...@apache.org>.
andygrove merged PR #18:
URL: https://github.com/apache/datafusion-java/pull/18
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org