You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "lgingerich (via GitHub)" <gi...@apache.org> on 2025/12/14 17:49:46 UTC

[PR] feat: support ArrowReaderOptions [iceberg-rust]

lgingerich opened a new pull request, #1935:
URL: https://github.com/apache/iceberg-rust/pull/1935

   ## Which issue does this PR close?
   
   - Closes #1934 .
   
   ## What changes are included in this PR?
   
   - Add support for `ArrowReaderOptions`
   - This PR does not actually hook up `ArrowReaderOptions` to be used anywhere as `get_metadata` is currently only called from `parquet_files_to_data_files()` which is marked as dead_code.
      - Given this, it's essentially a no-op change, though fixes a minor issue for future integration with `ArrowFileReader`.
   
   ## Are these changes tested?
   
   Tested with existing test suite.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org

Re: [PR] feat: support ArrowReaderOptions [iceberg-rust]

Posted by "lgingerich (via GitHub)" <gi...@apache.org>.
lgingerich commented on PR #1935:
URL: https://github.com/apache/iceberg-rust/pull/1935#issuecomment-3651845665

   I've confirmed that the failing CI / Unit Tests passes for me locally (MacOS)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org

Re: [PR] feat: support ArrowReaderOptions [iceberg-rust]

Posted by "liurenjie1024 (via GitHub)" <gi...@apache.org>.
liurenjie1024 commented on code in PR #1935:
URL: https://github.com/apache/iceberg-rust/pull/1935#discussion_r2618692237


##########
Cargo.toml:
##########
@@ -101,7 +101,7 @@ num-bigint = "0.4.6"
 once_cell = "1.20"
 opendal = "0.55.0"
 ordered-float = "4"
-parquet = "57.0"
+parquet = "57.1.0"

Review Comment:
   Why we need to change this? We expect to keep parquet version same as arrow.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org

Re: [PR] feat: support ArrowReaderOptions [iceberg-rust]

Posted by "lgingerich (via GitHub)" <gi...@apache.org>.
lgingerich commented on code in PR #1935:
URL: https://github.com/apache/iceberg-rust/pull/1935#discussion_r2621195196


##########
Cargo.toml:
##########
@@ -101,7 +101,7 @@ num-bigint = "0.4.6"
 once_cell = "1.20"
 opendal = "0.55.0"
 ordered-float = "4"
-parquet = "57.0"
+parquet = "57.1.0"

Review Comment:
   I didn't realize it initially, but the necessary `ParquetMetaDataReader` method `with_metadata_options` did not get added until 57.1.0. You can see the initial clippy failure here: https://github.com/apache/iceberg-rust/actions/runs/20211830111/job/58018793697#step:4:854
   
   Given this, I am fine if you'd prefer to just close this PR for now. I assume upgrading Arrow also isn't a preferred solution for such a minor PR.
   
   Also, sorry for forgetting to update the PR comment with these details!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org

Re: [PR] feat: support ArrowReaderOptions [iceberg-rust]

Posted by "lgingerich (via GitHub)" <gi...@apache.org>.
lgingerich commented on code in PR #1935:
URL: https://github.com/apache/iceberg-rust/pull/1935#discussion_r2623172629


##########
Cargo.toml:
##########
@@ -101,7 +101,7 @@ num-bigint = "0.4.6"
 once_cell = "1.20"
 opendal = "0.55.0"
 ordered-float = "4"
-parquet = "57.0"
+parquet = "57.1.0"

Review Comment:
   Sounds good, will close this.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org

Re: [PR] feat: support ArrowReaderOptions [iceberg-rust]

Posted by "liurenjie1024 (via GitHub)" <gi...@apache.org>.
liurenjie1024 commented on code in PR #1935:
URL: https://github.com/apache/iceberg-rust/pull/1935#discussion_r2622759147


##########
Cargo.toml:
##########
@@ -101,7 +101,7 @@ num-bigint = "0.4.6"
 once_cell = "1.20"
 opendal = "0.55.0"
 ordered-float = "4"
-parquet = "57.0"
+parquet = "57.1.0"

Review Comment:
   Thanks @lgingerich , I prefer to wait for next round upgrading of arrow to add this change.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org

Re: [PR] feat: support ArrowReaderOptions [iceberg-rust]

Posted by "lgingerich (via GitHub)" <gi...@apache.org>.
lgingerich closed pull request #1935: feat: support ArrowReaderOptions
URL: https://github.com/apache/iceberg-rust/pull/1935


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org