You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "aokolnychyi (via GitHub)" <gi...@apache.org> on 2023/09/15 16:41:06 UTC
[GitHub] [iceberg] aokolnychyi opened a new pull request, #8568: Arrow: Propagate correct field info while reading metadata columns
aokolnychyi opened a new pull request, #8568:
URL: https://github.com/apache/iceberg/pull/8568
As a follow-up to #8466, this PR propagates correct field info while reading metadata columns.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] advancedxy commented on a diff in pull request #8568: Arrow: Propagate correct field info while reading metadata columns
Posted by "advancedxy (via GitHub)" <gi...@apache.org>.
advancedxy commented on code in PR #8568:
URL: https://github.com/apache/iceberg/pull/8568#discussion_r1328717392
##########
arrow/src/main/java/org/apache/iceberg/arrow/vectorized/VectorizedArrowReader.java:
##########
@@ -605,7 +606,9 @@ public void setBatchSize(int batchSize) {}
* Holder which indicates whether a given row is deleted.
*/
public static class DeletedVectorReader extends VectorizedArrowReader {
- public DeletedVectorReader() {}
+ public DeletedVectorReader() {
Review Comment:
do you think it's worth to add a unit test for this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] aokolnychyi commented on a diff in pull request #8568: Arrow: Propagate correct field info while reading metadata columns
Posted by "aokolnychyi (via GitHub)" <gi...@apache.org>.
aokolnychyi commented on code in PR #8568:
URL: https://github.com/apache/iceberg/pull/8568#discussion_r1329248037
##########
arrow/src/main/java/org/apache/iceberg/arrow/vectorized/VectorizedArrowReader.java:
##########
@@ -605,7 +606,9 @@ public void setBatchSize(int batchSize) {}
* Holder which indicates whether a given row is deleted.
*/
public static class DeletedVectorReader extends VectorizedArrowReader {
- public DeletedVectorReader() {}
+ public DeletedVectorReader() {
Review Comment:
Overall, it is nice to have and is not strictly required. It is being tested mostly via Spark so adding a brand-new test suite would not be worth the effort, probably.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] aokolnychyi merged pull request #8568: Arrow: Propagate correct field info while reading metadata columns
Posted by "aokolnychyi (via GitHub)" <gi...@apache.org>.
aokolnychyi merged PR #8568:
URL: https://github.com/apache/iceberg/pull/8568
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] aokolnychyi commented on pull request #8568: Arrow: Propagate correct field info while reading metadata columns
Posted by "aokolnychyi (via GitHub)" <gi...@apache.org>.
aokolnychyi commented on PR #8568:
URL: https://github.com/apache/iceberg/pull/8568#issuecomment-1724324992
This does not break any tests because our Spark readers are mindful of these special vector holder types. This change is for consistency and to avoid NPE in custom implementations.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] aokolnychyi commented on a diff in pull request #8568: Arrow: Propagate correct field info while reading metadata columns
Posted by "aokolnychyi (via GitHub)" <gi...@apache.org>.
aokolnychyi commented on code in PR #8568:
URL: https://github.com/apache/iceberg/pull/8568#discussion_r1329248037
##########
arrow/src/main/java/org/apache/iceberg/arrow/vectorized/VectorizedArrowReader.java:
##########
@@ -605,7 +606,9 @@ public void setBatchSize(int batchSize) {}
* Holder which indicates whether a given row is deleted.
*/
public static class DeletedVectorReader extends VectorizedArrowReader {
- public DeletedVectorReader() {}
+ public DeletedVectorReader() {
Review Comment:
Overall, it is nice to have and is not strictly required. It is being tested mostly via Spark so adding a dedicated test would not be worth the effort, probably.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
[GitHub] [iceberg] advancedxy commented on a diff in pull request #8568: Arrow: Propagate correct field info while reading metadata columns
Posted by "advancedxy (via GitHub)" <gi...@apache.org>.
advancedxy commented on code in PR #8568:
URL: https://github.com/apache/iceberg/pull/8568#discussion_r1329483873
##########
arrow/src/main/java/org/apache/iceberg/arrow/vectorized/VectorizedArrowReader.java:
##########
@@ -605,7 +606,9 @@ public void setBatchSize(int batchSize) {}
* Holder which indicates whether a given row is deleted.
*/
public static class DeletedVectorReader extends VectorizedArrowReader {
- public DeletedVectorReader() {}
+ public DeletedVectorReader() {
Review Comment:
👌
##########
arrow/src/main/java/org/apache/iceberg/arrow/vectorized/VectorizedArrowReader.java:
##########
@@ -605,7 +606,9 @@ public void setBatchSize(int batchSize) {}
* Holder which indicates whether a given row is deleted.
*/
public static class DeletedVectorReader extends VectorizedArrowReader {
- public DeletedVectorReader() {}
+ public DeletedVectorReader() {
Review Comment:
👌
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org