You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/09/06 11:24:30 UTC
[GitHub] [spark] cxzl25 opened a new pull request, #37808: [SPARK-39830][SQL][TESTS][3.3] Add a test case to read ORC table that requires type promotion
cxzl25 opened a new pull request, #37808:
URL: https://github.com/apache/spark/pull/37808
### What changes were proposed in this pull request?
Increase ORC test coverage.
[ORC-1205](https://issues.apache.org/jira/browse/ORC-1205) Size of batches in some ConvertTreeReaders should be ensured before using
### Why are the changes needed?
When spark reads an orc with type promotion, an `ArrayIndexOutOfBoundsException` may be thrown, which has been fixed in version 1.7.6 and 1.8.0.
```java
java.lang.ArrayIndexOutOfBoundsException: 1
at org.apache.orc.impl.TreeReaderFactory$TreeReader.nextVector(TreeReaderFactory.java:387)
at org.apache.orc.impl.TreeReaderFactory$LongTreeReader.nextVector(TreeReaderFactory.java:740)
at org.apache.orc.impl.ConvertTreeReaderFactory$StringGroupFromAnyIntegerTreeReader.nextVector(ConvertTreeReaderFactory.java:1069)
at org.apache.orc.impl.reader.tree.StructBatchReader.readBatchColumn(StructBatchReader.java:65)
```
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
add UT
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun closed pull request #37808: [SPARK-39830][SQL][TESTS][3.3] Add a test case to read ORC table that requires type promotion
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun closed pull request #37808: [SPARK-39830][SQL][TESTS][3.3] Add a test case to read ORC table that requires type promotion
URL: https://github.com/apache/spark/pull/37808
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #37808: [SPARK-39830][SQL][TESTS][3.3] Add a test case to read ORC table that requires type promotion
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on PR #37808:
URL: https://github.com/apache/spark/pull/37808#issuecomment-1239235817
Can one of the admins verify this patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] cxzl25 commented on pull request #37808: [SPARK-39830][SQL][TESTS][3.3] Add a test case to read ORC table that requires type promotion
Posted by GitBox <gi...@apache.org>.
cxzl25 commented on PR #37808:
URL: https://github.com/apache/spark/pull/37808#issuecomment-1238021583
https://github.com/apache/spark/pull/37800
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org