You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/03/08 08:43:14 UTC
[GitHub] [spark] LuciferYang opened a new pull request #31776: [SPARK-34661][SQL] Replaces `OriginalType` with `LogicalTypeAnnotation` in VectorizedColumnReader
LuciferYang opened a new pull request #31776:
URL: https://github.com/apache/spark/pull/31776
### What changes were proposed in this pull request?
`OriginalType` has been marked as '@Deprecated', The main change of this pr is follow the advice of Apache Parquet to replaces `OriginalType` usage with `LogicalTypeAnnotation` in `VectorizedColumnReader`.
### Why are the changes needed?
Cleanup deprecated api usage.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass the Jenkins or GitHub Action
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31776: [WIP][SPARK-34661][SQL] Clean up deprecated `OriginalType` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-799238872
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-828348565
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42559/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31776: [WIP][SPARK-34661][SQL] Replaces `OriginalType` with `LogicalTypeAnnotation` in VectorizedColumnReader
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-792681547
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40444/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31776: [WIP][SPARK-34661][SQL] Replaces `OriginalType` with `LogicalTypeAnnotation` in VectorizedColumnReader
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-792746768
**[Test build #135861 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135861/testReport)** for PR 31776 at commit [`34c8d2e`](https://github.com/apache/spark/commit/34c8d2eb6089bfc0ae12c639d10585553e7608e3).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #31776:
URL: https://github.com/apache/spark/pull/31776#discussion_r627050278
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala
##########
@@ -105,23 +105,28 @@ class ParquetFilters(
fieldType: ParquetSchemaType)
private case class ParquetSchemaType(
- originalType: OriginalType,
+ logicalTypeAnnotation: LogicalTypeAnnotation,
primitiveTypeName: PrimitiveTypeName,
- length: Int,
- decimalMetadata: DecimalMetadata)
-
- private val ParquetBooleanType = ParquetSchemaType(null, BOOLEAN, 0, null)
- private val ParquetByteType = ParquetSchemaType(INT_8, INT32, 0, null)
- private val ParquetShortType = ParquetSchemaType(INT_16, INT32, 0, null)
- private val ParquetIntegerType = ParquetSchemaType(null, INT32, 0, null)
- private val ParquetLongType = ParquetSchemaType(null, INT64, 0, null)
- private val ParquetFloatType = ParquetSchemaType(null, FLOAT, 0, null)
- private val ParquetDoubleType = ParquetSchemaType(null, DOUBLE, 0, null)
- private val ParquetStringType = ParquetSchemaType(UTF8, BINARY, 0, null)
- private val ParquetBinaryType = ParquetSchemaType(null, BINARY, 0, null)
- private val ParquetDateType = ParquetSchemaType(DATE, INT32, 0, null)
- private val ParquetTimestampMicrosType = ParquetSchemaType(TIMESTAMP_MICROS, INT64, 0, null)
- private val ParquetTimestampMillisType = ParquetSchemaType(TIMESTAMP_MILLIS, INT64, 0, null)
+ length: Int)
Review comment:
cc @wangyum FYI
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-836281556
**[Test build #138328 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138328/testReport)** for PR 31776 at commit [`2bc0391`](https://github.com/apache/spark/commit/2bc0391a879c009f5a1b43bb5ddd1d6c78af6c22).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-827182527
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137966/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sunchao commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
sunchao commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-810346655
> > I think this is related to #31685 too
>
> @sunchao Do you know why is this PR no longer updated? thx ~
@LuciferYang I don't know. That PR is also based on another one back in 2019.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sunchao commented on a change in pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
sunchao commented on a change in pull request #31776:
URL: https://github.com/apache/spark/pull/31776#discussion_r604249720
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala
##########
@@ -105,23 +109,29 @@ class ParquetFilters(
fieldType: ParquetSchemaType)
private case class ParquetSchemaType(
- originalType: OriginalType,
+ logicalTypeAnnotation: LogicalTypeAnnotation,
primitiveTypeName: PrimitiveTypeName,
length: Int,
- decimalMetadata: DecimalMetadata)
+ decimalLogicalType: DecimalLogicalTypeAnnotation)
Review comment:
this may no longer needed since we can just use `logicalTypeAnnotation`?
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala
##########
@@ -105,23 +109,29 @@ class ParquetFilters(
fieldType: ParquetSchemaType)
private case class ParquetSchemaType(
- originalType: OriginalType,
+ logicalTypeAnnotation: LogicalTypeAnnotation,
primitiveTypeName: PrimitiveTypeName,
length: Int,
- decimalMetadata: DecimalMetadata)
+ decimalLogicalType: DecimalLogicalTypeAnnotation)
private val ParquetBooleanType = ParquetSchemaType(null, BOOLEAN, 0, null)
- private val ParquetByteType = ParquetSchemaType(INT_8, INT32, 0, null)
- private val ParquetShortType = ParquetSchemaType(INT_16, INT32, 0, null)
+ private val ParquetByteType =
+ ParquetSchemaType(LogicalTypeAnnotation.intType(8, true), INT32, 0, null)
+ private val ParquetShortType =
+ ParquetSchemaType(LogicalTypeAnnotation.intType(16, true), INT32, 0, null)
private val ParquetIntegerType = ParquetSchemaType(null, INT32, 0, null)
private val ParquetLongType = ParquetSchemaType(null, INT64, 0, null)
private val ParquetFloatType = ParquetSchemaType(null, FLOAT, 0, null)
private val ParquetDoubleType = ParquetSchemaType(null, DOUBLE, 0, null)
- private val ParquetStringType = ParquetSchemaType(UTF8, BINARY, 0, null)
+ private val ParquetStringType =
+ ParquetSchemaType(LogicalTypeAnnotation.stringType(), BINARY, 0, null)
private val ParquetBinaryType = ParquetSchemaType(null, BINARY, 0, null)
- private val ParquetDateType = ParquetSchemaType(DATE, INT32, 0, null)
- private val ParquetTimestampMicrosType = ParquetSchemaType(TIMESTAMP_MICROS, INT64, 0, null)
- private val ParquetTimestampMillisType = ParquetSchemaType(TIMESTAMP_MILLIS, INT64, 0, null)
+ private val ParquetDateType =
+ ParquetSchemaType(LogicalTypeAnnotation.dateType(), INT32, 0, null)
+ private val ParquetTimestampMicrosType =
+ ParquetSchemaType(LogicalTypeAnnotation.timestampType(true, TimeUnit.MICROS), INT64, 0, null)
+ private val ParquetTimestampMillisType =
+ ParquetSchemaType(LogicalTypeAnnotation.timestampType(true, TimeUnit.MILLIS), INT64, 0, null)
Review comment:
I think there is also `TimeUnit.NANOS` but seems Spark doesn't support it yet.
##########
File path: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedColumnReader.java
##########
@@ -894,4 +904,15 @@ private void readPageV2(DataPageV2 page) throws IOException {
throw new IOException("could not read page " + page + " in col " + descriptor, e);
}
}
+
+ private boolean isTimestampTypeMatched(TimeUnit unit) {
Review comment:
nit: these can be static methods
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-809695248
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41246/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31776: [WIP][SPARK-34661][SQL] Clean up deprecated `OriginalType` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-800006805
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136096/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-810717272
**[Test build #136745 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136745/testReport)** for PR 31776 at commit [`9a7ec8c`](https://github.com/apache/spark/commit/9a7ec8cafe80a89b45ed02affd8e8355304c17ce).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-832663431
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42694/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-836281556
**[Test build #138328 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138328/testReport)** for PR 31776 at commit [`2bc0391`](https://github.com/apache/spark/commit/2bc0391a879c009f5a1b43bb5ddd1d6c78af6c22).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen closed pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
srowen closed pull request #31776:
URL: https://github.com/apache/spark/pull/31776
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31776: [WIP][SPARK-34661][SQL] Replaces `OriginalType` with `LogicalTypeAnnotation` in VectorizedColumnReader
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-792681547
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40444/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-810529249
**[Test build #136726 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136726/testReport)** for PR 31776 at commit [`9835eeb`](https://github.com/apache/spark/commit/9835eeb332764ec220e975f3fe07329730be7d37).
* This patch **fails SparkR unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang edited a comment on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
LuciferYang edited a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-802570617
> I don't really know enough to evaluate this. It looks reasonable and tests pass. Are there any compatibility concerns, or possible changes to behavior you can think of?
https://github.com/apache/parquet-mr/blob/d96b19bb97caf6f358579c9e22626553e8dc986d/parquet-column/src/main/java/org/apache/parquet/schema/LogicalTypeAnnotation.java#L187-L245
@srowen From the above code, it seems that the types of `OriginalType` and `LogicalTypeAnnotation` correspond one by one, but I also have some doubts, although it doesn't seem to cause compatibility problems in Spark UTs now
- `OrginalType.TIME_MILLIS` is converted to `TimestampLogicalTypeAnnotation(isAdjustedToUTC = true, TimeUnit.MILLIS)` and ignores the scenario where `isAdjustedToUTC` is `false`.
- `TimestampLogicalTypeAnnotation.toOriginalType()` method will return `OriginalType.TIMESTAMP_MILLIS` when unit is TimeUnit.MILLIS and ignores values of `isAdjustedToUTC` also.
- A similar situation exists between `OriginalType.TIME_MILLIS` and `TimeLogicalTypeAnnotation`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-809695248
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41246/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31776: [WIP][SPARK-34661][SQL] Clean up deprecated api usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-800280969
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136116/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-809528551
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136665/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-810530280
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136726/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-828515614
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138040/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31776: [WIP][SPARK-34661][SQL] Clean up deprecated `OriginalType` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-799360662
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40648/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-828515614
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138040/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-810849938
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136745/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-833194844
I guess it's fine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31776: [WIP][SPARK-34661][SQL] Clean up deprecated `OriginalType` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-799360662
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40648/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-801792506
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40787/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-810849938
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136745/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-810124913
> I think this is related to #31685 too
@sunchao Do you know why is this PR no longer updated? thx ~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-827182527
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137966/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-810735102
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41327/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-827016852
**[Test build #137966 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137966/testReport)** for PR 31776 at commit [`9a7ec8c`](https://github.com/apache/spark/commit/9a7ec8cafe80a89b45ed02affd8e8355304c17ce).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
srowen commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-801891092
I don't really know enough to evaluate this. It looks reasonable and tests pass. Are there any compatibility concerns, or possible changes to behavior you can think of?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31776: [WIP][SPARK-34661][SQL] Replaces `OriginalType` with `LogicalTypeAnnotation` in VectorizedColumnReader
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-792654979
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40444/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-827167481
**[Test build #137966 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137966/testReport)** for PR 31776 at commit [`9a7ec8c`](https://github.com/apache/spark/commit/9a7ec8cafe80a89b45ed02affd8e8355304c17ce).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-828310099
**[Test build #138040 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138040/testReport)** for PR 31776 at commit [`51f75b7`](https://github.com/apache/spark/commit/51f75b7cd8ca42bb1ad64f3cfc7a34fa90864121).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31776: [WIP][SPARK-34661][SQL] Replaces `OriginalType` with `LogicalTypeAnnotation` in VectorizedColumnReader
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-792671432
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40444/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-810114356
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136700/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-832819401
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138173/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31776: [WIP][SPARK-34661][SQL] Replaces `OriginalType` with `LogicalTypeAnnotation` in VectorizedColumnReader
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-799071354
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136048/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-832819401
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138173/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31776: [WIP][SPARK-34661][SQL] Clean up deprecated `OriginalType` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-800006805
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136096/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-801792506
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40787/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-810735102
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41327/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-810318491
**[Test build #136726 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136726/testReport)** for PR 31776 at commit [`9835eeb`](https://github.com/apache/spark/commit/9835eeb332764ec220e975f3fe07329730be7d37).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31776: [WIP][SPARK-34661][SQL] Clean up deprecated api usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-800282444
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40698/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31776: [WIP][SPARK-34661][SQL] Clean up deprecated `OriginalType` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-800007660
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40679/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-810365004
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41307/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
srowen commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-841822139
Merged to master
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31776: [WIP][SPARK-34661][SQL] Replaces `OriginalType` with `LogicalTypeAnnotation` in VectorizedColumnReader
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-792604258
**[Test build #135861 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135861/testReport)** for PR 31776 at commit [`34c8d2e`](https://github.com/apache/spark/commit/34c8d2eb6089bfc0ae12c639d10585553e7608e3).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-810357123
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41307/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-810364641
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41307/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
srowen commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-839792934
Unless @wangyum has comments, I can merge to master
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on a change in pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on a change in pull request #31776:
URL: https://github.com/apache/spark/pull/31776#discussion_r604555987
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala
##########
@@ -105,23 +109,29 @@ class ParquetFilters(
fieldType: ParquetSchemaType)
private case class ParquetSchemaType(
- originalType: OriginalType,
+ logicalTypeAnnotation: LogicalTypeAnnotation,
primitiveTypeName: PrimitiveTypeName,
length: Int,
- decimalMetadata: DecimalMetadata)
+ decimalLogicalType: DecimalLogicalTypeAnnotation)
private val ParquetBooleanType = ParquetSchemaType(null, BOOLEAN, 0, null)
- private val ParquetByteType = ParquetSchemaType(INT_8, INT32, 0, null)
- private val ParquetShortType = ParquetSchemaType(INT_16, INT32, 0, null)
+ private val ParquetByteType =
+ ParquetSchemaType(LogicalTypeAnnotation.intType(8, true), INT32, 0, null)
+ private val ParquetShortType =
+ ParquetSchemaType(LogicalTypeAnnotation.intType(16, true), INT32, 0, null)
private val ParquetIntegerType = ParquetSchemaType(null, INT32, 0, null)
private val ParquetLongType = ParquetSchemaType(null, INT64, 0, null)
private val ParquetFloatType = ParquetSchemaType(null, FLOAT, 0, null)
private val ParquetDoubleType = ParquetSchemaType(null, DOUBLE, 0, null)
- private val ParquetStringType = ParquetSchemaType(UTF8, BINARY, 0, null)
+ private val ParquetStringType =
+ ParquetSchemaType(LogicalTypeAnnotation.stringType(), BINARY, 0, null)
private val ParquetBinaryType = ParquetSchemaType(null, BINARY, 0, null)
- private val ParquetDateType = ParquetSchemaType(DATE, INT32, 0, null)
- private val ParquetTimestampMicrosType = ParquetSchemaType(TIMESTAMP_MICROS, INT64, 0, null)
- private val ParquetTimestampMillisType = ParquetSchemaType(TIMESTAMP_MILLIS, INT64, 0, null)
+ private val ParquetDateType =
+ ParquetSchemaType(LogicalTypeAnnotation.dateType(), INT32, 0, null)
+ private val ParquetTimestampMicrosType =
+ ParquetSchemaType(LogicalTypeAnnotation.timestampType(true, TimeUnit.MICROS), INT64, 0, null)
+ private val ParquetTimestampMillisType =
+ ParquetSchemaType(LogicalTypeAnnotation.timestampType(true, TimeUnit.MILLIS), INT64, 0, null)
Review comment:
Yes, it seems that spark is not yet supported `TimeUnit.NANOS`,
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala
##########
@@ -105,23 +109,29 @@ class ParquetFilters(
fieldType: ParquetSchemaType)
private case class ParquetSchemaType(
- originalType: OriginalType,
+ logicalTypeAnnotation: LogicalTypeAnnotation,
primitiveTypeName: PrimitiveTypeName,
length: Int,
- decimalMetadata: DecimalMetadata)
+ decimalLogicalType: DecimalLogicalTypeAnnotation)
private val ParquetBooleanType = ParquetSchemaType(null, BOOLEAN, 0, null)
- private val ParquetByteType = ParquetSchemaType(INT_8, INT32, 0, null)
- private val ParquetShortType = ParquetSchemaType(INT_16, INT32, 0, null)
+ private val ParquetByteType =
+ ParquetSchemaType(LogicalTypeAnnotation.intType(8, true), INT32, 0, null)
+ private val ParquetShortType =
+ ParquetSchemaType(LogicalTypeAnnotation.intType(16, true), INT32, 0, null)
private val ParquetIntegerType = ParquetSchemaType(null, INT32, 0, null)
private val ParquetLongType = ParquetSchemaType(null, INT64, 0, null)
private val ParquetFloatType = ParquetSchemaType(null, FLOAT, 0, null)
private val ParquetDoubleType = ParquetSchemaType(null, DOUBLE, 0, null)
- private val ParquetStringType = ParquetSchemaType(UTF8, BINARY, 0, null)
+ private val ParquetStringType =
+ ParquetSchemaType(LogicalTypeAnnotation.stringType(), BINARY, 0, null)
private val ParquetBinaryType = ParquetSchemaType(null, BINARY, 0, null)
- private val ParquetDateType = ParquetSchemaType(DATE, INT32, 0, null)
- private val ParquetTimestampMicrosType = ParquetSchemaType(TIMESTAMP_MICROS, INT64, 0, null)
- private val ParquetTimestampMillisType = ParquetSchemaType(TIMESTAMP_MILLIS, INT64, 0, null)
+ private val ParquetDateType =
+ ParquetSchemaType(LogicalTypeAnnotation.dateType(), INT32, 0, null)
+ private val ParquetTimestampMicrosType =
+ ParquetSchemaType(LogicalTypeAnnotation.timestampType(true, TimeUnit.MICROS), INT64, 0, null)
+ private val ParquetTimestampMillisType =
+ ParquetSchemaType(LogicalTypeAnnotation.timestampType(true, TimeUnit.MILLIS), INT64, 0, null)
Review comment:
Yes, it seems that spark is not yet supported `TimeUnit.NANOS`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-810734775
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41327/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-809485471
**[Test build #136665 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136665/testReport)** for PR 31776 at commit [`6a9d8c5`](https://github.com/apache/spark/commit/6a9d8c5ec67b498b67960720a08b5ab1714e415d).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-801793829
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136205/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-836335904
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42845/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31776: [WIP][SPARK-34661][SQL] Clean up deprecated `OriginalType` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-800007660
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31776: [WIP][SPARK-34661][SQL] Replaces `OriginalType` with `LogicalTypeAnnotation` in VectorizedColumnReader
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-799083658
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40631/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31776: [WIP][SPARK-34661][SQL] Clean up deprecated `OriginalType` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-799238872
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40641/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-810530280
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136726/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-832817814
**[Test build #138173 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138173/testReport)** for PR 31776 at commit [`f91b670`](https://github.com/apache/spark/commit/f91b6709f77b3c19a09069103c6382a4295720d9).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
* `case class ResolveHigherOrderFunctions(catalogManager: CatalogManager)`
* `case class ApplyFunctionExpression(`
* `case class V2Aggregator[BUF <: java.io.Serializable, OUT](`
* `trait ExtractValue extends Expression `
* ` implicit class FunctionIdentifierHelper(ident: FunctionIdentifier) `
* `case class AddJarsCommand(paths: Seq[String]) extends LeafRunnableCommand `
* `case class AddFilesCommand(paths: Seq[String]) extends LeafRunnableCommand `
* `case class AddArchivesCommand(paths: Seq[String]) extends LeafRunnableCommand `
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-800757254
cc @dongjoon-hyun @HyukjinKwon @srowen @maropu can you help to review this pr? thx ~
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31776: [WIP][SPARK-34661][SQL] Clean up deprecated api usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-800282444
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40698/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-802570617
> I don't really know enough to evaluate this. It looks reasonable and tests pass. Are there any compatibility concerns, or possible changes to behavior you can think of?
https://github.com/apache/parquet-mr/blob/d96b19bb97caf6f358579c9e22626553e8dc986d/parquet-column/src/main/java/org/apache/parquet/schema/LogicalTypeAnnotation.java#L187-L245
From the above code, it seems that the types of `OriginalType` and `LogicalTypeAnnotation` correspond one by one, but I also have some doubts, although it doesn't seem to cause compatibility problems
- `OrginalType.TIME_MILLIS` is converted to `TimestampLogicalTypeAnnotation(isAdjustedToUTC = true, TimeUnit.MILLIS)` and ignores the scenario where `isAdjustedToUTC` is `false`.
- `TimestampLogicalTypeAnnotation.toOriginalType()` method will return `OriginalType.TIMESTAMP_MILLIS` when unit is TimeUnit.MILLIS and ignores values of `isAdjustedToUTC` also.
- A similar situation exists between `OriginalType.TIME_MILLIS` and `TimeLogicalTypeAnnotation`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-836329378
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42845/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31776: [WIP][SPARK-34661][SQL] Clean up deprecated `OriginalType` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-799420771
Kubernetes integration test unable to build dist.
exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40649/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-809670885
Kubernetes integration test unable to build dist.
exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41246/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-836622574
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138328/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-810733141
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41327/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sunchao commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
sunchao commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-809554649
I think this is related to #31685 too
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31776: [WIP][SPARK-34661][SQL] Clean up deprecated `OriginalType` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-799155670
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31776: [WIP][SPARK-34661][SQL] Replaces `OriginalType` with `LogicalTypeAnnotation` in VectorizedColumnReader
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-792747928
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135861/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-809485471
**[Test build #136665 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136665/testReport)** for PR 31776 at commit [`6a9d8c5`](https://github.com/apache/spark/commit/6a9d8c5ec67b498b67960720a08b5ab1714e415d).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-809499947
**[Test build #136665 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136665/testReport)** for PR 31776 at commit [`6a9d8c5`](https://github.com/apache/spark/commit/6a9d8c5ec67b498b67960720a08b5ab1714e415d).
* This patch **fails to build**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-810365004
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41307/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-810114356
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136700/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-810032074
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41283/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-811642573
Thanks ~ @sunchao
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-809990853
**[Test build #136700 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136700/testReport)** for PR 31776 at commit [`66715c9`](https://github.com/apache/spark/commit/66715c92b1555afc9fdfe650f75c0e6fa13f88b6).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31776: [WIP][SPARK-34661][SQL] Clean up deprecated `OriginalType` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-800079794
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40688/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31776: [WIP][SPARK-34661][SQL] Clean up deprecated `OriginalType` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-799240161
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136058/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31776: [WIP][SPARK-34661][SQL] Replaces `OriginalType` with `LogicalTypeAnnotation` in VectorizedColumnReader
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-792604258
**[Test build #135861 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135861/testReport)** for PR 31776 at commit [`34c8d2e`](https://github.com/apache/spark/commit/34c8d2eb6089bfc0ae12c639d10585553e7608e3).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-809528551
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136665/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-828310099
**[Test build #138040 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138040/testReport)** for PR 31776 at commit [`51f75b7`](https://github.com/apache/spark/commit/51f75b7cd8ca42bb1ad64f3cfc7a34fa90864121).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-810095773
**[Test build #136700 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136700/testReport)** for PR 31776 at commit [`66715c9`](https://github.com/apache/spark/commit/66715c92b1555afc9fdfe650f75c0e6fa13f88b6).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-828348520
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-841931849
thx ~ @srowen @HyukjinKwon @sunchao
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
srowen commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-841822139
Merged to master
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-832658703
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-810023714
Kubernetes integration test unable to build dist.
exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41283/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-810717272
**[Test build #136745 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136745/testReport)** for PR 31776 at commit [`9a7ec8c`](https://github.com/apache/spark/commit/9a7ec8cafe80a89b45ed02affd8e8355304c17ce).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-836593649
**[Test build #138328 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138328/testReport)** for PR 31776 at commit [`2bc0391`](https://github.com/apache/spark/commit/2bc0391a879c009f5a1b43bb5ddd1d6c78af6c22).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] sunchao commented on a change in pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
sunchao commented on a change in pull request #31776:
URL: https://github.com/apache/spark/pull/31776#discussion_r605063241
##########
File path: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedColumnReader.java
##########
@@ -894,4 +904,15 @@ private void readPageV2(DataPageV2 page) throws IOException {
throw new IOException("could not read page " + page + " in col " + descriptor, e);
}
}
+
+ private boolean isTimestampTypeMatched(TimeUnit unit) {
Review comment:
I mean we can just change `private boolean` to `private static boolean` since they are not depending on any of the state from the class, but it's a really minor thing :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31776: [WIP][SPARK-34661][SQL] Replaces `OriginalType` with `LogicalTypeAnnotation` in VectorizedColumnReader
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-792747928
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135861/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on a change in pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on a change in pull request #31776:
URL: https://github.com/apache/spark/pull/31776#discussion_r604557010
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala
##########
@@ -105,23 +109,29 @@ class ParquetFilters(
fieldType: ParquetSchemaType)
private case class ParquetSchemaType(
- originalType: OriginalType,
+ logicalTypeAnnotation: LogicalTypeAnnotation,
primitiveTypeName: PrimitiveTypeName,
length: Int,
- decimalMetadata: DecimalMetadata)
+ decimalLogicalType: DecimalLogicalTypeAnnotation)
Review comment:
9a7ec8c try to remove this from `ParquetSchemaType`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31776: [WIP][SPARK-34661][SQL] Clean up deprecated `OriginalType` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-799359679
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136065/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31776: [WIP][SPARK-34661][SQL] Clean up deprecated `OriginalType` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-799426707
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang edited a comment on pull request #31776: [WIP][SPARK-34661][SQL] Clean up deprecated `OriginalType` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
LuciferYang edited a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-799061972
Will try to cleanup all deprecated `OriginalType` usage related parquet code in one pr, so change this pr to draft first.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-832663431
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42694/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31776: [WIP][SPARK-34661][SQL] Clean up deprecated api usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-800280969
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136116/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on a change in pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on a change in pull request #31776:
URL: https://github.com/apache/spark/pull/31776#discussion_r604552556
##########
File path: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedColumnReader.java
##########
@@ -894,4 +904,15 @@ private void readPageV2(DataPageV2 page) throws IOException {
throw new IOException("could not read page " + page + " in col " + descriptor, e);
}
}
+
+ private boolean isTimestampTypeMatched(TimeUnit unit) {
Review comment:
Should we extract these methods into util class?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-810829057
**[Test build #136745 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136745/testReport)** for PR 31776 at commit [`9a7ec8c`](https://github.com/apache/spark/commit/9a7ec8cafe80a89b45ed02affd8e8355304c17ce).
* This patch **fails PySpark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-801793829
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136205/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31776: [WIP][SPARK-34661][SQL] Clean up deprecated `OriginalType` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-800078679
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136106/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-827016852
**[Test build #137966 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137966/testReport)** for PR 31776 at commit [`9a7ec8c`](https://github.com/apache/spark/commit/9a7ec8cafe80a89b45ed02affd8e8355304c17ce).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang edited a comment on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
LuciferYang edited a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-802570617
> I don't really know enough to evaluate this. It looks reasonable and tests pass. Are there any compatibility concerns, or possible changes to behavior you can think of?
https://github.com/apache/parquet-mr/blob/d96b19bb97caf6f358579c9e22626553e8dc986d/parquet-column/src/main/java/org/apache/parquet/schema/LogicalTypeAnnotation.java#L187-L245
@srowen From the above code, it seems that the types of `OriginalType` and `LogicalTypeAnnotation` correspond one by one, but I also have some doubts, although it doesn't seem to cause compatibility problems
- `OrginalType.TIME_MILLIS` is converted to `TimestampLogicalTypeAnnotation(isAdjustedToUTC = true, TimeUnit.MILLIS)` and ignores the scenario where `isAdjustedToUTC` is `false`.
- `TimestampLogicalTypeAnnotation.toOriginalType()` method will return `OriginalType.TIMESTAMP_MILLIS` when unit is TimeUnit.MILLIS and ignores values of `isAdjustedToUTC` also.
- A similar situation exists between `OriginalType.TIME_MILLIS` and `TimeLogicalTypeAnnotation`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31776: [WIP][SPARK-34661][SQL] Clean up deprecated `OriginalType` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-799359679
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136065/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-809990853
**[Test build #136700 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136700/testReport)** for PR 31776 at commit [`66715c9`](https://github.com/apache/spark/commit/66715c92b1555afc9fdfe650f75c0e6fa13f88b6).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-841931849
thx ~ @srowen @HyukjinKwon @sunchao
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31776: [WIP][SPARK-34661][SQL] Clean up deprecated `OriginalType` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-799426706
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-832628468
**[Test build #138173 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138173/testReport)** for PR 31776 at commit [`f91b670`](https://github.com/apache/spark/commit/f91b6709f77b3c19a09069103c6382a4295720d9).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-810318491
**[Test build #136726 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136726/testReport)** for PR 31776 at commit [`9835eeb`](https://github.com/apache/spark/commit/9835eeb332764ec220e975f3fe07329730be7d37).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-836622574
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138328/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-812272546
Gentle ping, @wangyum @HyukjinKwon @dongjoon-hyun @maropu
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-836344080
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42845/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen closed pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
srowen closed pull request #31776:
URL: https://github.com/apache/spark/pull/31776
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-836344080
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42845/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-828348565
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42559/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-828513726
**[Test build #138040 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138040/testReport)** for PR 31776 at commit [`51f75b7`](https://github.com/apache/spark/commit/51f75b7cd8ca42bb1ad64f3cfc7a34fa90864121).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-810032074
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41283/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31776: [SPARK-34661][SQL] Clean up `OriginalType` and `DecimalMetadata ` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-832628468
**[Test build #138173 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138173/testReport)** for PR 31776 at commit [`f91b670`](https://github.com/apache/spark/commit/f91b6709f77b3c19a09069103c6382a4295720d9).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31776: [WIP][SPARK-34661][SQL] Clean up deprecated `OriginalType` usage in Parquet related code
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-799155670
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #31776: [WIP][SPARK-34661][SQL] Replaces `OriginalType` with `LogicalTypeAnnotation` in VectorizedColumnReader
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-799061972
Will try to cleanup all these deprecated api usage related parquet code in one pr, so change this pr to draft first.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31776: [WIP][SPARK-34661][SQL] Replaces `OriginalType` with `LogicalTypeAnnotation` in VectorizedColumnReader
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #31776:
URL: https://github.com/apache/spark/pull/31776#issuecomment-799071354
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org