You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/06/04 14:37:27 UTC
[GitHub] [spark] eejbyfeldt opened a new pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
eejbyfeldt opened a new pull request #32783:
URL: https://github.com/apache/spark/pull/32783
### What changes were proposed in this pull request?
Use the key/valueLambdaFunction to convert the elements instead of
using CatalystTypeConverters.createToScalaConverter. This is how it is
done in MapObjects and that correctly handles Arrays with case classes.
### Why are the changes needed?
Before these changes the added test cases would fail with the following:
```
[info] - encode/decode for map with case class as value: Map(1 -> IntAndString(1,a)) (interpreted path) *** FAILED *** (64 milliseconds)
[info] Encoded/Decoded data does not match input data
[info]
[info] in: Map(1 -> IntAndString(1,a))
[info] out: Map(1 -> [1,a])
[info] types: scala.collection.immutable.Map$Map1 [info]
[info] Encoded Data: [org.apache.spark.sql.catalyst.expressions.UnsafeMapData@5ecf5d9e]
[info] Schema: value#823
[info] root
[info] -- value: map (nullable = true)
[info] |-- key: integer
[info] |-- value: struct (valueContainsNull = true)
[info] | |-- i: integer (nullable = false)
[info] | |-- s: string (nullable = true)
[info]
[info]
[info] fromRow Expressions:
[info] catalysttoexternalmap(lambdavariable(CatalystToExternalMap_key, IntegerType, false, 178), lambdavariable(CatalystToExternalMap_key, IntegerType, false, 178), lambdavariable(CatalystToExternalMap_value, StructField(i,IntegerType,false), StructField(s,StringType,true), true, 179), if (isnull(lambdavariable(CatalystToExternalMap_value, StructField(i,IntegerType,false), StructField(s,StringType,true), true, 179))) null else newInstance(class org.apache.spark.sql.catalyst.encoders.IntAndString), input[0, map<int,struct<i:int,s:string>>, true], interface scala.collection.immutable.Map
[info] :- lambdavariable(CatalystToExternalMap_key, IntegerType, false, 178)
[info] :- lambdavariable(CatalystToExternalMap_key, IntegerType, false, 178)
[info] :- lambdavariable(CatalystToExternalMap_value, StructField(i,IntegerType,false), StructField(s,StringType,true), true, 179)
[info] :- if (isnull(lambdavariable(CatalystToExternalMap_value, StructField(i,IntegerType,false), StructField(s,StringType,true), true, 179))) null else newInstance(class org.apache.spark.sql.catalyst.encoders.IntAndString)
[info] : :- isnull(lambdavariable(CatalystToExternalMap_value, StructField(i,IntegerType,false), StructField(s,StringType,true), true, 179))
[info] : : +- lambdavariable(CatalystToExternalMap_value, StructField(i,IntegerType,false), StructField(s,StringType,true), true, 179)
[info] : :- null
[info] : +- newInstance(class org.apache.spark.sql.catalyst.encoders.IntAndString)
[info] : :- assertnotnull(lambdavariable(CatalystToExternalMap_value, StructField(i,IntegerType,false), StructField(s,StringType,true), true, 179).i)
[info] : : +- lambdavariable(CatalystToExternalMap_value, StructField(i,IntegerType,false), StructField(s,StringType,true), true, 179).i
[info] : : +- lambdavariable(CatalystToExternalMap_value, StructField(i,IntegerType,false), StructField(s,StringType,true), true, 179)
[info] : +- lambdavariable(CatalystToExternalMap_value, StructField(i,IntegerType,false), StructField(s,StringType,true), true, 179).s.toString
[info] : +- lambdavariable(CatalystToExternalMap_value, StructField(i,IntegerType,false), StructField(s,StringType,true), true, 179).s
[info] : +- lambdavariable(CatalystToExternalMap_value, StructField(i,IntegerType,false), StructField(s,StringType,true), true, 179)
[info] +- input[0, map<int,struct<i:int,s:string>>, true] (ExpressionEncoderSuite.scala:627)
```
So using a map with cases classes for keys or values and using the interpreted path would incorrect deserialize data from the catalyst representation.
### Does this PR introduce _any_ user-facing change?
Yes, it fixes the bug.
### How was this patch tested?
Existing and new unit tests in the ExpressionEncoderSuite
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-856527878
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43979/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
maropu commented on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-859152984
Thank you, all~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
maropu commented on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-856440404
cc: @viirya
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-855091656
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43874/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-857300522
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139513/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] eejbyfeldt commented on a change in pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
eejbyfeldt commented on a change in pull request #32783:
URL: https://github.com/apache/spark/pull/32783#discussion_r647149051
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala
##########
@@ -1181,21 +1176,25 @@ case class CatalystToExternalMap private(
newMapBuilderMethod.invoke(moduleField).asInstanceOf[Builder[AnyRef, AnyRef]]
}
+ private def keyValueIterator(md: MapData): Iterator[AnyRef] = {
+ val keyArray = md.keyArray()
+ val valueArray = md.valueArray()
+ val row = new GenericInternalRow(1)
+ 0.until(md.numElements()).iterator.map { i =>
+ row.update(0, keyArray.get(i, inputMapType.keyType))
+ val key = keyLambdaFunction.eval(row)
+ row.update(0, valueArray.get(i, inputMapType.valueType))
+ val value = valueLambdaFunction.eval(row)
+ Tuple2(key, value)
+ }
+ }
+
override def eval(input: InternalRow): Any = {
val result = inputData.eval(input).asInstanceOf[MapData]
if (result != null) {
val builder = newMapBuilder()
builder.sizeHint(result.numElements())
- val keyArray = result.keyArray()
- val valueArray = result.valueArray()
- var i = 0
- while (i < result.numElements()) {
Review comment:
I guess I am not sure whether it does or not. But I updated the PR to use a while instead, to be sure. Now the style is also more similar to what was there before.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #32783:
URL: https://github.com/apache/spark/pull/32783#discussion_r647110803
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala
##########
@@ -1181,21 +1176,25 @@ case class CatalystToExternalMap private(
newMapBuilderMethod.invoke(moduleField).asInstanceOf[Builder[AnyRef, AnyRef]]
}
+ private def keyValueIterator(md: MapData): Iterator[AnyRef] = {
+ val keyArray = md.keyArray()
+ val valueArray = md.valueArray()
+ val row = new GenericInternalRow(1)
+ 0.until(md.numElements()).iterator.map { i =>
+ row.update(0, keyArray.get(i, inputMapType.keyType))
+ val key = keyLambdaFunction.eval(row)
+ row.update(0, valueArray.get(i, inputMapType.valueType))
+ val value = valueLambdaFunction.eval(row)
+ Tuple2(key, value)
+ }
+ }
+
override def eval(input: InternalRow): Any = {
val result = inputData.eval(input).asInstanceOf[MapData]
if (result != null) {
val builder = newMapBuilder()
builder.sizeHint(result.numElements())
- val keyArray = result.keyArray()
- val valueArray = result.valueArray()
- var i = 0
- while (i < result.numElements()) {
Review comment:
We tend to use `while` for perf-intensive code. The proposed code does not cause any perf overhead?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-854800073
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-856527858
Kubernetes integration test unable to build dist.
exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43979/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-856490236
**[Test build #139456 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139456/testReport)** for PR 32783 at commit [`9cc2484`](https://github.com/apache/spark/commit/9cc2484600374022bb76a95039e22a8c232a4700).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-857281303
**[Test build #139513 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139513/testReport)** for PR 32783 at commit [`9cc2484`](https://github.com/apache/spark/commit/9cc2484600374022bb76a95039e22a8c232a4700).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-855160977
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139352/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-857320603
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44038/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-856647023
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139456/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-855021460
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43874/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya edited a comment on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
viirya edited a comment on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-858772822
Thanks @eejbyfeldt for your first contribution! Added you into contributor list on JIRA and assigned this ticket to you. Welcome to Spark community.
Thanks @maropu @cloud-fan for review. Merging to master, 3.1, 3.0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-857300522
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139513/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-854998579
Thank you for making a PR, @eejbyfeldt .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-855167204
Can one of the admins verify this patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-856527878
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43979/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-857320603
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44038/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-855051761
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43874/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-857309120
Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44038/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-858404807
I think the new behavior looks more correct. Just wondering how the change affects developers or users. Usually for user-facing change, we can document in migration guide. This is per Catalyst expression change, not sure if migration guide is suitable too.
cc @cloud-fan
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-855167204
Can one of the admins verify this patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] eejbyfeldt commented on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
eejbyfeldt commented on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-856787275
I would think the test failure was caused by the outage of python infrastructure (https://status.python.org/) and not my code. How can I retest this branch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-854998275
ok to test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] eejbyfeldt commented on a change in pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
eejbyfeldt commented on a change in pull request #32783:
URL: https://github.com/apache/spark/pull/32783#discussion_r647149051
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala
##########
@@ -1181,21 +1176,25 @@ case class CatalystToExternalMap private(
newMapBuilderMethod.invoke(moduleField).asInstanceOf[Builder[AnyRef, AnyRef]]
}
+ private def keyValueIterator(md: MapData): Iterator[AnyRef] = {
+ val keyArray = md.keyArray()
+ val valueArray = md.valueArray()
+ val row = new GenericInternalRow(1)
+ 0.until(md.numElements()).iterator.map { i =>
+ row.update(0, keyArray.get(i, inputMapType.keyType))
+ val key = keyLambdaFunction.eval(row)
+ row.update(0, valueArray.get(i, inputMapType.valueType))
+ val value = valueLambdaFunction.eval(row)
+ Tuple2(key, value)
+ }
+ }
+
override def eval(input: InternalRow): Any = {
val result = inputData.eval(input).asInstanceOf[MapData]
if (result != null) {
val builder = newMapBuilder()
builder.sizeHint(result.numElements())
- val keyArray = result.keyArray()
- val valueArray = result.valueArray()
- var i = 0
- while (i < result.numElements()) {
Review comment:
I guess I am not sure whether it does or not. But I updated the PR to use a while instead, to be sure. Now it the style is also more similar to what was there before.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-854998867
**[Test build #139352 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139352/testReport)** for PR 32783 at commit [`e95d7f4`](https://github.com/apache/spark/commit/e95d7f48c1d4a0a9a1dac1ecb6d0b32f895c4154).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on a change in pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
maropu commented on a change in pull request #32783:
URL: https://github.com/apache/spark/pull/32783#discussion_r647150106
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala
##########
@@ -1181,21 +1176,25 @@ case class CatalystToExternalMap private(
newMapBuilderMethod.invoke(moduleField).asInstanceOf[Builder[AnyRef, AnyRef]]
}
+ private def keyValueIterator(md: MapData): Iterator[AnyRef] = {
+ val keyArray = md.keyArray()
+ val valueArray = md.valueArray()
+ val row = new GenericInternalRow(1)
+ 0.until(md.numElements()).iterator.map { i =>
+ row.update(0, keyArray.get(i, inputMapType.keyType))
+ val key = keyLambdaFunction.eval(row)
+ row.update(0, valueArray.get(i, inputMapType.valueType))
+ val value = valueLambdaFunction.eval(row)
+ Tuple2(key, value)
+ }
+ }
+
override def eval(input: InternalRow): Any = {
val result = inputData.eval(input).asInstanceOf[MapData]
if (result != null) {
val builder = newMapBuilder()
builder.sizeHint(result.numElements())
- val keyArray = result.keyArray()
- val valueArray = result.valueArray()
- var i = 0
- while (i < result.numElements()) {
Review comment:
The latest one looks fine. Thanks ;)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
maropu commented on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-857264992
retest this please
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-854998867
**[Test build #139352 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139352/testReport)** for PR 32783 at commit [`e95d7f4`](https://github.com/apache/spark/commit/e95d7f48c1d4a0a9a1dac1ecb6d0b32f895c4154).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-857281303
**[Test build #139513 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139513/testReport)** for PR 32783 at commit [`9cc2484`](https://github.com/apache/spark/commit/9cc2484600374022bb76a95039e22a8c232a4700).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-855160728
**[Test build #139352 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139352/testReport)** for PR 32783 at commit [`e95d7f4`](https://github.com/apache/spark/commit/e95d7f48c1d4a0a9a1dac1ecb6d0b32f895c4154).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
maropu commented on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-857265235
> I would think the test failure was caused by the outage of python infrastructure (https://status.python.org/) and not my code. How can I retest this branch?
Yea, the PR looks fine cuz the GA tests passed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-856646244
**[Test build #139456 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139456/testReport)** for PR 32783 at commit [`9cc2484`](https://github.com/apache/spark/commit/9cc2484600374022bb76a95039e22a8c232a4700).
* This patch **fails PySpark pip packaging tests**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-855160977
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139352/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-854800073
Can one of the admins verify this patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-857300346
**[Test build #139513 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139513/testReport)** for PR 32783 at commit [`9cc2484`](https://github.com/apache/spark/commit/9cc2484600374022bb76a95039e22a8c232a4700).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya closed pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
viirya closed pull request #32783:
URL: https://github.com/apache/spark/pull/32783
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] eejbyfeldt commented on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
eejbyfeldt commented on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-858394340
> `StructConverter` converts from Catalyst struct to a `Row`. Will it be a behavior change? Although it is Catalyst expression.
From my understanding this is exactly the bug being fixed. So there will be behavior change. But the behavior is changed such that the interpreted path and the code gen path has the same behavior. The thinking being that the old behavior was undesired and incorrect.
My understanding of the failure the test cases (if added without the patch):
```
[info] - encode/decode for map with case class as value: Map(1 -> IntAndString(1,a)) (interpreted path) *** FAILED *** (64 milliseconds)
[info] Encoded/Decoded data does not match input data
[info]
[info] in: Map(1 -> IntAndString(1,a))
[info] out: Map(1 -> [1,a])
```
Is that the value of the `Map` was converted to an `[1,a]` which I believe is an `InternalRow` instead of the expected `IntAndString`. This sounds like the change you are mentioning?
The reason I believe my change is the correct is that using the `key/valueLambdaFunction` is since this is how it is done inside `MapObjects`:
https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala#L820-L826
Please correct me if I am wrong about anything or misunderstood the question as I am new to the code base.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya commented on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
viirya commented on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-858772822
Thanks @eejbyfeldt for your first contribution! Thanks @maropu @cloud-fan for review. Merging to master, 3.1, 3.0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-856490236
**[Test build #139456 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139456/testReport)** for PR 32783 at commit [`9cc2484`](https://github.com/apache/spark/commit/9cc2484600374022bb76a95039e22a8c232a4700).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] maropu commented on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
maropu commented on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-856440290
Nice catch, @eejbyfeldt and thank you for your contribution.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-857297160
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44038/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] viirya edited a comment on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
viirya edited a comment on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-858772822
Thanks @eejbyfeldt for your first contribution! Added you into contributor list on JIRA and assigned this ticket to you.
Thanks @maropu @cloud-fan for review. Merging to master, 3.1, 3.0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32783: [SPARK-35653][SQL] Fix CatalystToExternalMap interpreted path fails for Map with case classes as keys or values
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32783:
URL: https://github.com/apache/spark/pull/32783#issuecomment-856647023
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139456/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org