You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by viirya <gi...@git.apache.org> on 2015/11/14 07:22:50 UTC
[GitHub] spark pull request: [SPARK-11743][SQL] Add UserDefinedType support...
GitHub user viirya opened a pull request:
https://github.com/apache/spark/pull/9712
[SPARK-11743][SQL] Add UserDefinedType support to RowEncoder
JIRA: https://issues.apache.org/jira/browse/SPARK-11743
RowEncoder doesn't support UserDefinedType now. We should add the support for it.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/viirya/spark-1 rowencoder-udt
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/9712.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #9712
----
commit 27217416429c2e9a745ea1e9babdb7e82b350604
Author: Liang-Chi Hsieh <vi...@appier.com>
Date: 2015-11-14T06:17:43Z
Add UserDefinedType support to RowEncoder.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-11743][SQL] Add UserDefinedType support...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9712#issuecomment-157009731
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-11743][SQL] Add UserDefinedType support...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9712#issuecomment-156654406
**[Test build #45923 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45923/consoleFull)** for PR 9712 at commit [`2721741`](https://github.com/apache/spark/commit/27217416429c2e9a745ea1e9babdb7e82b350604).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-11743][SQL] Add UserDefinedType support...
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/9712#discussion_r48646084
--- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/encoders/RowEncoderSuite.scala ---
@@ -68,7 +117,36 @@ class RowEncoderSuite extends SparkFunSuite {
.add("structOfArray", new StructType().add("array", arrayOfString))
.add("structOfMap", new StructType().add("map", mapOfString))
.add("structOfArrayAndMap",
- new StructType().add("array", arrayOfString).add("map", mapOfString)))
+ new StructType().add("array", arrayOfString).add("map", mapOfString))
+ .add("structOfUDT", structOfUDT))
+
+ test(s"encode/decode: arrayOfUDT") {
--- End diff --
No. I moved it in #10538.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-11743][SQL] Add UserDefinedType support...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9712#issuecomment-157062183
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/45994/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-11743][SQL] Add UserDefinedType support...
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/9712#discussion_r44867544
--- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/encoders/RowEncoderSuite.scala ---
@@ -68,7 +117,36 @@ class RowEncoderSuite extends SparkFunSuite {
.add("structOfArray", new StructType().add("array", arrayOfString))
.add("structOfMap", new StructType().add("map", mapOfString))
.add("structOfArrayAndMap",
- new StructType().add("array", arrayOfString).add("map", mapOfString)))
+ new StructType().add("array", arrayOfString).add("map", mapOfString))
+ .add("structOfUDT", structOfUDT))
+
+ test(s"encode/decode: arrayOfUDT") {
+ val schema = new StructType()
+ .add("arrayOfUDT", arrayOfUDT)
+
+ val encoder = RowEncoder(schema)
+
+ val input: Row = Row(Seq(new ExamplePoint(0.1, 0.2), new ExamplePoint(0.3, 0.4)))
+ val row = encoder.toRow(input)
+ val convertedBack = encoder.fromRow(row)
+ assert(input.getSeq[ExamplePoint](0) == convertedBack.getSeq[ExamplePoint](0))
+ }
+
+ test(s"encode/decode: Product") {
+ val schema = new StructType()
+ .add("structAsProduct",
+ new StructType()
+ .add("int", IntegerType)
+ .add("string", StringType)
+ .add("double", DoubleType))
+
+ val encoder = RowEncoder(schema)
+
+ val input: Row = Row((100, "test", 0.123))
--- End diff --
Actually I found this problem when working on ScalaUDF. ScalaUDF will use `schemaFor` to obtain catalyst type for UDF input and output. The catalyst type returned by `schemaFor` for a `Product` is `StructType`. It is reasonable as we don't have other type to represent `Product` as I see.
So for a `StructType` field in an external `Row`, both `Row` and `Product` are possible values. When we call `extractorsFor` on the external `Row`, `externalDataTypeFor` will return `ObjectType(classOf[Row])` for this field. But the `get` accessor on the inputObject (i.e., the `Row`) will possibly return a `Product` for the ScalaUDF case and an exception will be thrown.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-11743][SQL] Add UserDefinedType support...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9712#issuecomment-156978797
**[Test build #45992 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45992/consoleFull)** for PR 9712 at commit [`a5fdbce`](https://github.com/apache/spark/commit/a5fdbcec468926134e32ea7c35d9bfc7a28debfa).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-11743][SQL] Add UserDefinedType support...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9712#issuecomment-156719745
**[Test build #45931 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45931/consoleFull)** for PR 9712 at commit [`db644fb`](https://github.com/apache/spark/commit/db644fb2be3a3fd32f7f62993575dc6d8cef594e).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-11743][SQL] Add UserDefinedType support...
Posted by davies <gi...@git.apache.org>.
Github user davies commented on a diff in the pull request:
https://github.com/apache/spark/pull/9712#discussion_r44886605
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects.scala ---
@@ -343,33 +343,50 @@ case class MapObjects(
private lazy val loopAttribute = AttributeReference("loopVar", elementType)()
private lazy val completeFunction = function(loopAttribute)
+ private def itemAccessorMethod(dataType: DataType): String => String = dataType match {
+ case IntegerType => (i: String) => s".getInt($i)"
+ case LongType => (i: String) => s".getLong($i)"
+ case FloatType => (i: String) => s".getFloat($i)"
+ case DoubleType => (i: String) => s".getDouble($i)"
+ case ByteType => (i: String) => s".getByte($i)"
+ case ShortType => (i: String) => s".getShort($i)"
+ case BooleanType => (i: String) => s".getBoolean($i)"
+ case StringType => (i: String) => s".getUTF8String($i)"
+ case s: StructType => (i: String) => s".getStruct($i, ${s.size})"
+ case a: ArrayType => (i: String) => s".getArray($i)"
+ case _: MapType => (i: String) => s".getMap($i)"
+ case udt: UserDefinedType[_] => itemAccessorMethod(udt.sqlType)
+ }
+
private lazy val (lengthFunction, itemAccessor, primitiveElement) = inputData.dataType match {
case ObjectType(cls) if classOf[Seq[_]].isAssignableFrom(cls) =>
(".size()", (i: String) => s".apply($i)", false)
case ObjectType(cls) if cls.isArray =>
(".length", (i: String) => s"[$i]", false)
case ArrayType(s: StructType, _) =>
- (".numElements()", (i: String) => s".getStruct($i, ${s.size})", false)
+ (".numElements()", itemAccessorMethod(s), false)
case ArrayType(a: ArrayType, _) =>
- (".numElements()", (i: String) => s".getArray($i)", true)
+ (".numElements()", itemAccessorMethod(a), true)
case ArrayType(IntegerType, _) =>
- (".numElements()", (i: String) => s".getInt($i)", true)
+ (".numElements()", itemAccessorMethod(IntegerType), true)
case ArrayType(LongType, _) =>
- (".numElements()", (i: String) => s".getLong($i)", true)
+ (".numElements()", itemAccessorMethod(LongType), true)
case ArrayType(FloatType, _) =>
- (".numElements()", (i: String) => s".getFloat($i)", true)
+ (".numElements()", itemAccessorMethod(FloatType), true)
case ArrayType(DoubleType, _) =>
- (".numElements()", (i: String) => s".getDouble($i)", true)
+ (".numElements()", itemAccessorMethod(DoubleType), true)
case ArrayType(ByteType, _) =>
- (".numElements()", (i: String) => s".getByte($i)", true)
+ (".numElements()", itemAccessorMethod(ByteType), true)
case ArrayType(ShortType, _) =>
- (".numElements()", (i: String) => s".getShort($i)", true)
+ (".numElements()", itemAccessorMethod(ShortType), true)
case ArrayType(BooleanType, _) =>
- (".numElements()", (i: String) => s".getBoolean($i)", true)
+ (".numElements()", itemAccessorMethod(BooleanType), true)
case ArrayType(StringType, _) =>
- (".numElements()", (i: String) => s".getUTF8String($i)", false)
- case ArrayType(_: MapType, _) =>
- (".numElements()", (i: String) => s".getMap($i)", false)
+ (".numElements()", itemAccessorMethod(StringType), false)
+ case ArrayType(m: MapType, _) =>
+ (".numElements()", itemAccessorMethod(m), false)
+ case ArrayType(udt: UserDefinedType[_], _) =>
+ (".numElements()", itemAccessorMethod(udt.sqlType), false)
--- End diff --
Should we merge these branches together?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-11743][SQL] Add UserDefinedType support...
Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/9712#discussion_r44860429
--- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/encoders/RowEncoderSuite.scala ---
@@ -68,7 +117,36 @@ class RowEncoderSuite extends SparkFunSuite {
.add("structOfArray", new StructType().add("array", arrayOfString))
.add("structOfMap", new StructType().add("map", mapOfString))
.add("structOfArrayAndMap",
- new StructType().add("array", arrayOfString).add("map", mapOfString)))
+ new StructType().add("array", arrayOfString).add("map", mapOfString))
+ .add("structOfUDT", structOfUDT))
+
+ test(s"encode/decode: arrayOfUDT") {
+ val schema = new StructType()
+ .add("arrayOfUDT", arrayOfUDT)
+
+ val encoder = RowEncoder(schema)
+
+ val input: Row = Row(Seq(new ExamplePoint(0.1, 0.2), new ExamplePoint(0.3, 0.4)))
+ val row = encoder.toRow(input)
+ val convertedBack = encoder.fromRow(row)
+ assert(input.getSeq[ExamplePoint](0) == convertedBack.getSeq[ExamplePoint](0))
+ }
+
+ test(s"encode/decode: Product") {
+ val schema = new StructType()
+ .add("structAsProduct",
+ new StructType()
+ .add("int", IntegerType)
+ .add("string", StringType)
+ .add("double", DoubleType))
+
+ val encoder = RowEncoder(schema)
+
+ val input: Row = Row((100, "test", 0.123))
--- End diff --
I have a question here. According to the Javadoc of `Row`, a user should use and only use `Row` for `StructType` field. It's ok to support `Product` too, but do we have a reason for this? Is it needed for the UDT stuff? Sorry I'm not familiar with UDT handling, it will be good if you can explain it in detail, thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-11743][SQL] Add UserDefinedType support...
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on the pull request:
https://github.com/apache/spark/pull/9712#issuecomment-157021799
retest this please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-11743][SQL] Add UserDefinedType support...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9712#issuecomment-157062181
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-11743][SQL] Add UserDefinedType support...
Posted by davies <gi...@git.apache.org>.
Github user davies commented on a diff in the pull request:
https://github.com/apache/spark/pull/9712#discussion_r44949708
--- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/encoders/RowEncoderSuite.scala ---
@@ -68,7 +117,36 @@ class RowEncoderSuite extends SparkFunSuite {
.add("structOfArray", new StructType().add("array", arrayOfString))
.add("structOfMap", new StructType().add("map", mapOfString))
.add("structOfArrayAndMap",
- new StructType().add("array", arrayOfString).add("map", mapOfString)))
+ new StructType().add("array", arrayOfString).add("map", mapOfString))
+ .add("structOfUDT", structOfUDT))
+
+ test(s"encode/decode: arrayOfUDT") {
+ val schema = new StructType()
+ .add("arrayOfUDT", arrayOfUDT)
+
+ val encoder = RowEncoder(schema)
+
+ val input: Row = Row(Seq(new ExamplePoint(0.1, 0.2), new ExamplePoint(0.3, 0.4)))
+ val row = encoder.toRow(input)
+ val convertedBack = encoder.fromRow(row)
+ assert(input.getSeq[ExamplePoint](0) == convertedBack.getSeq[ExamplePoint](0))
+ }
+
+ test(s"encode/decode: Product") {
+ val schema = new StructType()
+ .add("structAsProduct",
+ new StructType()
+ .add("int", IntegerType)
+ .add("string", StringType)
+ .add("double", DoubleType))
+
+ val encoder = RowEncoder(schema)
+
+ val input: Row = Row((100, "test", 0.123))
--- End diff --
I think the reason we still support Product for StructType is for backward-compatibility, we did not enforce the inbound type before, someone may reply one it (because it's easier than Row in Scala).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-11743][SQL] Add UserDefinedType support...
Posted by davies <gi...@git.apache.org>.
Github user davies commented on the pull request:
https://github.com/apache/spark/pull/9712#issuecomment-157098726
LGTM, merging this into master and 1.6 branch, thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-11743][SQL] Add UserDefinedType support...
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/9712#discussion_r44906230
--- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects.scala ---
@@ -343,33 +343,50 @@ case class MapObjects(
private lazy val loopAttribute = AttributeReference("loopVar", elementType)()
private lazy val completeFunction = function(loopAttribute)
+ private def itemAccessorMethod(dataType: DataType): String => String = dataType match {
+ case IntegerType => (i: String) => s".getInt($i)"
+ case LongType => (i: String) => s".getLong($i)"
+ case FloatType => (i: String) => s".getFloat($i)"
+ case DoubleType => (i: String) => s".getDouble($i)"
+ case ByteType => (i: String) => s".getByte($i)"
+ case ShortType => (i: String) => s".getShort($i)"
+ case BooleanType => (i: String) => s".getBoolean($i)"
+ case StringType => (i: String) => s".getUTF8String($i)"
+ case s: StructType => (i: String) => s".getStruct($i, ${s.size})"
+ case a: ArrayType => (i: String) => s".getArray($i)"
+ case _: MapType => (i: String) => s".getMap($i)"
+ case udt: UserDefinedType[_] => itemAccessorMethod(udt.sqlType)
+ }
+
private lazy val (lengthFunction, itemAccessor, primitiveElement) = inputData.dataType match {
case ObjectType(cls) if classOf[Seq[_]].isAssignableFrom(cls) =>
(".size()", (i: String) => s".apply($i)", false)
case ObjectType(cls) if cls.isArray =>
(".length", (i: String) => s"[$i]", false)
case ArrayType(s: StructType, _) =>
- (".numElements()", (i: String) => s".getStruct($i, ${s.size})", false)
+ (".numElements()", itemAccessorMethod(s), false)
case ArrayType(a: ArrayType, _) =>
- (".numElements()", (i: String) => s".getArray($i)", true)
+ (".numElements()", itemAccessorMethod(a), true)
case ArrayType(IntegerType, _) =>
- (".numElements()", (i: String) => s".getInt($i)", true)
+ (".numElements()", itemAccessorMethod(IntegerType), true)
case ArrayType(LongType, _) =>
- (".numElements()", (i: String) => s".getLong($i)", true)
+ (".numElements()", itemAccessorMethod(LongType), true)
case ArrayType(FloatType, _) =>
- (".numElements()", (i: String) => s".getFloat($i)", true)
+ (".numElements()", itemAccessorMethod(FloatType), true)
case ArrayType(DoubleType, _) =>
- (".numElements()", (i: String) => s".getDouble($i)", true)
+ (".numElements()", itemAccessorMethod(DoubleType), true)
case ArrayType(ByteType, _) =>
- (".numElements()", (i: String) => s".getByte($i)", true)
+ (".numElements()", itemAccessorMethod(ByteType), true)
case ArrayType(ShortType, _) =>
- (".numElements()", (i: String) => s".getShort($i)", true)
+ (".numElements()", itemAccessorMethod(ShortType), true)
case ArrayType(BooleanType, _) =>
- (".numElements()", (i: String) => s".getBoolean($i)", true)
+ (".numElements()", itemAccessorMethod(BooleanType), true)
case ArrayType(StringType, _) =>
- (".numElements()", (i: String) => s".getUTF8String($i)", false)
- case ArrayType(_: MapType, _) =>
- (".numElements()", (i: String) => s".getMap($i)", false)
+ (".numElements()", itemAccessorMethod(StringType), false)
+ case ArrayType(m: MapType, _) =>
+ (".numElements()", itemAccessorMethod(m), false)
+ case ArrayType(udt: UserDefinedType[_], _) =>
+ (".numElements()", itemAccessorMethod(udt.sqlType), false)
--- End diff --
yes. updated.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-11743][SQL] Add UserDefinedType support...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9712#issuecomment-156679592
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/45926/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-11743][SQL] Add UserDefinedType support...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9712#issuecomment-157009687
**[Test build #45992 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45992/consoleFull)** for PR 9712 at commit [`a5fdbce`](https://github.com/apache/spark/commit/a5fdbcec468926134e32ea7c35d9bfc7a28debfa).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:\n * `case class JSONOptions(`\n
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-11743][SQL] Add UserDefinedType support...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/9712
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-11743][SQL] Add UserDefinedType support...
Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/9712#discussion_r48645591
--- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/encoders/RowEncoderSuite.scala ---
@@ -68,7 +117,36 @@ class RowEncoderSuite extends SparkFunSuite {
.add("structOfArray", new StructType().add("array", arrayOfString))
.add("structOfMap", new StructType().add("map", mapOfString))
.add("structOfArrayAndMap",
- new StructType().add("array", arrayOfString).add("map", mapOfString)))
+ new StructType().add("array", arrayOfString).add("map", mapOfString))
+ .add("structOfUDT", structOfUDT))
+
+ test(s"encode/decode: arrayOfUDT") {
--- End diff --
any reason why we put this test here instead of adding arrayOfUDT type in `encodeDecodeTest` like `structOfUDT`?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-11743][SQL] Add UserDefinedType support...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9712#issuecomment-156655113
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/45923/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-11743][SQL] Add UserDefinedType support...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9712#issuecomment-156655090
**[Test build #45923 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45923/consoleFull)** for PR 9712 at commit [`2721741`](https://github.com/apache/spark/commit/27217416429c2e9a745ea1e9babdb7e82b350604).
* This patch **fails Scala style tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-11743][SQL] Add UserDefinedType support...
Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/9712#discussion_r44871929
--- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/encoders/RowEncoderSuite.scala ---
@@ -68,7 +117,36 @@ class RowEncoderSuite extends SparkFunSuite {
.add("structOfArray", new StructType().add("array", arrayOfString))
.add("structOfMap", new StructType().add("map", mapOfString))
.add("structOfArrayAndMap",
- new StructType().add("array", arrayOfString).add("map", mapOfString)))
+ new StructType().add("array", arrayOfString).add("map", mapOfString))
+ .add("structOfUDT", structOfUDT))
+
+ test(s"encode/decode: arrayOfUDT") {
+ val schema = new StructType()
+ .add("arrayOfUDT", arrayOfUDT)
+
+ val encoder = RowEncoder(schema)
+
+ val input: Row = Row(Seq(new ExamplePoint(0.1, 0.2), new ExamplePoint(0.3, 0.4)))
+ val row = encoder.toRow(input)
+ val convertedBack = encoder.fromRow(row)
+ assert(input.getSeq[ExamplePoint](0) == convertedBack.getSeq[ExamplePoint](0))
+ }
+
+ test(s"encode/decode: Product") {
+ val schema = new StructType()
+ .add("structAsProduct",
+ new StructType()
+ .add("int", IntegerType)
+ .add("string", StringType)
+ .add("double", DoubleType))
+
+ val encoder = RowEncoder(schema)
+
+ val input: Row = Row((100, "test", 0.123))
--- End diff --
If one of the input parameter is `Tuple2`, then we need to use the encoder to decode a catalyst value to external value, i.e. decode an `InternalRow` object to `Tuple2` object. I think this is hard for a `RowEncoder`(your change only makes it possible to encode a `Product` into `InternalRow`, but not vice versa), we should use `ProductEncoder` for this case.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-11743][SQL] Add UserDefinedType support...
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/9712#discussion_r44904260
--- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/encoders/RowEncoderSuite.scala ---
@@ -68,7 +117,36 @@ class RowEncoderSuite extends SparkFunSuite {
.add("structOfArray", new StructType().add("array", arrayOfString))
.add("structOfMap", new StructType().add("map", mapOfString))
.add("structOfArrayAndMap",
- new StructType().add("array", arrayOfString).add("map", mapOfString)))
+ new StructType().add("array", arrayOfString).add("map", mapOfString))
+ .add("structOfUDT", structOfUDT))
+
+ test(s"encode/decode: arrayOfUDT") {
+ val schema = new StructType()
+ .add("arrayOfUDT", arrayOfUDT)
+
+ val encoder = RowEncoder(schema)
+
+ val input: Row = Row(Seq(new ExamplePoint(0.1, 0.2), new ExamplePoint(0.3, 0.4)))
+ val row = encoder.toRow(input)
+ val convertedBack = encoder.fromRow(row)
+ assert(input.getSeq[ExamplePoint](0) == convertedBack.getSeq[ExamplePoint](0))
+ }
+
+ test(s"encode/decode: Product") {
+ val schema = new StructType()
+ .add("structAsProduct",
+ new StructType()
+ .add("int", IntegerType)
+ .add("string", StringType)
+ .add("double", DoubleType))
+
+ val encoder = RowEncoder(schema)
+
+ val input: Row = Row((100, "test", 0.123))
--- End diff --
Hmm, as I tried, I found a problem is we may not always be able to get the type `T` needed to construct `ProductEncoder` and `FlatEncoder`. Even we can get it, we can't keep it in ScalaUDF due to serialization issue.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-11743][SQL] Add UserDefinedType support...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9712#issuecomment-156655110
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-11743][SQL] Add UserDefinedType support...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9712#issuecomment-157061992
**[Test build #45994 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45994/consoleFull)** for PR 9712 at commit [`a5fdbce`](https://github.com/apache/spark/commit/a5fdbcec468926134e32ea7c35d9bfc7a28debfa).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:\n * `case class JSONOptions(`\n
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-11743][SQL] Add UserDefinedType support...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9712#issuecomment-156679591
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-11743][SQL] Add UserDefinedType support...
Posted by viirya <gi...@git.apache.org>.
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/9712#discussion_r44872999
--- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/encoders/RowEncoderSuite.scala ---
@@ -68,7 +117,36 @@ class RowEncoderSuite extends SparkFunSuite {
.add("structOfArray", new StructType().add("array", arrayOfString))
.add("structOfMap", new StructType().add("map", mapOfString))
.add("structOfArrayAndMap",
- new StructType().add("array", arrayOfString).add("map", mapOfString)))
+ new StructType().add("array", arrayOfString).add("map", mapOfString))
+ .add("structOfUDT", structOfUDT))
+
+ test(s"encode/decode: arrayOfUDT") {
+ val schema = new StructType()
+ .add("arrayOfUDT", arrayOfUDT)
+
+ val encoder = RowEncoder(schema)
+
+ val input: Row = Row(Seq(new ExamplePoint(0.1, 0.2), new ExamplePoint(0.3, 0.4)))
+ val row = encoder.toRow(input)
+ val convertedBack = encoder.fromRow(row)
+ assert(input.getSeq[ExamplePoint](0) == convertedBack.getSeq[ExamplePoint](0))
+ }
+
+ test(s"encode/decode: Product") {
+ val schema = new StructType()
+ .add("structAsProduct",
+ new StructType()
+ .add("int", IntegerType)
+ .add("string", StringType)
+ .add("double", DoubleType))
+
+ val encoder = RowEncoder(schema)
+
+ val input: Row = Row((100, "test", 0.123))
--- End diff --
If we have an input parameter mapping to a `StructType` field in an `InternalRow`, we will use `Row` as its input type. E.g., `sqlContext.udf.register("udfFunc", (ns: Row) => { (ns.getInt(0), ns.getString(1)) })`. But we can't use `Row` as output type for an UDF. Because we can still get the input schema of `ScalaUDF`'s children expressions later if we can't infer input types correctly by using `schemaFor`. However, the output types of the UDF can be only inferred by `schemaFor`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-11743][SQL] Add UserDefinedType support...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9712#issuecomment-156663451
**[Test build #45926 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45926/consoleFull)** for PR 9712 at commit [`bab5c5b`](https://github.com/apache/spark/commit/bab5c5ba8f388e466b9d56af20b2fb2f8fae43ce).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-11743][SQL] Add UserDefinedType support...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9712#issuecomment-156708957
**[Test build #45931 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45931/consoleFull)** for PR 9712 at commit [`db644fb`](https://github.com/apache/spark/commit/db644fb2be3a3fd32f7f62993575dc6d8cef594e).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-11743][SQL] Add UserDefinedType support...
Posted by cloud-fan <gi...@git.apache.org>.
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/9712#discussion_r44882651
--- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/encoders/RowEncoderSuite.scala ---
@@ -68,7 +117,36 @@ class RowEncoderSuite extends SparkFunSuite {
.add("structOfArray", new StructType().add("array", arrayOfString))
.add("structOfMap", new StructType().add("map", mapOfString))
.add("structOfArrayAndMap",
- new StructType().add("array", arrayOfString).add("map", mapOfString)))
+ new StructType().add("array", arrayOfString).add("map", mapOfString))
+ .add("structOfUDT", structOfUDT))
+
+ test(s"encode/decode: arrayOfUDT") {
+ val schema = new StructType()
+ .add("arrayOfUDT", arrayOfUDT)
+
+ val encoder = RowEncoder(schema)
+
+ val input: Row = Row(Seq(new ExamplePoint(0.1, 0.2), new ExamplePoint(0.3, 0.4)))
+ val row = encoder.toRow(input)
+ val convertedBack = encoder.fromRow(row)
+ assert(input.getSeq[ExamplePoint](0) == convertedBack.getSeq[ExamplePoint](0))
+ }
+
+ test(s"encode/decode: Product") {
+ val schema = new StructType()
+ .add("structAsProduct",
+ new StructType()
+ .add("int", IntegerType)
+ .add("string", StringType)
+ .add("double", DoubleType))
+
+ val encoder = RowEncoder(schema)
+
+ val input: Row = Row((100, "test", 0.123))
--- End diff --
Ah I see. We need the type tag to infer the return type of UDF, and if a `Row` is returned, there is no type information we can get. How about we use `ProductEncoder` or `FlatEncoder` for the return value?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-11743][SQL] Add UserDefinedType support...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9712#issuecomment-157009732
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/45992/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-11743][SQL] Add UserDefinedType support...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9712#issuecomment-157024019
**[Test build #45994 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45994/consoleFull)** for PR 9712 at commit [`a5fdbce`](https://github.com/apache/spark/commit/a5fdbcec468926134e32ea7c35d9bfc7a28debfa).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-11743][SQL] Add UserDefinedType support...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9712#issuecomment-156719774
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-11743][SQL] Add UserDefinedType support...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/9712#issuecomment-156679559
**[Test build #45926 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/45926/consoleFull)** for PR 9712 at commit [`bab5c5b`](https://github.com/apache/spark/commit/bab5c5ba8f388e466b9d56af20b2fb2f8fae43ce).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request: [SPARK-11743][SQL] Add UserDefinedType support...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/9712#issuecomment-156719775
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/45931/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org