You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by koertkuipers <gi...@git.apache.org> on 2017/02/10 22:27:48 UTC
[GitHub] spark pull request #16889: [SPARK-17668][SQL] Use Expressions for conversion...
GitHub user koertkuipers opened a pull request:
https://github.com/apache/spark/pull/16889
[SPARK-17668][SQL] Use Expressions for conversions to/from user types in UDFs
## What changes were proposed in this pull request?
do not merge
this is a first attempt at trying to address SPARK-17688. but i do no expect it to be sufficient.
things that bother me:
* i do not use codegen for the encoder expressions. instead i rely on fromRow and toRow in ExpressionEncoder. that seems inefficient.
* some unnecessary wrapping in InternalRows. this is probably related to the usage of fromRow and toRow
## How was this patch tested?
added TypedUDFSuite
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/tresata/spark feat-typed-udf
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/16889.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #16889
----
commit e73c84ff75e66dee9a395c9913109a965d7d68f4
Author: Koert Kuipers <ko...@tresata.com>
Date: 2017-02-09T20:04:36Z
got something working but not sure how good this is yet
commit bd111ae1f1a721ae2664d0e8a5810f018eb4c935
Author: Koert Kuipers <ko...@tresata.com>
Date: 2017-02-10T01:16:34Z
pattern match to create expr1
commit 1a257c366aa1c0181dd7791fdd6eb05140c48d7e
Author: Koert Kuipers <ko...@tresata.com>
Date: 2017-02-10T02:04:53Z
fix bug with internal row getting re-used and check results in unit tests
commit e1c337ae9e747a3dc02b939a87c9bb5c8605b86c
Author: Koert Kuipers <ko...@tresata.com>
Date: 2017-02-10T03:26:40Z
deal with annoying style rules
commit ac09ad519437fe8efb071f354cc4387a4a95c206
Author: Koert Kuipers <ko...@tresata.com>
Date: 2017-02-10T19:43:19Z
Merge branch 'master' into feat-typed-udf
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16889: [SPARK-17668][SQL] Use Expressions for conversions to/fr...
Posted by koertkuipers <gi...@git.apache.org>.
Github user koertkuipers commented on the issue:
https://github.com/apache/spark/pull/16889
i am going to close this for now since i dont think this is an optimal solution
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16889: [SPARK-17668][SQL] Use Expressions for conversions to/fr...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16889
**[Test build #72720 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72720/testReport)** for PR 16889 at commit [`ac09ad5`](https://github.com/apache/spark/commit/ac09ad519437fe8efb071f354cc4387a4a95c206).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16889: [SPARK-17668][SQL] Use Expressions for conversions to/fr...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16889
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72720/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16889: [SPARK-17668][SQL] Use Expressions for conversions to/fr...
Posted by kayousterhout <gi...@git.apache.org>.
Github user kayousterhout commented on the issue:
https://github.com/apache/spark/pull/16889
FYI the test that failed here seems to be flaky and probably not related to this PR: https://issues.apache.org/jira/browse/SPARK-19559
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16889: [SPARK-17668][SQL] Use Expressions for conversions to/fr...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16889
**[Test build #72720 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72720/testReport)** for PR 16889 at commit [`ac09ad5`](https://github.com/apache/spark/commit/ac09ad519437fe8efb071f354cc4387a4a95c206).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #16889: [SPARK-17668][SQL] Use Expressions for conversions to/fr...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16889
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #16889: [SPARK-17668][SQL] Use Expressions for conversion...
Posted by koertkuipers <gi...@git.apache.org>.
Github user koertkuipers closed the pull request at:
https://github.com/apache/spark/pull/16889
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org