You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by wangyum <gi...@git.apache.org> on 2017/05/29 17:06:38 UTC
[GitHub] spark pull request #18138: [SPARK-20915][SQL] Make lpad/rpad with empty pad ...
GitHub user wangyum opened a pull request:
https://github.com/apache/spark/pull/18138
[SPARK-20915][SQL] Make lpad/rpad with empty pad string same as MySQL.
## What changes were proposed in this pull request?
Spark SQL `rpad/lpad` with empty pad string:
```sql
spark-sql> select rpad('hello', -2, ''), rpad('hello', -1, '') , rpad('hello', 0, ''), rpad('hello', 1, ''), rpad('hello', 3, ''), rpad('hello', 5, ''), rpad('hello', 6, '');
h hel hello hello
spark-sql> select lpad('hello', -2, ''), lpad('hello', -1, '') , lpad('hello', 0, ''), lpad('hello', 1, ''), lpad('hello', 3, ''), lpad('hello', 5, ''), lpad('hello', 6, '');
h hel hello hello
spark-sql>
```
but the MySQL result:
```
mysql> select rpad('hello', -2, ''), rpad('hello', -1, '') , rpad('hello', 0, ''), rpad('hello', 1, ''), rpad('hello', 3, ''), rpad('hello', 5, ''), rpad('hello', 6, '') from dual;
+-----------------------+-----------------------+----------------------+----------------------+----------------------+----------------------+----------------------+
| rpad('hello', -2, '') | rpad('hello', -1, '') | rpad('hello', 0, '') | rpad('hello', 1, '') | rpad('hello', 3, '') | rpad('hello', 5, '') | rpad('hello', 6, '') |
+-----------------------+-----------------------+----------------------+----------------------+----------------------+----------------------+----------------------+
| NULL | NULL | | h | hel | hello | NULL |
+-----------------------+-----------------------+----------------------+----------------------+----------------------+----------------------+----------------------+
1 row in set (0.00 sec)
mysql> select lpad('hello', -2, ''), lpad('hello', -1, '') , lpad('hello', 0, ''), lpad('hello', 1, ''), lpad('hello', 3, ''), lpad('hello', 5, ''), lpad('hello', 6, '') from dual;
+-----------------------+-----------------------+----------------------+----------------------+----------------------+----------------------+----------------------+
| lpad('hello', -2, '') | lpad('hello', -1, '') | lpad('hello', 0, '') | lpad('hello', 1, '') | lpad('hello', 3, '') | lpad('hello', 5, '') | lpad('hello', 6, '') |
+-----------------------+-----------------------+----------------------+----------------------+----------------------+----------------------+----------------------+
| NULL | NULL | | h | hel | hello | NULL |
+-----------------------+-----------------------+----------------------+----------------------+----------------------+----------------------+----------------------+
1 row in set (0.01 sec)
```
This PR fix this issue, after this PR:
```sql
spark-sql> select rpad('hello', -2, ''), rpad('hello', -1, '') , rpad('hello', 0, ''), rpad('hello', 1, ''), rpad('hello', 3, ''), rpad('hello', 5, ''), rpad('hello', 6, '');
NULL NULL h hel hello NULL
spark-sql> select lpad('hello', -2, ''), lpad('hello', -1, '') , lpad('hello', 0, ''), lpad('hello', 1, ''), lpad('hello', 3, ''), lpad('hello', 5, ''), lpad('hello', 6, '');
NULL NULL h hel hello NULL
spark-sql>
```
## How was this patch tested?
unit tests
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/wangyum/spark SPARK-20915
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/18138.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #18138
----
commit 3ac9fb07ef2f53315247ad12d391b1bed92319e9
Author: Yuming Wang <wg...@gmail.com>
Date: 2017-05-29T16:47:55Z
Make lpad/rpad with empty pad string same as MySQL.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18138: [SPARK-20915][SQL] Make lpad/rpad with empty pad string ...
Posted by wangyum <gi...@git.apache.org>.
Github user wangyum commented on the issue:
https://github.com/apache/spark/pull/18138
Hive will throw `ArrayIndexOutOfBoundsException` at runtime: https://issues.apache.org/jira/browse/HIVE-17077
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18138: [SPARK-20915][SQL] Make lpad/rpad with empty pad string ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18138
**[Test build #77512 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77512/testReport)** for PR 18138 at commit [`895f414`](https://github.com/apache/spark/commit/895f414983250a708fee46b7879de1524f01c368).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18138: [SPARK-20915][SQL] Make lpad/rpad with empty pad string ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18138
**[Test build #77505 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77505/testReport)** for PR 18138 at commit [`3ac9fb0`](https://github.com/apache/spark/commit/3ac9fb07ef2f53315247ad12d391b1bed92319e9).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18138: [SPARK-20915][SQL] Make lpad/rpad with empty pad string ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18138
**[Test build #77512 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77512/testReport)** for PR 18138 at commit [`895f414`](https://github.com/apache/spark/commit/895f414983250a708fee46b7879de1524f01c368).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18138: [SPARK-20915][SQL] Make lpad/rpad with empty pad string ...
Posted by jiangxb1987 <gi...@git.apache.org>.
Github user jiangxb1987 commented on the issue:
https://github.com/apache/spark/pull/18138
Can you also examine the HIVE behavior under such cases?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18138: [SPARK-20915][SQL] Make lpad/rpad with empty pad string ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18138
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18138: [SPARK-20915][SQL] Make lpad/rpad with empty pad string ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18138
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77505/
Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18138: [SPARK-20915][SQL] Make lpad/rpad with empty pad string ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18138
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18138: [SPARK-20915][SQL] Make lpad/rpad with empty pad string ...
Posted by SparkQA <gi...@git.apache.org>.
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18138
**[Test build #77505 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/77505/testReport)** for PR 18138 at commit [`3ac9fb0`](https://github.com/apache/spark/commit/3ac9fb07ef2f53315247ad12d391b1bed92319e9).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18138: [SPARK-20915][SQL] Make lpad/rpad with empty pad string ...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18138
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/77512/
Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18138: [SPARK-20915][SQL] Make lpad/rpad with empty pad string ...
Posted by jiangxb1987 <gi...@git.apache.org>.
Github user jiangxb1987 commented on the issue:
https://github.com/apache/spark/pull/18138
Seems our current behavior doesn't follow each way :(
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #18138: [SPARK-20915][SQL] Make lpad/rpad with empty pad ...
Posted by wangyum <gi...@git.apache.org>.
Github user wangyum closed the pull request at:
https://github.com/apache/spark/pull/18138
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org