You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by heary-cao <gi...@git.apache.org> on 2017/05/23 08:36:59 UTC
[GitHub] spark pull request #18069: Improve division and multiplication mixing proces...
GitHub user heary-cao opened a pull request:
https://github.com/apache/spark/pull/18069
Improve division and multiplication mixing process the data
## What changes were proposed in this pull request?
spark-sql> select (1234567890123456789012 / 12345678901234567890120) * 12345678901234567890120;
NULL
spark-sql> select (12345678901234567890 / 123) * 123;
NULL
when the length of the getText is greater than 19, The result is not what we expected.
but mysql handle the value is ok.
mysql> select (1234567890123456789012 / 12345678901234567890120) * 12345678901234567890120;
+------------------------------------------------------------------------------+
| (1234567890123456789012 / 12345678901234567890120) * 12345678901234567890120 |
+------------------------------------------------------------------------------+
| 1234567890123456789012.0000 |
+------------------------------------------------------------------------------+
1 row in set (0.00 sec)
mysql>
mysql> select (12345678901234567890 / 123) * 123;
+------------------------------------+
| (12345678901234567890 / 123) * 123 |
+------------------------------------+
| 12345678901234567890.0000 |
+------------------------------------+
1 row in set (0.00 sec)
## How was this patch tested?
Supplement the unit test.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/heary-cao/spark slash_handle
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/18069.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #18069
----
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18069: [SPARK-20850][SQL]Improve division and multiplication mi...
Posted by heary-cao <gi...@git.apache.org>.
Github user heary-cao commented on the issue:
https://github.com/apache/spark/pull/18069
@hvanhovell
thank you for review it.
This is an extreme case. must be with `Longtype` type and `Decimal` type mixing test.
However, when test case:
spark-sql> select (1234567890123456789012 / 12345678901234567890120)
0.1
spark-sql> select 0.1 * 12345678901234567890120;
1234567890123456789012
the result is ok.
For this PR, It is not the best treatment method.
close it for a moment.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18069: Improve division and multiplication mixing process the d...
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18069
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #18069: [SPARK-20850][SQL]Improve division and multiplica...
Posted by heary-cao <gi...@git.apache.org>.
Github user heary-cao closed the pull request at:
https://github.com/apache/spark/pull/18069
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #18069: [SPARK-20850][SQL]Improve division and multiplication mi...
Posted by hvanhovell <gi...@git.apache.org>.
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/18069
I don't think this makes any sense:
- The parser is not the place where you should make this change. This should be done in `TypeCoercion`. Now we create different semantics between SQL and the DataFrame API.
- This is a breaking change to result type of `Divide`, and will probably break Hive compatibility. This is going break to user code in a lot of places. I also think it is weird that we are going to always return `Double` when a `Decimal` is completely valid.
- I think MySql returns a `Decimal` and not a `Double`.
So I am going to -1 this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org