You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by 351zyf <gi...@git.apache.org> on 2018/10/30 09:47:44 UTC
[GitHub] spark pull request #22891: SPARK-25881
GitHub user 351zyf opened a pull request:
https://github.com/apache/spark/pull/22891
SPARK-25881
## What changes were proposed in this pull request?
https://github.com/apache/spark/pull/22888
decimal type should consider as a number but not object (string)
(Please fill in changes proposed in this fix)
## How was this patch tested?
(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)
Please review http://spark.apache.org/contributing.html before opening a pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/351zyf/spark SPARK-25881
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/22891.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #22891
----
commit 5a73e1710bdb663cfd6fa4a3f228737dc309e0e4
Author: zhangyefei <zh...@...>
Date: 2018-10-30T09:28:37Z
[SPARK-25881] deal with decimal type
commit 403b4d00934de8e51b2c19c76170624fb91b1fb6
Author: zhangyefei <zh...@...>
Date: 2018-10-30T07:22:41Z
add parametere coerce_float
commit 11b7cf47e83018c1d9a4ae9bf8df4f507680e0c4
Author: zhangyefei <zh...@...>
Date: 2018-10-30T07:52:39Z
comment
commit 891a25f344db6d476e7d0b2857b09943d6c84720
Author: zhangyefei <zh...@...>
Date: 2018-10-30T09:35:28Z
[SPARK-25881] deal with decimal type
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22891: [SPARK-25881][pyspark] df.toPandas() convert decimal to ...
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/22891
I don't think we should map decimals to float. It will loses precisions and it's a breaking change.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22891: SPARK-25881
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22891
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #22891: [SPARK-25881][pyspark] df.toPandas() convert deci...
Posted by 351zyf <gi...@git.apache.org>.
Github user 351zyf closed the pull request at:
https://github.com/apache/spark/pull/22891
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22891: SPARK-25881
Posted by kiszk <gi...@git.apache.org>.
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/22891
Thank you for your contribution.
Could you please write appropriate title and descriptions based on http://spark.apache.org/contributing.html ?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22891: SPARK-25881
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22891
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark pull request #22891: [SPARK-25881][pyspark] df.toPandas() convert deci...
Posted by HyukjinKwon <gi...@git.apache.org>.
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22891#discussion_r229564044
--- Diff: python/pyspark/sql/dataframe.py ---
@@ -2064,6 +2064,7 @@ def toDF(self, *cols):
@since(1.3)
def toPandas(self):
"""
+ :param coerce_float: default False, if Ture, will handle decimal type to np.float64 instand of type object.
--- End diff --
There's no such param. Also, we don't convert it to string. Also, Arrow optimization code path should also be fixed BTW.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] spark issue #22891: SPARK-25881
Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22891
Can one of the admins verify this patch?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org