You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/03/13 03:15:03 UTC
[GitHub] [spark] beliefer opened a new pull request #35831: [SPARK-38539][SQL][DOC] Add document for ToNumber for DecimalFormat's compatibility
beliefer opened a new pull request #35831:
URL: https://github.com/apache/spark/pull/35831
### What changes were proposed in this pull request?
Currently, Spark choose `DecimalFormat` as the underlying implementation.
because `DecimalFormat` have better compatibility, so ToNumber could have the better compatibility than it in some database (e.g. Oracle).
### Why are the changes needed?
Explain why ToNumber have the better compatibility.
### Does this PR introduce _any_ user-facing change?
'No'. New feature.
### How was this patch tested?
No need.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #35831: [SPARK-38539][SQL][DOCS] Add document for ToNumber for DecimalFormat's compatibility
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #35831:
URL: https://github.com/apache/spark/pull/35831#discussion_r825586985
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/numberFormatExpressions.scala
##########
@@ -52,6 +52,11 @@ import org.apache.spark.unsafe.types.UTF8String
> SELECT _FUNC_('12,454.8-', '99G999D9S');
-12454.8
""",
+ note = """
+ ToNumber chose DecimalFormat as the underlying implementation, so ToNumber keep
Review comment:
OK
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #35831: [SPARK-38539][SQL][DOCS] Add document for ToNumber for DecimalFormat's compatibility
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #35831:
URL: https://github.com/apache/spark/pull/35831#discussion_r825587843
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/numberFormatExpressions.scala
##########
@@ -52,6 +52,11 @@ import org.apache.spark.unsafe.types.UTF8String
> SELECT _FUNC_('12,454.8-', '99G999D9S');
-12454.8
""",
+ note = """
+ ToNumber has better compatibility with number format based on the underlying implementation.
Review comment:
```suggestion
`to_number` has better compatibility with number format based on the underlying implementation.
```
Let's use `to_number` too since this is user-facing docs.
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/numberFormatExpressions.scala
##########
@@ -52,6 +52,11 @@ import org.apache.spark.unsafe.types.UTF8String
> SELECT _FUNC_('12,454.8-', '99G999D9S');
-12454.8
""",
+ note = """
+ ToNumber has better compatibility with number format based on the underlying implementation.
+ For example: to_number('1,11,6.11', '000,,0.00') works good and returns 1116.11. Please see
Review comment:
```suggestion
For example: `to_number('1,11,6.11', '000,,0.00')` works good and returns 1116.11. Please see
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on a change in pull request #35831: [SPARK-38539][SQL][DOCS] Add document for ToNumber for DecimalFormat's compatibility
Posted by GitBox <gi...@apache.org>.
beliefer commented on a change in pull request #35831:
URL: https://github.com/apache/spark/pull/35831#discussion_r835738017
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/numberFormatExpressions.scala
##########
@@ -52,6 +52,11 @@ import org.apache.spark.unsafe.types.UTF8String
> SELECT _FUNC_('12,454.8-', '99G999D9S');
-12454.8
""",
+ note = """
Review comment:
Updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #35831: [SPARK-38539][SQL][DOCS] Add document for ToNumber for DecimalFormat's compatibility
Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #35831:
URL: https://github.com/apache/spark/pull/35831#issuecomment-1072007599
ping @MaxGekk cc @cloud-fan
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] beliefer commented on pull request #35831: [SPARK-38539][SQL][DOCS] Add document for ToNumber for DecimalFormat's compatibility
Posted by GitBox <gi...@apache.org>.
beliefer commented on pull request #35831:
URL: https://github.com/apache/spark/pull/35831#issuecomment-1072007599
ping @MaxGekk cc @cloud-fan
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #35831: [SPARK-38539][SQL][DOC] Add document for ToNumber for DecimalFormat's compatibility
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #35831:
URL: https://github.com/apache/spark/pull/35831#discussion_r825547273
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/numberFormatExpressions.scala
##########
@@ -52,6 +52,11 @@ import org.apache.spark.unsafe.types.UTF8String
> SELECT _FUNC_('12,454.8-', '99G999D9S');
-12454.8
""",
+ note = """
+ ToNumber chose DecimalFormat as the underlying implementation, so ToNumber keep
Review comment:
Can we avoid using class names here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a change in pull request #35831: [SPARK-38539][SQL][DOCS] Add document for ToNumber for DecimalFormat's compatibility
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on a change in pull request #35831:
URL: https://github.com/apache/spark/pull/35831#discussion_r825587972
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/numberFormatExpressions.scala
##########
@@ -52,6 +52,11 @@ import org.apache.spark.unsafe.types.UTF8String
> SELECT _FUNC_('12,454.8-', '99G999D9S');
-12454.8
""",
+ note = """
+ ToNumber has better compatibility with number format based on the underlying implementation.
+ For example: to_number('1,11,6.11', '000,,0.00') works good and returns 1116.11. Please see
+ the implementation of NumberFormatter.
Review comment:
```suggestion
the implementation of `org.apache.spark.sql.catalyst.util.NumberFormatter`.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on a change in pull request #35831: [SPARK-38539][SQL][DOCS] Add document for ToNumber for DecimalFormat's compatibility
Posted by GitBox <gi...@apache.org>.
srowen commented on a change in pull request #35831:
URL: https://github.com/apache/spark/pull/35831#discussion_r834478942
##########
File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/numberFormatExpressions.scala
##########
@@ -52,6 +52,11 @@ import org.apache.spark.unsafe.types.UTF8String
> SELECT _FUNC_('12,454.8-', '99G999D9S');
-12454.8
""",
+ note = """
Review comment:
I think this is a vague statement, better compatibility, 'works good'? This isn't telling users something actionable; I'm actually not sure what to make of it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org