You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2020/04/14 17:25:50 UTC
[GitHub] [spark] MaxGekk opened a new pull request #28216:
[SPARK-31445][SQL][2.4] Avoid floating-point operations in `millisToDays`
MaxGekk opened a new pull request #28216: [SPARK-31445][SQL][2.4] Avoid floating-point operations in `millisToDays`
URL: https://github.com/apache/spark/pull/28216
### What changes were proposed in this pull request?
Replacing a floating-point division and Math.floor of Double value by `Math.floorDiv` for Long in `DateTimeUtils.millisToDays()`.
### Why are the changes needed?
The changes improve performance, in particular, it speeds up the conversion from java.sql.Date as the ported `DateTimeBenchmark` in https://github.com/MaxGekk/spark/pull/27 and comparisons in https://github.com/apache/spark/pull/28205 show.
### Does this PR introduce any user-facing change?
No
### How was this patch tested?
By existing test suite `DateTimeUtilsSuite`, `DateFunctionsSuite` and `DateExpressionsSuite`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #28216:
[SPARK-31445][SQL][2.4] Avoid floating-point operations in `millisToDays`
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on issue #28216: [SPARK-31445][SQL][2.4] Avoid floating-point operations in `millisToDays`
URL: https://github.com/apache/spark/pull/28216#issuecomment-613610610
+1 for @kiszk and @gatorsmile 's advice.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28216:
[SPARK-31445][SQL][2.4] Avoid floating-point operations in `millisToDays`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28216: [SPARK-31445][SQL][2.4] Avoid floating-point operations in `millisToDays`
URL: https://github.com/apache/spark/pull/28216#issuecomment-613576506
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25971/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28216:
[SPARK-31445][SQL][2.4] Avoid floating-point operations in `millisToDays`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28216: [SPARK-31445][SQL][2.4] Avoid floating-point operations in `millisToDays`
URL: https://github.com/apache/spark/pull/28216#issuecomment-613677488
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121285/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28216:
[SPARK-31445][SQL][2.4] Avoid floating-point operations in `millisToDays`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28216: [SPARK-31445][SQL][2.4] Avoid floating-point operations in `millisToDays`
URL: https://github.com/apache/spark/pull/28216#issuecomment-613677478
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28216:
[SPARK-31445][SQL][2.4] Avoid floating-point operations in `millisToDays`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on issue #28216: [SPARK-31445][SQL][2.4] Avoid floating-point operations in `millisToDays`
URL: https://github.com/apache/spark/pull/28216#issuecomment-613576498
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28216: [SPARK-31445][SQL][2.4]
Avoid floating-point operations in `millisToDays`
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #28216: [SPARK-31445][SQL][2.4] Avoid floating-point operations in `millisToDays`
URL: https://github.com/apache/spark/pull/28216#issuecomment-613575737
**[Test build #121285 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121285/testReport)** for PR 28216 at commit [`b453aa2`](https://github.com/apache/spark/commit/b453aa20c09c19ddd7bff2ce60f153a630f0b231).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #28216:
[SPARK-31445][SQL][2.4] Avoid floating-point operations in `millisToDays`
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on issue #28216: [SPARK-31445][SQL][2.4] Avoid floating-point operations in `millisToDays`
URL: https://github.com/apache/spark/pull/28216#issuecomment-613575737
**[Test build #121285 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121285/testReport)** for PR 28216 at commit [`b453aa2`](https://github.com/apache/spark/commit/b453aa20c09c19ddd7bff2ce60f153a630f0b231).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28216:
[SPARK-31445][SQL][2.4] Avoid floating-point operations in `millisToDays`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28216: [SPARK-31445][SQL][2.4] Avoid floating-point operations in `millisToDays`
URL: https://github.com/apache/spark/pull/28216#issuecomment-613576506
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25971/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk commented on issue #28216: [SPARK-31445][SQL][2.4]
Avoid floating-point operations in `millisToDays`
Posted by GitBox <gi...@apache.org>.
MaxGekk commented on issue #28216: [SPARK-31445][SQL][2.4] Avoid floating-point operations in `millisToDays`
URL: https://github.com/apache/spark/pull/28216#issuecomment-613615418
@kiszk @gatorsmile @dongjoon-hyun Thank you for your feedback. I am closing this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28216: [SPARK-31445][SQL][2.4]
Avoid floating-point operations in `millisToDays`
Posted by GitBox <gi...@apache.org>.
SparkQA commented on issue #28216: [SPARK-31445][SQL][2.4] Avoid floating-point operations in `millisToDays`
URL: https://github.com/apache/spark/pull/28216#issuecomment-613676940
**[Test build #121285 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/121285/testReport)** for PR 28216 at commit [`b453aa2`](https://github.com/apache/spark/commit/b453aa20c09c19ddd7bff2ce60f153a630f0b231).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28216:
[SPARK-31445][SQL][2.4] Avoid floating-point operations in `millisToDays`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28216: [SPARK-31445][SQL][2.4] Avoid floating-point operations in `millisToDays`
URL: https://github.com/apache/spark/pull/28216#issuecomment-613576498
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] kiszk commented on issue #28216: [SPARK-31445][SQL][2.4]
Avoid floating-point operations in `millisToDays`
Posted by GitBox <gi...@apache.org>.
kiszk commented on issue #28216: [SPARK-31445][SQL][2.4] Avoid floating-point operations in `millisToDays`
URL: https://github.com/apache/spark/pull/28216#issuecomment-613581416
According to https://spark.apache.org/versioning-policy.html, I think that maintenance release will have only patches for bug fix, not for performance.
>MAINTENANCE: Maintenance releases will occur more frequently and depend on specific patches introduced (e.g. bug fixes) and their urgency. In general these releases are designed to patch bugs. However, higher level libraries may introduce small features, such as a new algorithm, provided they are entirely additive and isolated from existing code paths. Spark core may not introduce any features.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] gatorsmile commented on issue #28216:
[SPARK-31445][SQL][2.4] Avoid floating-point operations in `millisToDays`
Posted by GitBox <gi...@apache.org>.
gatorsmile commented on issue #28216: [SPARK-31445][SQL][2.4] Avoid floating-point operations in `millisToDays`
URL: https://github.com/apache/spark/pull/28216#issuecomment-613605752
I would close this PR.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28216:
[SPARK-31445][SQL][2.4] Avoid floating-point operations in `millisToDays`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28216: [SPARK-31445][SQL][2.4] Avoid floating-point operations in `millisToDays`
URL: https://github.com/apache/spark/pull/28216#issuecomment-613677488
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/121285/
Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28216:
[SPARK-31445][SQL][2.4] Avoid floating-point operations in `millisToDays`
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on issue #28216: [SPARK-31445][SQL][2.4] Avoid floating-point operations in `millisToDays`
URL: https://github.com/apache/spark/pull/28216#issuecomment-613677478
Merged build finished. Test PASSed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] MaxGekk closed pull request #28216: [SPARK-31445][SQL][2.4]
Avoid floating-point operations in `millisToDays`
Posted by GitBox <gi...@apache.org>.
MaxGekk closed pull request #28216: [SPARK-31445][SQL][2.4] Avoid floating-point operations in `millisToDays`
URL: https://github.com/apache/spark/pull/28216
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org