You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/11/01 09:19:41 UTC
[GitHub] [spark] LuciferYang opened a new pull request, #38465: [SPARK-40985][BUILD] Upgrade RoaringBitmap to 0.9.35
LuciferYang opened a new pull request, #38465:
URL: https://github.com/apache/spark/pull/38465
### What changes were proposed in this pull request?
This pr aims upgrade RoaringBitmap 0.9.35
### Why are the changes needed?
This version bring some bug fix:
- https://github.com/RoaringBitmap/RoaringBitmap/pull/587
- https://github.com/RoaringBitmap/RoaringBitmap/issues/588
other changes as follows:
https://github.com/RoaringBitmap/RoaringBitmap/compare/0.9.32...0.9.35
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Pass GitHub Actions
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen closed pull request #38465: [SPARK-40985][BUILD] Upgrade RoaringBitmap to 0.9.35
Posted by GitBox <gi...@apache.org>.
srowen closed pull request #38465: [SPARK-40985][BUILD] Upgrade RoaringBitmap to 0.9.35
URL: https://github.com/apache/spark/pull/38465
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on pull request #38465: [SPARK-40985][BUILD] Upgrade RoaringBitmap to 0.9.35
Posted by GitBox <gi...@apache.org>.
srowen commented on PR #38465:
URL: https://github.com/apache/spark/pull/38465#issuecomment-1300728818
Merged to master
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on a diff in pull request #38465: [SPARK-40985][BUILD] Upgrade RoaringBitmap to 0.9.35
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on code in PR #38465:
URL: https://github.com/apache/spark/pull/38465#discussion_r1011088808
##########
core/benchmarks/MapStatusesConvertBenchmark-jdk11-results.txt:
##########
@@ -2,12 +2,12 @@
MapStatuses Convert Benchmark
================================================================================================
-OpenJDK 64-Bit Server VM 11.0.16+8-LTS on Linux 5.15.0-1019-azure
+OpenJDK 64-Bit Server VM 11.0.16.1+1 on Linux 5.15.0-1022-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
MapStatuses Convert: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Num Maps: 50000 Fetch partitions:500 1269 1276 8 0.0 1268666001.0 1.0X
-Num Maps: 50000 Fetch partitions:1000 2672 2695 39 0.0 2671542753.0 0.5X
-Num Maps: 50000 Fetch partitions:1500 4034 4069 50 0.0 4033696987.0 0.3X
+Num Maps: 50000 Fetch partitions:500 1227 1262 47 0.0 1226744907.0 1.0X
Review Comment:
`Stdev` refers to the standard deviation estimated based on the sample. The standard deviation reflects the dispersion of the value relative to the mean.
From `Per Row(ns)`, although the cpu frequency of the tester is slower than the base, the performance has not changed significantly
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #38465: [SPARK-40985][BUILD] Upgrade RoaringBitmap to 0.9.35
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on PR #38465:
URL: https://github.com/apache/spark/pull/38465#issuecomment-1301588097
Thanks @srowen @HyukjinKwon @itholic
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] itholic commented on a diff in pull request #38465: [SPARK-40985][BUILD] Upgrade RoaringBitmap to 0.9.35
Posted by GitBox <gi...@apache.org>.
itholic commented on code in PR #38465:
URL: https://github.com/apache/spark/pull/38465#discussion_r1011048696
##########
core/benchmarks/MapStatusesConvertBenchmark-jdk11-results.txt:
##########
@@ -2,12 +2,12 @@
MapStatuses Convert Benchmark
================================================================================================
-OpenJDK 64-Bit Server VM 11.0.16+8-LTS on Linux 5.15.0-1019-azure
+OpenJDK 64-Bit Server VM 11.0.16.1+1 on Linux 5.15.0-1022-azure
Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
MapStatuses Convert: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
-Num Maps: 50000 Fetch partitions:500 1269 1276 8 0.0 1268666001.0 1.0X
-Num Maps: 50000 Fetch partitions:1000 2672 2695 39 0.0 2671542753.0 0.5X
-Num Maps: 50000 Fetch partitions:1500 4034 4069 50 0.0 4033696987.0 0.3X
+Num Maps: 50000 Fetch partitions:500 1227 1262 47 0.0 1226744907.0 1.0X
Review Comment:
Just out of curiosity, seems like the `Stdev` slow-dowm almost 6x, it is acceptable performance degradation?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org