You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "panbingkun (via GitHub)" <gi...@apache.org> on 2023/07/25 01:29:13 UTC
[GitHub] [spark] panbingkun opened a new pull request, #42143: [SPARK-44539][BUILD] Upgrade RoaringBitmap to 0.9.46
panbingkun opened a new pull request, #42143:
URL: https://github.com/apache/spark/pull/42143
### What changes were proposed in this pull request?
The pr aims to upgrade RoaringBitmap from 0.9.45 to 0.9.46.
### Why are the changes needed?
The newest brings some improvments, eg:
More specialized method for value decrementation by @xtonik in https://github.com/RoaringBitmap/RoaringBitmap/pull/640
Duplicated small array sort routine by @xtonik in https://github.com/RoaringBitmap/RoaringBitmap/pull/638
Avoid intermediate byte array creation by @xtonik in https://github.com/RoaringBitmap/RoaringBitmap/pull/635
Useless back and forth BD bytes conversion by @xtonik in https://github.com/RoaringBitmap/RoaringBitmap/pull/636
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass GA.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen closed pull request #42143: [SPARK-44539][BUILD] Upgrade RoaringBitmap to 1.0.0
Posted by "srowen (via GitHub)" <gi...@apache.org>.
srowen closed pull request #42143: [SPARK-44539][BUILD] Upgrade RoaringBitmap to 1.0.0
URL: https://github.com/apache/spark/pull/42143
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] panbingkun commented on pull request #42143: [SPARK-44539][BUILD] Upgrade RoaringBitmap to 0.9.46
Posted by "panbingkun (via GitHub)" <gi...@apache.org>.
panbingkun commented on PR #42143:
URL: https://github.com/apache/spark/pull/42143#issuecomment-1648833100
Let me add the results of org.apache.spark.MapStatusesConvertBenchmark.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on a diff in pull request #42143: [SPARK-44539][BUILD] Upgrade RoaringBitmap to 0.9.46
Posted by "HyukjinKwon (via GitHub)" <gi...@apache.org>.
HyukjinKwon commented on code in PR #42143:
URL: https://github.com/apache/spark/pull/42143#discussion_r1272950835
##########
dev/deps/spark-deps-hadoop-3-hive-2.3:
##########
@@ -1,7 +1,7 @@
HikariCP/2.5.1//HikariCP-2.5.1.jar
JLargeArrays/1.5//JLargeArrays-1.5.jar
JTransforms/3.1//JTransforms-3.1.jar
-RoaringBitmap/0.9.45//RoaringBitmap-0.9.45.jar
+RoaringBitmap/0.9.46//RoaringBitmap-0.9.46.jar
Review Comment:
I think we should probably have a buferred update within individual release thread. Upgrading is good and a legitimate improvement but could be done in batch within each release window.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] panbingkun commented on pull request #42143: [SPARK-44539][BUILD] Upgrade RoaringBitmap to 1.0.0
Posted by "panbingkun (via GitHub)" <gi...@apache.org>.
panbingkun commented on PR #42143:
URL: https://github.com/apache/spark/pull/42143#issuecomment-1734690011
@HyukjinKwon
After the official release of version 1.0.0, I think we should upgrade this major version, which includes some changes that rely on jars.
cc @dongjoon-hyun @LuciferYang
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] panbingkun commented on a diff in pull request #42143: [SPARK-44539][BUILD] Upgrade RoaringBitmap to 0.9.46
Posted by "panbingkun (via GitHub)" <gi...@apache.org>.
panbingkun commented on code in PR #42143:
URL: https://github.com/apache/spark/pull/42143#discussion_r1274284668
##########
dev/deps/spark-deps-hadoop-3-hive-2.3:
##########
@@ -1,7 +1,7 @@
HikariCP/2.5.1//HikariCP-2.5.1.jar
JLargeArrays/1.5//JLargeArrays-1.5.jar
JTransforms/3.1//JTransforms-3.1.jar
-RoaringBitmap/0.9.45//RoaringBitmap-0.9.45.jar
+RoaringBitmap/0.9.46//RoaringBitmap-0.9.46.jar
Review Comment:
Maybe once a month? Or two months?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] yaooqinn commented on a diff in pull request #42143: [SPARK-44539][BUILD] Upgrade RoaringBitmap to 0.9.46
Posted by "yaooqinn (via GitHub)" <gi...@apache.org>.
yaooqinn commented on code in PR #42143:
URL: https://github.com/apache/spark/pull/42143#discussion_r1274277739
##########
dev/deps/spark-deps-hadoop-3-hive-2.3:
##########
@@ -1,7 +1,7 @@
HikariCP/2.5.1//HikariCP-2.5.1.jar
JLargeArrays/1.5//JLargeArrays-1.5.jar
JTransforms/3.1//JTransforms-3.1.jar
-RoaringBitmap/0.9.45//RoaringBitmap-0.9.45.jar
+RoaringBitmap/0.9.46//RoaringBitmap-0.9.46.jar
Review Comment:
+1 to @HyukjinKwon's
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] panbingkun commented on pull request #42143: [SPARK-44539][BUILD] Upgrade RoaringBitmap to 1.0.0
Posted by "panbingkun (via GitHub)" <gi...@apache.org>.
panbingkun commented on PR #42143:
URL: https://github.com/apache/spark/pull/42143#issuecomment-1732844716
From version 1.0.0, the `ArraysShim` class has been moved from `shims-x.x.x.jar` jar to `RoaringBitmap-x.x.x.jar` jar, so we no longer need to rely on it.
<img width="593" alt="image" src="https://github.com/apache/spark/assets/15246973/46161a83-ebf0-4c2a-88bc-bd7fe6bc9d2c">
<img width="1106" alt="image" src="https://github.com/apache/spark/assets/15246973/3db090a3-029b-45b6-98b9-a09c54ff7b23">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] srowen commented on pull request #42143: [SPARK-44539][BUILD] Upgrade RoaringBitmap to 1.0.0
Posted by "srowen (via GitHub)" <gi...@apache.org>.
srowen commented on PR #42143:
URL: https://github.com/apache/spark/pull/42143#issuecomment-1737761781
Merged to master
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org