You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/10/25 03:53:46 UTC
[GitHub] [spark] LuciferYang opened a new pull request, #38382: [SPARK-40905][BUILD] Upgrade rocksdbjni to 7.7.3
LuciferYang opened a new pull request, #38382:
URL: https://github.com/apache/spark/pull/38382
### What changes were proposed in this pull request?
This pr aims to upgrade rocksdbjni from 7.6.0 to 7.7.3.
### Why are the changes needed?
This version bring the performance of `DeleteRange()` behavior and fixes some bugs, the release notes as follows:
- https://github.com/facebook/rocksdb/releases/tag/v7.7.2
- https://github.com/facebook/rocksdb/releases/tag/v7.7.3
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
- Pass GitHub Actions
- Manual test `RocksDBBenchmark`:
**7.6.0**
```
[INFO] Running org.apache.spark.util.kvstore.RocksDBBenchmark
count mean min max 95th
dbClose 4 0.374 0.286 0.591 0.591
dbCreation 4 67.738 3.709 259.012 259.012
naturalIndexCreateIterator 1024 0.006 0.002 1.595 0.007
naturalIndexDescendingCreateIterator 1024 0.007 0.006 0.068 0.008
naturalIndexDescendingIteration 1024 0.006 0.004 0.060 0.008
naturalIndexIteration 1024 0.006 0.004 0.119 0.009
randomDeleteIndexed 1024 0.028 0.021 0.295 0.038
randomDeletesNoIndex 1024 0.015 0.013 0.043 0.018
randomUpdatesIndexed 1024 0.082 0.032 31.250 0.086
randomUpdatesNoIndex 1024 0.033 0.030 0.661 0.036
randomWritesIndexed 1024 0.119 0.034 52.561 0.118
randomWritesNoIndex 1024 0.039 0.033 1.894 0.043
refIndexCreateIterator 1024 0.005 0.005 0.020 0.007
refIndexDescendingCreateIterator 1024 0.003 0.003 0.027 0.005
refIndexDescendingIteration 1024 0.007 0.005 0.046 0.008
refIndexIteration 1024 0.007 0.005 0.321 0.010
sequentialDeleteIndexed 1024 0.021 0.017 0.099 0.026
sequentialDeleteNoIndex 1024 0.015 0.012 0.043 0.019
sequentialUpdatesIndexed 1024 0.042 0.036 0.899 0.049
sequentialUpdatesNoIndex 1024 0.039 0.031 0.945 0.047
sequentialWritesIndexed 1024 0.048 0.041 2.022 0.056
sequentialWritesNoIndex 1024 0.039 0.031 2.419 0.041
```
**7.7.3**
```
[INFO] Running org.apache.spark.util.kvstore.RocksDBBenchmark
count mean min max 95th
dbClose 4 0.365 0.272 0.531 0.531
dbCreation 4 66.947 3.915 258.255 258.255
naturalIndexCreateIterator 1024 0.005 0.002 1.378 0.006
naturalIndexDescendingCreateIterator 1024 0.005 0.005 0.061 0.007
naturalIndexDescendingIteration 1024 0.005 0.004 0.033 0.008
naturalIndexIteration 1024 0.006 0.004 0.048 0.009
randomDeleteIndexed 1024 0.026 0.018 0.271 0.034
randomDeletesNoIndex 1024 0.015 0.012 0.044 0.018
randomUpdatesIndexed 1024 0.079 0.032 29.256 0.083
randomUpdatesNoIndex 1024 0.035 0.032 0.521 0.039
randomWritesIndexed 1024 0.115 0.033 49.182 0.118
randomWritesNoIndex 1024 0.040 0.034 1.876 0.045
refIndexCreateIterator 1024 0.004 0.004 0.015 0.005
refIndexDescendingCreateIterator 1024 0.003 0.002 0.025 0.004
refIndexDescendingIteration 1024 0.006 0.005 0.053 0.008
refIndexIteration 1024 0.007 0.005 0.259 0.010
sequentialDeleteIndexed 1024 0.021 0.017 0.118 0.027
sequentialDeleteNoIndex 1024 0.015 0.012 0.043 0.018
sequentialUpdatesIndexed 1024 0.044 0.037 0.902 0.053
sequentialUpdatesNoIndex 1024 0.041 0.030 1.155 0.050
sequentialWritesIndexed 1024 0.049 0.038 2.132 0.059
sequentialWritesNoIndex 1024 0.039 0.030 2.527 0.042
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #38382: [SPARK-40905][BUILD] Upgrade rocksdbjni to 7.7.3
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on PR #38382:
URL: https://github.com/apache/spark/pull/38382#issuecomment-1291417756
Thanks @HyukjinKwon @itholic @dongjoon-hyun
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun closed pull request #38382: [SPARK-40905][BUILD] Upgrade rocksdbjni to 7.7.3
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun closed pull request #38382: [SPARK-40905][BUILD] Upgrade rocksdbjni to 7.7.3
URL: https://github.com/apache/spark/pull/38382
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org