You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2022/05/12 09:59:50 UTC
[GitHub] [spark] LuciferYang opened a new pull request, #36522: [SPARK-39161][BUILD] Upgrade rocksdbjni to 7.2.2
LuciferYang opened a new pull request, #36522:
URL: https://github.com/apache/spark/pull/36522
### What changes were proposed in this pull request?
This PR aims to upgrade RocksDB JNI library from 7.1.2 to 7.2.2.
### Why are the changes needed?
This will bring improvements and bug fix of RocksDB JNI.
- https://github.com/facebook/rocksdb/releases/tag/v7.2.2
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
- Pass GA
The benchmark result :
**Before 7.1.2**
```
[INFO] Running org.apache.spark.util.kvstore.RocksDBBenchmark
count mean min max 95th
dbClose 4 0.285 0.199 0.492 0.492
dbCreation 4 76.967 3.187 300.938 300.938
naturalIndexCreateIterator 1024 0.006 0.002 1.391 0.007
naturalIndexDescendingCreateIterator 1024 0.005 0.005 0.063 0.007
naturalIndexDescendingIteration 1024 0.006 0.004 0.247 0.008
naturalIndexIteration 1024 0.006 0.004 0.053 0.009
randomDeleteIndexed 1024 0.027 0.019 0.277 0.037
randomDeletesNoIndex 1024 0.015 0.013 0.031 0.017
randomUpdatesIndexed 1024 0.081 0.033 30.561 0.085
randomUpdatesNoIndex 1024 0.036 0.033 0.465 0.039
randomWritesIndexed 1024 0.118 0.035 52.557 0.123
randomWritesNoIndex 1024 0.042 0.036 1.439 0.048
refIndexCreateIterator 1024 0.004 0.004 0.017 0.006
refIndexDescendingCreateIterator 1024 0.003 0.003 0.028 0.004
refIndexDescendingIteration 1024 0.006 0.005 0.045 0.008
refIndexIteration 1024 0.007 0.005 0.073 0.009
sequentialDeleteIndexed 1024 0.021 0.018 0.108 0.025
sequentialDeleteNoIndex 1024 0.015 0.012 0.039 0.017
sequentialUpdatesIndexed 1024 0.045 0.039 0.732 0.056
sequentialUpdatesNoIndex 1024 0.042 0.031 0.812 0.054
sequentialWritesIndexed 1024 0.050 0.044 1.881 0.057
sequentialWritesNoIndex 1024 0.039 0.032 2.160 0.042
```
**After 7.2.2**
```
[INFO] Running org.apache.spark.util.kvstore.RocksDBBenchmark
count mean min max 95th
dbClose 4 0.397 0.280 0.719 0.719
dbCreation 4 77.931 3.410 300.959 300.959
naturalIndexCreateIterator 1024 0.006 0.002 1.433 0.007
naturalIndexDescendingCreateIterator 1024 0.005 0.005 0.064 0.007
naturalIndexDescendingIteration 1024 0.006 0.004 0.253 0.009
naturalIndexIteration 1024 0.006 0.004 0.069 0.010
randomDeleteIndexed 1024 0.027 0.019 0.243 0.037
randomDeletesNoIndex 1024 0.015 0.013 0.042 0.017
randomUpdatesIndexed 1024 0.082 0.033 30.571 0.087
randomUpdatesNoIndex 1024 0.034 0.031 0.625 0.037
randomWritesIndexed 1024 0.118 0.034 52.687 0.128
randomWritesNoIndex 1024 0.040 0.034 1.923 0.045
refIndexCreateIterator 1024 0.004 0.004 0.018 0.006
refIndexDescendingCreateIterator 1024 0.003 0.002 0.033 0.004
refIndexDescendingIteration 1024 0.007 0.005 0.047 0.010
refIndexIteration 1024 0.007 0.005 0.077 0.010
sequentialDeleteIndexed 1024 0.021 0.017 0.103 0.025
sequentialDeleteNoIndex 1024 0.015 0.012 0.067 0.017
sequentialUpdatesIndexed 1024 0.042 0.037 0.817 0.052
sequentialUpdatesNoIndex 1024 0.040 0.030 1.067 0.049
sequentialWritesIndexed 1024 0.048 0.039 2.049 0.055
sequentialWritesNoIndex 1024 0.039 0.032 2.421 0.041
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] LuciferYang commented on pull request #36522: [SPARK-39161][BUILD] Upgrade rocksdbjni to 7.2.2
Posted by GitBox <gi...@apache.org>.
LuciferYang commented on PR #36522:
URL: https://github.com/apache/spark/pull/36522#issuecomment-1125580153
thanks @srowen @dongjoon-hyun
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun closed pull request #36522: [SPARK-39161][BUILD] Upgrade rocksdbjni to 7.2.2
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun closed pull request #36522: [SPARK-39161][BUILD] Upgrade rocksdbjni to 7.2.2
URL: https://github.com/apache/spark/pull/36522
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org