You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "neilramaswamy (via GitHub)" <gi...@apache.org> on 2024/01/10 23:36:08 UTC
[PR] [WIP][DO-NOT-MERGE] Upgrade rocksjni to 8.9.1 [spark]
neilramaswamy opened a new pull request, #44674:
URL: https://github.com/apache/spark/pull/44674
### What changes were proposed in this pull request?
- Upgrade `rocksjni` dependency to 8.9.1
### Why are the changes needed?
Due to the performance regression in [JDK 17 attach/detach calls](https://bugs.openjdk.org/browse/JDK-8314859), JDK 17 Structured Streaming workloads that stress RocksDB also experience a performance regression. We plan to mitigate this issue by writing RocksDB logs to `stderr` instead of the JVM Log4J logger, as implemented in [this RocksDB PR](https://github.com/facebook/rocksdb/pull/12213).
However, as that will likely come out in RocksDB 8.10, we'd like to incrementally upgrade to verify that there aren't issues with bumping all the way from 8.3.2 to 8.9.1.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
WIP. Likely:
- Benchmark tool for KV store
### Was this patch authored or co-authored using generative AI tooling?
<!--
If generative AI tooling has been used in the process of authoring this patch, please include the
phrase: 'Generated-by: ' followed by the name of the tool and its version.
If no, write 'No'.
Please refer to the [ASF Generative Tooling Guidance](https://www.apache.org/legal/generative-tooling.html) for details.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [WIP][DO-NOT-MERGE] Upgrade rocksjni to 8.9.1 [spark]
Posted by "neilramaswamy (via GitHub)" <gi...@apache.org>.
neilramaswamy closed pull request #44674: [WIP][DO-NOT-MERGE] Upgrade rocksjni to 8.9.1
URL: https://github.com/apache/spark/pull/44674
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [PR] [WIP][DO-NOT-MERGE] Upgrade rocksjni to 8.9.1 [spark]
Posted by "neilramaswamy (via GitHub)" <gi...@apache.org>.
neilramaswamy commented on PR #44674:
URL: https://github.com/apache/spark/pull/44674#issuecomment-1885941820
Oops, closing as there is existing work in SPARK-45110 to upgrade to 8.8.1, which is already doing plenty of validation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org