You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "swamirishi (via GitHub)" <gi...@apache.org> on 2025/12/22 16:36:49 UTC
[PR] HDDS-14226. [DO NOT MERGE]Get rid of Native Libs and use JNI for SSTFileTableIterator in rocksdb [ozone]
swamirishi opened a new pull request, #9544:
URL: https://github.com/apache/ozone/pull/9544
## What changes were proposed in this pull request?
Patch : https://github.com/facebook/rocksdb/pull/12511 in rocksdb creates a new JNI for the tableIterator API in SSTFileReader thus eliminating the need for the patched rocksdb native lib written in ozone for efficient snapshot diff. This patch would be responsible for bumping up rocksdb to a version containing the afforementioned rocksdb patch and removing all usages of the native lib from the code base which needs to be done in a single patch since the native lib won't build with the new rocksdb version and making the native lib compatible with newer rocksdb version is a duplication of effort.
This patch will not build currently since this needs a future version of rocksdb with the patch. This code has been tested with a local build of rocksdb containing the patch.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-14226
## How was this patch tested?
Added unit tests for the changed code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14226. [DO NOT MERGE] Get rid of Native Libs and use JNI for SSTFileTableIterator in rocksdb [ozone]
Posted by "swamirishi (via GitHub)" <gi...@apache.org>.
swamirishi commented on PR #9544:
URL: https://github.com/apache/ozone/pull/9544#issuecomment-3690135722
Thanks @peterxcli I will take a look at this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14226. [DO NOT MERGE] Get rid of Native Libs and use JNI for SSTFileTableIterator in rocksdb [ozone]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #9544: HDDS-14226. [DO NOT MERGE] Get rid of Native Libs and use JNI for SSTFileTableIterator in rocksdb
URL: https://github.com/apache/ozone/pull/9544
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14226. [DO NOT MERGE] Get rid of Native Libs and use JNI for SSTFileTableIterator in rocksdb [ozone]
Posted by "swamirishi (via GitHub)" <gi...@apache.org>.
swamirishi commented on PR #9544:
URL: https://github.com/apache/ozone/pull/9544#issuecomment-3682899420
@adoroszlai I was just wondering if I can do something similar for running CI with newer rocksdb version from a different maven repo. Any idea how I can do this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14226. [DO NOT MERGE] Get rid of Native Libs and use JNI for SSTFileTableIterator in rocksdb [ozone]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #9544: HDDS-14226. [DO NOT MERGE] Get rid of Native Libs and use JNI for SSTFileTableIterator in rocksdb
URL: https://github.com/apache/ozone/pull/9544
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14226. [DO NOT MERGE] Get rid of Native Libs and use JNI for SSTFileTableIterator in rocksdb [ozone]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #9544:
URL: https://github.com/apache/ozone/pull/9544#issuecomment-4227417413
Thank you for your contribution. This PR is being closed due to inactivity. Please contact a maintainer if you would like to reopen it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14226. [DO NOT MERGE] Get rid of Native Libs and use JNI for SSTFileTableIterator in rocksdb [ozone]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #9544:
URL: https://github.com/apache/ozone/pull/9544#issuecomment-4001106927
This PR has been marked as stale due to 21 days of inactivity. Please comment or remove the stale label to keep it open. Otherwise, it will be automatically closed in 7 days.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14226. [DO NOT MERGE] Get rid of Native Libs and use JNI for SSTFileTableIterator in rocksdb [ozone]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #9544:
URL: https://github.com/apache/ozone/pull/9544#issuecomment-3821055123
This PR has been marked as stale due to 21 days of inactivity. Please comment or remove the stale label to keep it open. Otherwise, it will be automatically closed in 7 days.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14226. [DO NOT MERGE] Get rid of Native Libs and use JNI for SSTFileTableIterator in rocksdb [ozone]
Posted by "swamirishi (via GitHub)" <gi...@apache.org>.
swamirishi commented on PR #9544:
URL: https://github.com/apache/ozone/pull/9544#issuecomment-3711492141
@rnblough This is the patch for upgrading our rocksdb version
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14226. [DO NOT MERGE] Get rid of Native Libs and use JNI for SSTFileTableIterator in rocksdb [ozone]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #9544:
URL: https://github.com/apache/ozone/pull/9544#issuecomment-4185746688
This PR has been marked as stale due to 21 days of inactivity. Please comment or remove the stale label to keep it open. Otherwise, it will be automatically closed in 7 days.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14226. [DO NOT MERGE] Get rid of Native Libs and use JNI for SSTFileTableIterator in rocksdb [ozone]
Posted by "smengcl (via GitHub)" <gi...@apache.org>.
smengcl commented on PR #9544:
URL: https://github.com/apache/ozone/pull/9544#issuecomment-3883368376
Still blocked on https://github.com/facebook/rocksdb/pull/12511
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14226. [DO NOT MERGE] Get rid of Native Libs and use JNI for SSTFileTableIterator in rocksdb [ozone]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #9544:
URL: https://github.com/apache/ozone/pull/9544#issuecomment-4051230241
Thank you for your contribution. This PR is being closed due to inactivity. Please contact a maintainer if you would like to reopen it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14226. [DO NOT MERGE] Get rid of Native Libs and use JNI for SSTFileTableIterator in rocksdb [ozone]
Posted by "smengcl (via GitHub)" <gi...@apache.org>.
smengcl commented on code in PR #9544:
URL: https://github.com/apache/ozone/pull/9544#discussion_r2674265698
##########
hadoop-ozone/integration-test/ozone_rocksdb_tools3729431883314919784/libozone_rocksdb_tools.dylib:
##########
Review Comment:
TODO: Remove this
##########
hadoop-ozone/integration-test/ozone_rocksdb_tools3729431883314919784/librocksdbjni-osx-arm64.jnilib:
##########
Review Comment:
TODO: Remove this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14226. [DO NOT MERGE] Get rid of Native Libs and use JNI for SSTFileTableIterator in rocksdb [ozone]
Posted by "peterxcli (via GitHub)" <gi...@apache.org>.
peterxcli commented on PR #9544:
URL: https://github.com/apache/ozone/pull/9544#issuecomment-3685214733
@swamirishi I tried to the same thing before to build my own rocksdb include the patch to fix the bug in the `getPropertiesOfTablesInRange` JNI: https://github.com/peterxcli/rocksdb/commit/d9abdf0a1e0c84bc6a3b5f82c16d31a1162eef67
- https://github.com/peterxcli/rocksdb/packages/2527529 (maven repo on github, include the build step)
- https://github.com/peterxcli/rocksdb/releases/tag/v7.7.3.2
It was used in my PR: https://github.com/peterxcli/ozone/pull/2 (able to pass the ci: https://github.com/peterxcli/ozone/actions/runs/15602888062)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org