You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "swamirishi (via GitHub)" <gi...@apache.org> on 2025/12/24 13:24:43 UTC
[PR] HDDS-14238. Move RDBBatchOperation Byte comparison to native comparison for optimization [ozone]
swamirishi opened a new pull request, #9550:
URL: https://github.com/apache/ozone/pull/9550
## What changes were proposed in this pull request?
Instead of performing ByteWise comparison on Java side moving the comparison to native side would more optimal since most of the transactions are going to use direct byte buffers. Here we intend to use the rocksdb Slice to perform bytewise comparisons. RocksDB Bytewise comparators also uses the same comparator.
https://github.com/facebook/rocksdb/blob/c110091d368b8a01b5be36a14198769e60786c05/util/comparator.cc#L36-L38
Unfortunately currently in 7.7.3 there is no direct native comparison that has been implemented. We can look into contributing to Rocksdb by having a JNI implementation for
https://github.com/facebook/rocksdb/blob/0bf9079d44eea91afda7151306d3a3439a39511b/java/src/main/java/org/rocksdb/NativeComparatorWrapper.java#L16
Till then this is a nice workaround to have.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-14238
## How was this patch tested?
Update existing unit tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14238. Move RDBBatchOperation Byte comparison to native comparison for optimization [ozone]
Posted by "swamirishi (via GitHub)" <gi...@apache.org>.
swamirishi commented on PR #9550:
URL: https://github.com/apache/ozone/pull/9550#issuecomment-3695742750
> > One can always check the CI run on the fork. ...
>
> Please provide the link then. I would like to save my time for finding it.
https://github.com/swamirishi/ozone/actions/runs/20561058088
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14238. Move RDBBatchOperation Byte comparison to native comparison for optimization [ozone]
Posted by "swamirishi (via GitHub)" <gi...@apache.org>.
swamirishi commented on code in PR #9550:
URL: https://github.com/apache/ozone/pull/9550#discussion_r2651491930
##########
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/RDBBatchOperation.java:
##########
@@ -127,12 +131,21 @@ public int hashCode() {
@Override
public String toString() {
- return array != null ? bytes2String(array)
- : bytes2String(asReadOnlyByteBuffer());
+ return slice.toString();
Review Comment:
If we want the binary value then we have to print the hex value.
slice.toString(true) would do that
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14238. Move RDBBatchOperation Byte comparison to native comparison for optimization [ozone]
Posted by "szetszwo (via GitHub)" <gi...@apache.org>.
szetszwo commented on code in PR #9550:
URL: https://github.com/apache/ozone/pull/9550#discussion_r2651410596
##########
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/RDBBatchOperation.java:
##########
@@ -127,12 +131,21 @@ public int hashCode() {
@Override
public String toString() {
- return array != null ? bytes2String(array)
- : bytes2String(asReadOnlyByteBuffer());
+ return slice.toString();
Review Comment:
Isn't the data support to be binary? This seems not working.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14238. Move RDBBatchOperation Byte comparison to native comparison for optimization [ozone]
Posted by "szetszwo (via GitHub)" <gi...@apache.org>.
szetszwo commented on code in PR #9550:
URL: https://github.com/apache/ozone/pull/9550#discussion_r2651410596
##########
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/RDBBatchOperation.java:
##########
@@ -127,12 +131,21 @@ public int hashCode() {
@Override
public String toString() {
- return array != null ? bytes2String(array)
- : bytes2String(asReadOnlyByteBuffer());
+ return slice.toString();
Review Comment:
> To String Heap Value: test
> To String Direct Value: test
Isn't the data supposed to be binary? This seems not working.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14238. Move RDBBatchOperation Byte comparison to native comparison for optimization [ozone]
Posted by "swamirishi (via GitHub)" <gi...@apache.org>.
swamirishi commented on PR #9550:
URL: https://github.com/apache/ozone/pull/9550#issuecomment-3695133816
> > ... I will click on ready for review when I get a +1. I don't want to run the CI multiple times.
>
> @swamirishi , we should pass the CI first before asking for reviewing -- if the change cannot pass CI, it may be fundamentally incorrect. What is the point for reviewing?
>
> ⚠️ Also, reviewer's time/effort is much more important than saving the machines' time/effort.
One can always check the CI run on the fork
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14238. Move RDBBatchOperation Byte comparison to native comparison for optimization [ozone]
Posted by "swamirishi (via GitHub)" <gi...@apache.org>.
swamirishi commented on code in PR #9550:
URL: https://github.com/apache/ozone/pull/9550#discussion_r2651427857
##########
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/RDBBatchOperation.java:
##########
@@ -127,12 +131,21 @@ public int hashCode() {
@Override
public String toString() {
- return array != null ? bytes2String(array)
- : bytes2String(asReadOnlyByteBuffer());
+ return slice.toString();
Review Comment:
It is supposed to be the UTF-8 Decoded value. https://github.com/facebook/rocksdb/blob/1cfdece85ddc9450d24c3007222195e46e1cc0a8/util/slice.cc#L303-L304
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14238. Move RDBBatchOperation Byte comparison to native comparison for optimization [ozone]
Posted by "swamirishi (via GitHub)" <gi...@apache.org>.
swamirishi commented on code in PR #9550:
URL: https://github.com/apache/ozone/pull/9550#discussion_r2651436549
##########
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/RDBBatchOperation.java:
##########
@@ -127,12 +131,21 @@ public int hashCode() {
@Override
public String toString() {
- return array != null ? bytes2String(array)
- : bytes2String(asReadOnlyByteBuffer());
+ return slice.toString();
Review Comment:
When you say it is not working. Maybe you are not loading the rocksdb libs before this. I forgot there was static codeblock loading the rocksdb lib.
This should work:
```
public static void main(String[] args) throws CodecException {
ManagedRocksObjectUtils.loadRocksDBLibrary();
String value = "test";
Bytes bytes = new Bytes(value.getBytes(UTF_8));
System.out.println("To String Heap Value: " + bytes);
Bytes dbytes = newBytes(CodecBufferCodec.get(true).fromPersistedFormat(value.getBytes(UTF_8)));
System.out.println("To String Direct Value: " + dbytes);
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14238. Move RDBBatchOperation Byte comparison to native comparison for optimization [ozone]
Posted by "swamirishi (via GitHub)" <gi...@apache.org>.
swamirishi commented on code in PR #9550:
URL: https://github.com/apache/ozone/pull/9550#discussion_r2651446208
##########
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/RDBBatchOperation.java:
##########
@@ -127,12 +131,21 @@ public int hashCode() {
@Override
public String toString() {
- return array != null ? bytes2String(array)
- : bytes2String(asReadOnlyByteBuffer());
+ return slice.toString();
Review Comment:
The output before the change and after will be the same since StringUtils.byte2String() also decodes the byte value to String value
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14238. Move RDBBatchOperation Byte comparison to native comparison for optimization [ozone]
Posted by "szetszwo (via GitHub)" <gi...@apache.org>.
szetszwo commented on code in PR #9550:
URL: https://github.com/apache/ozone/pull/9550#discussion_r2651502983
##########
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/RDBBatchOperation.java:
##########
@@ -127,12 +131,21 @@ public int hashCode() {
@Override
public String toString() {
- return array != null ? bytes2String(array)
- : bytes2String(asReadOnlyByteBuffer());
+ return slice.toString();
Review Comment:
You are right -- it is the same as before.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14238. Move RDBBatchOperation Byte comparison to native comparison for optimization [ozone]
Posted by "szetszwo (via GitHub)" <gi...@apache.org>.
szetszwo commented on PR #9550:
URL: https://github.com/apache/ozone/pull/9550#issuecomment-3695114351
> ... I will click on ready for review when I get a +1. I don't want to run the CI multiple times.
@swamirishi , we should pass the CI first before asking for reviewing -- if the change cannot pass CI, it may be fundamentally incorrect. What is the point for reviewing?
⚠️ Also, reviewer's time/effort is much more important than saving the machines' time/effort.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14238. Move RDBBatchOperation Byte comparison to native comparison for optimization [ozone]
Posted by "swamirishi (via GitHub)" <gi...@apache.org>.
swamirishi commented on PR #9550:
URL: https://github.com/apache/ozone/pull/9550#issuecomment-3694885897
@szetszwo This is good to be reviewed. I will click on ready for review when I get a +1. I don't want to run the CI multiple times.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14238. Move RDBBatchOperation Byte comparison to native comparison for optimization [ozone]
Posted by "szetszwo (via GitHub)" <gi...@apache.org>.
szetszwo commented on code in PR #9550:
URL: https://github.com/apache/ozone/pull/9550#discussion_r2649967812
##########
hadoop-hdds/framework/src/test/java/org/apache/hadoop/hdds/utils/db/TestCodec.java:
##########
@@ -295,14 +298,13 @@ public static <T> void runTest(Codec<T> codec, T original,
static <T> void runTestBytes(T object, Codec<T> codec) throws IOException {
final byte[] array = codec.toPersistedFormat(object);
final Bytes fromArray = new Bytes(array);
-
- try (CodecBuffer buffer = codec.toCodecBuffer(object,
- CodecBuffer.Allocator.HEAP)) {
- final Bytes fromBuffer = new Bytes(buffer);
-
- assertEquals(fromArray.hashCode(), fromBuffer.hashCode());
- assertEquals(fromArray, fromBuffer);
- assertEquals(fromBuffer, fromArray);
+ for (CodecBuffer.Allocator allocator : ImmutableList.of(CodecBuffer.Allocator.HEAP, CodecBuffer.Allocator.DIRECT)) {
Review Comment:
Pass allocator instead of adding a loop.
```java
runTestBytes(original, codec, CodecBuffer.Allocator.HEAP);
runTestBytes(original, codec, CodecBuffer.Allocator.DIRECT);
}
static <T> void runTestBytes(T object, Codec<T> codec, CodecBuffer.Allocator allocator) throws IOException {
final byte[] array = codec.toPersistedFormat(object);
final Bytes fromArray = new Bytes(array);
try (CodecBuffer buffer = codec.toCodecBuffer(object, allocator)) {
```
##########
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/RDBBatchOperation.java:
##########
@@ -80,26 +81,33 @@ private static String countSize2String(int count, long size) {
* To implement {@link #equals(Object)} and {@link #hashCode()}
* based on the contents of the bytes.
*/
- static final class Bytes {
- private final byte[] array;
- private final CodecBuffer buffer;
+ static final class Bytes implements Closeable {
+ private AbstractSlice<?> slice;
/** Cache the hash value. */
- private final int hash;
+ private int hash;
Bytes(CodecBuffer buffer) {
- this.array = null;
- this.buffer = Objects.requireNonNull(buffer, "buffer == null");
- this.hash = buffer.asReadOnlyByteBuffer().hashCode();
+ Objects.requireNonNull(buffer, "buffer == null");
+ if (buffer.isDirect()) {
+ initWithDirectByteBuffer(buffer.asReadOnlyByteBuffer());
+ } else {
+ initWithByteArray(buffer.getArray());
+ }
}
Bytes(byte[] array) {
- this.array = array;
- this.buffer = null;
+ Objects.requireNonNull(array, "array == null");
+ initWithByteArray(array);
+ }
+
+ private void initWithByteArray(byte[] array) {
+ this.slice = new ManagedSlice(array);
this.hash = ByteBuffer.wrap(array).hashCode();
}
- ByteBuffer asReadOnlyByteBuffer() {
- return buffer.asReadOnlyByteBuffer();
+ private void initWithDirectByteBuffer(ByteBuffer byteBuffer) {
+ this.slice = new ManagedDirectSlice(byteBuffer);
+ this.hash = byteBuffer.hashCode();
}
Review Comment:
Constructors should not call any functions. Otherwise, the fields cannot be final, which is very useful for reducing bugs and for understanding the code..
```java
private final AbstractSlice<?> slice;
/** Cache the hash value. */
private final int hash;
static Bytes newBytes(CodecBuffer buffer) {
Objects.requireNonNull(buffer, "buffer == null");
return buffer.isDirect() ? new Bytes(buffer.asReadOnlyByteBuffer()) : new Bytes(buffer.getArray());
}
Bytes(ByteBuffer buffer) {
Objects.requireNonNull(buffer, "buffer == null");
assertTrue(buffer.isDirect(), "buffer is not direct");
this.slice = new ManagedDirectSlice(buffer);
this.hash = buffer.hashCode();
}
Bytes(byte[] array) {
Objects.requireNonNull(array, "array == null");
this.slice = new ManagedSlice(array);
this.hash = ByteBuffer.wrap(array).hashCode();
}
```
##########
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/RDBBatchOperation.java:
##########
@@ -127,12 +131,21 @@ public int hashCode() {
@Override
public String toString() {
- return array != null ? bytes2String(array)
- : bytes2String(asReadOnlyByteBuffer());
+ return slice.toString();
Review Comment:
Have you checked what will it return? Could you show an example output?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14238. Move RDBBatchOperation Byte comparison to native comparison for optimization [ozone]
Posted by "swamirishi (via GitHub)" <gi...@apache.org>.
swamirishi merged PR #9550:
URL: https://github.com/apache/ozone/pull/9550
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14238. Move RDBBatchOperation Byte comparison to native comparison for optimization [ozone]
Posted by "swamirishi (via GitHub)" <gi...@apache.org>.
swamirishi commented on code in PR #9550:
URL: https://github.com/apache/ozone/pull/9550#discussion_r2651427857
##########
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/RDBBatchOperation.java:
##########
@@ -127,12 +131,21 @@ public int hashCode() {
@Override
public String toString() {
- return array != null ? bytes2String(array)
- : bytes2String(asReadOnlyByteBuffer());
+ return slice.toString();
Review Comment:
It is supposed to be the char decoded value. https://github.com/facebook/rocksdb/blob/1cfdece85ddc9450d24c3007222195e46e1cc0a8/util/slice.cc#L303-L304
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14238. Move RDBBatchOperation Byte comparison to native comparison for optimization [ozone]
Posted by "swamirishi (via GitHub)" <gi...@apache.org>.
swamirishi commented on PR #9550:
URL: https://github.com/apache/ozone/pull/9550#issuecomment-3697188933
Thank you @szetszwo for reviewing the patch
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14238. Move RDBBatchOperation Byte comparison to native comparison for optimization [ozone]
Posted by "swamirishi (via GitHub)" <gi...@apache.org>.
swamirishi commented on code in PR #9550:
URL: https://github.com/apache/ozone/pull/9550#discussion_r2649991569
##########
hadoop-hdds/framework/src/test/java/org/apache/hadoop/hdds/utils/db/TestCodec.java:
##########
@@ -295,14 +298,13 @@ public static <T> void runTest(Codec<T> codec, T original,
static <T> void runTestBytes(T object, Codec<T> codec) throws IOException {
final byte[] array = codec.toPersistedFormat(object);
final Bytes fromArray = new Bytes(array);
-
- try (CodecBuffer buffer = codec.toCodecBuffer(object,
- CodecBuffer.Allocator.HEAP)) {
- final Bytes fromBuffer = new Bytes(buffer);
-
- assertEquals(fromArray.hashCode(), fromBuffer.hashCode());
- assertEquals(fromArray, fromBuffer);
- assertEquals(fromBuffer, fromArray);
+ for (CodecBuffer.Allocator allocator : ImmutableList.of(CodecBuffer.Allocator.HEAP, CodecBuffer.Allocator.DIRECT)) {
Review Comment:
done
##########
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/RDBBatchOperation.java:
##########
@@ -80,26 +81,33 @@ private static String countSize2String(int count, long size) {
* To implement {@link #equals(Object)} and {@link #hashCode()}
* based on the contents of the bytes.
*/
- static final class Bytes {
- private final byte[] array;
- private final CodecBuffer buffer;
+ static final class Bytes implements Closeable {
+ private AbstractSlice<?> slice;
/** Cache the hash value. */
- private final int hash;
+ private int hash;
Bytes(CodecBuffer buffer) {
- this.array = null;
- this.buffer = Objects.requireNonNull(buffer, "buffer == null");
- this.hash = buffer.asReadOnlyByteBuffer().hashCode();
+ Objects.requireNonNull(buffer, "buffer == null");
+ if (buffer.isDirect()) {
+ initWithDirectByteBuffer(buffer.asReadOnlyByteBuffer());
+ } else {
+ initWithByteArray(buffer.getArray());
+ }
}
Bytes(byte[] array) {
- this.array = array;
- this.buffer = null;
+ Objects.requireNonNull(array, "array == null");
+ initWithByteArray(array);
+ }
+
+ private void initWithByteArray(byte[] array) {
+ this.slice = new ManagedSlice(array);
this.hash = ByteBuffer.wrap(array).hashCode();
}
- ByteBuffer asReadOnlyByteBuffer() {
- return buffer.asReadOnlyByteBuffer();
+ private void initWithDirectByteBuffer(ByteBuffer byteBuffer) {
+ this.slice = new ManagedDirectSlice(byteBuffer);
+ this.hash = byteBuffer.hashCode();
}
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14238. Move RDBBatchOperation Byte comparison to native comparison for optimization [ozone]
Posted by "swamirishi (via GitHub)" <gi...@apache.org>.
swamirishi commented on PR #9550:
URL: https://github.com/apache/ozone/pull/9550#issuecomment-3696287017
@szetszwo i have addressed all your review comments is ithis good to be merged?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14238. Move RDBBatchOperation Byte comparison to native comparison for optimization [ozone]
Posted by "szetszwo (via GitHub)" <gi...@apache.org>.
szetszwo commented on PR #9550:
URL: https://github.com/apache/ozone/pull/9550#issuecomment-3695164410
> One can always check the CI run on the fork. ...
Please provide the link then. I would like to save my time for finding it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14238. Move RDBBatchOperation Byte comparison to native comparison for optimization [ozone]
Posted by "swamirishi (via GitHub)" <gi...@apache.org>.
swamirishi commented on code in PR #9550:
URL: https://github.com/apache/ozone/pull/9550#discussion_r2649990243
##########
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/RDBBatchOperation.java:
##########
@@ -127,12 +131,21 @@ public int hashCode() {
@Override
public String toString() {
- return array != null ? bytes2String(array)
- : bytes2String(asReadOnlyByteBuffer());
+ return slice.toString();
Review Comment:
```
public static void main(String[] args) throws CodecException {
String value = "test";
Bytes bytes = new Bytes(value.getBytes(UTF_8));
System.out.println("To String Heap Value: " + bytes);
Bytes dbytes = newBytes(CodecBufferCodec.get(true).fromPersistedFormat(value.getBytes(UTF_8)));
System.out.println("To String Direct Value: " + dbytes);
}
```
```
2025-12-28 18:25:34,786 [main] INFO db.CodecBuffer (CodecBuffer.java:set(85)) - Successfully set constructor to LeakDetector::newCodecBuffer: org.apache.hadoop.hdds.utils.db.CodecBuffer$$Lambda$4/1694556038@7085bdee
To String Heap Value: test
To String Direct Value: test
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14238. Move RDBBatchOperation Byte comparison to native comparison for optimization [ozone]
Posted by "swamirishi (via GitHub)" <gi...@apache.org>.
swamirishi commented on code in PR #9550:
URL: https://github.com/apache/ozone/pull/9550#discussion_r2651446208
##########
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/RDBBatchOperation.java:
##########
@@ -127,12 +131,21 @@ public int hashCode() {
@Override
public String toString() {
- return array != null ? bytes2String(array)
- : bytes2String(asReadOnlyByteBuffer());
+ return slice.toString();
Review Comment:
The output before the change and after will be the same
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14238. Move RDBBatchOperation Byte comparison to native comparison for optimization [ozone]
Posted by "szetszwo (via GitHub)" <gi...@apache.org>.
szetszwo commented on code in PR #9550:
URL: https://github.com/apache/ozone/pull/9550#discussion_r2651410596
##########
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/RDBBatchOperation.java:
##########
@@ -127,12 +131,21 @@ public int hashCode() {
@Override
public String toString() {
- return array != null ? bytes2String(array)
- : bytes2String(asReadOnlyByteBuffer());
+ return slice.toString();
Review Comment:
Isn't the data supposed to be binary? This seems not working.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-14238. Move RDBBatchOperation Byte comparison to native comparison for optimization [ozone]
Posted by "swamirishi (via GitHub)" <gi...@apache.org>.
swamirishi commented on code in PR #9550:
URL: https://github.com/apache/ozone/pull/9550#discussion_r2651492678
##########
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/db/RDBBatchOperation.java:
##########
@@ -127,12 +131,21 @@ public int hashCode() {
@Override
public String toString() {
- return array != null ? bytes2String(array)
- : bytes2String(asReadOnlyByteBuffer());
+ return slice.toString();
Review Comment:
I did the above to keep the behaviour same as before.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org