You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by "dlmarion (via GitHub)" <gi...@apache.org> on 2025/12/10 18:16:32 UTC
[PR] Modify internals to use byte[] instead of String [accumulo-access]
dlmarion opened a new pull request, #89:
URL: https://github.com/apache/accumulo-access/pull/89
The current implementation does not allow for non-UTF8 characters to be used. This change alters the internal code to use byte[] instead of String to allow for that in AccessExpressions and Authorizations objecs.
Related to #88
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Modify internals to use byte[] instead of String [accumulo-access]
Posted by "dlmarion (via GitHub)" <gi...@apache.org>.
dlmarion commented on code in PR #89:
URL: https://github.com/apache/accumulo-access/pull/89#discussion_r2608448920
##########
src/main/java/org/apache/accumulo/access/StringUtils.java:
##########
@@ -0,0 +1,36 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.accumulo.access;
+
+import static java.nio.charset.StandardCharsets.UTF_8;
+
+/**
+ * Utilities for String to byte[] conversion and vice-versa
+ */
+public class StringUtils {
Review Comment:
I'm using it in the tests, which are in a different package. I guess I could *not* use it in the tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Modify internals to use byte[] instead of String [accumulo-access]
Posted by "keith-turner (via GitHub)" <gi...@apache.org>.
keith-turner commented on code in PR #89:
URL: https://github.com/apache/accumulo-access/pull/89#discussion_r2608441749
##########
src/main/java/org/apache/accumulo/access/BytesWrapper.java:
##########
@@ -22,10 +22,12 @@
import static java.util.Objects.checkFromIndexSize;
import static java.util.Objects.checkIndex;
+import java.io.Serializable;
import java.util.Arrays;
-final class BytesWrapper implements Comparable<BytesWrapper> {
+public final class BytesWrapper implements Comparable<BytesWrapper>, Serializable {
Review Comment:
If this will be in the public api then could just call it `Bytes`. It would need a since tag. Also could add a `byte[] toArray()` method that returns a copy of the array. The `byteAt()` method should be made public.
##########
src/main/java/org/apache/accumulo/access/StringUtils.java:
##########
@@ -0,0 +1,36 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.accumulo.access;
+
+import static java.nio.charset.StandardCharsets.UTF_8;
+
+/**
+ * Utilities for String to byte[] conversion and vice-versa
+ */
+public class StringUtils {
Review Comment:
Anything public in this package will be in the public API. Is this intended to be in the public API? Feels out of scope to me.
```suggestion
class StringUtils {
```
##########
src/main/java/org/apache/accumulo/access/AccessExpressionImpl.java:
##########
@@ -18,40 +18,35 @@
*/
package org.apache.accumulo.access;
-import static java.nio.charset.StandardCharsets.UTF_8;
+import static org.apache.accumulo.access.ByteUtils.EMPTY_BYTES;
+import java.util.Arrays;
import java.util.concurrent.atomic.AtomicReference;
final class AccessExpressionImpl extends AccessExpression {
private static final long serialVersionUID = 1L;
- public static final AccessExpression EMPTY = new AccessExpressionImpl("");
+ public static final AccessExpression EMPTY = new AccessExpressionImpl(EMPTY_BYTES);
- private final String expression;
+ private final byte[] expression;
private final AtomicReference<ParsedAccessExpression> parseTreeRef = new AtomicReference<>();
- AccessExpressionImpl(String expression) {
- validate(expression);
- this.expression = expression;
- }
-
AccessExpressionImpl(byte[] expression) {
validate(expression);
- this.expression = new String(expression, UTF_8);
+ this.expression = Arrays.copyOf(expression, expression.length);
}
@Override
- public String getExpression() {
+ public byte[] getExpression() {
return expression;
Review Comment:
This needs to copy if we want to keep AccessExpressions immutable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Modify internals to use byte[] instead of String [accumulo-access]
Posted by "dlmarion (via GitHub)" <gi...@apache.org>.
dlmarion commented on code in PR #89:
URL: https://github.com/apache/accumulo-access/pull/89#discussion_r2611202812
##########
src/build/ci/find-unapproved-public.sh:
##########
@@ -24,9 +24,20 @@ count=$(grep -E "public.*(class|interface|enum|record)" src/main/java/org/apache
grep -v " class ParsedAccessExpression " |
grep -v " enum ExpressionType " |
grep -v " class Authorizations " |
+ grep -v " class Bytes " |
grep -c -v " class InvalidAccessExpressionException ")
+files=$(grep -E "public.*(class|interface|enum|record)" src/main/java/org/apache/accumulo/access/*.java |
Review Comment:
There is probably an easier way to get the count of the `files` array, but I didn't look that hard at the `bash` man page.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Modify internals to use byte[] instead of String [accumulo-access]
Posted by "dlmarion (via GitHub)" <gi...@apache.org>.
dlmarion commented on PR #89:
URL: https://github.com/apache/accumulo-access/pull/89#issuecomment-3670116067
Closing this based on discussions regarding continuing to use String for Accumulo Access and potentlally moving any byte[] handling code for backwards compatibility with existing Accumulo instances into the Accumulo codebase.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Modify internals to use byte[] instead of String [accumulo-access]
Posted by "dlmarion (via GitHub)" <gi...@apache.org>.
dlmarion commented on PR #89:
URL: https://github.com/apache/accumulo-access/pull/89#issuecomment-3638381521
The unit test pass, but I have not run the benchmark yet. I have not modified any of the ANTLR example code down in src/it yet, but that likely doesn't need to be modified until the specification is altered to remove UTF-8.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Modify internals to use byte[] instead of String [accumulo-access]
Posted by "dlmarion (via GitHub)" <gi...@apache.org>.
dlmarion commented on PR #89:
URL: https://github.com/apache/accumulo-access/pull/89#issuecomment-3639260145
Ran the benchmark on the main branch and this branch. Results are very similar:
```
Main branch
----
Benchmark Mode Cnt Score Error Units
AccessExpressionBenchmark.measureBytesValidation thrpt 12 36.558 ± 0.440 ops/us
AccessExpressionBenchmark.measureLegacyEvaluationOnly thrpt 12 22.242 ± 0.467 ops/us
AccessExpressionBenchmark.measureLegacyParseAndEvaluation thrpt 12 9.696 ± 0.343 ops/us
AccessExpressionBenchmark.measureParseAndEvaluation thrpt 12 16.697 ± 0.062 ops/us
AccessExpressionBenchmark.measureStringValidation thrpt 12 29.350 ± 0.336 ops/us
PR Branch
----
Benchmark Mode Cnt Score Error Units
AccessExpressionBenchmark.measureBytesValidation thrpt 12 36.803 ± 0.143 ops/us
AccessExpressionBenchmark.measureLegacyEvaluationOnly thrpt 12 22.231 ± 0.196 ops/us
AccessExpressionBenchmark.measureLegacyParseAndEvaluation thrpt 12 9.545 ± 0.049 ops/us
AccessExpressionBenchmark.measureParseAndEvaluation thrpt 12 16.007 ± 0.401 ops/us
AccessExpressionBenchmark.measureStringValidation thrpt 12 29.027 ± 0.077 ops/us
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Modify internals to use byte[] instead of String [accumulo-access]
Posted by "dlmarion (via GitHub)" <gi...@apache.org>.
dlmarion commented on code in PR #89:
URL: https://github.com/apache/accumulo-access/pull/89#discussion_r2611199565
##########
src/main/java/org/apache/accumulo/access/StringUtils.java:
##########
@@ -0,0 +1,36 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.accumulo.access;
+
+import static java.nio.charset.StandardCharsets.UTF_8;
+
+/**
+ * Utilities for String to byte[] conversion and vice-versa
+ */
+public class StringUtils {
Review Comment:
Reverted this in e04689e. Modified the tests to have their own methods.
##########
src/main/java/org/apache/accumulo/access/AccessExpressionImpl.java:
##########
@@ -18,40 +18,35 @@
*/
package org.apache.accumulo.access;
-import static java.nio.charset.StandardCharsets.UTF_8;
+import static org.apache.accumulo.access.ByteUtils.EMPTY_BYTES;
+import java.util.Arrays;
import java.util.concurrent.atomic.AtomicReference;
final class AccessExpressionImpl extends AccessExpression {
private static final long serialVersionUID = 1L;
- public static final AccessExpression EMPTY = new AccessExpressionImpl("");
+ public static final AccessExpression EMPTY = new AccessExpressionImpl(EMPTY_BYTES);
- private final String expression;
+ private final byte[] expression;
private final AtomicReference<ParsedAccessExpression> parseTreeRef = new AtomicReference<>();
- AccessExpressionImpl(String expression) {
- validate(expression);
- this.expression = expression;
- }
-
AccessExpressionImpl(byte[] expression) {
validate(expression);
- this.expression = new String(expression, UTF_8);
+ this.expression = Arrays.copyOf(expression, expression.length);
}
@Override
- public String getExpression() {
+ public byte[] getExpression() {
return expression;
Review Comment:
Implemented in e04689e.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Modify internals to use byte[] instead of String [accumulo-access]
Posted by "dlmarion (via GitHub)" <gi...@apache.org>.
dlmarion commented on code in PR #89:
URL: https://github.com/apache/accumulo-access/pull/89#discussion_r2611198341
##########
src/main/java/org/apache/accumulo/access/BytesWrapper.java:
##########
@@ -22,10 +22,12 @@
import static java.util.Objects.checkFromIndexSize;
import static java.util.Objects.checkIndex;
+import java.io.Serializable;
import java.util.Arrays;
-final class BytesWrapper implements Comparable<BytesWrapper> {
+public final class BytesWrapper implements Comparable<BytesWrapper>, Serializable {
Review Comment:
Backed out this change in e04689e. Renamed BytesWrapper to BytesImpl and created a Bytes interface.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Modify internals to use byte[] instead of String [accumulo-access]
Posted by "dlmarion (via GitHub)" <gi...@apache.org>.
dlmarion closed pull request #89: Modify internals to use byte[] instead of String
URL: https://github.com/apache/accumulo-access/pull/89
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org