You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "peterxcli (via GitHub)" <gi...@apache.org> on 2025/04/30 20:53:09 UTC
[PR] Introduce EventExecutorMetrics instead of set the metrics props unsafely [ozone]
peterxcli opened a new pull request, #8371:
URL: https://github.com/apache/ozone/pull/8371
## What changes were proposed in this pull request?
The error
```java
2025-04-24 16:22:54,652 [main] ERROR utils.MetricsUtil (MetricsUtil.java:updateAnnotation(97)) - Update Metrics annotation failed.
java.lang.reflect.InaccessibleObjectException: Unable to make private java.lang.Class$AnnotationData java.lang.Class.annotationData() accessible: module java.base does not "opens java.lang" to unnamed module @2c7b84de
```
keeps appearing in test log
And the root cause and solution can be found from https://stackoverflow.com/a/41265267
>The exception is caused by the [Java Platform Module System](http://openjdk.java.net/projects/jigsaw/spec/sotms) that was introduced in Java 9, particularly its implementation of strong encapsulation. It only allows [access](http://openjdk.java.net/projects/jigsaw/spec/sotms/#accessibility) under certain conditions,
> - the most prominent ones are: the type has to be public
> - the owning package has to be exported
And I had tried add some argument from the suggestion:
```bash
# --add-opens has the following syntax: {A}/{package}={B}
java --add-opens java.base/java.lang=ALL-UNNAMED
```
But the exception still appear in the test log, I think it might also not a good long term solution. So I created a common metrics class for those two type of event related thread executor. Removing the metrics code from them and the metrics also support customize name for every of its instance.
### Check that exception log is gone
Ran some example test
```bash
mvn test -pl hadoop-ozone/integration-test "-Dtest=TestValidateBCSIDOnRestart#testValidateBCSIDOnDnRestart"
```
test output:
[org.apache.hadoop.ozone.client.rpc.TestValidateBCSIDOnRestart-output.txt](https://github.com/user-attachments/files/19984184/org.apache.hadoop.ozone.client.rpc.TestValidateBCSIDOnRestart-output.txt)
### Metrics name correctness
```bash
ozone freon stb --scm-host=scm --benchmark=AllocateBlocks
```

## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-12901
## How was this patch tested?
CI:
https://github.com/peterxcli/ozone/actions/runs/14763747318
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12901. Introduce EventExecutorMetrics instead of setting the metrics props unsafely [ozone]
Posted by "peterxcli (via GitHub)" <gi...@apache.org>.
peterxcli commented on code in PR #8371:
URL: https://github.com/apache/ozone/pull/8371#discussion_r2069884829
##########
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/events/SingleThreadExecutor.java:
##########
@@ -77,42 +59,43 @@ public SingleThreadExecutor(String name, String threadNamePrefix) {
@Override
public void onMessage(EventHandler<P> handler, P message, EventPublisher
publisher) {
- queued.incr();
+ metrics.incrementQueued();
executor.execute(() -> {
- scheduled.incr();
+ metrics.incrementScheduled();
try {
handler.onMessage(message, publisher);
- done.incr();
+ metrics.incrementDone();
} catch (Exception ex) {
LOG.error("Error on execution message {}", message, ex);
- failed.incr();
+ metrics.incrementFailed();
}
});
}
@Override
public long failedEvents() {
- return failed.value();
+ return metrics.getFailed();
}
@Override
public long successfulEvents() {
- return done.value();
+ return metrics.getDone();
}
@Override
public long queuedEvents() {
- return queued.value();
+ return metrics.getQueued();
}
@Override
public long scheduledEvents() {
- return scheduled.value();
+ return metrics.getScheduled();
}
@Override
public void close() {
Review Comment:
The `eventQueue` is also explicitly closed by `StorageContainerManager`:
https://github.com/apache/ozone/blob/5cb6dd84d89f962e01500109db89f0da0ff84d29/hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/server/StorageContainerManager.java#L1694
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12901. Introduce EventExecutorMetrics instead of setting the metrics props unsafely [ozone]
Posted by "peterxcli (via GitHub)" <gi...@apache.org>.
peterxcli commented on PR #8371:
URL: https://github.com/apache/ozone/pull/8371#issuecomment-2846419956
Thanks @jojochuang for reviewing and merging!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12901. Introduce EventExecutorMetrics instead of setting the metrics props unsafely [ozone]
Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang merged PR #8371:
URL: https://github.com/apache/ozone/pull/8371
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12901. Introduce EventExecutorMetrics instead of set the metrics props unsafely [ozone]
Posted by "peterxcli (via GitHub)" <gi...@apache.org>.
peterxcli commented on PR #8371:
URL: https://github.com/apache/ozone/pull/8371#issuecomment-2843262579
trigger
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12901. Introduce EventExecutorMetrics instead of setting the metrics props unsafely [ozone]
Posted by "peterxcli (via GitHub)" <gi...@apache.org>.
peterxcli commented on code in PR #8371:
URL: https://github.com/apache/ozone/pull/8371#discussion_r2069882219
##########
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/events/SingleThreadExecutor.java:
##########
@@ -77,42 +59,43 @@ public SingleThreadExecutor(String name, String threadNamePrefix) {
@Override
public void onMessage(EventHandler<P> handler, P message, EventPublisher
publisher) {
- queued.incr();
+ metrics.incrementQueued();
executor.execute(() -> {
- scheduled.incr();
+ metrics.incrementScheduled();
try {
handler.onMessage(message, publisher);
- done.incr();
+ metrics.incrementDone();
} catch (Exception ex) {
LOG.error("Error on execution message {}", message, ex);
- failed.incr();
+ metrics.incrementFailed();
}
});
}
@Override
public long failedEvents() {
- return failed.value();
+ return metrics.getFailed();
}
@Override
public long successfulEvents() {
- return done.value();
+ return metrics.getDone();
}
@Override
public long queuedEvents() {
- return queued.value();
+ return metrics.getQueued();
}
@Override
public long scheduledEvents() {
- return scheduled.value();
+ return metrics.getScheduled();
}
@Override
public void close() {
Review Comment:
I guess they are closed at:
https://github.com/apache/ozone/blob/5cb6dd84d89f962e01500109db89f0da0ff84d29/hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/events/EventQueue.java#L286-L302
Those executors are init and bind to to the map when calling `executors.putIfAbsent(event, new HashMap<>()); `:
https://github.com/apache/ozone/blob/5cb6dd84d89f962e01500109db89f0da0ff84d29/hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/events/EventQueue.java#L158-L175
##########
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/events/SingleThreadExecutor.java:
##########
@@ -77,42 +59,43 @@ public SingleThreadExecutor(String name, String threadNamePrefix) {
@Override
public void onMessage(EventHandler<P> handler, P message, EventPublisher
publisher) {
- queued.incr();
+ metrics.incrementQueued();
executor.execute(() -> {
- scheduled.incr();
+ metrics.incrementScheduled();
try {
handler.onMessage(message, publisher);
- done.incr();
+ metrics.incrementDone();
} catch (Exception ex) {
LOG.error("Error on execution message {}", message, ex);
- failed.incr();
+ metrics.incrementFailed();
}
});
}
@Override
public long failedEvents() {
- return failed.value();
+ return metrics.getFailed();
}
@Override
public long successfulEvents() {
- return done.value();
+ return metrics.getDone();
}
@Override
public long queuedEvents() {
- return queued.value();
+ return metrics.getQueued();
}
@Override
public long scheduledEvents() {
- return scheduled.value();
+ return metrics.getScheduled();
}
@Override
public void close() {
Review Comment:
I think they are closed at:
https://github.com/apache/ozone/blob/5cb6dd84d89f962e01500109db89f0da0ff84d29/hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/events/EventQueue.java#L286-L302
Those executors are init and bind to to the map when calling `executors.putIfAbsent(event, new HashMap<>()); `:
https://github.com/apache/ozone/blob/5cb6dd84d89f962e01500109db89f0da0ff84d29/hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/events/EventQueue.java#L158-L175
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-12901. Introduce EventExecutorMetrics instead of setting the metrics props unsafely [ozone]
Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang commented on code in PR #8371:
URL: https://github.com/apache/ozone/pull/8371#discussion_r2069873000
##########
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/server/events/SingleThreadExecutor.java:
##########
@@ -77,42 +59,43 @@ public SingleThreadExecutor(String name, String threadNamePrefix) {
@Override
public void onMessage(EventHandler<P> handler, P message, EventPublisher
publisher) {
- queued.incr();
+ metrics.incrementQueued();
executor.execute(() -> {
- scheduled.incr();
+ metrics.incrementScheduled();
try {
handler.onMessage(message, publisher);
- done.incr();
+ metrics.incrementDone();
} catch (Exception ex) {
LOG.error("Error on execution message {}", message, ex);
- failed.incr();
+ metrics.incrementFailed();
}
});
}
@Override
public long failedEvents() {
- return failed.value();
+ return metrics.getFailed();
}
@Override
public long successfulEvents() {
- return done.value();
+ return metrics.getDone();
}
@Override
public long queuedEvents() {
- return queued.value();
+ return metrics.getQueued();
}
@Override
public long scheduledEvents() {
- return scheduled.value();
+ return metrics.getScheduled();
}
@Override
public void close() {
Review Comment:
I'm looking for the usage of these two thread executors and whether or not they are closed explicitly.
It looks like they are not closed explicitly so this may not be called properly. Probably ok now because they are only initialized once for Recon and SCM.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org