You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by "wu-sheng (via GitHub)" <gi...@apache.org> on 2025/08/26 19:31:37 UTC
[I] [Bug] `persistence_timer_bulk_execute_latency` in PersistentTimer is counterintuitive [skywalking]
wu-sheng opened a new issue, #13441:
URL: https://github.com/apache/skywalking/issues/13441
### Search before asking
- [x] I had searched in the [issues](https://github.com/apache/skywalking/issues?q=is%3Aissue) and found no similar issues.
### Apache SkyWalking Component
OAP server (apache/skywalking)
### What happened
During reviewing `Persistence Execution Latency Per Metric Type (ms / 5min)` on so11y, I noticed the max(p99) is always reaching 25000ms(25s).
Furthermore, I checked the codes and noticed that, as the following ways to create a timer
```java
// Execution stage
HistogramMetrics.Timer executeLatencyTimer = executeLatency.createTimer();
batchDAO.flush(innerPrepareRequests)
.whenComplete(($1, $2) -> executeLatencyTimer.close());
```
This thing relies on flush implementations. In BanyanDB implementation(maybe in Elasticsearch as well), it doesn't always call real flushing as the method called. There is a `#flushIfNeeded` and `#doPeriodicalFlush` in BanyanDB client(`AbstractBulkWriteProcessor`).
So, this metric doesn't measure the writing performance well.
### What you expected to happen
We should create a way to measure writing performance better and accurately. And we are lacking a way to measure record(trace and logs) writing.
### How to reproduce
<img width="2922" height="1208" alt="Image" src="https://github.com/user-attachments/assets/688a3096-b885-4835-b11b-4ac74cea098e" />
### Anything else
_No response_
### Are you willing to submit a pull request to fix on your own?
- [ ] Yes I am willing to submit a pull request on my own!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] `persistence_timer_bulk_execute_latency` in PersistentTimer is counterintuitive [skywalking]
Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng commented on issue #13441:
URL: https://github.com/apache/skywalking/issues/13441#issuecomment-3225735168
Of course, from OAP `persistence_timer_bulk_execute_latency` point of view, yes, the duration from retrieving from cache to flushing executed, it really takes 25s. But this is not a performance issue. Instead, this indicates the payload is not high, so most data is pending for nthe ext round of periodical flushing
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] `persistence_timer_bulk_execute_latency` in PersistentTimer is counterintuitive [skywalking]
Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng closed issue #13441: [Bug] `persistence_timer_bulk_execute_latency` in PersistentTimer is counterintuitive
URL: https://github.com/apache/skywalking/issues/13441
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] `persistence_timer_bulk_execute_latency` in PersistentTimer is counterintuitive [skywalking]
Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng commented on issue #13441:
URL: https://github.com/apache/skywalking/issues/13441#issuecomment-3225820474
As a fix consideration, I feel it may be better to add metrics to the BanyanDB Java client directly within another 0.9 RC5
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] `persistence_timer_bulk_execute_latency` in PersistentTimer is counterintuitive [skywalking]
Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng commented on issue #13441:
URL: https://github.com/apache/skywalking/issues/13441#issuecomment-3226083193
API is being added through https://github.com/apache/skywalking-banyandb-java-client/pull/91. Note for OAP implementation, the `opt.PrometheusMetricsOpts.clientID` is expected to add, as to align with the OAP telemetry implementations(label`sw_backend_instance`=`oap instance ID`, ref to `TelemetryRelatedContext`).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org