You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by "Dev-next-gen (via GitHub)" <gi...@apache.org> on 2026/10/15 01:42:08 UTC
[PR] Fix ES metrics reads locating indices with superDatasetDayStep instead of dayStep [skywalking]
Dev-next-gen opened a new pull request, #14139:
URL: https://github.com/apache/skywalking/pull/14139
### Fix the Elasticsearch storage reading metrics from the wrong daily indices when `dayStep` and `superDatasetDayStep` differ
- [x] Add a unit test to verify that the fix works.
- [x] Explain briefly why the bug exists and how to fix it.
Metrics are written to indices rolled by `dayStep` (`TimeSeriesUtils.writeIndexName`), and `readMetricsValues` reads them back the same way through `queryIndexName`. Four other metrics reads locate their indices with `TimeRangeIndexNameGenerator`, which calls `superDatasetIndexNames` and rolls by `superDatasetDayStep`:
- `AggregationQueryEsDAO.sortMetrics` (top N)
- `MetricsQueryEsDAO.readLabeledMetricsValuesWithoutEntity`
- `MetricsQueryEsDAO.listEntityIdsInRange`
- both `TagAutoCompleteQueryDAO` queries (`TagAutocompleteData` is a metrics model, not a super dataset)
With the default `superDatasetDayStep: -1` both steps are equal and nothing shows. Once they differ, these reads search indices that do not hold the data. The search is sent with `ignore_unavailable=true`, so the missing indices are skipped silently and the query returns nothing or only part of the range.
Concrete case, the configuration `TimeSeriesUtilsTest` already uses: `dayStep: 3`, `superDatasetDayStep: 1`. The minutes of 2020-08-09 are written to `<metric>-20200807`, but a top N over 2020-08-09 10:10 to 10:20 searches only `<metric>-20200809`.
The fix adds `TimeSeriesUtils.metricsIndexNames`, the same range walk as `superDatasetIndexNames` but rolled by `dayStep`, and these four reads use it. The super dataset readers (traces, logs, Zipkin, OTLP spans, AI agent and GenAI evaluation records) keep `TimeRangeIndexNameGenerator`. In `TagAutoCompleteQueryDAO`, the time bounds become final so the lambda can capture them.
`AggregationQueryEsDAOTest` runs `sortMetrics` against a mocked client with the steps above and checks the indices it searches. Before the fix it fails with `["service_resp_time-20200809"]` where `["service_resp_time-20200807"]` is expected. After it, the test passes, as do the 34 tests of `storage-elasticsearch-plugin`, and checkstyle is clean.
- [ ] If this pull request closes/resolves/fixes an existing issue, replace the issue number. Closes #<issue number>.
- [x] Update the [`CHANGES` log](https://github.com/apache/skywalking/blob/master/docs/en/changes/changes.md).
Found by a defect-hunting pipeline I build and run ([Dev-next-gen](https://github.com/Dev-next-gen)), using Claude Code with Anthropic's Claude Opus 5.5.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix ES metrics reads locating indices with superDatasetDayStep instead of dayStep [skywalking]
Posted by "Dev-next-gen (via GitHub)" <gi...@apache.org>.
Dev-next-gen commented on PR #14139:
URL: https://github.com/apache/skywalking/pull/14139#issuecomment-6184663476
Thanks for the review and for taking the fix ahead of the API unification. I checked the five red checks on 673b912 since they are what keeps `Required` red; none of them comes from the changed ES read paths.
`Alarm ES` and `Alarm ES Sharding` also failed on #14135 a few hours earlier, which does not touch ES storage ([Alarm ES](https://github.com/apache/skywalking/actions/runs/38657210628/job/116176941620), [Alarm ES Sharding](https://github.com/apache/skywalking/actions/runs/38657210628/job/116176941636)), and on runs 38340517699 and 38181306104 as well; here `Alarm ES` passed 13 cases and failed one after 30 retries on `queryAlarms` returning `msgs: []`. `Agent Python` failed on `instance list`: the consumer's container hostname was `845886322e50`, which the expected-file template read as a number (`float64(8.45886322e+58)` in the diff). `Rover with Istio Process 1.28.0` stopped at Maven resolution, `No versions available for io.grpc:grpc-core:jar:[1.43.2] within specified range`, before any test ran. In the same run `Storage ES 9.3.0`, `Storage ES 8.18.8`, `Storage ES Sharding 8.18.8`, the three `Storage OpenSearch` jobs and `Baseline-driven Alarm ES` are green.
I tried `gh run rerun 38667069305 --failed` but it needs admin rights, so if a re-run of the failed jobs is the simplest way to get `Required` green, it would have to come from your side.
Found by a defect-hunting pipeline I build and run ([Dev-next-gen](https://github.com/Dev-next-gen)), using Claude Code with Anthropic's Claude Opus 5.5.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix ES metrics reads locating indices with superDatasetDayStep instead of dayStep [skywalking]
Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng merged PR #14139:
URL: https://github.com/apache/skywalking/pull/14139
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org