You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by "hanahmily (via GitHub)" <gi...@apache.org> on 2025/12/16 00:19:04 UTC
[I] [Feature] Add Optional Time Range in `manifest.json` for Secondary Index [skywalking]
hanahmily opened a new issue, #13620:
URL: https://github.com/apache/skywalking/issues/13620
### Search before asking
- [x] I had searched in the [issues](https://github.com/apache/skywalking/issues?q=is%3Aissue) and found no similar feature requirement.
### Description
We need to enhance the `manifest.json` file of the secondary index by introducing two optional fields for time range, specifically `max_timestamp` and `min_timestamp`.
- The new fields are optional because, in some cases, the key itself might be the timestamp, which means that the max/min keys will also correspond to the max/min timestamps.
- They are used to improve the part selection algorithm.
### Use case
_No response_
### Related issues
_No response_
### Are you willing to submit a pull request to implement this on your own?
- [ ] Yes I am willing to submit a pull request on my own!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Feature] Add Optional Time Range in `manifest.json` for Secondary Index [skywalking]
Posted by "hanahmily (via GitHub)" <gi...@apache.org>.
hanahmily closed issue #13620: [Feature] Add Optional Time Range in `manifest.json` for Secondary Index
URL: https://github.com/apache/skywalking/issues/13620
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Feature] Add Optional Time Range in `manifest.json` for Secondary Index [skywalking]
Posted by "onkar717 (via GitHub)" <gi...@apache.org>.
onkar717 commented on issue #13620:
URL: https://github.com/apache/skywalking/issues/13620#issuecomment-3711818129
Hi @hanahmily,
I’m preparing to work on this feature and had a few clarification questions to align with the expected design:
Timestamp availability:
In the current codebase, secondary index building (sidx) only receives the Key and not the original record timestamp.
Would you prefer:
extending sidx.WriteRequest to explicitly carry the timestamp, or
propagating timestamp metadata alongside keys during part building (without changing the public write API)?
Timestamp-as-key case:
When the secondary index key itself is a timestamp (e.g. ValueTypeTimestamp), is it acceptable to omit min_timestamp / max_timestamp in manifest.json and rely on MinKey / MaxKey only?
Query pruning scope:
Should time-range pruning be applied only at part-selection level (before reading blocks), or do you expect any deeper integration (e.g. block-level filtering)?
I want to ensure the implementation matches the intended direction before submitting a PR. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Feature] Add Optional Time Range in `manifest.json` for Secondary Index [skywalking]
Posted by "hanahmily (via GitHub)" <gi...@apache.org>.
hanahmily commented on issue #13620:
URL: https://github.com/apache/skywalking/issues/13620#issuecomment-3712592429
@onkar717,
Thank you for taking on this task.
I would like to suggest refactoring the `sidx.WriteRequest` to explicitly include the timestamp. Additionally, I recommend keeping the `min_timestamp` and `max_timestamp` fields in `manifest.json`, even if the keys are the timestamps themselves. This approach will simplify both writing and querying, and simplification often leads to a more robust system.
In the querying process, these two fields should be used for part selection and not in the block scope. Furthermore, please ensure these fields are updated after merging the two parts.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org