You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kvrocks.apache.org by "yezhizi (via GitHub)" <gi...@apache.org> on 2025/11/19 08:57:59 UTC
[I] TimeSeries: Support `LATEST` option in queries [kvrocks]
yezhizi opened a new issue, #3265:
URL: https://github.com/apache/kvrocks/issues/3265
### Search before asking
- [x] I had searched in the [issues](https://github.com/apache/kvrocks/issues) and found no similar issues.
### Motivation
When using `TS.GET/TS.RANGE` to query a compacted time series, the `LATEST` option can be specified. This causes `TS.GET/TS.RANGE` to return the compacted value of the latest bucket, which may be a partial bucket.
Redis reference:
[redis.io/docs/latest/commands/ts.create](https://redis.io/docs/latest/commands/ts.create/)
### Solution
Modify [`rangeCommon`](https://github.com/apache/kvrocks/blob/e19164f032b43c9408a5a0badf53aebeb092d97c/src/types/redis_timeseries.h#L318-L319) and [`getCommon`](https://github.com/apache/kvrocks/blob/e19164f032b43c9408a5a0badf53aebeb092d97c/src/types/redis_timeseries.h#L322-L323) to retrieve the latest bucket of source series when the latest sample is requested.
Note: Since partial aggregations are stored in `DOWNSTREAM` sub keys, we only need to fetch the last chunk of the source series to retrieve the latest bucket's aggregated value.
Refer to [`DOWNSTREAM` sub keys](https://kvrocks.apache.org/community/data-structure-on-rocksdb#downstream-sub-keys)
Refer to https://github.com/apache/kvrocks/pull/3151
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] TimeSeries: Support `LATEST` option in queries [kvrocks]
Posted by "var-nan (via GitHub)" <gi...@apache.org>.
var-nan commented on issue #3265:
URL: https://github.com/apache/kvrocks/issues/3265#issuecomment-3745864554
Hi @yezhizi , does the `TSDownStreamMeta` store the auxiliary information about the partial buckets of downstream series?
Say, there exist a rule from `src` to `dst` series to aggregate max values over duration of 10. When I insert new samples to `src`, I can't find the information about the partial bucket in `dst`'s TSDownStreamMeta object.
```cpp
const auto& src_key = metadata.source_key; // metadata is metadata of the dst series.
auto ns_src_key = AppendNamespacePrefix(src_key);
TimeSeriesMetadata src_metadata;
auto s = getTimeSeriesMetadata(ctx, ns_src_key, &src_metadata);
std::vector<std::string> ds_keys;
std::vector<TSDownStreamMeta> ds_metas;
std::vector<TimeSeriesMetadata> ds_series_meta;
s = getDownStreamRules(ctx, ns_src_key, src_metadata, &ds_keys, &ds_metas, &ds_series_meta);
```
When I try to access `f64_auxs` vector from `ds_metas[0]`, it contains `nan`.
Is this the right way to access auxiliary info of a destination series?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] TimeSeries: Support `LATEST` option in queries [kvrocks]
Posted by "yezhizi (via GitHub)" <gi...@apache.org>.
yezhizi commented on issue #3265:
URL: https://github.com/apache/kvrocks/issues/3265#issuecomment-3750526729
@var-nan
`auxInfo` stores the aggregated value from the preceding chunks (excluding the current/last one) that belong to the latest bucket.
For example, the latest bucket is 10, `auxInfo` would store the aggregated value of 11–18 in the following case:
```
| chunk 0 | | chunk 1 | | chunk 2 |
| 0 4 11 | | 15 16 18 | | 19 |
|_______________|
|
```
If the latest bucket has no data in the preceding chunks, `auxInfo` will be [“reset”](https://github.com/apache/kvrocks/blob/b5b419995c8327bd07a6d63090da367d98f59b72/src/types/redis_timeseries.cc#L377)(For the MAX aggregator, the value is NaN in the case you mentioned).
```
| chunk 0 | | chunk 1 | | chunk 2 | (latest bucket is 20)
| 0 4 11 | | 15 16 18 | | 21 22 |
```
Your approach to accessing the auxiliary info is correct. For the MAX aggregator, if the auxInfo value is NaN, it should indicate that samples from the latest bucket are only contained within the last/current chunk; therefore, the auxiliary info can be ignored in this case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] TimeSeries: Support `LATEST` option in queries [kvrocks]
Posted by "var-nan (via GitHub)" <gi...@apache.org>.
var-nan commented on issue #3265:
URL: https://github.com/apache/kvrocks/issues/3265#issuecomment-3752564843
Thanks for the response.
In that case, a possible solution is to retrieve auxiliary info from destination metadata, the latest chunk from the source series, and reduce these two results?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] TimeSeries: Support `LATEST` option in queries [kvrocks]
Posted by "yezhizi (via GitHub)" <gi...@apache.org>.
yezhizi commented on issue #3265:
URL: https://github.com/apache/kvrocks/issues/3265#issuecomment-3651380008
> If I understand it correctly, to report the latest value from the downstream key , we need to get the auxiliary info from the source's downstream metadata. This auxiliary info contains the partial values of the latest (open) bucket.
>
> ```
> source_key | version | downstream_key => | aggregator |bucket_duration | alignment |latest_bucket_index |auxinfo|
> ```
>
> is that correct?
@var-nan Yes, exactly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] TimeSeries: Support `LATEST` option in queries [kvrocks]
Posted by "var-nan (via GitHub)" <gi...@apache.org>.
var-nan commented on issue #3265:
URL: https://github.com/apache/kvrocks/issues/3265#issuecomment-3651264549
Hi @yezhizi , I'm interested in working on this.
If I understand it correctly, to report the latest value from the downstream key , we need to get the auxiliary info from the source's downstream metadata. This auxiliary info contains the partial values of the latest (open) bucket.
```
source_key | version | downstream_key => | aggregator |bucket_duration | alignment |latest_bucket_index |auxinfo|
```
is that correct?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org