You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by "mrproliu (via GitHub)" <gi...@apache.org> on 2026/06/14 07:42:42 UTC
[PR] Align default BanyanDB stage segmentInterval to integer multiples [skywalking]
mrproliu opened a new pull request, #13910:
URL: https://github.com/apache/skywalking/pull/13910
### Improve the efficiency of BanyanDB hot → warm → cold lifecycle stage migration (default config)
Each coarser lifecycle stage's `segmentInterval` should be an integer multiple of the immediately finer (preceding) stage's. BanyanDB segments are epoch-anchored, so when the coarser interval is a multiple of the finer one, every source segment nests entirely inside exactly one target segment and the tier migration stays on the cheap **whole-segment fast path** (byte-copy). A non-multiple interval makes source segments straddle target boundaries, forcing a row-by-row decode-and-re-bucket **slow path** that costs significantly more CPU, IO, and peak memory, and produces more small parts for the background merge loop to compact. It does not affect correctness — only migration efficiency.
This aligns the shipped defaults that previously violated the rule:
| Group | Field | Old → New |
|-------|-------|-----------|
| records | cold `segmentInterval` | 3 → 4 (multiple of warm=2) |
| metricsMinute | cold `segmentInterval` | 5 → 6 (multiple of warm=3) |
| metricsHour | warm `segmentInterval` | 7 → 10 (multiple of hot=5) |
| metricsHour | cold `segmentInterval` | 15 → 20 (multiple of warm=10) |
All adjusted values still satisfy `hot ≤ warm ≤ cold` and `segmentInterval ≤ that stage's ttl`. The other groups already comply and are unchanged.
- [x] The improvement is structural (whole-segment byte-copy vs row-by-row re-bucketing), not a micro-optimization, so no micro-benchmark applies; the fast/slow path mechanism is documented in the stages guide.
- [x] Theory proof: see the new "integer multiple" guideline in `docs/en/banyandb/stages.md`, and the BanyanDB migration fast/slow path behavior.
- [ ] If this pull request closes/resolves/fixes an existing issue, replace the issue number. Closes #<issue number>.
- [x] Update the [`CHANGES` log](https://github.com/apache/skywalking/blob/master/docs/en/changes/changes.md).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Align default BanyanDB stage segmentInterval to integer multiples [skywalking]
Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng merged PR #13910:
URL: https://github.com/apache/skywalking/pull/13910
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Align default BanyanDB stage segmentInterval to integer multiples [skywalking]
Posted by "mrproliu (via GitHub)" <gi...@apache.org>.
mrproliu commented on code in PR #13910:
URL: https://github.com/apache/skywalking/pull/13910#discussion_r3409224176
##########
docs/en/banyandb/stages.md:
##########
@@ -31,14 +31,15 @@ The lifecycle stages configuration is under each group settings of the `bydb.yml
nodeSelector: ${SW_STORAGE_BANYANDB_GM_MINUTE_WARM_NODE_SELECTOR:"type=warm"}
cold:
shardNum: ${SW_STORAGE_BANYANDB_GM_MINUTE_COLD_SHARD_NUM:2}
- segmentInterval: ${SW_STORAGE_BANYANDB_GM_MINUTE_COLD_SI_DAYS:5}
+ segmentInterval: ${SW_STORAGE_BANYANDB_GM_MINUTE_COLD_SI_DAYS:6}
Review Comment:
Update the `stages.md` from `metricsMin` to `metricsMinute`.
##########
docs/en/setup/backend/configuration-vocabulary.md:
##########
@@ -432,7 +432,7 @@ OAP will query the data from the "hot and warm" stage by default if the "warm" s
| - | - | nodeSelector | Specifying target nodes for this stage. | SW_STORAGE_BANYANDB_RECORDS_WARM_NODE_SELECTOR | type=warm |
| - | cold | - | The cold stage settings. | - | - |
| - | - | shardNum | Shards Number for for this stage. | SW_STORAGE_BANYANDB_RECORDS_COLD_SHARD_NUM | 1 |
-| - | - | segmentInterval | Segment Interval Days for this stage. | SW_STORAGE_BANYANDB_RECORDS_COLD_SI_DAYS | 3 |
+| - | - | segmentInterval | Segment Interval Days for this stage. | SW_STORAGE_BANYANDB_RECORDS_COLD_SI_DAYS | 4 |
Review Comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Align default BanyanDB stage segmentInterval to integer multiples [skywalking]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #13910:
URL: https://github.com/apache/skywalking/pull/13910#discussion_r3409207492
##########
docs/en/setup/backend/configuration-vocabulary.md:
##########
@@ -432,7 +432,7 @@ OAP will query the data from the "hot and warm" stage by default if the "warm" s
| - | - | nodeSelector | Specifying target nodes for this stage. | SW_STORAGE_BANYANDB_RECORDS_WARM_NODE_SELECTOR | type=warm |
| - | cold | - | The cold stage settings. | - | - |
| - | - | shardNum | Shards Number for for this stage. | SW_STORAGE_BANYANDB_RECORDS_COLD_SHARD_NUM | 1 |
-| - | - | segmentInterval | Segment Interval Days for this stage. | SW_STORAGE_BANYANDB_RECORDS_COLD_SI_DAYS | 3 |
+| - | - | segmentInterval | Segment Interval Days for this stage. | SW_STORAGE_BANYANDB_RECORDS_COLD_SI_DAYS | 4 |
Review Comment:
This page updates the default `segmentInterval` values, but `docs/en/setup/backend/storages/banyandb.md` still shows the old defaults (`records` cold 3, `metricsMinute` cold 5, `metricsHour` warm 7 and cold 15). Please update that doc too so users see consistent defaults across the docs.
##########
docs/en/banyandb/stages.md:
##########
@@ -31,14 +31,15 @@ The lifecycle stages configuration is under each group settings of the `bydb.yml
nodeSelector: ${SW_STORAGE_BANYANDB_GM_MINUTE_WARM_NODE_SELECTOR:"type=warm"}
cold:
shardNum: ${SW_STORAGE_BANYANDB_GM_MINUTE_COLD_SHARD_NUM:2}
- segmentInterval: ${SW_STORAGE_BANYANDB_GM_MINUTE_COLD_SI_DAYS:5}
+ segmentInterval: ${SW_STORAGE_BANYANDB_GM_MINUTE_COLD_SI_DAYS:6}
Review Comment:
The example snippet in this doc uses a `metricsMin` group and `SW_STORAGE_BANYANDB_GM_MINUTE_*` env vars, but the shipped `bydb.yml` uses `metricsMinute` and `SW_STORAGE_BANYANDB_METRICS_MINUTE_*`. Keeping this example aligned with the actual config avoids confusing users and makes the updated default (`..._COLD_SI_DAYS:6`) discoverable in the right place.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org