You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by "eye-gu (via GitHub)" <gi...@apache.org> on 2026/03/25 14:22:13 UTC
[PR] Fix the reuse of byte arrays in the underlying min/max implementation [skywalking-banyandb]
eye-gu opened a new pull request, #1022:
URL: https://github.com/apache/skywalking-banyandb/pull/1022
<!--
⚠️ Please make sure to read this template first, pull requests that don't accord with this template
maybe closed without notice.
Texts surrounded by `<` and `>` are meant to be replaced by you, e.g. <framework name>, <issue number>.
Put an `x` in the `[ ]` to mark the item as CHECKED. `[x]`
-->
### Fix <bug description or the bug issue number or bug issue link>
- [ ] Add a unit test to verify that the fix works.
- [ ] Explain briefly why the bug exists and how to fix it.
<!-- ==== 🆕 Remove this line WHEN AND ONLY WHEN you're adding a new feature, follow the checklist 👇 ====
### <Feature description>
- [ ] If this is non-trivial feature, paste the links/URLs to the design doc.
- [ ] Update the documentation to include this new feature.
- [ ] Tests(including UT, IT, E2E) are added to verify the new feature.
- [ ] If it's UI related, attach the screenshots below.
==== 🆕 Remove this line WHEN AND ONLY WHEN you're adding a new feature, follow the checklist 👆 ==== -->
- [ ] If this pull request closes/resolves/fixes an existing issue, replace the issue number. Fixes apache/skywalking#<issue number>.
- [ ] Update the [`CHANGES` log](https://github.com/apache/skywalking-banyandb/blob/main/CHANGES.md).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix the reuse of byte arrays in the underlying min/max implementation [skywalking-banyandb]
Posted by "hanahmily (via GitHub)" <gi...@apache.org>.
hanahmily commented on PR #1022:
URL: https://github.com/apache/skywalking-banyandb/pull/1022#issuecomment-4131083918
@eye-gu tag_metadata.go in the trace module has the same issue. Would you please fix it accordingly?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix the reuse of byte arrays in the underlying min/max implementation [skywalking-banyandb]
Posted by "eye-gu (via GitHub)" <gi...@apache.org>.
eye-gu commented on PR #1022:
URL: https://github.com/apache/skywalking-banyandb/pull/1022#issuecomment-4133951733
> @eye-gu, please fix the CI failure.
Fixed. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix the reuse of byte arrays in the underlying min/max implementation [skywalking-banyandb]
Posted by "hanahmily (via GitHub)" <gi...@apache.org>.
hanahmily commented on PR #1022:
URL: https://github.com/apache/skywalking-banyandb/pull/1022#issuecomment-4131104636
@eye-gu, please fix the CI failure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix the reuse of byte arrays in the underlying min/max implementation [skywalking-banyandb]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #1022:
URL: https://github.com/apache/skywalking-banyandb/pull/1022#discussion_r2992088106
##########
banyand/stream/tag_metadata.go:
##########
@@ -79,10 +80,12 @@ func (tm *tagMetadata) unmarshal(src []byte) ([]byte, error) {
if err != nil {
return nil, fmt.Errorf("cannot unmarshal tagMetadata.min: %w", err)
}
+ tm.min = bytes.Clone(tm.min)
src, tm.max, err = encoding.DecodeBytes(src)
if err != nil {
return nil, fmt.Errorf("cannot unmarshal tagMetadata.max: %w", err)
}
+ tm.max = bytes.Clone(tm.max)
Review Comment:
`bytes.Clone` always allocates for non-nil slices. Since `min`/`max` are only meaningful when they’re actually populated (e.g., for int64 tags), consider cloning conditionally (e.g., only when `len(tm.min) > 0` / `len(tm.max) > 0` or based on `valueType`) to avoid extra allocations on metadata unmarshalling paths that don’t use min/max.
```suggestion
if len(tm.min) > 0 {
tm.min = bytes.Clone(tm.min)
} else {
tm.min = nil
}
src, tm.max, err = encoding.DecodeBytes(src)
if err != nil {
return nil, fmt.Errorf("cannot unmarshal tagMetadata.max: %w", err)
}
if len(tm.max) > 0 {
tm.max = bytes.Clone(tm.max)
} else {
tm.max = nil
}
```
##########
banyand/stream/tag_filter_test.go:
##########
@@ -472,6 +472,49 @@ func BenchmarkTagFamilyFiltersUnmarshal(b *testing.B) {
}
}
+func TestTagFamilyFilterUnmarshalMinMaxCorruption(t *testing.T) {
+ const (
+ minValue int64 = 100
+ maxValue int64 = 200
+ )
+
+ tfm := generateTagFamilyMetadata()
+ defer releaseTagFamilyMetadata(tfm)
+
+ bf := filter.NewBloomFilter(10)
+ bf.Add([]byte("test-value"))
+ filterBuf := make([]byte, 0)
+ filterBuf = encodeBloomFilter(filterBuf, bf)
+
+ tfm.tagMetadata = append(tfm.tagMetadata, tagMetadata{
+ name: "numeric-tag",
+ valueType: pbv1.ValueTypeInt64,
+ min: convert.Int64ToBytes(minValue),
+ max: convert.Int64ToBytes(maxValue),
+ filterBlock: dataBlock{
+ offset: 0,
+ size: uint64(len(filterBuf)),
+ },
+ })
+
+ metaBuf := make([]byte, 0)
+ metaBuf = tfm.marshal(metaBuf)
+
+ tff := generateTagFamilyFilter()
+ defer releaseTagFamilyFilter(tff)
+ tff.unmarshal(
+ &dataBlock{offset: 0, size: uint64(len(metaBuf))},
+ &mockReader{data: metaBuf},
+ &mockReader{data: filterBuf},
+ &mockReader{data: []byte{}},
+ )
+
+ tagFilter, _ := (*tff)["numeric-tag"]
+
+ assert.Equal(t, minValue, convert.BytesToInt64(tagFilter.min), "min value should not be corrupted by bb.Buf reuse")
+ assert.Equal(t, maxValue, convert.BytesToInt64(tagFilter.max), "max value should not be corrupted by bb.Buf reuse")
Review Comment:
The test doesn’t verify that the map lookup succeeded. If `numeric-tag` isn’t present (or `tagFilter` is nil), the subsequent dereference will panic and produce a less-informative failure. Capture the boolean from the lookup and assert it (and `tagFilter != nil`) before checking `min`/`max`.
```suggestion
tagFilter, ok := (*tff)["numeric-tag"]
if assert.True(t, ok, "tag filter for numeric-tag should exist") && assert.NotNil(t, tagFilter, "tag filter for numeric-tag should not be nil") {
assert.Equal(t, minValue, convert.BytesToInt64(tagFilter.min), "min value should not be corrupted by bb.Buf reuse")
assert.Equal(t, maxValue, convert.BytesToInt64(tagFilter.max), "max value should not be corrupted by bb.Buf reuse")
}
```
##########
banyand/stream/tag_metadata.go:
##########
@@ -79,10 +80,12 @@ func (tm *tagMetadata) unmarshal(src []byte) ([]byte, error) {
if err != nil {
return nil, fmt.Errorf("cannot unmarshal tagMetadata.min: %w", err)
}
+ tm.min = bytes.Clone(tm.min)
src, tm.max, err = encoding.DecodeBytes(src)
if err != nil {
return nil, fmt.Errorf("cannot unmarshal tagMetadata.max: %w", err)
}
+ tm.max = bytes.Clone(tm.max)
Review Comment:
Same concern as `min`: cloning `max` here introduces an allocation per unmarshal. If min/max are only needed for certain value types, consider cloning only for those cases (or only when non-empty) to reduce allocation overhead in hot paths.
```suggestion
if len(tm.min) > 0 {
tm.min = bytes.Clone(tm.min)
}
src, tm.max, err = encoding.DecodeBytes(src)
if err != nil {
return nil, fmt.Errorf("cannot unmarshal tagMetadata.max: %w", err)
}
if len(tm.max) > 0 {
tm.max = bytes.Clone(tm.max)
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix the reuse of byte arrays in the underlying min/max implementation [skywalking-banyandb]
Posted by "hanahmily (via GitHub)" <gi...@apache.org>.
hanahmily merged PR #1022:
URL: https://github.com/apache/skywalking-banyandb/pull/1022
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix the reuse of byte arrays in the underlying min/max implementation [skywalking-banyandb]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #1022:
URL: https://github.com/apache/skywalking-banyandb/pull/1022#issuecomment-4134176224
## [Codecov](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/1022?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 30.13%. Comparing base ([`3530dd9`](https://app.codecov.io/gh/apache/skywalking-banyandb/commit/3530dd9b5cefaa4fef60eec91ecbfbdfcaf243c5?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`4486ecf`](https://app.codecov.io/gh/apache/skywalking-banyandb/commit/4486ecfd1e441be5909d08571d20181ae20b8fbd?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 171 commits behind head on main.
> :exclamation: There is a different number of reports uploaded between BASE (3530dd9) and HEAD (4486ecf). Click for more details.
>
> <details><summary>HEAD has 1 upload less than BASE</summary>
>
>| Flag | BASE (3530dd9) | HEAD (4486ecf) |
>|------|------|------|
>||1|0|
></details>
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## main #1022 +/- ##
===========================================
- Coverage 45.97% 30.13% -15.85%
===========================================
Files 328 124 -204
Lines 55505 12901 -42604
===========================================
- Hits 25520 3888 -21632
+ Misses 27909 8641 -19268
+ Partials 2076 372 -1704
```
| [Flag](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/1022/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [pkg](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/1022/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `30.13% <ø> (?)` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/1022?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org