You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by "ButterBright (via GitHub)" <gi...@apache.org> on 2024/01/24 06:58:02 UTC
[PR] Remove maxBlockLength in stream [skywalking-banyandb]
ButterBright opened a new pull request, #381:
URL: https://github.com/apache/skywalking-banyandb/pull/381
<!--
⚠️ Please make sure to read this template first, pull requests that don't accord with this template
maybe closed without notice.
Texts surrounded by `<` and `>` are meant to be replaced by you, e.g. <framework name>, <issue number>.
Put an `x` in the `[ ]` to mark the item as CHECKED. `[x]`
-->
<!-- ==== 🐛 Remove this line WHEN AND ONLY WHEN you're fixing a bug, follow the checklist 👇 ====
### Fix <bug description or the bug issue number or bug issue link>
- [ ] Add a unit test to verify that the fix works.
- [ ] Explain briefly why the bug exists and how to fix it.
==== 🐛 Remove this line WHEN AND ONLY WHEN you're fixing a bug, follow the checklist 👆 ==== -->
<!-- ==== 🆕 Remove this line WHEN AND ONLY WHEN you're adding a new feature, follow the checklist 👇 ====
### <Feature description>
- [ ] If this is non-trivial feature, paste the links/URLs to the design doc.
- [ ] Update the documentation to include this new feature.
- [ ] Tests(including UT, IT, E2E) are added to verify the new feature.
- [ ] If it's UI related, attach the screenshots below.
==== 🆕 Remove this line WHEN AND ONLY WHEN you're adding a new feature, follow the checklist 👆 ==== -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove maxBlockLength in stream [skywalking-banyandb]
Posted by "hanahmily (via GitHub)" <gi...@apache.org>.
hanahmily commented on code in PR #381:
URL: https://github.com/apache/skywalking-banyandb/pull/381#discussion_r1464949063
##########
banyand/stream/merger.go:
##########
@@ -300,31 +300,15 @@ func mergeBlocks(closeCh <-chan struct{}, bw *blockWriter, br *blockReader) (*pa
tmpBlock.bm.seriesID = b.bm.seriesID
br.loadBlockData(getDecoder())
mergeTwoBlocks(tmpBlock, pendingBlock, b)
- if len(tmpBlock.timestamps) <= maxBlockLength && tmpBlock.uncompressedSizeBytes() <= maxUncompressedBlockSize {
+ if tmpBlock.uncompressedSizeBytes() <= maxUncompressedBlockSize {
if len(tmpBlock.timestamps) == 0 {
pendingBlockIsEmpty = true
}
pendingBlock, tmpBlock = tmpBlock, pendingBlock
- continue
- }
-
- if len(tmpBlock.timestamps) <= maxBlockLength {
+ } else {
bw.mustWriteBlock(tmpBlock.bm.seriesID, &tmpBlock.block)
Review Comment:
The "pendingBlock" and "tmpBlock" should be reset after writing. "pendingBlockIsEmpty" should be set to "true"
##########
banyand/stream/merger.go:
##########
@@ -300,31 +300,15 @@ func mergeBlocks(closeCh <-chan struct{}, bw *blockWriter, br *blockReader) (*pa
tmpBlock.bm.seriesID = b.bm.seriesID
br.loadBlockData(getDecoder())
mergeTwoBlocks(tmpBlock, pendingBlock, b)
- if len(tmpBlock.timestamps) <= maxBlockLength && tmpBlock.uncompressedSizeBytes() <= maxUncompressedBlockSize {
+ if tmpBlock.uncompressedSizeBytes() <= maxUncompressedBlockSize {
if len(tmpBlock.timestamps) == 0 {
pendingBlockIsEmpty = true
}
pendingBlock, tmpBlock = tmpBlock, pendingBlock
- continue
- }
-
- if len(tmpBlock.timestamps) <= maxBlockLength {
+ } else {
Review Comment:
remove "else", add "continue" at the end of "if"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove maxBlockLength in stream [skywalking-banyandb]
Posted by "hanahmily (via GitHub)" <gi...@apache.org>.
hanahmily commented on code in PR #381:
URL: https://github.com/apache/skywalking-banyandb/pull/381#discussion_r1465897270
##########
banyand/stream/part_metadata.go:
##########
@@ -32,8 +32,7 @@ type partMetadata struct {
BlocksCount uint64 `json:"blocksCount"`
MinTimestamp int64 `json:"minTimestamp"`
MaxTimestamp int64 `json:"maxTimestamp"`
- // TODO: remove this?
- ID uint64 `json:"-"`
+ ID uint64 `json:"-"`
Review Comment:
You should raise another PR to remove it and the relevant "partID" field since the stream doesn't have to duplicate data.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove maxBlockLength in stream [skywalking-banyandb]
Posted by "hanahmily (via GitHub)" <gi...@apache.org>.
hanahmily commented on code in PR #381:
URL: https://github.com/apache/skywalking-banyandb/pull/381#discussion_r1464399046
##########
banyand/stream/part.go:
##########
@@ -325,8 +325,7 @@ func (mp *memPart) mustInitFromElements(es *elements) {
sidPrev = sid
}
- if uncompressedBlockSizeBytes >= maxUncompressedBlockSize ||
- (i-indexPrev) > maxBlockLength || sid != sidPrev {
+ if uncompressedBlockSizeBytes >= maxUncompressedBlockSize || sid != sidPrev {
Review Comment:
You should remove "indexPrev" as well
##########
banyand/stream/merger.go:
##########
@@ -300,20 +300,14 @@ func mergeBlocks(closeCh <-chan struct{}, bw *blockWriter, br *blockReader) (*pa
tmpBlock.bm.seriesID = b.bm.seriesID
br.loadBlockData(getDecoder())
mergeTwoBlocks(tmpBlock, pendingBlock, b)
- if len(tmpBlock.timestamps) <= maxBlockLength && tmpBlock.uncompressedSizeBytes() <= maxUncompressedBlockSize {
+ if tmpBlock.uncompressedSizeBytes() <= maxUncompressedBlockSize {
if len(tmpBlock.timestamps) == 0 {
pendingBlockIsEmpty = true
}
pendingBlock, tmpBlock = tmpBlock, pendingBlock
continue
}
- if len(tmpBlock.timestamps) <= maxBlockLength {
- bw.mustWriteBlock(tmpBlock.bm.seriesID, &tmpBlock.block)
- releaseDecoder()
- continue
- }
- tmpBlock.idx = maxBlockLength
Review Comment:
You can't simply remove it.
When the tmpBlock's size is larger than maxUncompressedBlockSize, it should be written. The rest of the pieces in the loop will be unnecessary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove maxBlockLength in stream [skywalking-banyandb]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #381:
URL: https://github.com/apache/skywalking-banyandb/pull/381#issuecomment-1907489247
## [Codecov](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/381?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Comparison is base [(`97e9111`)](https://app.codecov.io/gh/apache/skywalking-banyandb/commit/97e911121c4f9ec7ad07ac1580f50b39a03de644?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) 47.31% compared to head [(`98e64d8`)](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/381?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) 47.34%.
> Report is 1 commits behind head on main.
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## main #381 +/- ##
==========================================
+ Coverage 47.31% 47.34% +0.02%
==========================================
Files 172 172
Lines 21296 21290 -6
==========================================
+ Hits 10077 10079 +2
+ Misses 10377 10370 -7
+ Partials 842 841 -1
```
</details>
[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/skywalking-banyandb/pull/381?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Remove maxBlockLength in stream [skywalking-banyandb]
Posted by "hanahmily (via GitHub)" <gi...@apache.org>.
hanahmily merged PR #381:
URL: https://github.com/apache/skywalking-banyandb/pull/381
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org