You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by GitBox <gi...@apache.org> on 2022/12/11 07:07:35 UTC
[GitHub] [skywalking-banyandb] hanahmily opened a new pull request, #227: Enhance the inverted index, metadata and logger
hanahmily opened a new pull request, #227:
URL: https://github.com/apache/skywalking-banyandb/pull/227
There are several enhancements in this PR:
## Improve the inverted index
`SeriesID` is a mandatory field that is scanned in each equal operation. In this PR, I composite `SeriesID` and `Term` to a composite index. The equivalent process gets rid of roaringbitmap's intersection operation. The new path improves the ID matching's performance.
The side effect is: If the tag is marked as an `Analyzer`, the inverted index engine will split this composited index for analyzing the `Term`. That if you do an equal query on a tag with an `Analyzer`, the performance would be worse a bit.
This is related to https://github.com/apache/skywalking/issues/10119.
## Metadata
The Etcd-powered metadata registry closing process will wait till all running operations are done. It will prevent the retry interceptor from repeatedly connecting to a closed server, which will slow down the testing.
## Logger
Etcd uses the Zap logger, while others pick up the Zerologger. The PR introduces a Zap config generator that outputs a similar log configuration with the Zerolog.
In the development mode, both loggers will print their stacks and callers will help developers diagnose the code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking-banyandb] wu-sheng commented on pull request #227: Enhance the inverted index, metadata and logger
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on PR #227:
URL: https://github.com/apache/skywalking-banyandb/pull/227#issuecomment-1345476944
> If the tag is marked as an Analyzer
What does this mean? Is this a new attribute of the tag? Is there any docs update about this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking-banyandb] wu-sheng merged pull request #227: Enhance the inverted index, metadata and logger
Posted by GitBox <gi...@apache.org>.
wu-sheng merged PR #227:
URL: https://github.com/apache/skywalking-banyandb/pull/227
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking-banyandb] hanahmily commented on pull request #227: Enhance the inverted index, metadata and logger
Posted by GitBox <gi...@apache.org>.
hanahmily commented on PR #227:
URL: https://github.com/apache/skywalking-banyandb/pull/227#issuecomment-1345556366
> > If the tag is marked as an Analyzer
>
> What does this mean? Is this a new attribute of the tag? Is there any docs update about this?
No new attributes here. It means a tag's index rule sets a specific Analyzer in https://github.com/apache/skywalking-banyandb/blob/main/docs/api-reference.md#indexruleanalyzer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking-banyandb] codecov-commenter commented on pull request #227: Enhance the inverted index, metadata and logger
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #227:
URL: https://github.com/apache/skywalking-banyandb/pull/227#issuecomment-1345476823
# [Codecov](https://codecov.io/gh/apache/skywalking-banyandb/pull/227?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#227](https://codecov.io/gh/apache/skywalking-banyandb/pull/227?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (498b3b0) into [main](https://codecov.io/gh/apache/skywalking-banyandb/commit/005b02210caacee0141de8085edebed367ef5a6f?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (005b022) will **increase** coverage by `0.22%`.
> The diff coverage is `57.29%`.
```diff
@@ Coverage Diff @@
## main #227 +/- ##
==========================================
+ Coverage 43.84% 44.06% +0.22%
==========================================
Files 88 88
Lines 9035 9094 +59
==========================================
+ Hits 3961 4007 +46
- Misses 4697 4705 +8
- Partials 377 382 +5
```
| [Impacted Files](https://codecov.io/gh/apache/skywalking-banyandb/pull/227?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [banyand/metadata/schema/error.go](https://codecov.io/gh/apache/skywalking-banyandb/pull/227/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmFueWFuZC9tZXRhZGF0YS9zY2hlbWEvZXJyb3IuZ28=) | `0.00% <ø> (ø)` | |
| [banyand/tsdb/bucket/queue.go](https://codecov.io/gh/apache/skywalking-banyandb/pull/227/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmFueWFuZC90c2RiL2J1Y2tldC9xdWV1ZS5nbw==) | `40.34% <0.00%> (ø)` | |
| [banyand/tsdb/indexdb.go](https://codecov.io/gh/apache/skywalking-banyandb/pull/227/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmFueWFuZC90c2RiL2luZGV4ZGIuZ28=) | `7.24% <0.00%> (+0.75%)` | :arrow_up: |
| [banyand/tsdb/tsdb.go](https://codecov.io/gh/apache/skywalking-banyandb/pull/227/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmFueWFuZC90c2RiL3RzZGIuZ28=) | `53.23% <ø> (ø)` | |
| [banyand/liaison/grpc/measure.go](https://codecov.io/gh/apache/skywalking-banyandb/pull/227/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmFueWFuZC9saWFpc29uL2dycGMvbWVhc3VyZS5nbw==) | `2.32% <10.00%> (+2.32%)` | :arrow_up: |
| [banyand/liaison/grpc/stream.go](https://codecov.io/gh/apache/skywalking-banyandb/pull/227/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmFueWFuZC9saWFpc29uL2dycGMvc3RyZWFtLmdv) | `2.85% <10.00%> (+2.85%)` | :arrow_up: |
| [banyand/metadata/schema/etcd.go](https://codecov.io/gh/apache/skywalking-banyandb/pull/227/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmFueWFuZC9tZXRhZGF0YS9zY2hlbWEvZXRjZC5nbw==) | `58.14% <62.50%> (-2.11%)` | :arrow_down: |
| [pkg/index/inverted/inverted.go](https://codecov.io/gh/apache/skywalking-banyandb/pull/227/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGtnL2luZGV4L2ludmVydGVkL2ludmVydGVkLmdv) | `70.00% <75.94%> (-1.92%)` | :arrow_down: |
| [banyand/liaison/grpc/server.go](https://codecov.io/gh/apache/skywalking-banyandb/pull/227/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmFueWFuZC9saWFpc29uL2dycGMvc2VydmVyLmdv) | `75.69% <100.00%> (+0.34%)` | :arrow_up: |
| [banyand/measure/service.go](https://codecov.io/gh/apache/skywalking-banyandb/pull/227/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-YmFueWFuZC9tZWFzdXJlL3NlcnZpY2UuZ28=) | `74.02% <100.00%> (+0.34%)` | :arrow_up: |
| ... and [11 more](https://codecov.io/gh/apache/skywalking-banyandb/pull/227/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org