You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by GitBox <gi...@apache.org> on 2020/05/15 09:32:22 UTC
[GitHub] [skywalking] EvanLjp opened a new issue #4790: why not distinguish trace segment index config and metrics index config
EvanLjp opened a new issue #4790:
URL: https://github.com/apache/skywalking/issues/4790
Please answer these questions before submitting your issue.
- Why do you submit this issue?
- [ ] Question or discussion
___
### Question
- What do you want to know?
why not distinguish trace segment index config and metrics index config
One shards normally contains data is about 10G-30G
metrics data is usually smaller than above capacity ,trace segment data is is usually bigger than above capacity
Too many shards will affect es performance
why not distinguish the setting?Although I understand that manual templates can be controlled
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] EvanLjp commented on issue #4790: why not distinguish trace segment index config and metrics index config
Posted by GitBox <gi...@apache.org>.
EvanLjp commented on issue #4790:
URL: https://github.com/apache/skywalking/issues/4790#issuecomment-629167328
I konw it,I will consider it
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on issue #4790: why not distinguish trace segment index config and metrics index config
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on issue #4790:
URL: https://github.com/apache/skywalking/issues/4790#issuecomment-629183294
I think you could do these
1. Add an annotation `@SuperDataSet`, this annotation indicates the `StorageData` implementation having super size data set.
1. Add `@SuperDataSet` to `SegmentRecord`.
1. Change the model definition with `superDataSet` flag.
1. Add `superDataSetShardsNumberFector` to give the `multiplier` of `indexShardsNumber` for the `@SuperDataSet` annotated type.
@kezhenxu94 @dmsolr @JaredTan95 @arugal WDYT?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] EvanLjp commented on issue #4790: why not distinguish trace segment index config and metrics index config
Posted by GitBox <gi...@apache.org>.
EvanLjp commented on issue #4790:
URL: https://github.com/apache/skywalking/issues/4790#issuecomment-629185079
It's a good idea for implement it
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on issue #4790: why not distinguish trace segment index config and metrics index config
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on issue #4790:
URL: https://github.com/apache/skywalking/issues/4790#issuecomment-629141983
What do you want to propose? About how in the codes level? How to make people understand the config? There is not much special about the segment index from my understanding.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on issue #4790: why not distinguish trace segment index config and metrics index config
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on issue #4790:
URL: https://github.com/apache/skywalking/issues/4790#issuecomment-629151202
I know your case, I just consider the implementation. One, this is only an ES specific thing, the other is, this is only related to segment, but there are other record type entities.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] EvanLjp removed a comment on issue #4790: why not distinguish trace segment index config and metrics index config
Posted by GitBox <gi...@apache.org>.
EvanLjp removed a comment on issue #4790:
URL: https://github.com/apache/skywalking/issues/4790#issuecomment-629156497
I think this function should be the same as the number of shards and should only work in es storage config
it only works here
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng closed issue #4790: why not distinguish trace segment index config and metrics index config
Posted by GitBox <gi...@apache.org>.
wu-sheng closed issue #4790:
URL: https://github.com/apache/skywalking/issues/4790
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on issue #4790: why not distinguish trace segment index config and metrics index config
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on issue #4790:
URL: https://github.com/apache/skywalking/issues/4790#issuecomment-629166327
About config, yes. But you need to figure out how make it works for segment only? Have you considered this?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on issue #4790: why not distinguish trace segment index config and metrics index config
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on issue #4790:
URL: https://github.com/apache/skywalking/issues/4790#issuecomment-629568773
@EvanLjp Could you follow this to send a pull request?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] EvanLjp commented on issue #4790: why not distinguish trace segment index config and metrics index config
Posted by GitBox <gi...@apache.org>.
EvanLjp commented on issue #4790:
URL: https://github.com/apache/skywalking/issues/4790#issuecomment-629156497
I think this function should be the same as the number of shards and should only work in es storage config
it only works here
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] EvanLjp commented on issue #4790: why not distinguish trace segment index config and metrics index config
Posted by GitBox <gi...@apache.org>.
EvanLjp commented on issue #4790:
URL: https://github.com/apache/skywalking/issues/4790#issuecomment-629149153
1. The segment index is much larger than the other indexes
2. An index of 30-50g data can guarantee good query performance,and If a shards is too large, the es cluster will restart
So unless you manually control the number of shards, why not config it in code
And need to think again about how to implement code
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] EvanLjp edited a comment on issue #4790: why not distinguish trace segment index config and metrics index config
Posted by GitBox <gi...@apache.org>.
EvanLjp edited a comment on issue #4790:
URL: https://github.com/apache/skywalking/issues/4790#issuecomment-629157232
I think this function should be the same as the number of shards and should only work in es storage config
it only works here
so i think the config will not affect others
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] EvanLjp commented on issue #4790: why not distinguish trace segment index config and metrics index config
Posted by GitBox <gi...@apache.org>.
EvanLjp commented on issue #4790:
URL: https://github.com/apache/skywalking/issues/4790#issuecomment-629577291
ok,I would like to do it
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] EvanLjp commented on issue #4790: why not distinguish trace segment index config and metrics index config
Posted by GitBox <gi...@apache.org>.
EvanLjp commented on issue #4790:
URL: https://github.com/apache/skywalking/issues/4790#issuecomment-629157232
I think this function should be the same as the number of shards and should only work in es storage config
it only works here
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] kezhenxu94 commented on issue #4790: why not distinguish trace segment index config and metrics index config
Posted by GitBox <gi...@apache.org>.
kezhenxu94 commented on issue #4790:
URL: https://github.com/apache/skywalking/issues/4790#issuecomment-629187972
> I think you could do these
>
> 1. Add an annotation `@SuperDataSet`, this annotation indicates the `StorageData` implementation having super size data set.
> 2. Add `@SuperDataSet` to `SegmentRecord`.
> 3. Change the model definition with `superDataSet` flag.
> 4. Add `superDataSetShardsNumberFector` to give the `multiplier` of `indexShardsNumber` for the `@SuperDataSet` annotated type.
>
> @kezhenxu94 @dmsolr @JaredTan95 @arugal WDYT?
Sounds good enough to me,
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org