You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by "BigSecret1 (via GitHub)" <gi...@apache.org> on 2023/05/11 05:20:25 UTC
[GitHub] [druid] BigSecret1 opened a new issue, #14253: Implemention of new metric
BigSecret1 opened a new issue, #14253:
URL: https://github.com/apache/druid/issues/14253
As there are many metrics are available which apach druid exposes, I would like to propose one in addition to that which allows us to monitor each running task memory consumption.
This task can be index/compaction/kill so and so and by use of this metric we should in situation to understand the memory consumption of each task. We can observe a particular task for particular data source by this and if there are memory enhancement required we can do a proper analysis for that particular data source and do a proactive monitoring.
I would like to contribute on this, little bit guidance would be really helpful for me.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [I] Implemention of new metric (druid)
Posted by "FrankChen021 (via GitHub)" <gi...@apache.org>.
FrankChen021 closed issue #14253: Implemention of new metric
URL: https://github.com/apache/druid/issues/14253
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [I] Implemention of new metric (druid)
Posted by "FrankChen021 (via GitHub)" <gi...@apache.org>.
FrankChen021 commented on issue #14253:
URL: https://github.com/apache/druid/issues/14253#issuecomment-1855131690
this metric shows the used heap memory of each druid process, including ingestion tasks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [I] Implemention of new metric (druid)
Posted by "FrankChen021 (via GitHub)" <gi...@apache.org>.
FrankChen021 commented on issue #14253:
URL: https://github.com/apache/druid/issues/14253#issuecomment-1855125489
@BigSecret1 BTW, Druid provides memory metric like _jvm/mem/used_ , what's the difference betwen this metric and your proposal?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [I] Implemention of new metric (druid)
Posted by "BigSecret1 (via GitHub)" <gi...@apache.org>.
BigSecret1 commented on issue #14253:
URL: https://github.com/apache/druid/issues/14253#issuecomment-1855175359
@FrankChen021 okay then it should be enough if it provides those details.
Also I want to start contributing on compaction side. Where I can find src related to compaction in druid repo?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [I] Implemention of new metric (druid)
Posted by "BigSecret1 (via GitHub)" <gi...@apache.org>.
BigSecret1 commented on issue #14253:
URL: https://github.com/apache/druid/issues/14253#issuecomment-1855130628
@FrankChen021 but does this metric gives the memory consumption of each running task? That's where I came up with proposal
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [I] Implemention of new metric (druid)
Posted by "FrankChen021 (via GitHub)" <gi...@apache.org>.
FrankChen021 commented on issue #14253:
URL: https://github.com/apache/druid/issues/14253#issuecomment-1855938210
For compaction, you can get started from the `CompactSegments` which is running on coordinator to submit compaction task periodically, and the `CompactionTask` which is scheduled on middle managers to run the compaction task.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [I] Implemention of new metric (druid)
Posted by "BigSecret1 (via GitHub)" <gi...@apache.org>.
BigSecret1 commented on issue #14253:
URL: https://github.com/apache/druid/issues/14253#issuecomment-1855119684
@FrankChen021 Nope I am not working on this as of know
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [I] Implemention of new metric (druid)
Posted by "FrankChen021 (via GitHub)" <gi...@apache.org>.
FrankChen021 commented on issue #14253:
URL: https://github.com/apache/druid/issues/14253#issuecomment-1840017811
Hi @BigSecret1 are you still working on this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [I] Implemention of new metric (druid)
Posted by "BigSecret1 (via GitHub)" <gi...@apache.org>.
BigSecret1 commented on issue #14253:
URL: https://github.com/apache/druid/issues/14253#issuecomment-1855131720
In our project it was a need to track this and eventually found we don't have any metrics as of now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [I] Implemention of new metric (druid)
Posted by "FrankChen021 (via GitHub)" <gi...@apache.org>.
FrankChen021 commented on issue #14253:
URL: https://github.com/apache/druid/issues/14253#issuecomment-1855940219
I will close this issue. If you find that the _jvm/mem/*_ metrics don't meet your need, you can open a new issue or PR. Thank you @BigSecret1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org