You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by "IgorBerman (via GitHub)" <gi...@apache.org> on 2024/04/30 16:39:52 UTC
[I] Kill task using exclusive locks currently and no way to change it (druid)
IgorBerman opened a new issue, #16361:
URL: https://github.com/apache/druid/issues/16361
### Description
Currently kill tasks take exclusive locks only(hardcoded). I believe there will be benefit to permit it in some scenarios to run without lock at all or append lock(TBD)
### Motivation
In concurrent append and replace lock system we can run kafka ingestors in parallel to compaction tasks. However multiple compaction will create more unused segments, so to keep it controllable it will be good to clean every segment after compaction task finished. However currently kafka ingestors lock intervals with append lock, so kill task can't cleanup unused segments.
So my proposition is to permit kill task to run with replace lock(same as of compaction task) or no lock at all if we not marking segments as used(for any reason).
This should permit kill , compaction and kafka ingestors tasks to run concurrently, thus achieving best parallelism
@AmatyaAvadhanula cc
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [I] Kill task using exclusive locks currently and no way to change it (druid)
Posted by "kfaraz (via GitHub)" <gi...@apache.org>.
kfaraz closed issue #16361: Kill task using exclusive locks currently and no way to change it
URL: https://github.com/apache/druid/issues/16361
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org