You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2022/02/09 18:05:36 UTC
[GitHub] [druid] mustafakmal opened a new issue #12247: Automated Cleanup of 'druid_supervisors' metadata table not working
mustafakmal opened a new issue #12247:
URL: https://github.com/apache/druid/issues/12247
### Affected Version
I am using druid-0.22.1
### Description
I have a 5 node cluster
- 1 master node (overlord/coordinator)
- 1 query node (router/broker)
- 3 data nodes (historical/middlemanager)
I am trying to enable 'Automated Cleanup' of metadata tables. I have configured Druid to use Postgres as a Metadata store. To enable automated cleanup I went through the [documentation](https://druid.apache.org/docs/latest/operations/clean-metadata-store.html#indexer-task-logs) and have configured the following configurations on the coordinator.
```
druid.service=druid/coordinator
# HTTP server threads
druid.coordinator.startDelay=PT30S
druid.coordinator.period=PT30S
# Configure this coordinator to also run as Overlord
druid.coordinator.asOverlord.enabled=true
druid.coordinator.asOverlord.overlordService=druid/overlord
druid.indexer.queue.startDelay=PT30S
druid.indexer.runner.type=remote
#Metadata Autocleanup
druid.coordinator.period.indexingPeriod=PT5M
druid.coordinator.kill.on=true
druid.coordinator.kill.period=PT10M
druid.coordinator.kill.durationToRetain=PT10M
druid.coordinator.kill.maxSegments=1000
druid.coordinator.period.metadataStoreManagementPeriod=PT10M
killAllDataSources=true
druid.coordinator.kill.supervisor.on=true
druid.coordinator.kill.audit.on=true
druid.coordinator.kill.rule.on=true
druid.coordinator.kill.compaction.on=true
druid.coordinator.kill.datasource.on=true
druid.coordinator.kill.supervisor.period=PT10M
druid.coordinator.kill.audit.period=PT10M
druid.coordinator.kill.rule.period=PT10M
druid.coordinator.kill.compaction.period=PT10M
druid.coordinator.kill.datasource.period=PT10M
druid.coordinator.kill.supervisor.durationToRetain=PT10M
druid.coordinator.kill.audit.durationToRetain=PT10M
druid.coordinator.kill.rule.durationToRetain=PT10M
druid.coordinator.kill.datasource.durationToRetain=PT10M
```
The 'druid_segments' & 'druid_pendingsegments' tables do get cleaned according to the 'durationToRetain' set and once the segments are in an unused state.
However, the 'druid_supervisors' retains all records even though I have terminated the supervisor and the 'durationToRetain' has passed. The coordinator logs dont show any errors. If there are any details needed let me know.
I am attaching a screenshot of the 'druid_supervisors' table.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] mustafakmal closed issue #12247: Automated Cleanup of 'druid_supervisors' metadata table not working
Posted by GitBox <gi...@apache.org>.
mustafakmal closed issue #12247:
URL: https://github.com/apache/druid/issues/12247
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org