[GitHub] jihoonson edited a comment on issue #5979: Kafka Indexing Service lagging every hour

[GitBox <gi...@apache.org>]

jihoonson edited a comment on issue #5979: Kafka Indexing Service lagging every hour
URL: https://github.com/apache/incubator-druid/issues/5979#issuecomment-404276658
 
 
   @erankor 
   
   1. That SQL is to list all _used_ (live) segments falling in an interval, and it can be called 1) whenever a new segment is allocated in KIS and 2) when a taskLock is acquired in batch ingestion (like index or merge tasks). 
   a. Yes, you can delete them, but I would recommend to use the kill task rather than manually deleting rows. You can set `druid.coordinator.kill.on` to true for coordinators to periodically kill segments. Please check http://druid.io/docs/0.10.0/configuration/coordinator.html#coordinator-operation for details.
   b. I haven't tested, but sounds making sense. 
   2. The `druid_tasklogs` is not actually being used, so you can freely delete all entries in that table. Please check https://github.com/apache/incubator-druid/issues/5859. For `druid_tasks`, yes you can delete old entries from that table. One thing I'm curious is, do you have an actual issue of too large `druid_tasks` table? or are you just checking all tables? If you have an actual issue, it may make sense to support auto cleanup for `druid_tasks` table too.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@druid.apache.org
For additional commands, e-mail: dev-help@druid.apache.org