You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by GitBox <gi...@apache.org> on 2021/10/07 17:08:50 UTC
[GitHub] [druid] pchang388 opened a new issue #11780: Persisting Druid Peon Task Logs
pchang388 opened a new issue #11780:
URL: https://github.com/apache/druid/issues/11780
We have our middle manager and historical proceeses running on the same VMs. When trouble shooting, we take a look at the druid peon logs in `{druid_base_dir}/var/druid/tasks/index_kafka_*/log`.
After the task is done (success or fail), the directory in the tasks dir gets cleaned up instantly and the log file is no longer present. This can make it hard to troubleshoot since you can't look at past peon logs.
Is there a configuration option to retain these logs for a longer period of time or is the recommendation to use it's own log4j file to log to the common var/sv directory?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] pchang388 commented on issue #11780: Persisting Druid Peon Task Logs
Posted by GitBox <gi...@apache.org>.
pchang388 commented on issue #11780:
URL: https://github.com/apache/druid/issues/11780#issuecomment-940165451
Hi @ludun624 - thanks for the response. My goal is to actually use a log shipping agent like splunk/log stash to a remote store or querying/alerting but I also want the ability to troubleshoot locally if needed (in case of shipping issues, etc.).
So S3 or remote would not be preferred. Using local files are okay and I'm looking to see how we can persist these peon tasks logs for a few hours/days after completion instead of being deleted right after.
But I appreciate the input you gave.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] pchang388 edited a comment on issue #11780: Persisting Druid Peon Task Logs
Posted by GitBox <gi...@apache.org>.
pchang388 edited a comment on issue #11780:
URL: https://github.com/apache/druid/issues/11780#issuecomment-940165451
Hi @ludun624 - thanks for the response. My goal is to actually use a log shipping agent like splunk/log stash to a remote store for querying/alerting but I also want the ability to troubleshoot locally if needed (in case of shipping issues, etc.).
So S3 or remote would not be preferred. Using local files are okay and I'm looking to see how we can persist these peon tasks logs for a few hours/days after completion instead of being deleted right after.
But I appreciate the input you gave.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] pchang388 edited a comment on issue #11780: Persisting Druid Peon Task Logs
Posted by GitBox <gi...@apache.org>.
pchang388 edited a comment on issue #11780:
URL: https://github.com/apache/druid/issues/11780#issuecomment-940165451
Hi @ludun624 - thanks for the response. My goal is to actually use a log shipping agent like splunk/log stash to a remote store for querying/alerting but I also want the ability to troubleshoot locally if needed (in case of shipping issues, etc.).
So S3 or remote would not be preferred. Using local files and remote shipping is what I'm thinking we need to avoid issues with remote systems being down and losing logs, etc. I'm looking to see how we can persist these peon tasks logs for a few hours/days in local files after completion instead of being deleted right after.
But I appreciate the input you gave.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] pchang388 edited a comment on issue #11780: Persisting Druid Peon Task Logs
Posted by GitBox <gi...@apache.org>.
pchang388 edited a comment on issue #11780:
URL: https://github.com/apache/druid/issues/11780#issuecomment-940165451
Hi @ludun624 - thanks for the response. My goal is to actually use a log shipping agent like splunk/log stash to a remote store for querying/alerting but I also want the ability to troubleshoot locally if needed (in case of shipping issues, etc.).
So S3 or remote would not be preferred. Using local files and remote shipping is what I'm thinking we need to avoid issues with remote systems being down and losing logs, etc. I'm looking to see how we can persist these peon tasks logs for a few hours/days in local files after completion instead of being deleted right after. So if there are log shipping issues, we can still see the logs and troubleshoot.
But I appreciate the input you gave.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
[GitHub] [druid] ludun624 commented on issue #11780: Persisting Druid Peon Task Logs
Posted by GitBox <gi...@apache.org>.
ludun624 commented on issue #11780:
URL: https://github.com/apache/druid/issues/11780#issuecomment-939826537
You actually have a few decent options here if you are ok with persist logs externally, you can find the config [here](https://druid.apache.org/docs/latest/configuration/index.html#task-logging), I tried s3 and it works, please update here if that solved your problem @pchang388
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org