You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Sriram Ganesh <sr...@gmail.com> on 2022/05/25 05:29:33 UTC

Issue in seeing job manager and task manager logs in history sever

Hi Everyone,

I am Sriram Ganesh part of the Microsoft Flink platform team. I recently
went through the history server setup in Flink.

I found an issue in the history server that was not showing the job manager
and task manager logs. We are running clusters in session mode and using
native Kubernetes orchestration.

Is there a way we can get Job manager and task manager logs in the history
server?. Couldn't find any proper documentation regarding this.

Any help would be appreciated. Thanks in advance.

Thanks,
Sriram G

Re: Issue in seeing job manager and task manager logs in history sever

Posted by Sriram Ganesh <sr...@gmail.com>.
Thank you for the clarification. Sure will use your inputs.

On Wed, May 25, 2022, 12:34 Chesnay Schepler <ch...@apache.org> wrote:

> This is currently not supported, no.
>
> There are some conceptual issues.
> For example, session mode can't isolate the logs between jobs, sou you'd
> end up with gargantuan logs being archived for every job.
> Similarly, and form of log rotation would reduce the value of such a
> mechanism.
>
> I would recommend to set up some logging storage instead, e.g., logstash.
>
> On 25/05/2022 07:29, Sriram Ganesh wrote:
> > Hi Everyone,
> >
> > I am Sriram Ganesh part of the Microsoft Flink platform team. I recently
> > went through the history server setup in Flink.
> >
> > I found an issue in the history server that was not showing the job
> manager
> > and task manager logs. We are running clusters in session mode and using
> > native Kubernetes orchestration.
> >
> > Is there a way we can get Job manager and task manager logs in the
> history
> > server?. Couldn't find any proper documentation regarding this.
> >
> > Any help would be appreciated. Thanks in advance.
> >
> > Thanks,
> > Sriram G
> >
>
>

Re: Issue in seeing job manager and task manager logs in history sever

Posted by Chesnay Schepler <ch...@apache.org>.
This is currently not supported, no.

There are some conceptual issues.
For example, session mode can't isolate the logs between jobs, sou you'd 
end up with gargantuan logs being archived for every job.
Similarly, and form of log rotation would reduce the value of such a 
mechanism.

I would recommend to set up some logging storage instead, e.g., logstash.

On 25/05/2022 07:29, Sriram Ganesh wrote:
> Hi Everyone,
>
> I am Sriram Ganesh part of the Microsoft Flink platform team. I recently
> went through the history server setup in Flink.
>
> I found an issue in the history server that was not showing the job manager
> and task manager logs. We are running clusters in session mode and using
> native Kubernetes orchestration.
>
> Is there a way we can get Job manager and task manager logs in the history
> server?. Couldn't find any proper documentation regarding this.
>
> Any help would be appreciated. Thanks in advance.
>
> Thanks,
> Sriram G
>