You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by bat man <ti...@gmail.com> on 2021/09/13 14:59:26 UTC
Flink Native Kubernetes - Configuration kubernetes.flink.log.dir not working
Hi,
I am running a POC to evaluate Flink on Native Kubernetes. I tried changing
the default log location by using the configuration -
kubernetes.flink.log.dir
However, the job in application mode fails after bringing up the task
manager. This is the command I use -
./bin/flink run-application --target kubernetes-application
-Dkubernetes.cluster-id=flink-k8s-poc-app
-Dkubernetes.container.image=<custom-image>
-Dkubernetes.flink.log.dir="/var/log/containers"
local:///opt/flink/usrlib/uber.jar
Is there something else which needs to be done to write logs to a
different location like creating the folders in the custom image.
Thanks.
Re: Flink Native Kubernetes - Configuration kubernetes.flink.log.dir
not working
Posted by bat man <ti...@gmail.com>.
I was able to verify that this works fine
Dkubernetes.flink.log.dir="/var/log/containers" with Flink 1.13.1.
Initially there were some issues with the deployment, once fixed it worked
fine.
Cheers,
Hemant
On Sat, Sep 18, 2021 at 1:58 PM Yang Wang <da...@gmail.com> wrote:
> I think it might be a bug that "kubernetes.flink.log.dir" could not take
> effect. I have created a ticket[1].
>
> Could you please try with "-Denv.log.dir=/var/log/containers"?
>
> [1]. https://issues.apache.org/jira/browse/FLINK-24334
>
> Best,
> Yang
>
> Guowei Ma <gu...@gmail.com> 于2021年9月14日周二 下午4:48写道：
>
>> Hi
>>
>> Maybe you could try the `kubectl describe pod -n ${namespace}
>> ${podname}` to see what happened atm.
>>
>> Best,
>> Guowei
>>
>>
>> On Tue, Sep 14, 2021 at 2:58 PM bat man <ti...@gmail.com> wrote:
>>
>>> Hello Guowei,
>>>
>>> The pods terminate almost within a second so am unable to pull any logs.
>>> Is there any way I can pull the logs?
>>>
>>> Thanks,
>>> Hemant
>>>
>>> On Tue, Sep 14, 2021 at 12:22 PM Guowei Ma <gu...@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> Could you share some logs when the job fails?
>>>>
>>>> Best,
>>>> Guowei
>>>>
>>>>
>>>> On Mon, Sep 13, 2021 at 10:59 PM bat man <ti...@gmail.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I am running a POC to evaluate Flink on Native Kubernetes. I tried
>>>>> changing the default log location by using the configuration -
>>>>> kubernetes.flink.log.dir
>>>>> However, the job in application mode fails after bringing up the task
>>>>> manager. This is the command I use -
>>>>>
>>>>> ./bin/flink run-application --target kubernetes-application
>>>>> -Dkubernetes.cluster-id=flink-k8s-poc-app
>>>>> -Dkubernetes.container.image=<custom-image>
>>>>> -Dkubernetes.flink.log.dir="/var/log/containers"
>>>>> local:///opt/flink/usrlib/uber.jar
>>>>>
>>>>> Is there something else which needs to be done to write logs to a
>>>>> different location like creating the folders in the custom image.
>>>>>
>>>>> Thanks.
>>>>>
>>>>
Re: Flink Native Kubernetes - Configuration kubernetes.flink.log.dir
not working
Posted by Yang Wang <da...@gmail.com>.
I think it might be a bug that "kubernetes.flink.log.dir" could not take
effect. I have created a ticket[1].
Could you please try with "-Denv.log.dir=/var/log/containers"?
[1]. https://issues.apache.org/jira/browse/FLINK-24334
Best,
Yang
Guowei Ma <gu...@gmail.com> 于2021年9月14日周二 下午4:48写道：
> Hi
>
> Maybe you could try the `kubectl describe pod -n ${namespace} ${podname}`
> to see what happened atm.
>
> Best,
> Guowei
>
>
> On Tue, Sep 14, 2021 at 2:58 PM bat man <ti...@gmail.com> wrote:
>
>> Hello Guowei,
>>
>> The pods terminate almost within a second so am unable to pull any logs.
>> Is there any way I can pull the logs?
>>
>> Thanks,
>> Hemant
>>
>> On Tue, Sep 14, 2021 at 12:22 PM Guowei Ma <gu...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> Could you share some logs when the job fails?
>>>
>>> Best,
>>> Guowei
>>>
>>>
>>> On Mon, Sep 13, 2021 at 10:59 PM bat man <ti...@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> I am running a POC to evaluate Flink on Native Kubernetes. I tried
>>>> changing the default log location by using the configuration -
>>>> kubernetes.flink.log.dir
>>>> However, the job in application mode fails after bringing up the task
>>>> manager. This is the command I use -
>>>>
>>>> ./bin/flink run-application --target kubernetes-application
>>>> -Dkubernetes.cluster-id=flink-k8s-poc-app
>>>> -Dkubernetes.container.image=<custom-image>
>>>> -Dkubernetes.flink.log.dir="/var/log/containers"
>>>> local:///opt/flink/usrlib/uber.jar
>>>>
>>>> Is there something else which needs to be done to write logs to a
>>>> different location like creating the folders in the custom image.
>>>>
>>>> Thanks.
>>>>
>>>
Re: Flink Native Kubernetes - Configuration kubernetes.flink.log.dir
not working
Posted by Guowei Ma <gu...@gmail.com>.
Hi
Maybe you could try the `kubectl describe pod -n ${namespace} ${podname}`
to see what happened atm.
Best,
Guowei
On Tue, Sep 14, 2021 at 2:58 PM bat man <ti...@gmail.com> wrote:
> Hello Guowei,
>
> The pods terminate almost within a second so am unable to pull any logs.
> Is there any way I can pull the logs?
>
> Thanks,
> Hemant
>
> On Tue, Sep 14, 2021 at 12:22 PM Guowei Ma <gu...@gmail.com> wrote:
>
>> Hi,
>>
>> Could you share some logs when the job fails?
>>
>> Best,
>> Guowei
>>
>>
>> On Mon, Sep 13, 2021 at 10:59 PM bat man <ti...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I am running a POC to evaluate Flink on Native Kubernetes. I tried
>>> changing the default log location by using the configuration -
>>> kubernetes.flink.log.dir
>>> However, the job in application mode fails after bringing up the task
>>> manager. This is the command I use -
>>>
>>> ./bin/flink run-application --target kubernetes-application
>>> -Dkubernetes.cluster-id=flink-k8s-poc-app
>>> -Dkubernetes.container.image=<custom-image>
>>> -Dkubernetes.flink.log.dir="/var/log/containers"
>>> local:///opt/flink/usrlib/uber.jar
>>>
>>> Is there something else which needs to be done to write logs to a
>>> different location like creating the folders in the custom image.
>>>
>>> Thanks.
>>>
>>
Re: Flink Native Kubernetes - Configuration kubernetes.flink.log.dir
not working
Posted by bat man <ti...@gmail.com>.
Hello Guowei,
The pods terminate almost within a second so am unable to pull any logs. Is
there any way I can pull the logs?
Thanks,
Hemant
On Tue, Sep 14, 2021 at 12:22 PM Guowei Ma <gu...@gmail.com> wrote:
> Hi,
>
> Could you share some logs when the job fails?
>
> Best,
> Guowei
>
>
> On Mon, Sep 13, 2021 at 10:59 PM bat man <ti...@gmail.com> wrote:
>
>> Hi,
>>
>> I am running a POC to evaluate Flink on Native Kubernetes. I tried
>> changing the default log location by using the configuration -
>> kubernetes.flink.log.dir
>> However, the job in application mode fails after bringing up the task
>> manager. This is the command I use -
>>
>> ./bin/flink run-application --target kubernetes-application
>> -Dkubernetes.cluster-id=flink-k8s-poc-app
>> -Dkubernetes.container.image=<custom-image>
>> -Dkubernetes.flink.log.dir="/var/log/containers"
>> local:///opt/flink/usrlib/uber.jar
>>
>> Is there something else which needs to be done to write logs to a
>> different location like creating the folders in the custom image.
>>
>> Thanks.
>>
>
Re: Flink Native Kubernetes - Configuration kubernetes.flink.log.dir
not working
Posted by Guowei Ma <gu...@gmail.com>.
Hi,
Could you share some logs when the job fails?
Best,
Guowei
On Mon, Sep 13, 2021 at 10:59 PM bat man <ti...@gmail.com> wrote:
> Hi,
>
> I am running a POC to evaluate Flink on Native Kubernetes. I tried
> changing the default log location by using the configuration -
> kubernetes.flink.log.dir
> However, the job in application mode fails after bringing up the task
> manager. This is the command I use -
>
> ./bin/flink run-application --target kubernetes-application
> -Dkubernetes.cluster-id=flink-k8s-poc-app
> -Dkubernetes.container.image=<custom-image>
> -Dkubernetes.flink.log.dir="/var/log/containers"
> local:///opt/flink/usrlib/uber.jar
>
> Is there something else which needs to be done to write logs to a
> different location like creating the folders in the custom image.
>
> Thanks.
>