You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flink.apache.org by Luan Cooper <gc...@gmail.com> on 2020/09/21 04:37:58 UTC

How to disconnect taskmanager via rest api?

Hi

We're running flink standalone cluster on k8s
when deleting a taskmanager pod manually, jobmanager *should disconnect it
immediately*

however no such rest api available right now
we have to wait `akka.tcp.timeout` which means 30s later or more

What if I want to disconnect tm via rest api
Which way did you suggest ?

1. add disconnectTaskManager to
org.apache.flink.runtime.dispatcher.Dispatcher
which means a new Interface

CompletableFuture<Acknowledge> disconnectTaskManager(JobID jobId,
ResourceID resourceId);

in org.apache.flink.runtime.webmonitor.RestfulGateway

2. Any other suggestions?

Thanks

Re: How to disconnect taskmanager via rest api?

Posted by Yang Wang <da...@gmail.com>.

I think this is an interesting feature, especially when deploying Flink
standalone clusters on K8s.
The TaskManager pods are started/stopped externally via kubectl or other
tools. When we need to stop
a TaskManager pod, even though the pod is deleted quickly, we have to wait
for a timeout so that it
disappears in the JobManager.

Only the ActiveResourceManager(e.g. YarnResourceManager,
KubernetesResourceManager) have
the ability to allocate/release TaskManagers. Maybe in the standalone mode,
we need to make the
TaskManager could deregister themselves during stop.

Best,
Yang

Luan Cooper <gc...@gmail.com> 于2020年9月23日周三 下午4:30写道：

> thanks
> I'll create a new issue for this feature on github
>
>
> On Mon, Sep 21, 2020 at 11:51 PM Timo Walther <tw...@apache.org> wrote:
>
>> Hi Luan,
>>
>> this sound more of a new feature request to me. Maybe you can already
>> open an issue for it.
>>
>> I will loop in Chesnay in CC if there is some possibility to achieve
>> this already?
>>
>> Regards,
>> Timo
>>
>> On 21.09.20 06:37, Luan Cooper wrote:
>> > Hi
>> >
>> > We're running flink standalone cluster on k8s
>> > when deleting a taskmanager pod manually, jobmanager *should disconnect
>> > it immediately*
>> >
>> > however no such rest api available right now
>> > we have to wait `akka.tcp.timeout` which means 30s later or more
>> >
>> > What if I want to disconnect tm via rest api
>> > Which way did you suggest ?
>> >
>> > 1. add disconnectTaskManager to
>> > org.apache.flink.runtime.dispatcher.Dispatcher
>> > which means a new Interface
>> >
>> > CompletableFuture<Acknowledge>disconnectTaskManager(JobID jobId,
>> ResourceID resourceId);
>> >
>> > in org.apache.flink.runtime.webmonitor.RestfulGateway
>> >
>> > 2. Any other suggestions?
>> >
>> > Thanks
>>
>>

Re: How to disconnect taskmanager via rest api?

Posted by Luan Cooper <gc...@gmail.com>.

thanks
I'll create a new issue for this feature on github


On Mon, Sep 21, 2020 at 11:51 PM Timo Walther <tw...@apache.org> wrote:

> Hi Luan,
>
> this sound more of a new feature request to me. Maybe you can already
> open an issue for it.
>
> I will loop in Chesnay in CC if there is some possibility to achieve
> this already?
>
> Regards,
> Timo
>
> On 21.09.20 06:37, Luan Cooper wrote:
> > Hi
> >
> > We're running flink standalone cluster on k8s
> > when deleting a taskmanager pod manually, jobmanager *should disconnect
> > it immediately*
> >
> > however no such rest api available right now
> > we have to wait `akka.tcp.timeout` which means 30s later or more
> >
> > What if I want to disconnect tm via rest api
> > Which way did you suggest ?
> >
> > 1. add disconnectTaskManager to
> > org.apache.flink.runtime.dispatcher.Dispatcher
> > which means a new Interface
> >
> > CompletableFuture<Acknowledge>disconnectTaskManager(JobID jobId,
> ResourceID resourceId);
> >
> > in org.apache.flink.runtime.webmonitor.RestfulGateway
> >
> > 2. Any other suggestions?
> >
> > Thanks
>
>

Re: How to disconnect taskmanager via rest api?

Posted by Timo Walther <tw...@apache.org>.

Hi Luan,

this sound more of a new feature request to me. Maybe you can already 
open an issue for it.

I will loop in Chesnay in CC if there is some possibility to achieve 
this already?

Regards,
Timo

On 21.09.20 06:37, Luan Cooper wrote:
> Hi
> 
> We're running flink standalone cluster on k8s
> when deleting a taskmanager pod manually, jobmanager *should disconnect 
> it immediately*
> 
> however no such rest api available right now
> we have to wait `akka.tcp.timeout` which means 30s later or more
> 
> What if I want to disconnect tm via rest api
> Which way did you suggest ?
> 
> 1. add disconnectTaskManager to 
> org.apache.flink.runtime.dispatcher.Dispatcher
> which means a new Interface
> 
> CompletableFuture<Acknowledge>disconnectTaskManager(JobID jobId, ResourceID resourceId);
> 
> in org.apache.flink.runtime.webmonitor.RestfulGateway
> 
> 2. Any other suggestions?
> 
> Thanks