You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Bariša <ba...@gmail.com> on 2018/10/09 10:21:13 UTC
Wondering how to check if taskmanager is registered with the
jobmanager, without asking job manager
As part of deploying task managers and job managers, I'd like to expose
healthcheck on both task managers and job managers.
For the task managers, one of the requirements that they are healthy, is
that they have successfully registered themselves with the job manager.
Is there a way to achieve this, without making a call to job manager ( to
do that, I first need to make a call to the zookeeper to find the job
manager, so I'm trying to simplify the health check ).
Ideally, taskmanager would have a metric that says, ( am registered ), but
afaik, that doesn't exist
https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/metrics.html#cluster
P.S.
This is my first post in the email list, happy to update/change my
question, if I messed up, or misunderstood something.
Cheers,
Barisa
Re: Wondering how to check if taskmanager is registered with the
jobmanager, without asking job manager
Posted by Barisa Obradovic <bb...@gmail.com>.
Version of flink I'm using is 1.6.1, if that helps.
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
Re: Wondering how to check if taskmanager is registered with the
jobmanager, without asking job manager
Posted by Piotr Nowojski <pi...@data-artisans.com>.
You’re welcome :)
> On 10 Oct 2018, at 10:28, Bariša <ba...@gmail.com> wrote:
>
> Thnx Piotr. I agree, that would work. It's a bit chicken and the egg problem, since at that point we can't just spin up a task manager, and have it register itself, we need to have flinkmanager know how many task managers should be there. Bit more logic, but doable. Thnx for the tip.
>
> Cheers,
> Barisa
>
> On Wed, 10 Oct 2018 at 09:05, Piotr Nowojski <piotr@data-artisans.com <ma...@data-artisans.com>> wrote:
> Hi,
>
> I don’t think that’s exposed on the TaskManager.
>
> Maybe it would simplify things a bit if you implement this as a single “JobManager” health check, not multiple TaskManagers health check - for example verify that there are expected number of registered TaskManagers. It might cover your case.
>
> Piotrek
>
>> On 9 Oct 2018, at 12:21, Bariša <barisa.obradovic@gmail.com <ma...@gmail.com>> wrote:
>>
>> As part of deploying task managers and job managers, I'd like to expose healthcheck on both task managers and job managers.
>>
>> For the task managers, one of the requirements that they are healthy, is that they have successfully registered themselves with the job manager.
>>
>> Is there a way to achieve this, without making a call to job manager ( to do that, I first need to make a call to the zookeeper to find the job manager, so I'm trying to simplify the health check ).
>>
>> Ideally, taskmanager would have a metric that says, ( am registered ), but afaik, that doesn't exist https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/metrics.html#cluster <https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/metrics.html#cluster>
>>
>>
>> P.S.
>> This is my first post in the email list, happy to update/change my question, if I messed up, or misunderstood something.
>>
>> Cheers,
>> Barisa
>
Re: Wondering how to check if taskmanager is registered with the
jobmanager, without asking job manager
Posted by Bariša <ba...@gmail.com>.
Thnx Piotr. I agree, that would work. It's a bit chicken and the egg
problem, since at that point we can't just spin up a task manager, and have
it register itself, we need to have flinkmanager know how many task
managers should be there. Bit more logic, but doable. Thnx for the tip.
Cheers,
Barisa
On Wed, 10 Oct 2018 at 09:05, Piotr Nowojski <pi...@data-artisans.com>
wrote:
> Hi,
>
> I don’t think that’s exposed on the TaskManager.
>
> Maybe it would simplify things a bit if you implement this as a single
> “JobManager” health check, not multiple TaskManagers health check - for
> example verify that there are expected number of registered TaskManagers.
> It might cover your case.
>
> Piotrek
>
> On 9 Oct 2018, at 12:21, Bariša <ba...@gmail.com> wrote:
>
> As part of deploying task managers and job managers, I'd like to expose
> healthcheck on both task managers and job managers.
>
> For the task managers, one of the requirements that they are healthy, is
> that they have successfully registered themselves with the job manager.
>
> Is there a way to achieve this, without making a call to job manager ( to
> do that, I first need to make a call to the zookeeper to find the job
> manager, so I'm trying to simplify the health check ).
>
> Ideally, taskmanager would have a metric that says, ( am registered ), but
> afaik, that doesn't exist
> https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/metrics.html#cluster
>
>
> P.S.
> This is my first post in the email list, happy to update/change my
> question, if I messed up, or misunderstood something.
>
> Cheers,
> Barisa
>
>
>
Re: Wondering how to check if taskmanager is registered with the
jobmanager, without asking job manager
Posted by Piotr Nowojski <pi...@data-artisans.com>.
Hi,
I don’t think that’s exposed on the TaskManager.
Maybe it would simplify things a bit if you implement this as a single “JobManager” health check, not multiple TaskManagers health check - for example verify that there are expected number of registered TaskManagers. It might cover your case.
Piotrek
> On 9 Oct 2018, at 12:21, Bariša <ba...@gmail.com> wrote:
>
> As part of deploying task managers and job managers, I'd like to expose healthcheck on both task managers and job managers.
>
> For the task managers, one of the requirements that they are healthy, is that they have successfully registered themselves with the job manager.
>
> Is there a way to achieve this, without making a call to job manager ( to do that, I first need to make a call to the zookeeper to find the job manager, so I'm trying to simplify the health check ).
>
> Ideally, taskmanager would have a metric that says, ( am registered ), but afaik, that doesn't exist https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/metrics.html#cluster <https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/metrics.html#cluster>
>
>
> P.S.
> This is my first post in the email list, happy to update/change my question, if I messed up, or misunderstood something.
>
> Cheers,
> Barisa