How long it took a Flink Job to start up ?

[Vijay Balakrishnan <bv...@gmail.com>]

Hi,
I am trying to figure how long it took a Flink Job to start up ?
I used /jobs/overview and it gave me just the start-time as a long value.
The Flink DashBoard UI shows the Start-Time as auser friendly date. I am
trying to find or compute how long it took for the Flink Job to start up ?
Right now, I am manually looking at when the JobManager is up and running
in the logs and then subtracting the UI start time to get an approximate
idea. Trying to automate the process.

Attaching 2 screenshots of the /jobs/overview response and the Flink
DashBoard UI Start Time.


TIA,

Re: How long it took a Flink Job to start up ?

[Till Rohrmann <tr...@apache.org>]

Hi Vijay,

one way to solve this problem is to use Flink's REST API. By querying
REST_ENDPOINT:PORT/jobs/JOB_ID [1] you obtain the job information. The job
information contains the start-time for the whole job and the start-time
for every vertex of the job. By subtracting the job's start time from the
maximum start time of all vertices you should obtain the time it takes to
bring up the whole topology (this only applies to streaming jobs since in
batch jobs not all vertices/operators have to run at the same time).

[1]
https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/rest_api.html#jobs-jobid

Cheers,
Till

On Fri, Jul 31, 2020 at 7:56 PM Vijay Balakrishnan <bv...@gmail.com>
wrote:

> Hi,
> I am trying to figure how long it took a Flink Job to start up ?
> I used /jobs/overview and it gave me just the start-time as a long value.
> The Flink DashBoard UI shows the Start-Time as auser friendly date. I am
> trying to find or compute how long it took for the Flink Job to start up ?
> Right now, I am manually looking at when the JobManager is up and running
> in the logs and then subtracting the UI start time to get an approximate
> idea. Trying to automate the process.
>
> Attaching 2 screenshots of the /jobs/overview response and the Flink
> DashBoard UI Start Time.
>
>
> TIA,
>