Suggest to refactor the definition of batch job

[Yin Wan <ev...@gmail.com>]

Hi all,

once a user  create a job by user guide, it is actually bund to a scheduler
and then there may be some batch jobs running. seems i can not get the
metrics via batch job directly.

i also have a suggestion that you can refactor your batch job
implementation with spring batch, if that, you can easily add listeners to
update job status & metrics , that would be more simple and easy for us to
contribute it.

thanks,
Evan

Re: Suggest to refactor the definition of batch job

[William Guo <gu...@apache.org>]

hi Evan,

For batch job life cycle management, I agree spring batch has a great
solution for it.

But currently we are leveraging spark cluster to manage jobs,

Could you tell us , specifically, what are your requirements when using
spark jobs, what problems you have encountered?


Thanks,
William

On Thu, Apr 12, 2018 at 2:24 PM, Yin Wan <ev...@gmail.com> wrote:

> Hi all,
>
> once a user  create a job by user guide, it is actually bund to a
> scheduler and then there may be some batch jobs running. seems i can not
> get the metrics via batch job directly.
>
> i also have a suggestion that you can refactor your batch job
> implementation with spring batch, if that, you can easily add listeners to
> update job status & metrics , that would be more simple and easy for us to
> contribute it.
>
> thanks,
> Evan
>